LLama.cpp

Installation

apt install hipcc rocm-opencl-icd rocminfo clinfo apt install libvulkan-dev glslc spirv-headers vulkan-tools

git clone git@github.com:ggml-org/llama.cpp.git —depth=3 cd llama.cpp cmake -B build -DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release cmake —build build —config Release -j$(nproc)

./build/bin/llama-cli
—hf-repo unsloth/Qwen3.6-27B-GGUF —hf-file Qwen3.6-27B-Q4_K_M.gguf
-p “Write a quick Python script to parse JSON.”
-ngl 99

./build/bin/llama-cli
-hf DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:Q4_K_M
-p “Explain quantum computing briefly.”
-ngl 99

./build/bin/llama-cli
-hf DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:Q5_K_M
-t 12
-c 16384
-p “You are an expert developer. Explain how lock-free ring buffers work in C++.”

./build/bin/llama-server
-hf DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:Q5_K_M
-t 12
-c 16384
—port 8080

Overview of source code

---

1. Web Server (llama-server)

Located in tools/server:

• HTTP/HTTPS Server Engine: cpp-httplib
  A header-only C++11 HTTP/HTTPS library supporting multithreading, chunked transfer, and Server-Sent Events (SSE) for streaming model responses.
• TLS / Security: OpenSSL (optional CMake build dependency via LLAMA_OPENSSL) for enabling HTTPS support in cpp-httplib.
• JSON Processing: nlohmann/json
  Modern C++ header-only library for JSON parsing, serialization, and JSON-Schema validation.
• Chat Templates & Templating: Custom in-house Jinja engine (common/jinja) for rendering Jinja chat templates.
• Structured Output & Parsing:
  • In-house PEG Parser (common/chat-peg-parser.cpp, common/peg-parser.cpp) for parsing model outputs.
  • llguidance (optional integration with Guidance-AI library written in Rust for constrained sampling).

---

2. Embedded Web UI (llama-ui)

Located in tools/ui, configured via package.json. The UI is compiled into static assets and embedded directly into the C++ binary via embed.cpp:

• Framework & Build System:
  • Svelte 5 (svelte, @sveltejs/kit, @sveltejs/adapter-static)
  • Vite 7 (vite)
• Styling & UI Primitives:
  • Tailwind CSS v4 (tailwindcss, @tailwindcss/vite, @tailwindcss/typography, @tailwindcss/forms)
  • Bits UI (bits-ui - unstyled component primitives)
  • Lucide Svelte (@lucide/svelte - SVG icons)
  • clsx & tailwind-merge & tailwind-variants (dynamic class management)
  • mode-watcher (theme toggle management)
• Markdown, Syntax Highlighting & Math:
  • unified / remark / rehype (Markdown AST processing)
  • highlight.js (code syntax highlighting)
  • KaTeX (rehype-katex for rendering LaTeX math formulas)
  • Mermaid (mermaid for rendering diagrams in chat responses)
• Storage & Client Protocols:
  • Dexie.js (dexie - IndexedDB wrapper for local browser storage)
  • DOMPurify (dompurify - XSS sanitization)
  • Model Context Protocol SDK (@modelcontextprotocol/sdk)
  • Workbox & @vite-pwa (Progressive Web App support)
  • fflate (client-side compression)
  • pdfjs-dist (client-side PDF rendering/extraction)

---

3. Other Core C/C++ Vendor Libraries

Located in vendor:

• sheredom/subprocess.h: Cross-platform C library for subprocess execution and IPC.
• stb (stb_image.h, stb_image_write.h): Single-header image decoding and encoding.
• miniaudio: Single-header audio playback and capture library (used for speech/multimodal features).