Quick Start

srz connects to any OpenAI API-compliant LLM server. While cloud provider APIs are supported, running a local LLM is strongly preferred for complete privacy, zero subscription fees, and offline capability.

💡 New to Local LLMs?
A local LLM runner (such as Llama GUI, llama.cpp, or Ollama) is an application that runs language models directly on your computer’s CPU or GPU. This ensures 100% privacy—no text, summaries, transcripts, or personal reading habits ever leave your machine.

1. LLM Backend Setup

Recommended Local Backends (Default Port 9931)

srz connects by default to http://localhost:9931/v1, requiring zero configuration for official desktop runners:

• macOS: Install Llama-macOS. Launch the application, load any GGUF model, and srz connects immediately.
• Windows: Install Llama-Windows. Launch the application, load a model, and start summarizing.
• Linux: Build or run llama.cpp:

  # Start llama-server on port 9931 for zero configuration:
  llama-server -m model.gguf --port 9931
  
  # Or on default port 8080:
  llama-server -m model.gguf --port 8080
  export OPENAI_BASE_URL="http://localhost:8080/v1"
  

Other Local & Cloud Backends

• Ollama: ollama run mistral (connects on port 11434):

  export OPENAI_BASE_URL="http://localhost:11434/v1" OPENAI_MODEL="mistral"
  

• LM Studio: Start Local Server on port 1234:

  export OPENAI_BASE_URL="http://localhost:1234/v1"
  

• Cloud Providers (OpenAI, Anthropic, Google, Groq):

  export OPENAI_BASE_URL="https://api.openai.com/v1" OPENAI_MODEL="gpt-4o-mini" OPENAI_API_KEY="sk-..."
  

Verify Connectivity & Recommendations

Get an instant recommendation for your hardware and test connection health:

# Instant model and context window recommendation
srz recommend

# Full connectivity, cache health, and database diagnostics
srz doctor

# List models reported by the backend
srz models

2. Recommended Models & Context Length by Hardware

Before running your first summary, you can easily determine which model and context window to select based on your computer’s RAM.

1-Command Auto-Advisor: srz recommend

Run srz recommend to inspect your host hardware and receive customized model and context length recommendations:

srz recommend

=== srz Recommended Model & Context Length ===

Host Machine:
  • Device: Apple M5 Pro
  • Memory: 24.0 GB RAM (24–36 GB Tier (High-End Pro / Max Laptop))

Recommended Local LLM Setup (Llama GUI / llama.cpp):
  • Model:         gemma-4
  • Size:          E4B
  • Quant:         Q8_0
  • Disk Size:     ~8.6 GB
  • Context:       64k ctx
  • Estimated RAM: ~10 GB

Hardware Tier Reference Matrix

Single-Pass vs. Map-Reduce Chunking

How srz processes your document depends on the relationship between the document length and the server’s available context window (n_ctx).

When a document’s token count is within approximately 70% of the active context window, srz synthesizes the text in a single pass (1-Pass Mode). This mode yields the highest-quality summaries because the model maintains complete narrative and thematic awareness across all chapters simultaneously.

When a document exceeds this 70% threshold—such as a 300-page book or a large collection of research papers—srz automatically transitions into Chunking Mode. The engine splits the text into token-budgeted chunks with overlapping margins, summarizes each chunk in parallel across available server worker slots (total_slots), and finally executes a reduction pass to produce a cohesive, unified synthesis without running out of memory.

Why Google Gemma 4 (Q8_0) on Laptops

Selecting the appropriate local model requires balancing comprehension quality with battery efficiency and RAM usage. For everyday reading on laptops and workstations, srz recommends Google’s Gemma 4 series with 8-bit quantization (Q8_0).

Gemma 4 models offer exceptional reasoning density and instruction adherence per parameter, outperforming earlier-generation 7B and 8B models while drawing less electrical power. While 4-bit quantizations (Q4_K_M) reduce download sizes, they often degrade nuanced arguments and introduce hallucinations during long-form synthesis. In contrast, 8-bit Q8_0 preserves near-lossless precision, fits comfortably in unified memory, and scales natively to 128k context windows without degradation. Furthermore, querying the runner’s native /tokenize endpoint ensures zero tokenization drift between the CLI and the inference engine.

3. Basic Usage Examples

Summarize a Webpage or Article

Pass any URL directly. srz extracts clean text using Mozilla Readability, stripping ads and sidebars:

srz "https://en.wikipedia.org/wiki/Rust_(programming_language)"

Summarize a YouTube Video

Extract and synthesize closed-caption transcripts from YouTube URLs:

srz "https://youtu.be/5C_HPTJg5ek"

Summarize a Local PDF or Document

Pass a path to a .pdf, .html, .md, or .txt file:

srz assets/document.pdf

Summarize Unix Standard Input

Pipe terminal command output, logs, or git diffs directly into srz:

git diff | srz -
cat server.log | srz -

Summarize a Book by Title

srz queries Open Library and Wikipedia to enrich book metadata and synopses before summarizing:

srz "The Rust Programming Language" --open

Summarize Live News & RSS Feeds

Aggregate and synthesize news stories from Google News RSS or summarize direct syndication feeds:

# Recent news on a topic (past week by default)
srz news "Rust language updates"

# Filter by recency (day, week, month, all)
srz news "artificial intelligence" -t day

# Or summarize any RSS, Atom, or JSON feed directly
srz "https://blog.rust-lang.org/feed.xml"

4. Interactive Grounded Q&A (srz ask)

Ask targeted questions or start an interactive chat session grounded directly in document text (persisted to DuckDB):

# Ask a question about your most recently summarized item
srz ask "What are the main findings?"

# Ask a question about a specific document or file
srz ask assets/document.pdf "What are the primary conclusions?"

# Launch an interactive multi-turn chat REPL (type 'exit' or Ctrl+D to quit)
srz ask -i

# Inspect and resume saved chat sessions from DuckDB
srz ask --chats
srz ask --resume 1
srz ask --show-chat 1

5. Terminal Explorer & Offline Library Search

Full-Screen Ratatui Terminal Dashboard (srz browse)

Explore your reading history, filter items by category, copy summaries to clipboard, or read in fullscreen:

srz browse

Fast Offline BM25 Search (srz find)

Search through past summaries stored in DuckDB without network access:

srz find "memory safety"

Analytics & Maintenance

srz history        # View recent reads and word counts
srz stats          # Reading streak, time saved, and model token speeds
srz cache size     # View disk usage
srz cache prune    # Evict items older than 14 days