03 / Research / retrieval / AI

Search finds documents.
Research needs evidence.

A Blazor Server application that crawls a subreddit, embeds the discussion into a vector store, and answers questions with citations through a real-time chat interface.

Live product Production

Source Answer Rerank Information pipeline / evidence path

Information pipeline
Source material

PGVECTOR / SIGNALR

Each moving block is a chunk of source material. It is collected, split, embedded, retrieved, reranked, and only then used as context for an answer.

01 / Context

A community is not a search box

Researchers and founders want signal from a subreddit — recurring pain points, emerging trends, common questions — without reading thousands of posts by hand.

Keyword search on Reddit surfaces individual threads, not synthesis. Generic scraping produces raw text with no way to ask it a question and no obligation to show where an answer came from.

02 / Constraints

Volume, noise, and a live wait

The crawler has to respect Reddit API rate limits. Short comments make naive vector search noisy. Embedding volume is continuous, not a one-off batch.

A crawl of a large community cannot hide behind a spinner. The operator needs to see ingestion progress while the corpus is still being shaped.

Answers have to carry citations back to threads. A fluent summary without evidence is not research.

03 / System

An information-processing pipeline

Posts and comments enter through the Reddit API. Discussion is split into retrieval-sized chunks and embedded with a local Ollama model. Chunks and vectors land in PostgreSQL with pgvector, still linked to the source thread.

A question triggers vector retrieval and reranking before any context reaches the language model. The chat response streams back with citations instead of a bare summary.

Ingestion is a streaming background pipeline. Live progress is pushed to the browser over SignalR.

04 / Decisions

Decisions made against constraints.

Decision / 01

Embed locally, generate elsewhere

Problem: Ingestion embeds continuously as posts arrive. A metered hosted embedding API would add cost and a hard dependency to a step that does not need frontier-model quality.
Decision: Run embeddings through a local Ollama model. Keep synthesis on OpenRouter so the generation model can change without a code change.
Why: Embedding is a volume problem. Synthesis is a quality problem. Those workloads do not want the same vendor, the same cost model, or the same failure domain.

Decision / 02

Retrieve, then rerank, then generate

Problem: Raw vector similarity on short Reddit comments returns near-duplicates and tangential matches. A larger K would only pour more noise into the context window.
Decision: Take an initial candidate set from pgvector, rerank it, then send the reduced context to the language model.
Why: Reranking improved answer relevance without widening K. Retrieval and generation stay separate so each step can be judged on its own.

Decision / 03

Pick the architecture from the workload

Problem: The product is UI-light and update-heavy: a chat stream plus a live ingestion feed. A separate SPA would mean a second build pipeline and a hand-rolled WebSocket layer.
Decision: Use Blazor Server so SignalR is the circuit for both chat tokens and crawl progress.
Why: The interface is a window onto a long-running pipeline. Real-time status is the interface, not a decoration on a static form.

Decision / 04

Deploy the pipeline as one production unit

Problem: Crawler, database, embeddings, and the chat UI have to move together. A workstation setup is not a research tool other people can run.
Decision: Run the stack in Docker behind Nginx, matching the rest of the production infrastructure on the domain.
Why: Containerized deployment keeps the retrieval system operable as a service, not as a notebook.

05 / Implementation

A retrieval system with a thin UI

Blazor Server hosts the chat and the status feed on a SignalR circuit. PostgreSQL plus pgvector holds chunks, vectors, and source links. Ollama embeds on the same machine. OpenRouter is the swappable synthesis step.

Retrieval parameters — chunk size, top-K, rerank depth — are configurable per deployment rather than hardcoded. The chat endpoint is rate-limited per IP so one client cannot exhaust the model budget or the crawl queue.

Optional lead capture persists in PostgreSQL and notifies by email through Resend. The whole stack runs in Docker behind Nginx.

06 / Behavior

How the system behaves.

Ingestion A crawl reports progress as posts are fetched, chunked, and embedded. The operator sees a pipeline, not a blank wait.
Ask A question retrieves candidates, reranks them, and only then synthesizes. The answer names its sources.
Evidence Each used chunk remains tied to a thread and comment, so a claim can be opened in the original discussion.

07 / Result

What improved.

A subreddit becomes a retrievable corpus instead of a scroll of threads.

Answers are generated from ranked evidence, not from the model's prior alone.

Ingestion and chat share a real-time circuit, so long jobs stay visible.

08 / Reflection

What the work demonstrates.

The system is not 'an AI app.' It is a research pipeline: collect, shape, retrieve, rerank, cite, then write.

Architecture followed the workload. Local embeddings for volume, a swappable model for synthesis, and a server-rendered UI because the UI is a live view of a backend job.

Next system

04 / Local AI / document processing

PDF Read Aloud

Open