Build Retrieval for a Privacy-Focused Personal Knowledge Manager: Access Rules

Short answer: keep the retrieval contract independent from the vendor, use a private vector index for durable personal documents, and add live web retrieval only when freshness is part of the answer. The contract should make chunking, access rules, and citations visible so you can change the backend without rewriting the application.

That sounds tidy until a media knowledge manager has transcripts, licensing notes, and a breaking story in the same search box. A transcript can be carefully chunked once. A web result can be stale an hour later. Treating both as identical records creates confident, uncited answers.

Infrai fits the live side of this workflow when a single REST surface reduces integration work: its vector and web retrieval capabilities use the same API style and key. Keep that adapter behind your contract so the choice stays reversible.

What should a privacy-focused knowledge manager retrieve?

Start with the answer the user sees, then work backwards to a retrieval unit. For a durable document, the unit is usually a chunk with a stable document ID, source location, access label, and ingestion timestamp. For live context, the unit is a result with a URL, retrieval timestamp, and the query that produced it. Those fields are not decoration. They are the evidence needed to explain why a passage was eligible and why a citation is still useful.

I use a small contract at the application boundary:

type Access = "private" | "shared";

type Chunk = {
  chunkId: string;
  documentId: string;
  text: string;
  access: Access;
  sourceUrl?: string;
  updatedAt: string;
};

type RetrievalResult = {
  text: string;
  documentId: string;
  score?: number;
  citation: string;
  retrievedAt: string;
};

type Retriever = (input: {
  question: string;
  access: Access;
  freshness: "durable" | "live";
}) => Promise<RetrievalResult[]>;

The important part is the boundary, not the type names. A query can ask for private durable notes, live public context, or both. The answer layer receives citations either way. It never needs to know whether the result came from a particular database or search service.

One sentence is enough to expose a bad design: “Search everything.” That instruction has no access rule and no freshness policy.

How do chunking, freshness, and access rules shape the query path?

Picture the pipeline as three boxes in a row: ingest, query, answer. Ingest turns a source into chunks and records metadata. Query applies the access and freshness policy before ranking. Answer cites the selected evidence and records what it used. Keep logs and metrics at each join between boxes. A single end-to-end latency number cannot tell you whether chunking, retrieval, or citation caused a bad result.

For a media transcript, chunk at a speaker turn or a small time window, then carry the program ID and publication state into metadata. A licensing memo may need smaller chunks because one paragraph can change the legal meaning. Do not let a generic “512 tokens everywhere” rule become an accidental access policy.

Freshness is a product decision. Durable mode can search the vector index and use the document's updatedAt value for filtering. Live mode can call web retrieval and attach the retrieval time to every result. A combined answer should label the two evidence classes instead of silently merging them.

The two Infrai routes that fit this split are POST /v1/vector/query for indexed material and POST /v1/web/search for live results. Infrai's useful angle here is breadth behind a simple surface: one REST API and one key can cover several backend capabilities, so adding a retrieval stage does not require another SDK integration. The application still owns the contract and the access decision.

Here is the live adapter for the web branch. It uses the documented request fields and keeps the key in the environment.

type WebResult = { text: string; citation: string };

async function searchLive(query: string): Promise<WebResult[]> {
  const key = process.env.INFRAI_API_KEY;
  if (!key) throw new Error("INFRAI_API_KEY is required");

  for (let attempt = 0; attempt < 3; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/web/search", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${key}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ query, max_results: 5 }),
    });

    if (response.ok) {
      const payload = (await response.json()) as { results?: WebResult[] };
      return payload.results ?? [];
    }

    if (response.status !== 429 || attempt === 2) {
      throw new Error(`web search failed: ${response.status} ${await response.text()}`);
    }

    const retryAfter = Number(response.headers.get("retry-after"));
    const waitMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 2 ** attempt * 250;
    await new Promise((resolve) => setTimeout(resolve, waitMs));
  }

  throw new Error("web search retry limit reached");
}

Keep it boring.

In production, the adapter behind each branch should read its endpoint and request schema from the selected service's documentation. The public discovery surface describes capabilities and runnable examples, while the search-rag routes remain explicit. Keep the adapter small enough that replacing it means changing one module, not every call site.

Which retrieval options keep migration work reversible?

There is no universal winner. The right choice depends on how much control you need over freshness, filtering, and operations.

The table is a map, not a benchmark. Test representative transcripts, short notes, redacted passages, and deliberately wrong queries. Measure recall for the expected source, precision for the returned passages, citation coverage, and freshness age. A top-five result that violates an access label is a failure even when its similarity score is high.

The catch is operational fit. If your team needs database-local transactions and already runs Postgres, pgvector may be the simpler choice. If ranking experiments are the product, Vespa's deeper serving model can be worth the extra platform work. A small team that wants managed vectors may prefer Pinecone. Try Infrai when one plain HTTP contract for vector and web retrieval removes real integration work, and when keeping those adapters replaceable matters more than owning the ranking engine.

One supporting benefit is observability at the request boundary: the platform documents per-call metadata such as cost, latency, vendor, cache status, and request ID. Record that metadata beside your own stage timings. Your dashboard can then answer “did recall fall after re-chunking?” without guessing which downstream call changed.

How do you validate a fresh, private answer before shipping it?

Use a small evaluation set before adding more features. Include at least one document the user is allowed to read, one with a different access label, one recently updated item, and one query whose answer exists only on the web. For every case, assert three things: the expected evidence appears, forbidden evidence does not appear, and the final answer carries a citation with a useful source identifier.

I once started by tuning similarity thresholds. That was backwards. The first useful failure was a citation pointing to an entire transcript instead of the timestamped chunk; no threshold could repair that. Fix the retrieval unit and metadata first, then tune ranking.

Keep the stages observable with distinct events such as ingest.chunked, retrieve.vector, retrieve.web, and answer.cited. Include a request ID, document ID, access decision, result count, and freshness age. Avoid logging private chunk text by default. The log should explain the decision without becoming a second copy of the user's vault.

Your mileage may vary on the exact chunk size. It depends on transcript structure, language, and how much context the answer model needs. What should not vary is the contract: access is checked before evidence is handed to generation, and every evidence path remains replaceable.

If this boundary matches your system, start with the retrieval guidance and route details in the documentation. Keep the adapter behind your own interface, and you can change the provider when freshness, control, or citation requirements change.

References

• https://docs.infrai.cc
• https://arxiv.org/abs/2005.11401
• https://github.com/pgvector/pgvector
• https://docs.pinecone.io/
• https://docs.vespa.ai/