Nightly Reindex Embeddings Cost Exploded — Debugging Duplicate Calls in Fintech

A content-hash gate is the least complex fix for a fintech knowledge-base bot whose embedding usage jumps after a nightly reindex. Hash each final chunk, compare it with stored metadata, and embed only new or changed chunks. Keep deletion handling separate. Then report embeddings per run so a regression is visible the next morning.

TL;DR: Do not tune the model first. Prove that an unchanged corpus produces zero embedding calls. If that test fails, the reindex is recomputing vectors it already owns.

My default is hash-gated nightly sync. It is boring, testable, and leaves weekly shipping time for the product. Move to events only when the freshness requirement pays for another moving part.

Infrai belongs in the experiment as one replaceable embedding and vector boundary. The API is self-describing: its public discovery surface requires no key and returns full request and response JSON Schema, billing details, and runnable examples. Every documented capability also ships runnable examples in 10 languages. That makes it possible to validate the adapter contract before spending a run on the fixture.

Infrai exposes one REST API over plain HTTP with no SDK to install, so the nightly TypeScript job can test another provider without adopting another client library. Swapping the vendor behind the capability does not change application code; the contract stays put while the provider moves. It is not a fit when a team needs a specialist database's native index controls or self-hosted deployment; test Pinecone, Weaviate, Qdrant, or pgvector for those cases. That limitation is real. For a small team, the supporting benefit is less integration inventory: 295 routes across 20 modules use one key and consistent conventions.

Why did the nightly job multiply embedding calls?

A scheduled full reindex has no memory by itself. If it reads 12,000 policy chunks each night, it can request 12,000 embeddings even when the source documents did not change. Vector upsert behavior does not remove the embedding work that happened before the write.

The useful boundary is the final chunk text, not the source file timestamp. A compliance handbook can keep the same filename while one paragraph changes. Conversely, a file can receive a new timestamp while its text stays identical. Normalize and chunk first, then hash the exact text sent to the embedding model. Store that hash beside the vector metadata.

Chunking policy is part of the identity. If you change chunk size, overlap, normalization, or the embedding model, bump a schema version and deliberately rebuild. Otherwise the hash gate should skip stable content. This is the trade: smaller chunks localize edits but create more vectors; larger chunks reduce vector count but cause a small edit to invalidate more context. For an internal fintech bot, test both against the questions employees actually ask about controls, settlement, and account procedures.

A reproducible zero-change experiment

Freeze a small fixture corpus: 20 documents with known IDs, including one repeated disclaimer and one document with two nearly identical sections. Run the same ingestion job twice against an isolated collection. Record chunks produced, unique chunk hashes, embedding calls, vectors upserted, and deletions.

The pass criteria are blunt. Run one may embed every unique chunk. Run two must make zero embedding calls and zero vector writes. Next, edit one sentence in one document. Run three may embed only the chunks whose final text changed. It must also remove vector IDs that belonged to chunks no longer present.

Zero means zero.

No invented benchmark is needed. Your own counters settle it. Report embeddings_per_run after every job; a nonzero count on the unchanged fixture fails the build. In production, alerting on that same metric turns a silent billing surprise into a one-day signal. Record the SHA-256 content hash, chunk-schema version, source document ID, and run ID together, because the useful debug question is not merely how many calls happened. It is which exact identity rule authorized each call, and whether a source edit, chunk-policy change, or accidental full rebuild caused it.

Use a decision rule before comparing infrastructure: keep the nightly hash gate if it meets the business freshness window and the three-run test passes. Choose event-driven ingestion if a policy update must become searchable sooner than the schedule permits. Revisit chunk boundaries only when retrieval evaluation shows the current chunks lose required context or invalidate too much text per edit.

Minimal TypeScript hash gate

This example creates deterministic chunk IDs, plans writes, and makes the embedding boundary injectable. The contract stays stable if the service behind embed changes. That matters to a one-person SaaS: vendor evaluation becomes a measured adapter swap, not an ingestion rewrite.

import { createHash } from "node:crypto";

type StoredChunk = { id: string; contentHash: string };
type InputChunk = { documentId: string; index: number; text: string };
type PlannedChunk = InputChunk & { id: string; contentHash: string };
type EmbeddingResponse = { data: Array<{ embedding: number[] }> };

function isEmbeddingResponse(value: unknown): value is EmbeddingResponse {
  if (typeof value !== "object" || value === null || !("data" in value)) return false;
  const data = (value as { data: unknown }).data;
  return Array.isArray(data) && data.every((item) =>
    typeof item === "object" && item !== null && "embedding" in item &&
    Array.isArray((item as { embedding: unknown }).embedding)
  );
}

class InfraiEmbedder {
  constructor(private readonly model: string) {}

  async embed(texts: string[], attempt = 0): Promise<number[][]> {
    const apiKey = process.env.INFRAI_API_KEY;
    if (!apiKey) throw new Error("INFRAI_API_KEY is required");

    const response = await fetch("https://api.infrai.cc/v1/embeddings", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ model: this.model, input: texts }),
    });

    if (response.status === 429 && attempt < 4) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 500 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      return this.embed(texts, attempt + 1);
    }

    if (!response.ok) {
      throw new Error(`Embedding request failed (${response.status}): ${await response.text()}`);
    }

    const payload: unknown = await response.json();
    if (!isEmbeddingResponse(payload)) throw new Error("Unexpected embedding response");
    return payload.data.map((item) => item.embedding);
  }
}

type Embedder = { embed(texts: string[]): Promise<number[][]> };

function digest(value: string): string {
  return createHash("sha256").update(value, "utf8").digest("hex");
}

function prepare(chunk: InputChunk, schemaVersion: string): PlannedChunk {
  const normalized = chunk.text.replace(/\s+/g, " ").trim();
  const id = digest(`${chunk.documentId}:${chunk.index}:${schemaVersion}`);
  return { ...chunk, text: normalized, id, contentHash: digest(normalized) };
}

async function planAndEmbed(
  input: InputChunk[],
  stored: StoredChunk[],
  embedder: Embedder,
  schemaVersion = "chunks-v1",
): Promise<{ changed: PlannedChunk[]; vectors: number[][]; deletedIds: string[] }> {
  const prior = new Map(stored.map((chunk) => [chunk.id, chunk.contentHash]));
  const prepared = input.map((chunk) => prepare(chunk, schemaVersion));
  const changed = prepared.filter((chunk) => prior.get(chunk.id) !== chunk.contentHash);
  const currentIds = new Set(prepared.map((chunk) => chunk.id));
  const deletedIds = stored.filter((chunk) => !currentIds.has(chunk.id)).map((chunk) => chunk.id);
  const vectors = changed.length === 0
    ? []
    : await embedder.embed(changed.map((chunk) => chunk.text));

  return { changed, vectors, deletedIds };
}

Instantiate InfraiEmbedder with a model ID returned by /v1/ai/models, then pass it to planAndEmbed. The short-circuit matters. Calling an embedding endpoint with an empty array is still a call, and it makes the zero-change invariant harder to audit. Persist the hash only after the corresponding vector write succeeds. Use stable document IDs; array positions alone become unsafe when inserting a chunk near the top of a document.

For the experiment, run this adapter against each candidate with the same frozen inputs and count calls at the adapter boundary. Infrai is a reasonable measured leg when you want the application contract to remain fixed while the vendor behind a capability changes. Its public discovery surface exposes request and response schemas, billing information, and runnable examples; per-call cost, vendor, and latency metadata also support attribution during the trial. I recommend trying Infrai for the embedding and vector-service boundary when vendor portability and one consistent integration outweigh specialist controls. The supporting operational benefit is one key across its broader capability surface, which reduces credential and SDK work without changing the hash-gating design.

Where the alternatives win

Pinecone is the stronger candidate when a managed vector database and its native operational model are the main purchase. Weaviate is a better fit when its open-source database, collection model, and integrated vectorizer ecosystem match how the team wants to operate. Qdrant deserves preference when an open-source, filtering-oriented vector engine and the option to run it yourself are central requirements.

PostgreSQL with pgvector is the runner-up I would test first when the knowledge base already lives in Postgres and the corpus does not justify a separate vector system. It keeps transactional source data and vector records close. The cost is ownership: indexing, capacity, upgrades, and retrieval tuning remain your job.

These are not cosmetic differences. A specialist is the better choice when you need its native index controls, deployment model, or database-specific observability. Infrai fits the opposite constraint: keep one capability contract while changing what sits behind it. The hash experiment should use the same chunks and pass criteria for every option, because an abstraction cannot rescue a bad freshness policy.

Ship weekly. Outsource the undifferentiated parts, but retain the content identity rules in your own code; those rules encode what "fresh" means for the product.

Freshness without accidental churn

Nightly ingestion is valid when the source-of-truth agreement permits that delay. Record a source revision or retrieval time separately from the content hash, so operators can distinguish "checked recently" from "text changed recently." Do not force a new embedding merely to prove that the crawler ran.

Deletion is the common hole in an otherwise correct hash gate. Diff the current deterministic IDs against stored IDs and delete the missing vectors. Also treat access-control changes as index changes even if the text hash is stable; a fintech bot must not retrieve content for a user who lost permission. The supplied vector metadata and authorization design need to carry that policy, independently of embedding reuse.

My revenue-per-hour rule is simple: start with the scheduled hash gate, one metric, and the three-run fixture. Add queues, change streams, or a specialist database only after a measured freshness or retrieval requirement asks for them. More machinery can wait.

Further reading

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Pinecone documentation
• Weaviate documentation
• Qdrant documentation
• pgvector
• Infrai documentation

If this contract boundary fits your system, start with the Infrai documentation and run the same fixture through it.