How to Audit an AI Vendor's Data Path After the Notion-Fireworks Incident

A company can promise that an AI model provider does not retain prompts, and a separate feature can still send conversation data through another vendor workflow. That distinction is easy to miss when a product is described as one thing: “AI.” In a real system, a user request may pass through inference, classification, evaluation, logging, analytics, or support systems. Each path can have different data, vendors, and retention rules.

Notion’s November 11 advisory makes that distinction concrete. Notion says Fireworks AI notified it on the evening of November 2 that a small dataset had been accessed. Its initial review suggested that the data contained synthetic testing material, not customer data. On November 6, Fireworks notified Notion that a separate dataset might contain customer data; on November 8, Fireworks confirmed that dataset. Notion says it included AI chat transcripts used by the Insights feature. That feature used Fireworks for classification jobs to help customers understand AI usage. Notion says the transcripts were unattributable for the vast majority of affected workspaces, meaning they were not labelled with a user or workspace and did not include tool-call results. The advisory says fewer than 1% of Notion workspaces were affected, and that the investigation is ongoing. Notion’s incident advisory is the source for those details.

That is not the same as saying every Notion AI prompt was exposed, or that Fireworks’ inference service was involved. Notion says it disabled Insights and stopped serving open-weight models through Fireworks. Fireworks’ November 9 update separately says it has not detected impact to its Inference, Training SDK, and Nexus services, which it describes as zero-data-retention by default. Fireworks also says that some customer accounts were affected through other systems, including resource metadata, billing information, some secrets, datasets, and incomplete model weights. Its investigation remains active. Read Fireworks’ update alongside Notion’s advisory: the services and data categories are not identical.

The useful engineering question is not “Does the vendor have zero data retention?” It is: Can we name every path by which our data reaches a third party, and what happens to it on each path? Here is a practical way to answer that.

First, define the boundary you are auditing

Do not begin with a vendor list. Begin with one AI feature and one data type. For example: “Summarize support conversations” and “customer chat text.” This prevents a broad review from becoming an untestable claim that “AI is covered.”

Write down four things:

• The feature: What user-visible capability is being used?
• The input: What exact data can enter it, including attachments, retrieved documents, metadata, and tool results?
• The output: What is returned, stored, displayed, or used for a later decision?
• The actors: Which internal service and external organization can receive or access each item?

The aim is not to create a perfect map in one sitting. It is to establish a small, reviewable unit. NIST’s AI Risk Management Framework guidance says to document third-party AI resources and manage risks from them. Its Manage 3.1 playbook recommends applying and documenting risk-management plans for third-party AI technology, identifying third-party AI systems and components, and establishing testing and evaluation processes for them.

Step 1: Trace more than the model call

Draw the flow from the user action to deletion. Start with the obvious model request, then ask what else runs around it. A basic map might look like this:

User message
  -> product API
  -> prompt assembly and retrieval
  -> model inference provider
  -> response storage
  -> usage classification or analytics
  -> logs, evaluation, and support tools
  -> deletion or retention process

This is a template, not a claim about any specific product. Your real map may skip steps or contain more. For each arrow, identify the service, data fields, purpose, and whether the data is stored. Include asynchronous jobs: a feature can send data after the answer is returned, for a purpose such as classification or analytics.

Notion’s incident illustrates why this matters. The company says Fireworks served two roles: a general inference provider for open-weight models and a provider of classification jobs for Insights. The disclosed transcript exposure involved Insights. Notion’s report does not say that all ordinary inference requests were exposed. Keeping those paths separate makes the description more accurate and helps engineering teams ask the right questions of each service.

Step 2: Make a data inventory for every path

For each path, list the fields that leave your system. Avoid labels like “prompt” or “telemetry” unless you define them. A prompt might contain a user’s original message, retrieved internal documents, identifiers, and the results of previous tool calls. “Telemetry” could mean timing and token counts, or it could include text fragments. Those are very different data sets.

Use a simple record for each path:

• Path name: For example, support-summary.classification.
• Purpose: Why the data is sent.
• Data fields: Text, user or workspace identifiers, tool results, document excerpts, resource names, and any credentials that should never be present.
• Recipient: The vendor and the service or product feature receiving it.
• Processing location: The environment or region, if known and relevant to your requirements.
• Retention: Whether inputs, outputs, logs, datasets, and backups are retained, and for how long.
• Access: Which vendor personnel, service accounts, or internal teams can access the data.
• Control: The setting, contract, or code boundary that enforces the intended limit.
• Owner: The internal person or team responsible for reviewing changes and incidents.

If the vendor cannot answer a field, record “unknown” rather than assuming the safest answer. Unknowns are useful: they show where a contract, architecture diagram, or support response is needed.

Step 3: Treat retention as a property of a path, not a product

Ask the provider to define its retention statement precisely. Does it cover prompts and generated outputs only? Does it cover logs, evaluation runs, uploaded datasets, incomplete model artifacts, billing records, abuse monitoring, and support tickets? Is the rule different for batch processing, fine-tuning, or a hosted interface compared with an API?

Fireworks’ update draws this distinction in its own terms. It says it detected no impact to named zero-retention services, while reporting other categories of customer data affected in a limited set of accounts. The important point is not to infer that a zero-retention commitment is meaningless. It is to avoid applying a promise about one named service to a different feature or data path without evidence.

A useful vendor question is: “For this specific feature and endpoint, which data is processed, which data is retained, and which related services or subprocessors can access it?” Ask for the answer in writing, then match it to your data map. If the response uses broad product language, ask for the service names and exceptions.

Step 4: Check minimization and linkability

The safest data is data that never leaves your boundary. Before sending content to a vendor, ask whether the task can work with less information. Can you remove names, email addresses, workspace IDs, or irrelevant document context? Can you send a short classification label rather than the entire conversation? Can a stable identifier be replaced with a temporary token that only your service can resolve?

Removing a name is not automatically anonymization. A distinctive conversation may identify a person or company from its content. A pseudonymous ID may still be linkable if another system can map it back. In the Notion advisory, “unattributable” is carefully defined: the transcripts were not labelled with who wrote them or which workspace they came from, and Notion says there was no information beyond the transcript tying them to a workspace, user, or company. Preserve that level of precision when you document your own controls. Do not call a dataset anonymous merely because direct identifiers are absent.

Step 5: Verify controls in the running system

Policies and contracts describe intended behavior. Engineering controls should make that behavior observable and testable.

• Check egress: Confirm which service can call each vendor endpoint and block unapproved paths where possible.
• Inspect payloads safely: In a test environment, capture the request shape with synthetic content. Verify that sensitive fields are excluded or transformed before transmission.
• Check logs: Ensure application logs, traces, and error reports do not copy raw prompts or credentials by default.
• Test deletion: Follow one test record through the documented deletion flow. Identify any separate cache, analytics, or backup retention policy.
• Review changes: Alert on new vendors, endpoints, data fields, or AI features. A new “insights” job deserves its own review even if it uses a vendor already approved for inference.
• Keep a kill switch: Know how to disable the feature or route traffic elsewhere while you investigate.

This is where the data map becomes part of software maintenance, rather than a document made once for procurement. NIST’s AI RMF core describes monitoring plans that cover incident response, recovery, and change management. Its Manage playbook calls for documented risk practices and records for third-party AI components.

Step 6: Prepare for the incident before one happens

An incident playbook should answer operational questions, not only list a security contact. Decide who can disable the integration, who can identify affected records, who can rotate exposed credentials, and who communicates with customers or regulators when needed. Keep a copy of the vendor’s incident contact path and the contract’s notification language where the response team can find them.

For an AI integration, the first response checklist should include:

1. Pause the affected feature or data path without disabling unrelated functions if possible.
2. Identify the provider, service, dates, data fields, and customer groups involved.
3. Preserve relevant logs and configuration changes for investigation.
4. Determine whether prompts, tool results, identifiers, credentials, or uploaded datasets could be involved.
5. Rotate credentials if exposure is plausible, and check for unexpected usage.
6. Ask the provider for scope, evidence, containment actions, and the next update time.
7. Record what is confirmed, what remains unknown, and when the next decision will be made.

Notion’s advisory says affected customers should treat the data as exposed and rotate passwords, API keys, or other credentials pasted into Notion AI during the relevant period. On November 11, Notion also said it revoked exposed Notion API keys found in affected user messages during its investigation. That action is specific to this incident and its affected customers. The general lesson is to keep secrets out of prompts and to have a clear rotation path if they appear there.

A review checklist you can save

Before approving or renewing an AI integration, confirm:

• [ ] Each feature and data path has a named owner.
• [ ] Data sent on each path is listed at field level.
• [ ] Inference, classification, evaluation, analytics, logging, and support are not treated as one service by default.
• [ ] Retention and access commitments match the exact endpoint and product feature.
• [ ] Sensitive data is removed or minimized before leaving your system where feasible.
• [ ] Logs and traces are checked for prompt and credential leakage.
• [ ] Vendor and subprocessor changes trigger a review.
• [ ] There is a tested way to disable, replace, or isolate the integration.
• [ ] The incident process covers discovery, scoping, containment, credential rotation, and communication.

A vendor security page and a zero-retention statement are useful inputs, not a substitute for this map. The map gives your team a concrete way to compare the promise with the running system and to update the review when a new feature adds another route for data.

I write about practical AI systems and software engineering every week. Subscribe for free if you want more field guides like this.

Have you mapped every path your AI features use, including analytics and classification? What did you discover?

Sources

• Notion, Fireworks AI incident advisory, updated November 11, 2026
• Fireworks AI, Update on October 2026 security incident, updated November 9, 2026
• NIST AI Risk Management Framework Playbook, Manage 3
• NIST AI Risk Management Framework Core