DEV Community

Eqvero
Eqvero

Posted on

The SEC EDGAR Accession Number Trap: Disentangling Restatements and Comparative Reclassifications in Python

Financial data pipelines ingesting historical facts from SEC EDGAR routinely hit a subtle, pervasive bug: querying a single fiscal year returns multiple conflicting values under the same concept tag. When querying the SEC EDGAR Company Facts endpoint (/api/xbrl/companyfacts/CIK{cik}.json) for a concept such as us-gaap:Revenues for fiscal year 2024 (fy: 2024, fp: "FY"), a developer might expect a single canonical record. Instead, the endpoint frequently serves three or more distinct records across different accession numbers (accn), each with different filing dates (filed) and differing values (val).

If your ingestion logic simply selects the first record or blindly upserts chronologically, your pipeline introduces silent data contamination. In this article, we explain why multiple accession numbers exist for identical reporting periods, analyze the mechanical difference between point-in-time disclosures and comparative reclassifications, and implement a deterministic Python pipeline to resolve both views cleanly.


The Mechanism: Why Multiple Accession Numbers Collide

Under SEC Regulation S-X, public companies are required to present comparative financial statements. A standard Form 10-K must present audited statements of income, comprehensive income, cash flows, and stockholders' equity for the three most recent fiscal years, along with audited balance sheets for the two most recent fiscal years.

When a company files its 2026 Form 10-K, it submits XBRL tags for 2026 and repeats (or reclassifies) the facts for 2025 and 2024 under a new filing accession number.

Three distinct corporate actions cause historical values to diverge across accession numbers:

  1. Discontinued Operations and Reclassifications (ASC 205-20): When a company disposes of a component or classifies a business unit as held for sale, retrospective reclassification is required. Prior-period revenues from continuing operations are adjusted downward in subsequent filings to exclude the discontinued component.
  2. Accounting Principle Changes and Error Corrections (ASC 250): When new FASB standards are adopted retrospectively or prior accounting errors are corrected, previous period numbers are restated.
  3. Formal SEC Amendments (Form 10-K/A): If a company amends a filing to address SEC staff comments or disclosure discrepancies, a new accession number overwrites or supplements specific tagged concepts.

Production Tradeoff: Point-in-Time vs Restated Baselines

Every financial data engineering pipeline must explicitly choose an architectural ingestion mode:

Dimension Point-in-Time (As-Filed) Mode Restated (Latest Recast) Mode
Primary Use Case Quantitative backtesting, market-reaction modeling, ML feature store Fundamental analysis, long-term credit underwriting, valuation
Accession Selection Earliest valid accession number associated with the original filing Latest valid accession number sorted by filed date
Survivor Bias Zero (reflects strictly what the market knew on release day) Present (incorporates hindsight reclassifications)
Reconciliation Risk Requires tracking filing release timestamps against trading hours Prior-period comparatives seamlessly match the latest 10-K

Mixing these modes within the same pipeline corrupts derived metrics, such as calculating revenue growth by dividing a recast 2025 figure by an unadjusted 2024 figure.


Implementing the Disentanglement Pipeline in Python

The following script fetches company facts from the SEC EDGAR API, groups records by concept and fiscal period, and deterministically extracts both the Point-in-Time (original) and Latest-Recast (restated) timeseries.

import datetime
import gzip
import json
import urllib.request
from typing import Any, Dict, List, Optional, Tuple


def fetch_company_facts(cik: str, user_agent: str) -> Dict[str, Any]:
    """Fetch structured XBRL company facts from the SEC EDGAR API."""
    cik_padded = cik.strip().zfill(10)
    url = f"https://data.sec.gov/api/xbrl/companyfacts/CIK{cik_padded}.json"
    req = urllib.request.Request(
        url,
        headers={"User-Agent": user_agent, "Accept-Encoding": "gzip, deflate"},
    )
    with urllib.request.urlopen(req, timeout=15) as resp:
        raw = resp.read()
        if (
            resp.info().get("Content-Encoding") == "gzip"
            or raw[:2] == b"\x1f\x8b"
        ):
            raw = gzip.decompress(raw)
        return json.loads(raw.decode("utf-8"))


def resolve_fact_timeseries(
    facts: Dict[str, Any],
    concept: str,
    fiscal_period: str = "FY",
    unit: str = "USD",
) -> Dict[int, Dict[str, Any]]:
    """Group facts by fiscal year and resolve point-in-time vs latest restated values.

    Returns a dict keyed by fiscal year containing:
      - 'original': fact from the earliest filing date (point-in-time)
      - 'latest_recast': fact from the latest filing date (restated/reclassified)
      - 'accn_count': number of distinct accession filings referencing this period
      - 'variance': numeric delta between latest recast and original
    """
    raw_units = (
        facts.get("facts", {})
        .get("us-gaap", {})
        .get(concept, {})
        .get("units", {})
        .get(unit, [])
    )

    by_year: Dict[int, List[Dict[str, Any]]] = {}

    for record in raw_units:
        fy = record.get("fy")
        fp = record.get("fp")
        form = record.get("form", "")

        # Target full-year 10-K or amended 10-K/A filings
        if fp != fiscal_period or form not in ("10-K", "10-K/A"):
            continue
        if not fy or "filed" not in record or "val" not in record:
            continue

        by_year.setdefault(fy, []).append(record)

    resolved: Dict[int, Dict[str, Any]] = {}

    for fy, records in sorted(by_year.items()):
        # Sort chronologically by filing date, then accession number
        sorted_records = sorted(
            records, key=lambda r: (r["filed"], r["accn"])
        )

        original = sorted_records[0]
        latest = sorted_records[-1]

        unique_accessions = {r["accn"] for r in records}
        val_orig = float(original["val"])
        val_latest = float(latest["val"])

        resolved[fy] = {
            "fiscal_year": fy,
            "original_val": val_orig,
            "original_filed": original["filed"],
            "original_accn": original["accn"],
            "latest_val": val_latest,
            "latest_filed": latest["filed"],
            "latest_accn": latest["accn"],
            "accession_count": len(unique_accessions),
            "reclassified": val_orig != val_latest,
            "variance": val_latest - val_orig,
        }

    return resolved


def main() -> None:
    # Example: Berkshire Hathaway Inc. (CIK 0001067983)
    cik = "0001067983"
    ua = "FinancialDataPipeline/1.0 (dev@example.com)"
    concept = "Revenues"

    print(f"Fetching disclosures for CIK {cik}...")
    facts = fetch_company_facts(cik, ua)

    resolved = resolve_fact_timeseries(facts, concept, fiscal_period="FY")

    print(f"\n--- Resolved SEC EDGAR Facts: us-gaap:{concept} ---")
    print(
        f"{'FY':<6} | {'Original Filed':<14} | {'Orig Value ($B)':<15} | "
        f"{'Latest Filed':<14} | {'Latest Value ($B)':<16} | {'Variance ($M)':<14}"
    )
    print("-" * 90)

    for fy, row in sorted(resolved.items())[-5:]:
        orig_b = row["original_val"] / 1e9
        latest_b = row["latest_val"] / 1e9
        var_m = row["variance"] / 1e6
        status = " [RESTATED]" if row["reclassified"] else ""
        print(
            f"{fy:<6} | {row['original_filed']:<14} | ${orig_b:<14,.2f} | "
            f"{row['latest_filed']:<14} | ${latest_b:<15,.2f} | "
            f"${var_m:<13,.1f}{status}"
        )


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Production Edge Cases to Guard Against

When deploying accession resolution logic across thousands of CIKs, account for three failure modes:

  1. Multiple Filings on the Same Day: If a Form 10-K and Form 10-K/A are accepted on the identical calendar date, sorting solely by filed date creates indeterminate ordering. Use the EDGAR acceptance timestamp or sort on the submission accession sequence suffix to guarantee deterministic resolution.
  2. XBRL Concept Deprecation: A company may report an item under us-gaap:Revenues in earlier filings and transition to us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax after adopting ASC 606. Resolving accession numbers per tag does not catch concept migrations; robust pipelines must maintain taxonomic fallback chains.
  3. Unamended Segment Restatements: In notes to financial statements, operating segment revenues may be reclassified while consolidated revenues remain identical. Always track the dimensional member context (axis and member tags in XBRL) to ensure parent company consolidated facts are not mixed with segment disclosures.

Conclusion

Automated financial ingestion cannot treat SEC EDGAR as a flat key-value store. By explicitly modeling filing accession numbers and resolving point-in-time vs latest-recast records, data engineering pipelines eliminate silent timeseries corruption and ensure institutional-grade analytical reliability.

This post is part of the **SEC Filing Data Engineering* series on DEV. For previous installments, see Parsing SEC EDGAR Facts: Reconciling Cash Flow with Python and Why Quarter-over-Quarter SEC EDGAR Facts Break: Normalizing Discrete vs YTD Durations in XBRL.*

Top comments (0)