Senior Software Engineer focused on backend, databases, infrastructure, and AI coding agents. Sharing real systems, practical fixes, and reproducible results.
Location
Indonesia
Education
Mikroskil University: B.Sc. Information Systems, 2014
The npm package test gap is real. In agent-inspect@6.31.35, the monorepo's internal vitest suite passes, but npm pack && npm install -g followed by the same fixture shows the file-writer serializer drops error.code on the manual inspectRun() path — a field the instance-based createInspector() + fileWriter() path retains. The difference is visible in the schema version: manual path writes schema 0.1, instance path writes schema 1.0. Running the probe suite against the installed tarball catches this; running only against the monorepo source does not.
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@prascli Thanks for the precise reproduction. I haven’t verified the packed 6.31.35 behavior yet, so I won’t call it fixed or dismiss it. The manual inspectRun() versus createInspector() schema difference is exactly the installed-package boundary this article argues for. I’ll reproduce both paths from the tarball and add a regression if confirmed.
"Passes its internal tests while capturing nothing in an installed application" is such a common trap. Testing the packed tarball in a clean project is the only test that matches what users actually install. Good write up.
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@gautamtalksdev Exactly. The useful assertion is the combination of an independent SDK call counter and persisted trace readback; a green tool result alone can hide a missing span. I’m keeping the matrix explicit about which package, Node, module-system, and transport cells were actually exercised.
Product leader and systems analyst in fintech and edtech. Founder of AnalystCraft. Writing about requirements, acceptance criteria, agent evaluation, and Test-Driven Product Development.
The independent server counter makes missing spans visible. A further case I’d test is two concurrent runs with distinct run IDs and distinct echo values: both server calls should happen once, and each persisted trace should contain only its own run’s calls. That checks context isolation, not just total span count—a wrapper could capture both calls but attach one to the wrong run. Does the current fixture exercise overlapping runs?
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@innokentyb Good catch. The fixture shown here has one inspectRun, so it does not establish overlapping-run isolation. Two distinct run IDs and echo values, followed by per-trace assertions, would catch misattribution even when the total span count looks right. That is a valuable next matrix cell.
I measure AI coding claims in public. LGTM — Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
This one hit a nerve. I once shipped something where the monorepo tests were all green and the installed package silently did nothing, because two copies of the same context module were living side by side and neither one complained. The independent server counter is the part I'd copy, honestly, because "the trace has a span" proves nothing if you never counted the real calls. Do you run this consumer test on every release, or only when the adapter itself changes?
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@aheadkit That failure mode is exactly why the handler counter sits outside the adapter. This article shows an installed-package consumer test, but I don’t want to imply it currently runs on every release. It belongs in a publish gate, since bundling can change even when the adapter source does not.
the independent server counter is the strongest assertion here. i'd add a negative fixture that intentionally loads a second bundled run-context singleton: callTool still returns fixture-ok and the handler count stays at 1, while one of the two expected spans disappears. that would prove the test catches silent capture loss, not just a healthy package. do you also run a clean npm pack -> install-from-tarball smoke test before publishing?
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@ademisler The example installs the published 6.31.35 packages through a lockfile; that is different from saying a clean npm pack tarball test is already a prepublish gate on every release. Your deliberately duplicated run-context singleton is a strong negative control because the tool can still succeed while capture drops out. Thanks for sharpening that distinction.
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@kate_astrid Thanks for turning it into a checklist. I would keep trace readback and the independent server count side by side, so the test catches silent missing capture. If Docker becomes a CI claim, pin the image digest as well as the package lockfile.
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
@parasraju Yes—ESM and CommonJS can resolve different build outputs even when the source tests are green. Exercising both through the public package entry points is a small check with a lot of value.
We're a place where coders share, stay up-to-date and grow their careers.
The npm package test gap is real. In agent-inspect@6.31.35, the monorepo's internal vitest suite passes, but
npm pack && npm install -gfollowed by the same fixture shows the file-writer serializer dropserror.codeon the manualinspectRun()path — a field the instance-basedcreateInspector()+fileWriter()path retains. The difference is visible in the schema version: manual path writes schema0.1, instance path writes schema1.0. Running the probe suite against the installed tarball catches this; running only against the monorepo source does not.@prascli Thanks for the precise reproduction. I haven’t verified the packed 6.31.35 behavior yet, so I won’t call it fixed or dismiss it. The manual
inspectRun()versuscreateInspector()schema difference is exactly the installed-package boundary this article argues for. I’ll reproduce both paths from the tarball and add a regression if confirmed."Passes its internal tests while capturing nothing in an installed application" is such a common trap. Testing the packed tarball in a clean project is the only test that matches what users actually install. Good write up.
@gautamtalksdev Exactly. The useful assertion is the combination of an independent SDK call counter and persisted trace readback; a green tool result alone can hide a missing span. I’m keeping the matrix explicit about which package, Node, module-system, and transport cells were actually exercised.
The independent server counter makes missing spans visible. A further case I’d test is two concurrent runs with distinct run IDs and distinct echo values: both server calls should happen once, and each persisted trace should contain only its own run’s calls. That checks context isolation, not just total span count—a wrapper could capture both calls but attach one to the wrong run. Does the current fixture exercise overlapping runs?
@innokentyb Good catch. The fixture shown here has one
inspectRun, so it does not establish overlapping-run isolation. Two distinct run IDs and echo values, followed by per-trace assertions, would catch misattribution even when the total span count looks right. That is a valuable next matrix cell.This one hit a nerve. I once shipped something where the monorepo tests were all green and the installed package silently did nothing, because two copies of the same context module were living side by side and neither one complained. The independent server counter is the part I'd copy, honestly, because "the trace has a span" proves nothing if you never counted the real calls. Do you run this consumer test on every release, or only when the adapter itself changes?
@aheadkit That failure mode is exactly why the handler counter sits outside the adapter. This article shows an installed-package consumer test, but I don’t want to imply it currently runs on every release. It belongs in a publish gate, since bundling can change even when the adapter source does not.
the independent server counter is the strongest assertion here. i'd add a negative fixture that intentionally loads a second bundled run-context singleton: callTool still returns fixture-ok and the handler count stays at 1, while one of the two expected spans disappears. that would prove the test catches silent capture loss, not just a healthy package. do you also run a clean npm pack -> install-from-tarball smoke test before publishing?
@ademisler The example installs the published 6.31.35 packages through a lockfile; that is different from saying a clean
npm packtarball test is already a prepublish gate on every release. Your deliberately duplicated run-context singleton is a strong negative control because the tool can still succeed while capture drops out. Thanks for sharpening that distinction.Saving this as my release checklist:
Thanks for writing it up so clearly, Raju! 🧪
@kate_astrid Thanks for turning it into a checklist. I would keep trace readback and the independent server count side by side, so the test catches silent missing capture. If Docker becomes a CI claim, pin the image digest as well as the package lockfile.
Testing both ESM and CommonJS is a smart move.
@parasraju Yes—ESM and CommonJS can resolve different build outputs even when the source tests are green. Exercising both through the public package entry points is a small check with a lot of value.