Ask a coding agent how long a task will take and it may answer in human calendar units: “a few hours,” “one or two days,” or “about a week.” Then it finishes the code change in twelve minutes, or spends an afternoon chasing a broken test. The estimate sounds confident, but it may not be grounded in observed behavior from comparable tasks.
A small project called AI Time Estimator, shared on Hacker News on November 15, takes a different route. It uses local Claude Code transcripts to estimate the agent’s working time, then compares estimates with completed sessions and adjusts future ranges. Its author describes a read-only estimator, local data, and a feedback loop. Those are useful design ideas, not independent proof that its predictions are reliable for other people or projects.[1]
The broader lesson is worth borrowing even if you never install the plugin: measure the work your agent performs, separate that from human waiting and review, and calibrate ranges against your own history. This article builds a small, transparent version of that idea in Python. It is not a reproduction of the project, and it does not read Claude Code’s private transcript format.
Why a single number is the wrong target
A task estimate has at least two distinct meanings:
- Agent execution time: model turns, tool calls, shell commands, and time spent waiting for tools.
- Elapsed delivery time: execution plus your time clarifying requirements, reviewing diffs, rerunning checks, and unblocking the agent.
A tool that reports only the first can help answer “how long will this session run?” It cannot promise when a feature will be ready to merge. A developer may be reviewing another task while the agent works, or may need to answer a question immediately. Those human factors change elapsed time without changing the agent’s runtime.
There is also a difference between uncertainty in runtime and uncertainty in the task itself. “Add a field to a response DTO” is a narrow change. “Make exports reliable” may hide format decisions, legacy behavior, and tests no one has documented. A useful estimate should expose that uncertainty instead of laundering it into a precise-looking number.
That is why the example below returns a range. P50 is the median prediction: half of comparable tasks may finish sooner and half later, assuming the sample resembles the next task. P90 is a slower, more cautious bound. These are empirical percentiles, not guarantees.
A tiny estimator from your own history
Start with a small CSV you create after agent sessions. Keep the fields meaningful and avoid copying prompts or source code into the file. The point is to record task shape and measured time, not the contents of private work.
category,files,tool_calls,fix_loops,minutes
small_test,2,8,0,6
small_test,3,12,1,11
api_change,5,21,2,24
api_change,4,18,1,19
This sample is illustrative only. Do not use it as a benchmark or as evidence of accuracy. For a real experiment, record your own sessions consistently. “Fix loop” should mean a concrete cycle in which a check fails, the agent changes something, and the check is run again. Decide what counts before collecting data.
Here is a simple estimator that groups historical tasks by category, then adjusts each observed duration by the relative task size. It uses the median and 90th percentile of adjusted durations rather than asking a language model to invent a number.
import csv
import math
import statistics
import sys
def percentile(values, p):
"""Linear interpolation for a percentile in [0, 100]."""
values = sorted(values)
if not values:
raise ValueError("no observations")
if len(values) == 1:
return values[0]
position = (len(values) - 1) * p / 100
lower = math.floor(position)
upper = math.ceil(position)
fraction = position - lower
return values[lower] * (1 - fraction) + values[upper] * fraction
def estimate(path, category, files, tool_calls, fix_loops):
with open(path, newline="", encoding="utf-8") as f:
rows = [r for r in csv.DictReader(f) if r["category"] == category]
if len(rows) < 8:
raise ValueError("collect at least 8 comparable tasks first")
adjusted = []
for row in rows:
old_size = (
int(row["files"])
+ 0.15 * int(row["tool_calls"])
+ 3 * int(row["fix_loops"])
)
new_size = files + 0.15 * tool_calls + 3 * fix_loops
if old_size > 0:
adjusted.append(int(row["minutes"]) * new_size / old_size)
return percentile(adjusted, 50), percentile(adjusted, 90)
if __name__ == "__main__":
p50, p90 = estimate(sys.argv[1], sys.argv[2], *map(int, sys.argv[3:]))
print(f"Agent working time: P50 {p50:.0f} min, P90 {p90:.0f} min")
Run it with a category and a rough task-shape estimate:
python3 estimate.py history.csv api_change 5 20 1
The weights in old_size and new_size are deliberately simple example values, not research findings. They make the calculation concrete, but they should be tested and tuned, or removed, based on your data. If you have too few records, keep the category exact and avoid pretending the output is statistically sturdy.
What this model can and cannot tell you
It can give a rough starting point. If a new task looks similar to recent tasks, scaling observed durations by a coarse task-size measure may be better than an uncalibrated guess. Keeping the calculation visible makes it easy to challenge assumptions.
It cannot understand task complexity from a category label. Two “API change” tickets can differ radically. One may need a schema migration, another a new endpoint; the counts can look similar while the hidden work does not. Add categories only when you have enough observations, and split them when their tasks behave differently.
It cannot correct inconsistent measurement. If some sessions include idle time and others do not, the history mixes incompatible quantities. Define whether a pause, a model wait, or a human review counts. If the goal is agent runtime, record that. If the goal is delivery elapsed time, record that separately.
It can be distorted by rare failures. A network outage, a bad dependency release, or an unusually difficult bug can create a long tail. P90 is meant to reveal some of that risk, not erase it. Keep the raw observations and examine the outliers rather than deleting them merely because they make the estimate less attractive.
Calibrate instead of admiring the dashboard
A prediction system only earns trust by being tested on tasks it did not use to fit itself. A basic walk-forward evaluation works like this:
- Sort records by completion time.
- Use the first group as history.
- Predict the next task using only earlier records.
- Add the real result to history and continue.
- Count how often actual runtime falls between P50 and P90, and calculate the median absolute error of P50.
Do not evaluate on the same rows used to tune the weights and then call the score a forecast. That leaks the answer into the test. With a small dataset, the result will be noisy; label it as such. If you only have ten sessions, a claim like “this predicts reliably” is too strong.
The project that prompted this article illustrates why the distinction matters. Its README reports an author-run backtest on 72 held-out requests: 71% of actual times landed inside its P10–P90 range, while median P50 error was 67%.[1] Those numbers are not contradictory. A broad range can contain many outcomes while its central prediction remains far off. And because the report comes from the project author’s own machine and tasks, it is a hypothesis to test locally, not an independent benchmark.
Track at least three measurements:
- Range coverage: fraction of actual runtimes inside the predicted interval.
- Median absolute error: the typical distance between P50 and actual time.
- Range width: P90 minus P10, or another interval you choose.
A system that hits every actual by making a range enormous is not useful. A system with a narrow range but poor coverage is overconfident. Reporting all three makes the tradeoff visible.
Privacy is part of the design
Session logs can contain prompts, file paths, code fragments, tool outputs, and details about private projects. A local-only promise is meaningful, but it does not make a plugin automatically safe. Review the code that reads and writes data, check whether hooks execute on every session, and understand the permissions the extension needs before installing it.
The AI Time Estimator project says it reads Claude Code project transcripts, stores its own records locally, and makes no network calls.[1] Treat that as a project claim to verify against the current source and version you install. Do not assume a repository’s README is a security audit. In an organization, check policy before collecting developer activity, even if the data stays on an individual machine.
For your own minimal tracker, store only what you need: a pseudonymous task ID, category, counts, runtime, and whether the run succeeded. Avoid prompts, diffs, repository names, and usernames unless there is a clear reason and an approved retention policy. Make deletion straightforward. If a task is sensitive, leave it out.
A practical decision checklist
Before acting on any agent-time estimate, ask:
- Comparable data: Do I have enough previous tasks of this type?
- Same clock: Does history measure agent runtime, wall time, or delivery time consistently?
- Visible uncertainty: Is there a range, and can I see how wide it is?
- Honest evaluation: Were predictions tested on future or held-out tasks?
- Useful error reporting: Are coverage and median error both shown?
- Privacy boundary: What exact local files are read, and where is history stored?
- Human work: Does the estimate exclude clarification, review, and approval?
- Decision fit: Is this estimate being used for planning, or being mistaken for a deadline commitment?
A calibrated estimate is a planning aid, not a substitute for task discovery or engineering judgment. If the P90 is huge, that may be the most useful result: the request needs clarification or a short investigation before anyone promises a date.
The useful shift is from confidence to evidence
AI Time Estimator is a small, early tool, and the evidence around it is correspondingly limited. Its interesting contribution is not that it has solved project estimation. It is that it treats the agent’s own history as measurable data and closes the loop after each task. The same principle applies to any coding agent: collect consistent observations, keep the analysis local when possible, and show error instead of projecting certainty.
A tiny CSV and a transparent calculation will not forecast every surprise. But it can help distinguish a task that usually takes an agent ten minutes from one that regularly consumes an hour of tool calls and repair loops. Start small, inspect misses, and never turn an experimental P50 into a promise to a stakeholder.
Have you tried estimating coding-agent work from your own session history? What would you track, and what would you refuse to collect?
I write about Java, Spring Boot, and AI every week. Subscribe for practical engineering analysis and implementation patterns.
Top comments (1)
One useful addition is to track how estimates change as the task moves through the workflow. Agent execution time, repair-loop time, and human review or clarification time can have very different distributions, so combining them too early makes it difficult to identify where the uncertainty actually comes from.
I'd also watch for changes in the agent setup itself. A model upgrade, different tool configuration, or a change in test coverage can make older sessions less comparable even when the task category stays the same. A walk-forward evaluation is a good start, but tracking these shifts helps explain when historical calibration stops being useful.
The goal isn't just to predict a duration. It's to know whether the estimate is trustworthy enough for planning, and when a wide range is signaling that the task needs discovery before anyone commits to a deadline.