Evaluation - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Muhammed Rasin O M

Jul 10

Half the answer keys in text-to-SQL benchmarks are wrong. So I generated the database from the answer key.

#evaluation #dataagents #benchmarks #syntheticdata

7 min read

Jul 10

The regression your eval set is too small to catch

#machinelearning #statistics #evaluation #datascience

7 min read

Jul 8

One average eval score was hiding two different failure modes

#evaluation #statistics #analytics #datascience

5 min read

Jul 5

Evaluating LLM Apps in Java

#java #ai #llm #evaluation

10 min read

Jul 5

Evaluating LLM Apps in Python

#python #ai #llm #evaluation

9 min read

Saurav Bhattacharya

Jul 2

Short-Circuit Your Agent Evals: Tier Order Is a Latency Budget, Not a Preference

#ai #agents #evaluation #typescript

5 min read

Breach Protocol

Jul 1

Your AI judge might be reliable — and still be wrong

#evaluation #llmjudges #rlhf #methodology

3 min read

Breach Protocol

Jul 1

Reliable, and still wrong

#evaluation #llmasjudge #benchmarks

3 min read

Saurav Bhattacharya

Jun 29

Give Your Agent a Type Signature: Contract-First Output Beats a Smarter Judge

#ai #agents #evaluation #typescript

4 min read

Saurav Bhattacharya

Jun 28

Your Model-as-Judge Doesn't Belong in the Hot Path

#ai #agents #evaluation #observability

9 min read

Jun 28

Evaluating Large Language Models: The Pitfall of Overfitting in RAG

#llm #evaluation #overfitting #rag

2 min read

Jun 28

Evaluating Large Language Models: The Overfitting Problem

#llm #evaluation #overfitting #rag

2 min read

Jun 27

Building Evals That Don't Lie: How to Make AI Evaluation Reliable in Production

#ai #evaluation #production #llm

5 min read

Alex @ Vibe Agent Making

Jun 24

Our Quality Scores Were Precise, Useless, and Identical

#engineering #management #evaluation #codequality

8 min read

Saurav Bhattacharya

Jun 27

Who Grades the Grader? Your LLM Judge Is an Unvalidated Model in Production

#ai #evaluation #observability #testing

5 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.