Skip to content

DEV Community

Measuring ML systems honestly Series' Articles

Back to Raihan's Series

Cover image for How Accurate Is Your Model Right Now? Estimating Accuracy Without Labels

Raihan

Oct 4

How Accurate Is Your Model Right Now? Estimating Accuracy Without Labels

#machinelearning #mlops #python #monitoring

4 min read

Cover image for AWQ Looked 10 Points Better on GSM8K Until I Stopped Truncating the Answers

Raihan

Oct 5

AWQ Looked 10 Points Better on GSM8K Until I Stopped Truncating the Answers

#llm #machinelearning #python #testing

12 min read

Cover image for Do LLMs Invent Japanese Law Articles? A Bilingual Benchmark

Raihan

Oct 6

Do LLMs Invent Japanese Law Articles? A Bilingual Benchmark

#llm #nlp #japanese #opensource

4 min read

Cover image for Small LLM Judges Approved 11% and 41% of Wrong Answers. Then I Fixed My Own Pairwise Test.

Raihan

Oct 7

Small LLM Judges Approved 11% and 41% of Wrong Answers. Then I Fixed My Own Pairwise Test.

#llm #machinelearning #python #evaluation

5 min read

Cover image for GraphProof-QA: Teaching a Small Model to Prove Its Answers

Raihan

Oct 8

GraphProof-QA: Teaching a Small Model to Prove Its Answers

#llm #machinelearning #python #nlp

4 min read

Cover image for Mentioned Is Not Binding: Which Solicitation Clauses Actually Bind the Contract

Raihan

Oct 8

Mentioned Is Not Binding: Which Solicitation Clauses Actually Bind the Contract

#llm #machinelearning #python #nlp

3 min read

Cover image for A Grammar Stopped My Model Inventing FAR Clauses. It Also Made It Refuse the Real Ones

Raihan

Oct 9

A Grammar Stopped My Model Inventing FAR Clauses. It Also Made It Refuse the Real Ones

#llm #machinelearning #python #nlp

4 min read

Cover image for I Distilled a 568M Multilingual Model Into a 37M Japanese-English Encoder — Here's What Survived

Raihan

Oct 10

I Distilled a 568M Multilingual Model Into a 37M Japanese-English Encoder — Here's What Survived

#machinelearning #nlp #python #japanese

4 min read

Cover image for Teaching a Computer to Hear Japanese Pitch Accent

Raihan

Oct 10

Teaching a Computer to Hear Japanese Pitch Accent

#llm #machinelearning #python #japanese

2 min read

Cover image for My Triton Kernel Ties bitsandbytes — and Both Lose to Math

Raihan

Oct 12

My Triton Kernel Ties bitsandbytes — and Both Lose to Math

#gpu #llm #python #performance

4 min read

Cover image for Was My 2,103-Token Tokenizer a Mistake?

Raihan

Oct 13

Was My 2,103-Token Tokenizer a Mistake?

#machinelearning #llm #python #nlp

3 min read

Cover image for DemoDoctor: Bad Robot Demonstrations Are Detectable. Do They Matter?

Raihan

Oct 14

DemoDoctor: Bad Robot Demonstrations Are Detectable. Do They Matter?

#machinelearning #robotics #python #statistics

3 min read

Cover image for Invoice-Check JP: A 3B Model Reads the Invoice, Checks Decide Whether to Trust It

Raihan

Oct 15

Invoice-Check JP: A 3B Model Reads the Invoice, Checks Decide Whether to Trust It

#machinelearning #llm #python #japanese

4 min read

Cover image for PO Matching Looked Like It Halved Unsafe Invoice Approvals Until I Checked Why

Raihan

Oct 16

PO Matching Looked Like It Halved Unsafe Invoice Approvals Until I Checked Why

#langgraph #llm #python #evaluation

5 min read

Cover image for I Pre-Registered a Four-Way LangGraph Shootout. Plan-and-Execute Cost 2.3x and Bought Nothing I Could Detect

Raihan

Oct 17

I Pre-Registered a Four-Way LangGraph Shootout. Plan-and-Execute Cost 2.3x and Bought Nothing I Could Detect

#langgraph #llm #python #evaluation

6 min read

Cover image for Edit the Weights, Retrieve the Fact, or Look It Up in a Graph?

Raihan

Oct 18

Edit the Weights, Retrieve the Fact, or Look It Up in a Graph?

#llm #machinelearning #python #evaluation

2 min read

Cover image for Your LLM Judge Is Lying to You, a Little

Raihan

Oct 19

Your LLM Judge Is Lying to You, a Little

#llm #machinelearning #python #statistics

2 min read

Cover image for The Verb Comes Last: Measuring Why Japanese-English Live Translation Waits

Raihan

Oct 20

The Verb Comes Last: Measuring Why Japanese-English Live Translation Waits

#machinelearning #python #nlp #speech

3 min read

Cover image for My Japanese Tutor Agent Works — Except Its Brain Calls Everything a Particle

Raihan

Oct 21

My Japanese Tutor Agent Works — Except Its Brain Calls Everything a Particle

#machinelearning #python #nlp #langchain

3 min read

Cover image for Do Japanese and English Share Neurons? I Trained the Microscope to Check

Raihan

Oct 22

Do Japanese and English Share Neurons? I Trained the Microscope to Check

#machinelearning #python #nlp #llm

2 min read

Cover image for One Knob, 19x Latency: What I Learned Benchmarking vLLM on a Single GPU

Raihan

Oct 23

One Knob, 19x Latency: What I Learned Benchmarking vLLM on a Single GPU

#machinelearning #python #llm #devops

2 min read

Cover image for Correct, Cited, and Two Years Out of Date: Measuring Version Leakage in RAG

Raihan

Oct 24

Correct, Cited, and Two Years Out of Date: Measuring Version Leakage in RAG

#machinelearning #python #nlp #llm

2 min read

Raihan

Oct 25

Your RAG Has No Idea What Year It Is: A Frozen Benchmark for Version Leakage

#machinelearning #python #nlp #llm

2 min read

Raihan

Oct 26

Your MCP Tools Are Leaking: A Benchmark for Description Canonicalisation

#machinelearning #python #security #llm

3 min read