Goodreads is still where most readers log what they read, so it's where the numbers are: how many people rated a book, how those ratings split across one to five stars, what the most-liked reviews say, and which of an author's books carry their reputation. If you build a reading app, do comp titles for a publisher, or just want to know whether a book you're about to pitch is growing, you'll want that data in a script.
Then you search for "Goodreads API" and find out it's gone. This post covers what's left officially, what goodreads.com hands a plain script today, and then builds goodreads.mjs, one Node.js file with no npm dependencies (built-in fetch and node:sqlite, so Node 22.13+). It has six commands:
-
probechecks what goodreads.com returns to a script and reads the data embedded in a book page -
searchfinds books by title or ISBN -
bookprints details and the full one-to-five-star rating distribution -
reviewspages through community reviews with a cursor and summarizes them -
authorlists an author's books and shows how concentrated their ratings are -
trackstores rating snapshots in SQLite and reports new ratings per day between runs
Every command was run on November 13, 2026 on a rented Linux VM (Node 24.22.1), and each output block below is pasted unedited from that session.
Read this first. Goodreads' Terms of Use forbid this kind of collection. Section 1 ("Use of Our Service") of the terms at goodreads.com/about/terms (last revised April 28, 2021) says your license to use the site does not include "any collection and use of any book listings, descriptions, reviews or other material included in the Service" or "any use of data mining, robots, or similar data gathering and extraction tools." So scraping Goodreads is against its terms, whoever does the fetching. Read the terms yourself and think about the legal risk for your use case before you run anything here, and use the official options below wherever they cover what you need. Reviews are also personal writing by real people: don't republish them, and keep only what you need.
Is there still a Goodreads API?
Not for new projects. In December 2020 Goodreads stopped issuing new developer keys and said it planned to retire the public API. Today goodreads.com/api just redirects to the homepage (you'll see the 302 in Step 1). Old keys you may find in blog posts and GitHub repos don't help a new project.
What you can still use without scraping:
- Your own library export. Under My Books, "Import and export" gives you a CSV of your own shelves, ratings, and review dates. If you only need your own reading data, stop here.
- Open Library. The Internet Archive's Open Library API is open and has ISBN, work, and author lookups. It has ratings too, but far fewer: on November 13 it had about 200 ratings for Project Hail Mary, versus about 1.94 million on Goodreads.
If you need Goodreads' own ratings, distributions, or reviews, there's no official route left. That's what the rest of this post is about.
Step 1: See what goodreads.com sends a script
Book pages are a Next.js app. The HTML contains a <script id="__NEXT_DATA__"> tag with the page's Apollo GraphQL cache, so you don't need to parse markup. The cache holds Book:..., Work:..., Review:..., and User:... entries. The rating statistics live on the work (all editions together), not on the edition you asked for.
#!/usr/bin/env node
// goodreads.mjs - Goodreads book, review, and author data in Node.js (22.13+), no npm dependencies.
// Usage: node goodreads.mjs <probe|search|book|reviews|author|track> ...
import { DatabaseSync } from "node:sqlite";
import { writeFileSync } from "node:fs";
const KEY = process.env.SCRAPEATLAS_API_KEY ?? "";
const API = "https://api.scrapeatlas.com/v1/goodreads";
const UA = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0 Safari/537.36";
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function get(url, headers = {}) {
const r = await fetch(url, { headers, redirect: "manual", signal: AbortSignal.timeout(30_000) });
return { status: r.status, location: r.headers.get("location"), text: await r.text() };
}
function nextData(html) {
const m = html.match(/<script id="__NEXT_DATA__" type="application\/json">(.*?)<\/script>/s);
if (!m) throw new Error("no __NEXT_DATA__ on the page (layout changed or you got a block page)");
return JSON.parse(m[1]).props.pageProps.apolloState;
}
function bookFromApollo(state) {
const values = Object.entries(state);
const book = values.find(([k, v]) => k.startsWith("Book:") && v.title)?.[1];
const work = values.find(([k]) => k.startsWith("Work:"))?.[1];
const reviews = values.filter(([k]) => k.startsWith("Review:")).map(([, v]) => v);
return {
title: book.title,
isbn13: book.details?.isbn13,
pages: book.details?.numPages,
genres: (book.bookGenres ?? []).map((g) => g.genre.name),
averageRating: work.stats.averageRating,
ratingsCount: work.stats.ratingsCount,
ratingsCountDist: work.stats.ratingsCountDist, // [1-star, 2-star, 3-star, 4-star, 5-star]
textReviewsCount: work.stats.textReviewsCount,
embeddedReviews: reviews.length,
};
}
async function probe(bookId) {
const page = `https://www.goodreads.com/book/show/${bookId}`;
let real;
for (const ua of ["curl/8.5.0", "", null, UA]) {
const res = await get(page, ua === null ? {} : { "user-agent": ua, "accept-language": "en-US,en;q=0.9" });
const label = ua === null ? "Node default UA" : ua === "" ? "empty UA" : ua.slice(0, 15);
console.log(`GET /book/show/${bookId} [${label.padEnd(15)}] -> ${res.status}, ${res.text.length} bytes`);
if (ua === UA) real = res;
}
const b = bookFromApollo(nextData(real.text));
console.log(` __NEXT_DATA__: "${b.title}" avg ${b.averageRating}, ${b.ratingsCount} ratings, dist ${JSON.stringify(b.ratingsCountDist)}`);
console.log(` genres: ${b.genres.slice(0, 5).join(", ")} | ${b.textReviewsCount} text reviews, ${b.embeddedReviews} embedded in the HTML`);
const more = real.text.match(/\/book\/show\/\d+\/reviews\?reviewFilters=[^"]+/);
console.log(` "more reviews" link: ${more ? more[0].slice(0, 60) + "..." : "none"}`);
const ac = await get(`https://www.goodreads.com/book/auto_complete?format=json&q=${encodeURIComponent(b.title)}`, { "user-agent": UA });
const hits = JSON.parse(ac.text);
console.log(`GET /book/auto_complete?format=json -> ${ac.status}, ${hits.length} hits; top: ${hits[0]?.bookId} "${hits[0]?.title}" (${hits[0]?.ratingsCount} ratings)`);
const api = await get("https://www.goodreads.com/api", { "user-agent": UA });
console.log(`GET /api -> ${api.status} ${api.location ?? ""}`);
}
$ node goodreads.mjs probe 54493401
GET /book/show/54493401 [curl/8.5.0 ] -> 200, 744749 bytes
GET /book/show/54493401 [empty UA ] -> 403, 919 bytes
GET /book/show/54493401 [Node default UA] -> 503, 1356 bytes
GET /book/show/54493401 [Mozilla/5.0 (X1] -> 200, 745286 bytes
__NEXT_DATA__: "Project Hail Mary" avg 4.51, 1938684 ratings, dist [8979,28675,141888,549311,1209831]
genres: Science Fiction, Fiction, Audiobook, Book Club, Fantasy | 265795 text reviews, 30 embedded in the HTML
"more reviews" link: /book/show/54493401/reviews?reviewFilters=eyJhZnRlciI6Ik5EUT...
GET /book/auto_complete?format=json -> 200, 4 hits; top: 54493401 "Project Hail Mary" (1938672 ratings)
GET /api -> 302 https://www.goodreads.com/
What that tells you:
- The data is there. One request gets the average, the rating count, the five-bucket distribution, genres, and the text review count.
-
Only 30 reviews come with the page. The "more reviews" link is
/book/show/<id>/reviews?reviewFilters=<base64 cursor>, androbots.txtdisallows both/book/reviews/and/*reviewFilters. It also disallows/search,/work, and/review/list. -
/book/auto_complete?format=jsonis the search box's JSON endpoint. It's handy for title-to-ID lookups, but it only returns a handful of hits. -
Status codes are unreliable. An empty User-Agent got
403every time. Everything else changed from run to run: in that probe, the Node default UA got a503("Goodreads - unexpected error") and acurlUA got a200. A few minutes earlier, testing with thecurlcommand itself, thecurlUA got the503, while Node, Pythonrequests, and Go default UAs got200.
To see how bad that last point is, I requested ten book pages in a row, one second apart, for each of three User-Agents. Here are two runs a few minutes apart (the first version of the script only counted status codes):
$ node burst.mjs # run 1
{
'browser 200': 3,
'browser 202': 7,
'empty 403': 10,
'default 202': 10
}
$ node burst.mjs # run 2, also checks for __NEXT_DATA__
browser UA {"200 with data":10}
empty UA {"403 no data":10}
first non-data reply: 403 919B <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <HTML><
Node default {"200 with data":9,"503 no data":1}
first non-data reply: 503 1356B <!DOCTYPE html> <html> <head> <title>Goodreads - unexpected error</title> <link href="https://s.gr-assets.com/
So from one cloud IP, you get a mix of 200, 202, and 503 even at one request per second. My advice: check for __NEXT_DATA__ in the body rather than trusting the status code, and expect the Apollo keys to change whenever Goodreads ships a new frontend.
Step 2: A small API client
Reading the book page yourself works for a single book's stats. Paging through reviews means reverse-engineering the reviewFilters cursor that robots.txt asks crawlers to stay away from, and search results and author lists are more page types to parse, each of which can break on its own. So from here on, the script reads Goodreads through ScrapeAtlas*. Its OpenAPI spec lists five Goodreads routes: /v1/goodreads/search, /book, /book-reviews, /author, and /author-books. You call them with GET and pass your key in the x-api-key header. Each successful call, partial results included, costs 1 credit. Failed calls are free. Using an API doesn't change the terms: Goodreads' rules still apply to the data, whoever fetches it.
async function sa(path, params) {
if (!KEY) throw new Error("Set SCRAPEATLAS_API_KEY first.");
const qs = new URLSearchParams(Object.entries(params).filter(([, v]) => v != null && v !== ""));
for (let attempt = 1; ; attempt++) {
const r = await fetch(`${API}/${path}?${qs}`, { headers: { "x-api-key": KEY }, signal: AbortSignal.timeout(120_000) });
const body = await r.json().catch(() => ({}));
if (r.ok && body.success !== false) return body;
const code = body.failure?.code ?? `http_${r.status}`;
const retry = r.status === 429 || r.status >= 500;
if (!retry || attempt === 3) {
const err = new Error(`${path}: ${code} - ${body.failure?.message ?? r.statusText}`);
err.code = code;
throw err;
}
await sleep(2000 * attempt);
}
}
// Accepts 54493401, "54493401-project-hail-mary", or a full goodreads.com URL.
function idOf(x) {
const m = String(x).match(/(?:show\/)?(\d+)/);
if (!m) throw new Error(`not a Goodreads id or URL: ${x}`);
return m[1];
}
sa() retries 429 and 5xx up to three times and throws errors that carry the API's failure.code (for example not_found), so the commands can tell a missing book from a network problem. idOf() takes 54493401, 54493401-project-hail-mary, or a full book or author URL.
Step 3: Search by title or ISBN
async function search(query, page = 1) {
const res = await sa("search", { query, page });
console.log(`"${query}": ${res.total ?? "?"} results, page ${res.page}, hasMore=${res.hasMore}`);
for (const b of res.books) {
const author = b.authors.map((a) => a.name).join(", ");
console.log(`${String(b.id).padEnd(10)} ${b.averageRating.toFixed(2)} ${String(b.ratingsCount).padStart(9)} ${String(b.publishedYear ?? "----")} ${b.title.slice(0, 48)} - ${author}`);
}
}
$ node goodreads.mjs search project hail mary
"project hail mary": 107 results, page 1, hasMore=true
54493401 4.51 1938672 2021 Project Hail Mary - Andy Weir
119255205 4.45 869 ---- Project Hail Mary / Artemis/ The Martian - Andy Weir
58007522 4.04 25 ---- SUMMARY AND ANALYSIS OF PROJECT HAIL MARY BY AND - Good Reads Publishing
58057170 3.85 20 ---- Summary & Analysis of Andy Weir Book: Project Ha - PressPrint
58663158 3.79 14 ---- SUMMARY OF PROJECT HAIL MARY NOVEL: Detailed Sum - Evergreen Summaries
61157535 3.75 4 ---- Summary: Project Hail Mary: by Andy Weir - Zara Library
60211385 4.67 3 ---- Summary of Project Hail Mary: A Novel by Andy We - rah man
61213069 4.50 4 ---- Study Guide: Project Hail Mary by Andy Weir - SuperSummary
238724014 5.00 3 ---- Project Hail Mary Movie Review: From Page to Pro - Vivian A. Kendrick
58051187 3.60 5 ---- SYNOPSIS OF PROJECT HAIL MARY: An Epic Tale of R - Stephen White
60695472 4.00 1 ---- SUMMARY OF PROJECT HAIL MARY BY ANDY WEIR: A NOV - Harry Thomas
242321051 3.84 220 2026 Mars One: An explosive, race-against-time thrill - Charlotte Robinson
59821976 1.00 1 ---- Summary of Project Hail Mary: A Novel By Andy We - Anis Classics
60556727 2.00 1 ---- Summary Guide of Project Hail Mary by Andy Weir: - Unboxread
59743662 5.00 1 ---- Summary of Project Hail Mary by Andy Weir - Rita Bennet
60748861 3.00 4 ---- Epilogue for Project Hail Mary - Mark C. Tredecim
The real book is first. Most of the rest of page 1 is summary books, study guides, and "movie review" booklets: low-effort titles that piggyback on a bestseller's name. If you search by title in bulk, don't just take the first hit blindly. Check that the author matches and that ratingsCount is plausible, or search by ISBN instead:
$ node goodreads.mjs search 9780593135204
"9780593135204": 1 results, page 1, hasMore=false
54493401 4.51 1938684 2021 Project Hail Mary - Andy Weir
One exact hit. Two other things to notice. The title search reported 1938672 ratings and the ISBN search 1938684, a few minutes apart, because search results and book pages aren't refreshed at the same time. And publishedYear is null (printed as ----) for many editions and box sets. Here's the same kind of noise for another big book:
$ node goodreads.mjs search sunrise on the reaping
"sunrise on the reaping": 52 results, page 1, hasMore=true
214331246 4.50 1401387 2025 Sunrise on the Reaping - Suzanne Collins
217451201 4.66 1104 ---- Hunger Games 5-Book Hardcover Box Set - Suzanne Collins
231118820 4.49 207 ---- The Hunger Games 2 Book Collection Set - Suzanne Collins
231072856 4.44 40 ---- Sunrise on the Reaping: A gripping tale of coura - P. Love
244829102 4.35 20 ---- Sunrise on the Reaping: The Complete Synopsis an - Amy McMahon, Jonah Gathu
252237379 4.67 3 ---- The Hunger Games: Sunrise on the Reaping Movie R - James A. Schultz
238495844 5.00 2 ---- Sunrise on the Reaping: Weaving an unforgettable - Gift Betty
245877500 4.92 3 ---- SUNRISE ON THE REAPING BY SUZANNE COLLINS - Unknown Author
235026481 5.00 1 ---- The Hunger Games: Sunrise on the Reaping Pouch - Unknown Author
244086478 5.00 1 ---- Sunrise on the Reaping: Hunger Game - J Peace
245440175 3.00 1 ---- Sunrise on the Reaping: Hunger Games Book 5 - Unknown Author
231431811 0.00 0 ---- Study Guide: Sunrise on the Reaping by Suzanne C - SuperSummary
231575918 0.00 0 ---- Sinister Sunrise: Gothic Symbols, Rome, and Mode - Valerie Estelle Frankel
Step 4: Book details and the rating distribution
The average rating hides most of the story. Two books with a 4.2 can look completely different: one where nearly everyone gave 4 stars, and one that splits between 5 and 1. The distribution shows which one you have.
function ratingStats(dist) {
const counts = [1, 2, 3, 4, 5].map((s) => dist[s] ?? 0);
const total = counts.reduce((a, b) => a + b, 0);
const mean = counts.reduce((a, c, i) => a + c * (i + 1), 0) / total;
const share = (stars) => stars.reduce((a, s) => a + counts[s - 1], 0) / total;
return { counts, total, mean, five: share([5]), low: share([1, 2]) };
}
function bar(n, max, width = 30) {
return "#".repeat(Math.round((n / max) * width));
}
async function book(ref) {
const { book: b } = await sa("book", { id: idOf(ref) });
const st = ratingStats(b.ratingsDistribution);
const max = Math.max(...st.counts);
console.log(`${b.title} - ${b.authors.map((a) => a.name).join(", ")} (${b.work?.firstPublishedAt?.slice(0, 4) ?? "?"})`);
console.log(` ${b.format ?? "?"}, ${b.pages ?? "?"} pages, ISBN13 ${b.isbn13 ?? "-"}, ${b.publisher ?? "-"}`);
console.log(` avg ${b.averageRating} from ${b.ratingsCount} ratings (mean of distribution: ${st.mean.toFixed(3)}), ${b.reviewsCount} text reviews`);
for (let s = 5; s >= 1; s--) {
const n = st.counts[s - 1];
console.log(` ${s}* ${String(n).padStart(9)} ${(100 * n / st.total).toFixed(1).padStart(5)}% ${bar(n, max)}`);
}
console.log(` 5-star share ${(100 * st.five).toFixed(1)}%, 1-2 star share ${(100 * st.low).toFixed(1)}%`);
console.log(` genres: ${b.genres.slice(0, 6).map((g) => g.name).join(", ")}`);
const wins = (b.awards ?? []).filter((a) => a.designation === "WINNER");
if (b.awards?.length) console.log(` awards: ${b.awards.length} listed, ${wins.length} wins (${wins.slice(0, 3).map((a) => a.name).join(", ")})`);
if (b.series?.length) console.log(` series: ${b.series.map((s) => s.position ? `${s.title} #${s.position}` : `${s.title} (no position given)`).join(", ")}`);
return b;
}
$ node goodreads.mjs book 54493401 https://www.goodreads.com/book/show/214331246-sunrise-on-the-reaping
Project Hail Mary - Andy Weir (2021)
Hardcover, 476 pages, ISBN13 9780593135204, Ballantine Books
avg 4.51 from 1938684 ratings (mean of distribution: 4.507), 265795 text reviews
5* 1209831 62.4% ##############################
4* 549311 28.3% ##############
3* 141888 7.3% ####
2* 28675 1.5% #
1* 8979 0.5%
5-star share 62.4%, 1-2 star share 1.9%
genres: Science Fiction, Fiction, Audiobook, Book Club, Fantasy, Space
awards: 8 listed, 5 wins (Geffen Award, Audie Award, Nutmeg Book Award)
Sunrise on the Reaping - Suzanne Collins (2025)
Hardcover, 382 pages, ISBN13 9781546171461, Scholastic Press
avg 4.49 from 1401429 ratings (mean of distribution: 4.489), 231295 text reviews
5* 840556 60.0% ##############################
4* 430014 30.7% ###############
3* 110401 7.9% ####
2* 17177 1.2% #
1* 3281 0.2%
5-star share 60.0%, 1-2 star share 1.5%
genres: Dystopia, Young Adult, Fiction, Fantasy, Audiobook, Science Fiction
awards: 12 listed, 7 wins (British Book Award, Audie Award, Goodreads Choice Award)
series: The Hunger Games (no position given)
Some notes on the fields:
-
ratingsDistributionis keyed"1"to"5". Recomputing the mean from it (4.507) matches the roundedaverageRating(4.51), which is a cheap sanity check that you parsed the right object. -
reviewsCountcounts text reviews only (265,795). The other ~1.67 million people only clicked a star. -
The stats are for the work, so every edition of a book returns the same numbers. Use
work.idas your key if you collect several editions. -
seriesdoesn't always have a position. Sunrise on the Reaping is in "The Hunger Games" series, butpositioncame backnull, so don't count on it for reading order. -
Awards include nominations. Filter on
designation === "WINNER"if you only want wins.
Step 5: Reviews with cursor pagination
/v1/goodreads/book-reviews returns 30 reviews per page and a nextCursor. Pass it back as cursor to get the next page. The function below stops when there's no cursor or a page brings nothing new, deduplicates by review ID, and can write everything to JSONL.
async function reviews(ref, pages = 3, out = null) {
const id = idOf(ref);
const seen = new Map();
let cursor = null;
let total = null;
for (let p = 1; p <= pages; p++) {
const res = await sa("book-reviews", { id, cursor });
total = res.totalCount;
let fresh = 0;
for (const r of res.reviews) if (!seen.has(r.id)) { seen.set(r.id, r); fresh++; }
console.log(`page ${p}: ${res.reviews.length} reviews (${fresh} new), totalCount ${res.totalCount}`);
cursor = res.nextCursor;
if (!cursor || fresh === 0) break;
await sleep(500);
}
const all = [...seen.values()];
if (out) writeFileSync(out, all.map((r) => JSON.stringify(r)).join("\n") + "\n");
summarizeReviews(all, total);
if (out) console.log(`saved ${all.length} reviews to ${out}`);
return all;
}
function summarizeReviews(all, total) {
const byStar = [0, 1, 2, 3, 4, 5].map((s) => all.filter((r) => r.rating === s).length);
const words = all.map((r) => (r.text ?? "").split(/\s+/).filter(Boolean).length).sort((a, b) => a - b);
const median = words[Math.floor(words.length / 2)] ?? 0;
console.log(`\n${all.length} unique reviews of ~${total}; stars 5..1: ${byStar.slice(1).reverse().join(" / ")}; unrated: ${byStar[0]}`);
console.log(`median length ${median} words; spoiler-flagged ${all.filter((r) => r.spoiler).length}; written by Goodreads authors ${all.filter((r) => r.reviewer?.isAuthor).length}`);
const years = {};
for (const r of all) years[r.createdAt.slice(0, 4)] = (years[r.createdAt.slice(0, 4)] ?? 0) + 1;
console.log(`created per year: ${Object.entries(years).sort().map(([y, n]) => `${y}:${n}`).join(" ")}`);
console.log("most-liked:");
for (const r of [...all].sort((a, b) => b.likeCount - a.likeCount).slice(0, 3)) {
console.log(` ${r.rating}* ${String(r.likeCount).padStart(6)} likes ${r.reviewer.name.slice(0, 28)}: "${(r.text ?? "").replace(/\s+/g, " ").slice(0, 70)}..."`);
}
}
$ node goodreads.mjs reviews 54493401 --pages 4 --out runs/phm-reviews.jsonl
page 1: 30 reviews (30 new), totalCount 265185
page 2: 30 reviews (30 new), totalCount 265252
page 3: 30 reviews (30 new), totalCount 265185
page 4: 30 reviews (30 new), totalCount 265185
120 unique reviews of ~265185; stars 5..1: 71 / 20 / 11 / 8 / 6; unrated: 4
median length 223 words; spoiler-flagged 1; written by Goodreads authors 12
created per year: 2020:2 2021:56 2022:13 2023:4 2024:10 2025:9 2026:26
most-liked:
1* 6864 likes Julie In Wonderland: "Andy Weir cannot write and I'm tired of people pretending he can. Firs..."
5* 2954 likes Nataliya: "2025: Four years later this book is just as good and just as much fun...."
5* 2427 likes Yun: "Thoroughly imaginative, scientifically sound, and emotionally stirring..."
saved 120 reviews to runs/phm-reviews.jsonl
Two things here will mislead you if you don't know them:
- The order isn't random. Reviews come in Goodreads' default order, which puts popular, much-liked reviews first. In these 120, 14 of the 116 star-rated reviews are 1 or 2 stars (12%). Across all ratings, the distribution from Step 4 says it's 1.9%. The most-liked review of this 4.51-rated book is a 1-star rant with 6,864 likes. So a few pages of reviews tell you what people are reading about a book, not how readers rate it. Use the distribution for that.
-
totalCountmoves around. It came back as265185and265252on different pages of the same run, and the book'sreviewsCountwas265795. Treat it as an estimate.
Also: rating is 0 when someone wrote a review without picking stars (4 here). Each review has both text and textHtml, and the HTML can contain images and quotes. reviewer.isAuthor marks Goodreads author accounts, which is useful if you want to separate fellow writers from regular readers. Two reviews are from 2020, before the book came out in May 2021. Those are most likely advance-copy readers.
Step 6: An author's books
/v1/goodreads/author gives the author's totals. /v1/goodreads/author-books lists their books 30 per page with hasMore.
async function author(ref, pages = 2) {
const id = idOf(ref);
const { author: a } = await sa("author", { id });
console.log(`${a.name}: avg ${a.averageRating} over ${a.ratingsCount} ratings, ${a.worksCount} works, ${a.followersCount} followers`);
const books = [];
for (let page = 1; page <= pages; page++) {
const res = await sa("author-books", { id, page });
books.push(...res.books);
if (!res.hasMore) break;
}
const own = books.filter((b) => b.authors[0]?.id === Number(id));
const total = own.reduce((s, b) => s + b.ratingsCount, 0);
console.log(`${books.length} books listed (${books.length - own.length} where ${a.name} isn't first author)`);
let running = 0;
for (const b of own.sort((x, y) => y.ratingsCount - x.ratingsCount).slice(0, 8)) {
running += b.ratingsCount;
console.log(` ${String(b.publishedYear ?? "----")} ${b.averageRating.toFixed(2)} ${String(b.ratingsCount).padStart(9)} ${(100 * running / total).toFixed(1).padStart(5)}% cum ${b.title.slice(0, 50)}`);
}
const undated = own.filter((b) => !b.publishedYear);
console.log(` ${undated.length} titles without a publication year (often translations or box sets: ${undated.slice(0, 2).map((b) => b.title.slice(0, 30)).join(" | ")})`);
}
$ node goodreads.mjs author 6540057 --pages 2
Andy Weir: avg 4.37 over 3826923 ratings, 59 works, 77442 followers
59 books listed (16 where Andy Weir isn't first author)
2021 4.51 1938672 50.7% cum Project Hail Mary
2011 4.42 1396219 87.3% cum The Martian
2017 3.67 372324 97.0% cum Artemis
2009 4.20 43165 98.1% cum The Egg
2019 3.54 38389 99.1% cum Randomize
2017 3.92 7431 99.3% cum The Egg and Other Stories
2019 3.52 6912 99.5% cum Cheshire Crossing
2015 3.90 5574 99.7% cum Diary of an AssCan (The Martian #0.5)
16 titles without a publication year (often translations or box sets: Project Hail Mary / Artemis/ T | Rat)
The bibliography includes more than you'd expect. 16 of the 59 entries list someone else first: anthologies, a screenplay, a signed limited edition of Ready Player One, and foreign-language story collections. Filtering on the first author's ID removes those. The cumulative column is the interesting part. Two books hold 87% of all ratings for Andy Weir's own titles, and three books hold 97%. If you're comparing authors, that concentration tells you more than the author-wide average (4.37), which mixes the big novels with shorter works, some of them rated around 3.5.
Step 7: A ratings tracker in SQLite
A single snapshot can't tell you whether a book is gaining readers. Two snapshots can. track stores the totals and all five buckets for each book, then compares the newest snapshot with the previous one.
function openDb(file) {
const db = new DatabaseSync(file);
db.exec(`CREATE TABLE IF NOT EXISTS snapshot (
book_id INTEGER, taken_at TEXT, title TEXT, avg REAL, ratings INTEGER, reviews INTEGER,
s1 INTEGER, s2 INTEGER, s3 INTEGER, s4 INTEGER, s5 INTEGER,
PRIMARY KEY (book_id, taken_at))`);
return db;
}
async function track(refs, dbFile = "goodreads.db") {
const db = openDb(dbFile);
const prevStmt = db.prepare("SELECT * FROM snapshot WHERE book_id = ? ORDER BY taken_at DESC LIMIT 1");
const insert = db.prepare("INSERT INTO snapshot VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)");
const now = new Date().toISOString();
for (const ref of refs) {
let b;
try {
({ book: b } = await sa("book", { id: idOf(ref) }));
} catch (e) {
console.log(`${ref}: skipped (${e.code ?? e.message})`);
continue;
}
const d = b.ratingsDistribution;
const prev = prevStmt.get(b.id);
insert.run(b.id, now, b.title, b.averageRating, b.ratingsCount, b.reviewsCount, d[1], d[2], d[3], d[4], d[5]);
if (!prev) {
console.log(`${b.title.slice(0, 34).padEnd(34)} first snapshot: ${b.ratingsCount} ratings, avg ${b.averageRating}`);
continue;
}
const hours = (Date.parse(now) - Date.parse(prev.taken_at)) / 3.6e6;
const added = b.ratingsCount - prev.ratings;
const deltas = [1, 2, 3, 4, 5].map((s) => d[s] - prev[`s${s}`]);
console.log(`${b.title.slice(0, 34).padEnd(34)} +${added} ratings in ${hours.toFixed(1)}h (${(added / hours * 24).toFixed(0)}/day), +${b.reviewsCount - prev.reviews} reviews, avg ${prev.avg} -> ${b.averageRating}`);
console.log(`${"".padEnd(34)} new ratings by star 5..1: ${deltas.slice().reverse().map((x) => (x >= 0 ? "+" : "") + x).join(" / ")}`);
}
}
First run (999999999999 is a made-up ID, to show the skip path):
$ node goodreads.mjs track 54493401 214331246 18007564 999999999999
Project Hail Mary first snapshot: 1938684 ratings, avg 4.51
Sunrise on the Reaping first snapshot: 1401429 ratings, avg 4.49
The Martian first snapshot: 1396209 ratings, avg 4.42
999999999999: skipped (not_found)
Second run, 28 minutes later:
$ node goodreads.mjs track 54493401 214331246 18007564
Project Hail Mary +0 ratings in 0.5h (0/day), +0 reviews, avg 4.51 -> 4.51
new ratings by star 5..1: +0 / +0 / +0 / +0 / +0
Sunrise on the Reaping +0 ratings in 0.5h (0/day), +0 reviews, avg 4.49 -> 4.49
new ratings by star 5..1: +0 / +0 / +0 / +0 / +0
The Martian +15 ratings in 0.5h (768/day), +0 reviews, avg 4.42 -> 4.42
new ratings by star 5..1: +6 / +6 / +2 / +1 / +0
That's a useful result, because it shows a trap. Project Hail Mary and Sunrise on the Reaping didn't move at all in half an hour. That's almost certainly not because nobody rated them. Goodreads refreshes these counters in batches: between 04:01 and 04:04 UTC that morning, Project Hail Mary jumped from 1,938,620 to 1,938,684 ratings, then stayed exactly there for over half an hour, and the book page itself (fetched directly at 04:36 UTC) showed the same frozen number. The Martian happened to get a refresh in between, so its "768/day" is just 15 ratings extrapolated from a 28-minute window.
So don't run the tracker every few minutes. Run it once a day at the same time, and look at trends over a week or more. The per-star deltas are where it gets interesting over time: if a book's new ratings start landing in the 1- and 2-star buckets while the average still reads 4.4, the average is being held up by old ratings.
To run it every day, a cron line is enough:
15 6 * * * cd /opt/goodreads && SCRAPEATLAS_API_KEY=... node goodreads.mjs track 54493401 214331246 18007564 >> track.log 2>&1
Each track run costs one credit per book. The database is a single snapshot table, so SELECT taken_at, ratings, s5, s1 FROM snapshot WHERE book_id = 54493401 gives you a chart-ready series.
The CLI
const [cmd, ...args] = process.argv.slice(2);
const flag = (name, def) => {
const i = args.indexOf(`--${name}`);
if (i === -1) return def;
const v = args[i + 1];
args.splice(i, 2);
return v;
};
try {
if (cmd === "probe") await probe(idOf(args[0] ?? "54493401"));
else if (cmd === "search") { const page = Number(flag("page", 1)); await search(args.join(" "), page); }
else if (cmd === "book") for (const a of args) { await book(a); console.log(); }
else if (cmd === "reviews") { const pages = Number(flag("pages", 3)); const out = flag("out", null); await reviews(args[0], pages, out); }
else if (cmd === "author") { const pages = Number(flag("pages", 2)); await author(args[0], pages); }
else if (cmd === "track") { const db = flag("db", "goodreads.db"); await track(args, db); }
else console.log("usage: node goodreads.mjs <probe|search|book|reviews|author|track> ...");
} catch (e) {
console.error(`error: ${e.message}`);
process.exitCode = 1;
}
Gotchas I hit, collected
-
The official API is gone.
/apiredirects to the homepage. Old tutorials withkey=URLs won't help. -
Check the body, not the status. Empty UA →
403. Everything else gave a mix of200,202, and503within minutes, from the same IP. -
Only 30 reviews per book page. More need the
reviewFilterscursor, whichrobots.txtdisallows. -
Stats belong to the work. All editions share them. Key on
work.id. -
Title search returns summary spam. Search by ISBN, or check the author and
ratingsCount. - Counts differ between surfaces. Search, book page, and author list can be a dozen or so ratings apart at the same moment.
- Review order is popularity, not a sample. Low-star reviews are over-represented among the most-liked.
-
rating: 0means no stars, not a zero-star review. -
series[].positioncan benull, andpublishedYearis oftennullfor editions and box sets. - Bibliographies include anthologies and forewords. Filter on the first author's ID.
Wrapping up
- There's no Goodreads API for new projects. Your own CSV export and Open Library cover some needs.
- Book pages embed a full Apollo cache with rating distributions, genres, awards, and 30 reviews, but the site answers a script inconsistently.
- The five-bucket distribution says more than the average does, and it lets you check your parsing.
- Reviews show what's popular, not what's typical. Don't estimate sentiment from the first pages.
- Two SQLite snapshots are enough to measure new ratings per day and which star buckets they land in.
What do you use Goodreads data for: reading apps, comp titles, or something else? I'd like to hear how you deal with the summary-book spam in search.
*I build it.
Top comments (0)