DEV Community

Cover image for I built an API that searches 3 free libraries at once
Dani
Dani

Posted on

I built an API that searches 3 free libraries at once

#api #python #books #opensource

I wanted to read more classics, and the hardest part was finding them. Free, legal copies are spread across Project Gutenberg, Open Library / Internet Archive and LibriVox. Each has its own search, its own formats, and the same book shows up several times.

So I built Public Domain Book Finder: one API that searches all 3 at once and hands back one tidy list. It's the first of 3 reading APIs I'm building.

What it does

Endpoint What you get
GET /search One search across 78,000+ Gutenberg ebooks, Open Library's fully open scans and LibriVox audiobooks. Duplicates are merged.
GET /book/{id} Every free format for one book: EPUB, Kindle, HTML, plain text, or MP3 chapters for audiobooks.
GET /read/{id} The book as clean text: license pages removed, broken line wraps fixed, chapters listed, served page by page.

One search, 3 libraries 

import os
import requests

HEADERS = {
    "X-RapidAPI-Key": os.environ["RAPIDAPI_KEY"],
    "X-RapidAPI-Host": "public-domain-book-finder.p.rapidapi.com",
}

data = requests.get(
    "https://public-domain-book-finder.p.rapidapi.com/search",
    params={"q": "pride and prejudice", "language": "en", "limit": 3},
    headers=HEADERS,
).json()

print(data["sources"])
for book in data["results"]:
    print(book["title"], book["id"], "+", len(book["also_available"]), "more free copies")
Enter fullscreen mode Exit fullscreen mode 
{'gutenberg': '3 found', 'openlibrary': '2 found', 'librivox': '2 found'}
Pride and Prejudice gutenberg:1342 + 5 more free copies
Pride and Prejudice, a play founded on Jane Austen's novel gutenberg:37431 + 0 more free copies
Enter fullscreen mode Exit fullscreen mode

7 hits from 3 libraries became 2 results. The novel comes with its 5 other copies tucked underneath, including 2 audiobook recordings, so an app can offer "read it or listen to it" without a second search.

Clean text you can actually read

Raw Gutenberg text files start and end with pages of license text, and every line is hard-wrapped at about 70 characters. /read cleans that up:

page = requests.get(
    "https://public-domain-book-finder.p.rapidapi.com/read/gutenberg:216",
    params={"page": 1, "words_per_page": 900},
    headers=HEADERS,
).json()

print(page["title"], f'({page["total_pages"]} pages, {page["word_count"]} words)')
print(page["chapters"])
print(page["content"][6])
Enter fullscreen mode Exit fullscreen mode 
The Tao Teh King, or the Tao and its Characteristics (13 pages, 10657 words)
[{'title': 'PART 1.', 'page': 1}, {'title': 'PART II.', 'page': 6}]
Ch. 1. 1. The Tao that can be trodden is not the enduring and unchanging Tao. The name that can be named is not the enduring and unchanging name.
Enter fullscreen mode Exit fullscreen mode

Pages always end on a paragraph break, and poems keep their line breaks. That makes it a good fit for reading apps, e-ink devices, and AI tools that summarize or index books.

Audiobooks, chapter by chapter 

book = requests.get(
    "https://public-domain-book-finder.p.rapidapi.com/book/librivox:253",
    headers=HEADERS,
).json()
print(book["audio"]["total_time"], len(book["audio"]["chapters"]), "MP3 files")
Enter fullscreen mode Exit fullscreen mode 
13:06:44 37 MP3 files
Enter fullscreen mode Exit fullscreen mode

Every chapter has a direct MP3 link and its length, plus a zip of the whole book and a podcast feed you can paste into any podcast app.

How it works

  • Gutenberg straight from the source. Project Gutenberg publishes its whole catalog as one CSV file and asks programs to use it instead of crawling the site. The API downloads it at startup, refreshes it daily and searches it in memory, so Gutenberg results are instant.
  • The other 2 libraries are searched live, side by side. Open Library and LibriVox are called together with asyncio.gather, so a search takes about as long as the slowest library. If one doesn't answer, you still get results from the others, and sources tells you which one was unavailable.
  • Duplicates are matched by title and author surname, after dropping things like "The", subtitles and accents. The first copy found becomes the main result.
  • Only free, legal copies. Open Library results are limited to fully public scans (no borrowing), and anything still under copyright is refused.

It's written in Python with FastAPI and httpx, and runs on Render.

Try it

Next up in this series: a Clean PDF Reader API. If you build something with this one, I'd love to see it.

Top comments (0)