loading...

Faster Than Requests with Multi-Thread URL-to-SQLite Web Scraper

juancarlospaco profile image Juan Carlos ・1 min read

Alt Text

Faster Than Requests 0.9.6 🐍

✔️ 5 Web Scrapers, Built-in, One-Liner.
✔️ Multi-Threading mass Downloader.
✔️ GitHub Actions build from scratch on every Push.
✔️ GitHub Actions publish to PYPI on every Release.
✔️ Basic get(), post(), download(), etc.
✔️ Multi-Part data for all POST functions.
✔️ Complete documentation with examples.
✔️ Works on Alpine Linux and Dockers.
✔️ CI, PyPI, Docker, Tests, Fixes, etc.
✔️ 0 Dependency, Python 3.9 ready, native bare-metal C performance.

Multi-Threading Web Scraper

Alt Text

  • threads = True Multi-Threading, threads = False no Threads.

URL-to-SQLite Web Scraper, give it 1 URL, gives back 1 SQLite database

Alt Text

  • SQLite database can be visualized with any SQLite WYSIWYG, like https://sqlitebrowser.org
  • HTTP Headers are stored as Pretty-Printed JSON.
  • Date and Time are stored as Unix Timestamps.

Images and Photos Web Scraper

Alt Text

  • Can get the images from the new HTML5 <picture> element.

Gimme tha powah

https://github.com/juancarlospaco/faster-than-requests#extras-go-beyond-requests

  • Star faster_than_requests on GitHub!, awesome stuff incoming...

👑

Posted on by:

juancarlospaco profile

Juan Carlos

@juancarlospaco

.10x frAgile FullStuck Midend Devlooper, Python, Nim, Arch, OpenSource, EN|ES, Argentina, UTC-3, Atheist, WFH Nim Team Leader

Discussion

markdown guide
 

Sounds amazing. What are the draw backs of that approach?

 

Nim is good! Thank you for this library, but i still wanna ask, will you implement moar in it? because it hasn't still been like awesome requests.