DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» is a community of 967,911 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
songthamtung
songthamtung

Posted on

How to Web Scrape with Python Using 9 Lines of Code

Scraping is extracting data from websites. In this article, I will show you how to scrape links from a test e-commerce site with Python 3.

Prerequisites

If you haven't done so already, install beautifulsoup4 and requests.

pip install beautifulsoup4
pip install requests

Start Scraping!

Here is the full snippet that you can copy and paste directly to your terminal, favorite text editor, or jupyter notebook.

To check if you did it correctly, the output for data should be something similar to:

{'MSI GL62VR 7RFX': '/test-sites/e-commerce/allinone/product/326',Β 
'Dell Vostro 15…': '/test-sites/e-commerce/allinone/product/283',Β 
'Dell Inspiron 17…': '/test-sites/e-commerce/allinone/product/296'}

That's it

Web scraping is great and can save you plenty of time when you want to quickly extract data from websites. The examples above are used for you to quickly get started. Of course there's more to it than what I showed above e.g. (crawling, pagination, viewing the DOM, authentication, cookies, etc). This is only the tip of the iceberg πŸ˜‰.

Thanks for reading! Originally published on The Startup.

Top comments (0)

🌚 Browsing with dark mode makes you a better developer by a factor of exactly 40.

It's a scientific fact.