DEV Community

loading...

Discussion on: What is the web scraping and how it works?

Collapse
adamdsherman profile image
AdamDSherman • Edited

So ultimately you're using CSS classes and IDs to pull the data from an HTML element and save it?

What happens with React generated elements that don't have consistant CSS classes?

Collapse
florianrappl profile image
Florian Rappl

You can find any element via some selector. The only difference is the robustness of the solution. There is, however, no fully robust solution as everything (the DOM hierachy, the CSS classes, and the used IDs) may be changed from the site owner.

Just open your dev tools, click in the elements tab on some DOM node and select "copy selector".

Collapse
stalha97 profile image
stalha97

I have been seeing "Xpath" everywhere. Can it not be used as an absolute path? I am interested in web scraping.

P.S - Great article

Thread Thread
ahmedatefae profile image
Ahmed Atef Author • Edited

You can use css as selector for the target html elements but i advice you to use xml in web scraping it have alot of advantages