DEV Community

loading...

Day 71 Of 100DaysOfCode: Web Scraping

iamdurga profile image Durga Pokharel ・2 min read

This is my 71th day of #100daysofcode and #python learning. Like daily today also learned from Datacamp regarding to the topic web scraping. Today I learned about simple xpath, slasher double feature, (At)tribute, content with contains, extracting data from a selector list, inspecting the HTML.

Here is python code to extract data from selector list.

Python code

# Import a scrapy Selector
from scrapy import Selector

# Import requests
import requests

# Create the string html containing the HTML source
html = requests.get( url ).content

# Create the Selector object sel from html
sel = Selector( text = html )

# Print out the number of elements in the HTML document
print( "There are 1020 elements in the HTML document.")
print( "You have found: ", len( sel.xpath('//*') ) )
Enter fullscreen mode Exit fullscreen mode

Discussion (2)

pic
Editor guide
Collapse
otumianempire profile image
Otu Michael • Edited

71 one days.. awesome.. in a funny way you could become a full stack web developer with data science knowledge and experience too. You know html, css, javascript, SQL, numpy, scrapy... That's awesome.. I am cheering for you..

Collapse
iamdurga profile image
Durga Pokharel Author

Thank you for your support.