DEV Community

loading...

Beautiful Soup Hello World

vicentemaldonado profile image Vicente Maldonado Originally published at Medium on ・2 min read

Beautiful Soup is a Python library for working with HTML and XML files. You can use it to navigate a HTML document, search it, extract data from it and even change the document structure. Let’s see how it works:

from bs4 import BeautifulSoup

html = '''
    <html>
    <head>
        <title>Beautiful Soup Hello World</title>
    </head>
    <body>
        <h1>Header</h1>
        <p>Paragraph 1</p>
        <p>Paragraph 2</p>
        <p>Paragraph 3</p>
    </body>
    </html>
'''

soup = BeautifulSoup(html, 'html.parser')

print(soup.title)
print(soup.title.name)
print(soup.title.text)

print(soup.p.text)

for paragraph in soup.find\_all('p'):
    print(paragraph.text)

print(soup.get\_text())

It’s a really basic example, but before you can run it you first need to install Beautiful Soup:

pip install beautifulsoup4

While you’re at it, install another library as well:

pip install lxml

It’s a HTML/XML parser. Don’t worry about it.

Let’s start — import Beautiful Soup:

from bs4 import BeautifulSoup

Next, we need some HTML to work with:

html = '''
    <html>
    <head>
        <title>Beautifulsoup Hello World</title>
    </head>
    <body>
        <h1>Header</h1>
        <p>Paragraph 1</p>
        <p>Paragraph 2</p>
        <p>Paragraph 3</p>
    </body>
    </html>
'''

It is a basic HTML document stored in a Python string. Of course, working with HTML stored in a Python script is not very exciting, but this is a Hello, World, so hey.

Create an instance of the BeautifulSoup object, specifying the HTML document and the parser to be used (I said don’t worry about it):

soup = BeautifulSoup(html, 'html.parser')

Now we have our HTML parsed and stored in a variable named soup and we can play with it:

print(soup.title)

Use soup.title to access the HTML document’s

element. This prints:
<title>Beautifulsoup Hello World</title>

Sometimes you don’t want the HTML tag:

print(soup.title.text)

and get just the element text:

Beautifulsoup Hello World

Our document has just one

element so Beautiful Soup appropriately returns it if we use soup.title. But the documents has three

elements (paragraphs) so what happens when we try to pull the same trick?

print(soup.p.text)

It returns the first

element in the document:

Paragraph 1

If you want to get all paragraphs in a documents, well, just use find_all():

for paragraph in soup.find\_all('p'):
    print(paragraph.text)

find_all() returns all paragraphs in the document and you can iterate them using a simple for loop.

This is just scratching the surface with Beautiful Soup. At the end let’s see how simple it is to get all text (and only text) in the document:

print(soup.get\_text())

As expected, this prints

Beautifulsoup Hello World

Header
Paragraph 1
Paragraph 2
Paragraph 3

You can find the full script in my Github. ttfn.

Discussion (0)

pic
Editor guide