DEV Community

Cover image for Scrape DuckDuckGo Ad Results with Python
Dimitry Zub
Dimitry Zub

Posted on • Updated on • Originally published at serpapi.com

Scrape DuckDuckGo Ad Results with Python

Contents: intro, imports, what will be scraped, process, code, links, outro.

Intro

This blog post is a continuation of DuckDuckGo web scraping series. Here you'll see how to scrape Ad Results using Python with selenium library. An alternative API solution will be shown.

Prerequisites: familiarity with selenium library and regular expressions.

Imports

from selenium import webdriver
Enter fullscreen mode Exit fullscreen mode

What will be scraped

image

Process

Selecting title, link, snippet, source, sitelinks.

What I was doing is basically applying .results--ads selector before additional selector for title, link, snippet, source, and sitelinks to get most accurate selector.

A CSS selectors reference.

Code

from selenium import webdriver

driver = webdriver.Chrome(executable_path='path/to/chromedriver.exe')
driver.get('https://duckduckgo.com/?q=minecraft&kl=us-en&ia=web')

for result in driver.find_elements_by_css_selector('.results--ads .result__body.links_main.links_deep'):
    title = result.find_element_by_css_selector('.js-result-title-link').text
    link = result.find_element_by_css_selector('.js-result-title-link').get_attribute('href')
    source = result.find_element_by_css_selector('.js-result-extras-url').text
    snippet = result.find_element_by_css_selector('.js-result-snippet > a').text
    print(f'{title}\n{source}\n{snippet}\n{link}\n')

for sitelink in driver.find_elements_by_css_selector('.js-sitelinks-title'):
    sitelink_title = sitelink.text
    sitelink_url = sitelink.get_attribute('href')
    print(f'{sitelink_title}\n{sitelink_url}\n')

driver.quit()

---------------------
'''
Minecraft Video Games - Save On Minecraft Video Games
https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=g4RZhjUMFYuy1q3_sLWq3g%3D%3D&rut=8a6411c3b42b508666bdc647000f144258f958962f56d893c5620653f3ec2ebd&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8tlRvYW3t4dG5g%2DpvABGA4DVUCUyowbncAXntvrmjGeojwWrZFDK5oXEtgdBhIetjLMVkFSjF0OcmSEtTDNDjOilIW6B52S2hqo8v3CNZwdowb3t5LSt8wF5kfdECLDNLzZ7Js%2DlU0dmHtbUHnpUDy6SbSukR8g%2DVGUPe9w69ESsNB3%2DQnM9LOi5pAGiyHJ5pnYlulA%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjYlMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZzZWFyY2glMmYlM2ZxdWVyeSUzZG1pbmVjcmFmdCUyYnRoZSUyYmdhbWUlMjZhZGlkJTNkMjIyMjIyMjIyMjQ0MjU1OTM4MDIlMjZ3bWxzcGFydG5lciUzZHdtdGxhYnMlMjZ3bDAlM2RlJTI2d2wxJTNkcyUyNndsMiUzZGMlMjZ3bDMlM2Q3NTE3OTIwMzk4NDc3OSUyNndsNCUzZGt3ZC03NTE3OTI5MjkyMTM5OCUzYWxvYy0yMzUlMjZ3bDUlM2QxNDExNzElMjZ3bDYlM2QlMjZ3bDclM2QlMjZ3bDE0JTNkbWluZWNyYWZ0JTI2dmVoJTNkc2VtJTI2Z2NsaWQlM2Q1ODFiZmJjMGM2NGIxZDc1ZTk4YjliMzBlNWU3MmYxNiUyNmdjbHNyYyUzZDNwLmRzJTI2JTI2bXNjbGtpZCUzZDU4MWJmYmMwYzY0YjFkNzVlOThiOWIzMGU1ZTcyZjE2%26rlid%3D581bfbc0c64b1d75e98b9b30e5e72f16&vqd=3-61454750645793260001748454990905959497-291233585751825993113979507856558647155&iurl=%7B1%7DIG%3D67B7A1B78A264F3A851C7D08BD88545F%26CID%3D3BBC77207A5960C0343D67557B8661B7%26ID%3DDevEx%2C5642.1
Free 2-day Shipping On Millions of Items. No Membership Fee. Shop Now!
This collection includes: The Minecraft base game, 1000 Minecoins and the Starter ...
Curbside Pickup · Shop Everyday Low Prices
PC Gaming‧Virtual Reality‧Video Game Accessories‧Video Game Consoles
walmart.com

Save 5% w/ RedCard at Target™ - Mindcraft Game at Target™
https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=g4RZhjUMFYuy1q3_sLWq3g%3D%3D&rut=882b95c16f4fc14eddef3dd23b4753f8eb11a9051a45d274d9a412b416bcdfc8&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8G2FCJ0SWLPN18IxZVTwm5zVUCUz2jK7DkoJHozC22Ot_lZykLOLwJ_t0YFXATV0BgHJTj2gyEal6nI1B61u7ILh%2DjB8Rs9qklWqoOhqZcqIbL0qEFCL2itcztPQNH5PBoNrsojS_oRwHEPHy%2D77oS7JBFxJrssZHaQL3KwBdy_uIl3odAlaOJGj521JDMr6dhnO0aQ%26u%3DaHR0cHMlM2ElMmYlMmZjbGlja3NlcnZlLmRhcnRzZWFyY2gubmV0JTJmbGluayUyZmNsaWNrJTNmbGlkJTNkNDM3MDAwNTgyOTA5MjE0MTMlMjZkc19zX2t3Z2lkJTNkNTg3MDAwMDA4MDAyMzE1MjAlMjZkc19hX2NpZCUzZDQwNTUyNDU5NCUyNmRzX2FfY2FpZCUzZDkyMjQzMzE1OTIlMjZkc19hX2FnaWQlM2QxMDAzMDAyMTM2NjQlMjZkc19hX2xpZCUzZGt3ZC0yMDgzMTQ3MDEwMiUyNiUyNmRzX2VfYWRpZCUzZDgxMDIwMzM0NTUyMTgwJTI2ZHNfZV90YXJnZXRfaWQlM2Rrd2QtODEwMjA0Mjk3NjU2NDAlM2Fsb2MtMjM1JTI2JTI2ZHNfdXJsX3YlM2QyJTI2ZHNfZGVzdF91cmwlM2RodHRwcyUzYSUyZiUyZnd3dy50YXJnZXQuY29tJTJmcyUyZm1pbmRjcmFmdCUyYmdhbWUlM2ZyZWYlM2R0Z3RfYWR2X1hTMDAwMDAwJTI2QUZJRCUzZG1zbiUyNmZuZHNyYyUzZHRndGFvJTI2REZBJTNkNzE3MDAwMDAwMTI1MTk1MjIlMjZDUE5HJTNkVmlkZW8lMjUyQkdhbWVzX0NvbnNvbGUlMmJHYW1lcyUyNTdDVmlkZW8lMjUyQkdhbWVzX0Vjb21tX0hhcmRsaW5lcyUyNmFkZ3JvdXAlM2RDb25zb2xlJTJiVmlkZW8lMmJHYW1lXzIlMjZMSUQlM2Q3MDAwMDAwMDExNzYzMTAlMjZMTk0lM2RtaW5kY3JhZnQlMmJnYW1lJTI2TVQlM2RlJTI2bmV0d29yayUzZHMlMjZkZXZpY2UlM2RjJTI2bG9jYXRpb24lM2QxNDExNzElMjZ0YXJnZXRpZCUzZGt3ZC04MTAyMDQyOTc2NTY0MCUzYWxvYy0yMzUlMjZnY2xpZCUzZDliNzQ1YWE2MWYwZjEzYjEyOWQzM2E1NjNjYTNlZWM3JTI2Z2Nsc3JjJTNkM3AuZHMlMjZkc19ybCUzZDEyNDY5NzglMjZkc19ybCUzZDEyNDgwOTklMjZtc2Nsa2lkJTNkOWI3NDVhYTYxZjBmMTNiMTI5ZDMzYTU2M2NhM2VlYzc%26rlid%3D9b745aa61f0f13b129d33a563ca3eec7&vqd=3-205393906844187692533645762826856793138-291233585751825993113979507856558647155&iurl=%7B1%7DIG%3D67B7A1B78A264F3A851C7D08BD88545F%26CID%3D3BBC77207A5960C0343D67557B8661B7%26ID%3DDevEx%2C5661.1
Get the Latest Mindcraft Game. Lots of Mindcraft Game to Choose From. Fast & Free 2-Day Shipping on Orders $35+.
target.com

PC Gaming
https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=g4RZhjUMFYuy1q3_sLWq3g%3D%3D&rut=f9cd2547f922699dc0a2cfca5575b64a2d4a833cd2cac10c903d7bd44293c1d9&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8zWLFQwqJ1QTiTZTbYLfjXjVUCUy4NTNgKV3RmTlAZdQ%2DUZQnw4wf2ggXuBOMiKnU1TNZ4K5b1tgu_07X6M11e09DY71s2AMuS%2DOs94Pvgkxf19vVVStcYiNdkEdGDvSTMT6LX7zU4mzv9duuDrD6_QElBkpycUIcO_iWQH2FprtmtHTbAK793EnLQG5KaYQY0EzIfQ%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjZkc194X2FkeGlkJTNkODM3MDAwMDk1MjM4NjA0OTUlMjZkc194X2FkeHR5cGUlM2QxJTI2ZHNfYV9jaWQlM2Q2NjI1ODcxNDglMjZkc19hX2ZpaWQlM2QxNjQxMDQ3Njk0ODElMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZicm93c2UlMmZ2aWRlby1nYW1lcyUyZnBjLWdhbWluZyUyZjI2MzZfMTA3NzUwNCUzZmFkaWQlM2QyMjIyMjIyMjIyNDQyNTU5MzgwMiUyNndtbHNwYXJ0bmVyJTNkd210bGFicyUyNndsMCUzZGUlMjZ3bDElM2RzJTI2d2wyJTNkYyUyNndsMyUzZDc1MTc5MjAzOTg0Nzc5JTI2d2w0JTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNndsNSUzZDE0MTE3MSUyNndsNiUzZDczNTMwMTA1ODk2NDQlMjZ3bDclM2QlMjZ3bDE0JTNkbWluZWNyYWZ0JTI2dmVoJTNkc2VtJTI2Z2NsaWQlM2RkMjg1N2M5NThlMjgxOTNlNGQ3NTFjNjVhZDg0NTY0YyUyNmdjbHNyYyUzZDNwLmRzJTI2JTI2bXNjbGtpZCUzZGQyODU3Yzk1OGUyODE5M2U0ZDc1MWM2NWFkODQ1NjRj%26rlid%3Dd2857c958e28193e4d751c65ad84564c&vqd=3-1966876518034573175047393153744481480-291233585751825993113979507856558647155&iurl=%7B1%7DIG%3D67B7A1B78A264F3A851C7D08BD88545F%26CID%3D3BBC77207A5960C0343D67557B8661B7%26ID%3DDevEx%2C5647.1

Virtual Reality
https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=g4RZhjUMFYuy1q3_sLWq3g%3D%3D&rut=16e5db074dc2fc09f50561d0c1bc52559a473774d1a0c0e583cd4a27350dbdab&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8LyhkfwFWgmN0YU2ii3oCXjVUCUwHHiKzeQKJO8IZ2bAozw2nr3rdwptE%2D6O6e9OSSn7AyzP5XLHmKfOCV9Q1smXJOvnBQWvGnqa96sYZZcUgON4k6Qz4u8n0ZpWd2hvhUXPeqWC_Dfm1r4UFN3FIZN715EUuKTCVy1zV1fJanIeJN40x_M3aZP41N2CWtybi1z3nKA%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjZkc194X2FkeGlkJTNkODM3MDAwMDk1MjM4NjA0OTglMjZkc194X2FkeHR5cGUlM2QxJTI2ZHNfYV9jaWQlM2Q2NjI1ODcxNDglMjZkc19hX2ZpaWQlM2QxNjQxMDQ3Njk0ODQlMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZjcCUyZnZpcnR1YWwtcmVhbGl0eSUyZjU0NTQzMDElM2ZhZGlkJTNkMjIyMjIyMjIyMjQ0MjU1OTM4MDIlMjZ3bWxzcGFydG5lciUzZHdtdGxhYnMlMjZ3bDAlM2RlJTI2d2wxJTNkcyUyNndsMiUzZGMlMjZ3bDMlM2Q3NTE3OTIwMzk4NDc3OSUyNndsNCUzZGt3ZC03NTE3OTI5MjkyMTM5OCUzYWxvYy0yMzUlMjZ3bDUlM2QxNDExNzElMjZ3bDYlM2Q3MzUzMDEwNTg5NjQ1JTI2d2w3JTNkJTI2d2wxNCUzZG1pbmVjcmFmdCUyNnZlaCUzZHNlbSUyNmdjbGlkJTNkZmMyZGM3MGMwOGE4MTgzZTZkZTJhN2ZjMmIwNGE5MGMlMjZnY2xzcmMlM2QzcC5kcyUyNiUyNm1zY2xraWQlM2RmYzJkYzcwYzA4YTgxODNlNmRlMmE3ZmMyYjA0YTkwYw%26rlid%3Dfc2dc70c08a8183e6de2a7fc2b04a90c&vqd=3-137089535419097050706382597529438450029-291233585751825993113979507856558647155&iurl=%7B1%7DIG%3D67B7A1B78A264F3A851C7D08BD88545F%26CID%3D3BBC77207A5960C0343D67557B8661B7%26ID%3DDevEx%2C5649.1

Video Game Accessories
https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=g4RZhjUMFYuy1q3_sLWq3g%3D%3D&rut=ca63a9771de0e54103a90a811dd0ce07bd70f1c159fdb5cd137454cf58e2ebda&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8WDkz53ENl9Vn9gNtzmnmTjVUCUyqohFt10XkMWzIvjF7WnpaV07v7WA9aeyNFwOIRFn0AJgTgtv7sTAnurbHPtG0mcK5CGOXffNh5EACy3_UBRk0FC7JMCbz1NQZAJzCJidRLfKLfjbO2U8WN1kz8bcrAKm6epDBlUobUPgKP_hZoSpkv5XV4mkXPXLGrQ6Px1_%2DyA%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjZkc194X2FkeGlkJTNkODM3MDAwMDk1MjM4NjA0OTIlMjZkc194X2FkeHR5cGUlM2QxJTI2ZHNfYV9jaWQlM2Q2NjI1ODcxNDglMjZkc19hX2ZpaWQlM2QxNjQxMDQ3Njk0NzglMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZjcCUyZnZpZGVvLWdhbWUtYWNjZXNzb3JpZXMlMmYxMjI5MDE5JTNmYWRpZCUzZDIyMjIyMjIyMjI0NDI1NTkzODAyJTI2d21sc3BhcnRuZXIlM2R3bXRsYWJzJTI2d2wwJTNkZSUyNndsMSUzZHMlMjZ3bDIlM2RjJTI2d2wzJTNkNzUxNzkyMDM5ODQ3NzklMjZ3bDQlM2Rrd2QtNzUxNzkyOTI5MjEzOTglM2Fsb2MtMjM1JTI2d2w1JTNkMTQxMTcxJTI2d2w2JTNkNzM1MzAxMDU4OTY0MyUyNndsNyUzZCUyNndsMTQlM2RtaW5lY3JhZnQlMjZ2ZWglM2RzZW0lMjZnY2xpZCUzZDMzOTMyZjNmZmJlNTE2ZTRiMTg4NGZjZmQ4NDlhOTZkJTI2Z2Nsc3JjJTNkM3AuZHMlMjYlMjZtc2Nsa2lkJTNkMzM5MzJmM2ZmYmU1MTZlNGIxODg0ZmNmZDg0OWE5NmQ%26rlid%3D33932f3ffbe516e4b1884fcfd849a96d&vqd=3-129360418447431572000570497592988537963-291233585751825993113979507856558647155&iurl=%7B1%7DIG%3D67B7A1B78A264F3A851C7D08BD88545F%26CID%3D3BBC77207A5960C0343D67557B8661B7%26ID%3DDevEx%2C5651.1

Video Game Consoles
https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=g4RZhjUMFYuy1q3_sLWq3g%3D%3D&rut=ea9f551cf1965b8afd7f6ae3ce1ce7cc41ae5adac906667d499c042fb5c61c31&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De82MvYsV6CrsVtx4guYk_cSjVUCUxe_GSgLMhNbftx6vplN7hSUg%2DWEt3bwmwQXmRqKGccgyd4S6deyAa4qVcibAy%2DJ6hANa1TQJhruJEuEQdWs0dyggjvY29KW0yHLL6uzEk81JDc3iX%2Dixoeq_yQkUUe9ThBiDA9zCk%2DSpa%2D7_dhP7gTWzZtOxQo_KSuw_zVnndZ1w%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjZkc194X2FkeGlkJTNkODM3MDAwMDk1MjM4NjA0ODklMjZkc194X2FkeHR5cGUlM2QxJTI2ZHNfYV9jaWQlM2Q2NjI1ODcxNDglMjZkc19hX2ZpaWQlM2QxNjQxMDQ3Njk0NzUlMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZjcCUyZnZpZGVvLWdhbWVzJTJmMjYzNiUzZmFkaWQlM2QyMjIyMjIyMjIyNDQyNTU5MzgwMiUyNndtbHNwYXJ0bmVyJTNkd210bGFicyUyNndsMCUzZGUlMjZ3bDElM2RzJTI2d2wyJTNkYyUyNndsMyUzZDc1MTc5MjAzOTg0Nzc5JTI2d2w0JTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTIzNSUyNndsNSUzZDE0MTE3MSUyNndsNiUzZDczNTMwMTA1ODk2NDIlMjZ3bDclM2QlMjZ3bDE0JTNkbWluZWNyYWZ0JTI2dmVoJTNkc2VtJTI2Z2NsaWQlM2Q5NTE5MmJhOGVlMzAxYjVkMjFjMTRiNThjZDljMjZhMiUyNmdjbHNyYyUzZDNwLmRzJTI2JTI2bXNjbGtpZCUzZDk1MTkyYmE4ZWUzMDFiNWQyMWMxNGI1OGNkOWMyNmEy%26rlid%3D95192ba8ee301b5d21c14b58cd9c26a2&vqd=3-317251499328598894584215891959984246336-291233585751825993113979507856558647155&iurl=%7B1%7DIG%3D67B7A1B78A264F3A851C7D08BD88545F%26CID%3D3BBC77207A5960C0343D67557B8661B7%26ID%3DDevEx%2C5653.1
'''
Enter fullscreen mode Exit fullscreen mode

Using DuckDuckGo Ad Results API

SerpApi is a paid API with a free plan.

The difference is that is much faster, you don't need to render the page with selenium and you get an immediate structured JSON string.

import json
from serpapi import GoogleSearch

params = {
  "api_key": "YOUR_API_KEY",
  "engine": "duckduckgo",
  "q": "minecraft",
  "kl": "us-en"
}

search = GoogleSearch(params)
results = search.get_dict()

print(json.dumps(results['ads'], indent=2, ensure_ascii=False))

-------------------------
'''
[
  {
    "position": 1,
    "title": "Minecraft Video Games - Save On Minecraft Video Games",
    "link": "https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=0QdhC1y9z11BeQmahIracw%3D%3D&rut=a216fdd3b3188c92c2685df6a9583e121c134f2e105aeb38b383790e88600a0c&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8fgTI6SxasnT8IzGsTX1JCzVUCUxMOeN8XH_JyCXYHwFn9yeuq0eYsP2PfLKnYp3LD3CS5kmbcD1H8u51k8PPBW9DT_DUpkBQ5cUHNA9L232R4PfCEH26_I9uM_0tQpgEgrRdyXP9QUO_GQ9gGi570b_S06VtYpB6imjvtR77kU_r7jJankDSwMkwV195m8N_6IkGSw%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjYlMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTE5MCUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZzZWFyY2glMmYlM2ZxdWVyeSUzZG1pbmVjcmFmdCUyYnRoZSUyYmdhbWUlMjZhZGlkJTNkMjIyMjIyMjIyMjQ0MjU1OTM4MDIlMjZ3bWxzcGFydG5lciUzZHdtdGxhYnMlMjZ3bDAlM2RlJTI2d2wxJTNkcyUyNndsMiUzZGMlMjZ3bDMlM2Q3NTE3OTIwMzk4NDc3OSUyNndsNCUzZGt3ZC03NTE3OTI5MjkyMTM5OCUzYWxvYy0xOTAlMjZ3bDUlM2QxOTAlMjZ3bDYlM2QlMjZ3bDclM2QlMjZ3bDE0JTNkbWluZWNyYWZ0JTI2dmVoJTNkc2VtJTI2Z2NsaWQlM2RlNTY1NGI2MGMxMmYxZmZmNjQ1NzJiOTBlMmY4NWFiYSUyNmdjbHNyYyUzZDNwLmRzJTI2JTI2bXNjbGtpZCUzZGU1NjU0YjYwYzEyZjFmZmY2NDU3MmI5MGUyZjg1YWJh%26rlid%3De5654b60c12f1fff64572b90e2f85aba&vqd=3-87030390511408672693321996432917478138-182266568508120047137554084072326458307&iurl=%7B1%7DIG%3DB0E51F11F26E4D7F834E827F1130C03E%26CID%3D3AE9D303FD2763300696C376FCDA62DF%26ID%3DDevEx%2C5744.1",
    "source": "walmart.com",
    "snippet": "Free 2-day Shipping On Millions of Items. No Membership Fee. Shop Now!",
    "sitelinks": [
      {
        "title": "PC Gaming",
        "link": "https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=0QdhC1y9z11BeQmahIracw%3D%3D&rut=04e2b38a60eeb50ba10dccecfa7ad8cd86bc5b14745cc956182b9046e50cc0f3&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8jvj5mDwm7A8EWjuXtddsyjVUCUwxgrLM2RsbdFtuow%2DGguQDaQGvPeWxXitMufZG_oupGdkH%2D0JqG5tu_8LpfupXuIJnjgUWfg0EZOAsTSyFJtxOz5YJ5%2DsUoKDxr8ArveNQGoEI7BdU0IgoGXZjCiy_WVf0L0osJFCNQIx8BH1SkgDVm_kaP4x0nhEigFiZC38LEA%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjZkc194X2FkeGlkJTNkODM3MDAwMDk1MjM4NjA0OTUlMjZkc194X2FkeHR5cGUlM2QxJTI2ZHNfYV9jaWQlM2Q2NjI1ODcxNDglMjZkc19hX2ZpaWQlM2QxNjQxMDQ3Njk0ODElMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTE5MCUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZicm93c2UlMmZ2aWRlby1nYW1lcyUyZnBjLWdhbWluZyUyZjI2MzZfMTA3NzUwNCUzZmFkaWQlM2QyMjIyMjIyMjIyNDQyNTU5MzgwMiUyNndtbHNwYXJ0bmVyJTNkd210bGFicyUyNndsMCUzZGUlMjZ3bDElM2RzJTI2d2wyJTNkYyUyNndsMyUzZDc1MTc5MjAzOTg0Nzc5JTI2d2w0JTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTE5MCUyNndsNSUzZDE5MCUyNndsNiUzZDczNTMwMTA1ODk2NDQlMjZ3bDclM2QlMjZ3bDE0JTNkbWluZWNyYWZ0JTI2dmVoJTNkc2VtJTI2Z2NsaWQlM2RkMTU0ZTc5MjE2NDgxYWE2NGZkMDg3NjIzNTlmNWFjZSUyNmdjbHNyYyUzZDNwLmRzJTI2JTI2bXNjbGtpZCUzZGQxNTRlNzkyMTY0ODFhYTY0ZmQwODc2MjM1OWY1YWNl%26rlid%3Dd154e79216481aa64fd08762359f5ace&vqd=3-220109179843301254675732288972445008198-182266568508120047137554084072326458307&iurl=%7B1%7DIG%3DB0E51F11F26E4D7F834E827F1130C03E%26CID%3D3AE9D303FD2763300696C376FCDA62DF%26ID%3DDevEx%2C5749.1"
      },
      {
        "title": "Virtual Reality",
        "link": "https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=0QdhC1y9z11BeQmahIracw%3D%3D&rut=233ab76f169eac87c1482731a761f5b40bdf142a70d7804182cb7936070ede82&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De8HEspBp8JBHVa1ItwUzszTDVUCUy39V5wJqekOq3UHWELZ6v8PE7Nw0GmZUt8B_nUo_mt3lcfo8dqm8tE_nHmCtI_YUS5NSkp4im%2DqgRXe964XNGRIZ8yhuY3Jl5XcyjJtfSHS1crmNv4H6yxHYRd0pS7yV5PMaaeRQ7jI_v8icUTDuEjixnIyOaf3MmtY8AQyw30jw%26u%3DaHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA2MTI4NjIwMTM3OSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwNjc2NjE3NDYwNSUyNmRzX2FfY2lkJTNkNjYyNTg3MTQ4JTI2ZHNfYV9jYWlkJTNkMTI0MTgyNjIzMjQlMjZkc19hX2FnaWQlM2QxMjM1MjMxMDcyNTIlMjZkc19hX2xpZCUzZGt3ZC0zMDQ0Mzg5ODczNzglMjZkc194X2FkeGlkJTNkODM3MDAwMDk1MjM4NjA0OTglMjZkc194X2FkeHR5cGUlM2QxJTI2ZHNfYV9jaWQlM2Q2NjI1ODcxNDglMjZkc19hX2ZpaWQlM2QxNjQxMDQ3Njk0ODQlMjZkc19lX2FkaWQlM2Q3NTE3OTIwMzk4NDc3OSUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTc1MTc5MjkyOTIxMzk4JTNhbG9jLTE5MCUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cud2FsbWFydC5jb20lMmZjcCUyZnZpcnR1YWwtcmVhbGl0eSUyZjU0NTQzMDElM2ZhZGlkJTNkMjIyMjIyMjIyMjQ0MjU1OTM4MDIlMjZ3bWxzcGFydG5lciUzZHdtdGxhYnMlMjZ3bDAlM2RlJTI2d2wxJTNkcyUyNndsMiUzZGMlMjZ3bDMlM2Q3NTE3OTIwMzk4NDc3OSUyNndsNCUzZGt3ZC03NTE3OTI5MjkyMTM5OCUzYWxvYy0xOTAlMjZ3bDUlM2QxOTAlMjZ3bDYlM2Q3MzUzMDEwNTg5NjQ1JTI2d2w3JTNkJTI2d2wxNCUzZG1pbmVjcmFmdCUyNnZlaCUzZHNlbSUyNmdjbGlkJTNkYWM2MjU2MDFkNGU0MTY5ZTE1ZDMzNmYxZjczMzIyMjclMjZnY2xzcmMlM2QzcC5kcyUyNiUyNm1zY2xraWQlM2RhYzYyNTYwMWQ0ZTQxNjllMTVkMzM2ZjFmNzMzMjIyNw%26rlid%3Dac625601d4e4169e15d336f1f7332227&vqd=3-134614315498574247865815717038458011354-182266568508120047137554084072326458307&iurl=%7B1%7DIG%3DB0E51F11F26E4D7F834E827F1130C03E%26CID%3D3AE9D303FD2763300696C376FCDA62DF%26ID%3DDevEx%2C5751.1"
      }
...
 {
    "position": 2,
    "title": "Discover Savings Today - Get Free Shipping w/ RedCard",
    "link": "https://duckduckgo.com/y.js?ad_provider=bingv7aa&eddgt=0QdhC1y9z11BeQmahIracw%3D%3D&rut=04ff8dd1af518abff4214d1e44198e92cf70219ab9cc762bcbfc2f047af1c083&u3=https%3A%2F%2Fwww.bing.com%2Faclick%3Fld%3De84Ph8X4w260Gl92Z%2DIMdNlDVUCUxi6IcTvbnqI%2DRZRBILc0WlQAVwo6NvwHp54gu9smxxtlnCx_pc%2DHIb12pJzkYMFXQaTFi1wetFpfKNmUv68wi61B%2DOBrJ6Ja%2DqR86VrPNq6WCDA28OomiMpwZJy%2DI4oUuNsqwbD_OFazgGMvNjERIw4YRFJu%2De7nxmtFFQk0IIKg%26u%3DaHR0cHMlM2ElMmYlMmZjbGlja3NlcnZlLmRhcnRzZWFyY2gubmV0JTJmbGluayUyZmNsaWNrJTNmbGlkJTNkNDM3MDAwNTgyOTkyNzkyNjUlMjZkc19zX2t3Z2lkJTNkNTg3MDAwMDY0NTIwMTEwNzElMjZkc19hX2NpZCUzZDQwNTUwNzc5OSUyNmRzX2FfY2FpZCUzZDkyMjQ0MjI0NjQlMjZkc19hX2FnaWQlM2QxMTMxMTExNzMxNDAlMjZkc19hX2xpZCUzZGt3ZC0zMDAzOTg2MzAzNzIlMjYlMjZkc19lX2FkaWQlM2Q4MTg0NDk3MDA2MjIxMCUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTgxODQ1MDYyNjUyMDI2JTNhbG9jLTE5MCUyNiUyNmRzX3VybF92JTNkMiUyNmRzX2Rlc3RfdXJsJTNkaHR0cHMlM2ElMmYlMmZ3d3cudGFyZ2V0LmNvbSUyZnMlMmZtaW5lY3JhZnQlMmJtaW5lY3JhZnQlMmJtaW5lY3JhZnQlM2ZyZWYlM2R0Z3RfYWR2X1hTMDAwMDAwJTI2QUZJRCUzZG1zbiUyNmZuZHNyYyUzZHRndGFvJTI2REZBJTNkNzE3MDAwMDAwMTI1MzkwMzYlMjZDUE5HJTNkRWxlY3Ryb25pY3NfVmlkZW8lMmJHYW1lJTJiQ29uc29sZXMlMjU3Q0VsZWN0cm9uaWNzX0Vjb21tX0hhcmRsaW5lcyUyNmFkZ3JvdXAlM2RWaWRlbyUyYkdhbWUlMmJDb25zb2xlc18yJTI2TElEJTNkNzAwMDAwMDAxMTc2MjQ2JTI2TE5NJTNkbWluZWNyYWZ0JTJibWluZWNyYWZ0JTJibWluZWNyYWZ0JTI2TVQlM2RlJTI2bmV0d29yayUzZHMlMjZkZXZpY2UlM2RjJTI2bG9jYXRpb24lM2QxOTAlMjZ0YXJnZXRpZCUzZGt3ZC04MTg0NTA2MjY1MjAyNiUzYWxvYy0xOTAlMjZnY2xpZCUzZDM1MjExNTNiYmQxNDEyNDNhYWZhNzVjYjllYzNlY2NkJTI2Z2Nsc3JjJTNkM3AuZHMlMjZkc19ybCUzZDEyNDY5NzglMjZkc19ybCUzZDEyNDgwOTklMjZtc2Nsa2lkJTNkMzUyMTE1M2JiZDE0MTI0M2FhZmE3NWNiOWVjM2VjY2Q%26rlid%3D3521153bbd141243aafa75cb9ec3eccd&vqd=3-252781822407565903154811433641671895283-182266568508120047137554084072326458307&iurl=%7B1%7DIG%3DB0E51F11F26E4D7F834E827F1130C03E%26CID%3D3AE9D303FD2763300696C376FCDA62DF%26ID%3DDevEx%2C5763.1",
    "source": "target.com",
    "snippet": "Shop Minecraft Minecraft Minecraft & More. Get Minecraft Minecraft Minecraft at Target™ Today. Save on Minecraft Minecraft Minecraft. Quick & Easy Returns In-Store."
  }
]
'''
Enter fullscreen mode Exit fullscreen mode

Links

GitHub GistDuckDuckGo Ad Results API

Outro

If you have any questions or something isn't working correctly or you want to write something else, feel free to drop a comment in the comment section or via Twitter at @serp_api.

Yours,
Dimitry, and the rest of SerpApi Team.

Discussion (0)