Flutter Web Scraper Level 2: Etsy Product Finder Part 1

Intro

In my previous post (post) I showed how we can use web scraper to get the current world population number and display, however, this is a pretty simple and useless idea. So no I want to show how to use web scraper to make an app that is more like what you would be used to seeing. The application is called Etsy Product Finder and it will allow users to search for a product and view the top 5 Etsy results with their prices and if the user wants to view the product on the Etsy website, a clickthrough will be implemented. Since this will be a long walk through I will be posting 2 parts, the first one will cover installing all the packages and setting up the ui. The second will focus on how to use the data by web scraping the etsy website.

Tutorial Portion

Installing Packages

The first thing we need to do is add the packages that we need to our pubspec yaml. In this case it will be web_scraper and url_launcher, these packages allow us to do the necessary scraping data and also then allowing the user to clickthrough to open the item's page in the browser

Then in main.dart you do the usual and clear everything out leaving only the main function and a stateless widget that has it's home being you homescreen:

Now I will first breakdown what we are going to build in the first part:

• Main Screen with a search bar at the top
• List to contain search result
• List items for the result list

In this part we will be going through only the UI elements of the app

Creating Homescreen

We need to create a homescreen with a search bar at the top and a list underneath it.

So we are first going to have

This create a homescreen with a scaffold within a safe area (widget that avoid notches and notification bar), and it has a column, whose only child is the appbar currently.

_HomeScreen state contains the variables for if the search was has been made and the string for the search

As you will notice in the Homescreen class we create _HomeScreenState as a variable (state) then pass it on to createState(), this is because we are going to want to be able to change the state from the search bar class. which is what we will be doing next.

Creating the Seachbar

For the search bar, we will use a row that contains a texform field and a material button that will trigger the search.

In the code above you can see that we pass an instance of homescreen to the search bar, in turn when the material button is pressed, we change the homescreen's state so that searched is true and, the search text is the text from the textformfeild.

Next we will design the result item and the result list.

Result Item

The result item has to include variables for:

• String for item's image url
• String for item's description
• String for item's clickthrough url
• String for item's prices

For the text we use a null checker to make sure we are not passing null results. The whole thing is wraped in an GestureDetector that calls _lauchUrl(). This in turn ia an async method that calls launch(url) to open the clickthough link.

Now to create a list of these

Result List

For testing puproses we won't webscrape yet, instead for this part we generate 5 test list items just to see what it looks like.

Then in test_screen.dart we can add to the column:

if we run this and click search we will see:

Conclusion

In the next part I will show how we can add the webscraping to scrape search results from etsy

source code