DEV Community

loading...
Cover image for A python package with selenium to download high-res image using Google Search by Image

A python package with selenium to download high-res image using Google Search by Image

Dilli Babu R
I am working on RPA, love Python, Julia, JavaScript. Happy to contribute to opensource community in anyway I can.
・5 min read

highres

A python package to download high resolution image using Google Search by Image feature

Alt Text

Prerequisites:

Instructions are below

  • Python with pip selenium package
  • A browser with Webdriver

Install selenium via pip

pip install selenium
Enter fullscreen mode Exit fullscreen mode

Webdriver Setup

Click and download the driver from their official websites.

Add webdriver folder path to System path

Add downloaded driver folder path to System path.

Todo

Now, before we start coding, lets break down our goal into smaller components.

Set up Code for:

  • Input & Output folders
  • Identify image files in a folder.
  • Detect browser and launch
  • Navigate and use Google's 'Search by Image'
  • Download image based on it's URL and save in a folder
  • Create a pip package

Code for Input & Output folders

Nomally we can simply use string variable to store paths, but using Path from pathlib is OS neutral and has many useful features. Like for combining paths, we can simply use / instead + "//" + like we do with strings.

so, lets import that one.

import os
from pathlib import Path
# this is come as command line input for completed code.
images_folder_path = Path(r'/home/dr/Pictures/low-res')
highres_folder_name = Path(r'highres')
success_folder_name = Path(r'old')
error_folder_name = Path(r'error')

if not os.path.exists(images_folder_path / highres_folder_name):
    os.mkdir(images_folder_path / highres_folder_name)
if not os.path.exists(images_folder_path / success_folder_name):
    os.mkdir(images_folder_path / success_folder_name)
if not os.path.exists(images_folder_path / error_folder_name):
    os.mkdir(images_folder_path / error_folder_name)

image_extensions = [".jpeg", ".jpg", ".png", ".bmp"]

if not os.path.exists(images_folder_path):
    raise SystemExit("{} is a invalid folder".format(images_folder_path))
Enter fullscreen mode Exit fullscreen mode

Identify image files in a folder.

As I mentioned before Path from pathlib has lot of features,
one of them is getting the file extention easily.
like

Path('/home/dr/Pictures/image.jpeg').suffix # returns ".jpeg"
Enter fullscreen mode Exit fullscreen mode
# lets get all the images in the folder
files = os.listdir(images_folder_path)

# lets filter all the files having this extensions,
# using list comprehension here
image_extensions = [".jpeg", ".jpg", ".png", ".bmp"]

files = [file for file in files if Path(file).suffix.lower() in image_extensions]
if len(files) == 0:
    raise SystemExit("No images found in the input folder")
Enter fullscreen mode Exit fullscreen mode

Detect Browser and define code

Normally, we can launch a webpage simpy with following three lines...

from selenium import webdriver
browser = webdriver.Firefox()
browser.get("https://dev.to")
Enter fullscreen mode Exit fullscreen mode

But we want to support most popular browsers. So, will try to launch with a browser, if we get an error, we will try to check next browser.

browser = None
url = r"https://google.com/images"

if browser is None:
    try:
        browser = webdriver.Chrome()
        browser.get(google_images_url)
    except (Exception) as err:
        print(err)
if browser is None:
    try:
        browser = webdriver.Firefox()
        browser.get(google_images_url)
    except (Exception) as err:
        print(err)
if browser is None:
    try:
        browser = webdriver.Safari()
        browser.get(google_images_url)
    except (Exception) as err:
        print(err)
if browser is None:
    try:
        browser = webdriver.Edge()
        browser.get(google_images_url)
    except (Exception) as err:
        print(err)
if browser is None:
    raise SystemExit(
        "Browser is not available or Webdriver is not set up properly")
Enter fullscreen mode Exit fullscreen mode

Navigation in Google Image Search

Now, for this, we will be waiting for few elements to load completely, before we click on that.
For this reason, we will define a re-usable method, which we will reuse later.

def wait_for_element(driver, byType, byValue, maxWait: int):
    """
    This method, takes properties of an element and looks for the element in page.
    if the element is found, it will return the element else None is returned

    Args:
        driver ([webdriver]): [Selenium webdriver]
        byType ([By Type]): [Which property we are using to get element. ex: By.XPATH or BY.Id]
        byValue ([type]): [value of property]
        maxWait (int): [wait for element to be found]

    Returns:
        [webdriverelement]: [identified element]
    """
    try:
        element = WebDriverWait(driver, maxWait).until(
            EC.presence_of_element_located((byType, byValue)))
        return element
    except TimeoutException:
        print("Element not found")
        return None
Enter fullscreen mode Exit fullscreen mode

Now the actual code for navigation, we are just getting the element and clicking the element, one by one.

dr.get(url)
print("working on file:", file)
file_name = Path(file)
image_icon = wait_for_element(dr,
                              By.XPATH, "/html/body/div[2]/div[2]/div[2]/form/div[2]/div[1]/div[1]/div/div[3]/div[2", 10)
image_icon.click()
upload_image_link = wait_for_element(dr,
                                     By.XPATH, "/html/body/div[2]/div[2]/div[2]/div/div[2]/form/div[1]/div/a", 10)
upload_image_link.click()
browse_image_button = wait_for_element(dr,
                                       By.XPATH, '//*[@id="awyMjb"]', 10)
browse_image_button.send_keys(str(images_folder_path) + "/" + file)
try:
    all_sizes_button = wait_for_element(dr,
                                            By.XPATH, '/html/body/div[7]/div[2]/div[10]/div[1]/div[2]/div/div[2]/div[1]div/div[1]/div[2]/div[2]/span[1]/a', 10)
    all_sizes_button.click()
    first_image = wait_for_element(dr,
                                       By.XPATH, '/html/body/div[2]/c-wiz/div[3]/div[1]/div/div/div/div/div[1]/div[1]/di[1]/a[1]/div[1]/img', 10)
    first_image.click()
    time.sleep(wait_time)
    preview_image_link = wait_for_element(dr,
                                              By.XPATH, '/html/body/div[2]/c-wiz/div[3]/div[2]/div[3]/div/div/div[3]/di[2]/c-wiz/div[1]/div[1]/div/div[2]/a/img', 10)
    src = preview_image_link.get_attribute('src')
Enter fullscreen mode Exit fullscreen mode

Save image file

import urllib.request

urllib.request.urlretrieve(
    src, images_folder_path / highres_folder_name / file_name)
# os.renames is used to move the file, which will automatically create folders if not exits already.
os.renames(images_folder_path / file_name,
           images_folder_path / success_folder_name / file_name)
print("success for file:", file)
Enter fullscreen mode Exit fullscreen mode

creating a pip package

for this we want our script to execute from command line, so we will put above codes in appropriate functions and create a handler method, then put them in a module and create a command-line entry point as follows.

Note: for simplicity, the function definition and argparse code is not included here, completed code at Github is linked at the end.

# command_line.py
from highresmodule import highres

def main():
    highres.highres_handler()
Enter fullscreen mode Exit fullscreen mode

Finaly the most important pip setup code, we will use built-in setuptools to create a pip package, as follows.

# setup.py
import os
import setuptools


def read(fname):
    # for reading the description
    return open(os.path.join(os.path.dirname(__file__), fname)).read()


setuptools.setup(
    name='highres',
    version='1.0.3',
    author="Dilli Babu R",
    author_email="dillir07@outlook.com",
    description='A script to download hi-resolution images using google search by image feature',
    long_description=read('pip-readme.rst'),
    long_description_contect_type="text/markdown",
    url="https://dillir07.github.io/highres/",
    packages=['highresmodule'],
    install_requires=['selenium'], #specifying dependency
    entry_points={
        "console_scripts": ['highres=highresmodule.command_line:main']
    },
    classifiers=[
        "Programming Language :: Python :: 3",
        "Operating System :: OS Independent",
    ],
)
Enter fullscreen mode Exit fullscreen mode

now, finally we can have the package created and installed locally with the following command.

python setup.py develop
Enter fullscreen mode Exit fullscreen mode

Once we are okay, we can upload to pypi registry by following instructions at pip package creation

The final code is available at Github and demo at highres

As usaul comments are welcome, thanks for reading :)

Discussion (0)