It's usually preferable to write a function that downloads and decompresses data from github (or online data) rather than doing it manually.
This is especially true if the data changes frequently: you can write a small script that uses the function to retrieve the most recent data (of you can set up a scheduled job to do that automatically at regular intervals). Automating the data retrieval process is also useful if you need to install the dataset on multiple machines.
Here is the fuction to fetch and load data:
# ----- Libraries --------- from pathlib import Path import pandas as pd import tarfile import urllib.request # -------------------------- # Function to fetch and load data ---> def function_name(): zipped_path = Path("datasets/files.tgz") if not zipped_path.is_file(): Path("datasets").mkdir(parents=True, exist_ok=True) url = "https://github.com/****/Datasets/raw/main/files.tgz" urllib.request.urlretrieve(url, zipped_path) with tarfile.open(zipped_path) as file_name: file_name.extractall(path = "datasets") return pd.read_csv(Path("datasets/files/file.csv")) data_file = function_name()
function_name()is called, it will look for the dataset file in
datasets/files.tgz. If it does not find it, it will create a directory
datasets inside your working directory; then it will download the
files.tgz from the site
files.tgz contains the file
The function with then lod the CSV file into a Pandas DataFrame object containing all the data, and return it.
You can check your data by:
print(data_file[:10]) # Or print(housing.head())
To display the top 10 rows ( or 5 top rows) of your data.
Latest comments (0)