Today, I am thrilled to present you with a cost-free alternative to ChatGPT, which enables seamless document interaction akin to ChatGPT. Moreover, this solution ensures your privacy and operates offline, eliminating any concerns about data breaches.
PrivateGPT supports source documents in the following formats (.csv, .docx, .doc, .epub, .ppt, and .txt).
Depending on your Desktop, or laptop, PrivateGPT won't be as fast as ChatGPT, but it's free, offline secure, and I would encourage you to try it out.
These are the system requirements to hopefully save you some time and frustration later.
System Requirements
https://github.com/imartinez/privateGPT
Python Version
To use this software, you must have Python 3.10 or later installed. Earlier versions of Python will not compile.
C++ Compiler
If you encounter an error while building a wheel during the pip install process, you may need to install a C++ compiler on your computer.
For Windows 10/11
To install a C++ compiler on Windows 10/11, follow these steps:
Install Visual Studio 2022.
Make sure the following components are selected:
Universal Windows Platform development
C++ CMake tools for Windows
Download the MinGW installer from the MinGW website.
Run the installer and select the gcc component.
Where do I get PrivateGPT?
Go to this GitHub repo, click on the green button that says “Code” and copy the link inside.
If you are familiar with terminals, open up a new terminal and clone the GitHub repository below.
git clone https://github.com/imartinez/privateGPT.git
Go to this new folder
cd privateGPT/
Inside the privateGPT folder, there is a requirements.txt file. This file contains all the dependencies we have to install for privateGPT.
pip install -r requirements.txt
Once the installation is done, you need to rename the fileexample.env to .env. Since I am using Visual Studio Code, I just renamed the file there.
You may need to update the contents of the .env file, it should look like the one
PERSIST_DIRECTORY=db
MODEL_TYPE=GPT4All
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000%
Next, we need to download the LLM. What is an LLM, put simply, it's a concept in AI that expands the data we are using for training and inference. Just head back to the repo page https://github.com/imartinez/privateGPT, scroll down to the environment setup section, and you will find the LLM model to download.
After the LLM file(s) are downloaded, we need to create a folder named “models” inside the privateGPT folder and then copy the LLM we just downloaded inside the “models” folder.
Note: because of the way langchain loads the SentenceTransformers embeddings, the first time you run the script it will require an internet connection to download the embeddings model itself.
Now the fun part, make sure to copy the files you want to ingest and analyze and remember they must be in the following formats (.csv .docx .doc .enex .eml .epub .html .md .msg .odt .pdf .pptx .ppt .txt).
Now head back to your terminal, I have also just used the terminal in Visual Studio Code, we need to run the following command to ingest all the data.
python ingest.py
/opt/homebrew/bin/python3 /Users/mike/Documents/projects/privateGPT-main/ingest.py
The output will look something like this. I had already run the ingest command prior and had no new documents to load.
Appending to existing vectorstore at db
Using embedded DuckDB with persistence: data will be stored in: db
Loading documents from source_documents
Loading new documents: 0it [00:00, ?it/s]
No new documents to load
It will create a db folder containing the local vectorstore. You can ingest as many documents as you want, and all will be accumulated in the local embedding database. If you want to start from an empty database, just delete the db folder.
And now the moment you have been waiting for, Asking questions about your local documents by using the command below.
python privateGPT.py
After a brief time, you will be prompted to enter a query.
So how does this all work?
I think for many people, ChatGPT and other generative AI tools are still a bit of a magic trick. With PrivateGPT, if you head back to the repo https://github.com/imartinez/privateGPT, toward the bottom there is a “How does it work” section which is pretty interesting.
“Selecting the right local models and the power of LangChain you can run the entire pipeline locally, without any data leaving your environment, and with reasonable performance.”
ingest.py uses LangChain tools to parse the document and create embeddings locally using HuggingFaceEmbeddings (SentenceTransformers). It then stores the result in a local vector database using Chroma vector store.
privateGPT.py uses a local LLM based on GPT4All-J or LlamaCpp to understand questions and create answers. The context for the answers is extracted from the local vector store using a similarity search to locate the right piece of context from the docs.
GPT4All-J wrapper was introduced in LangChain 0.0.162.
Top comments (0)