On the eerie night of September 10th, as I delved deep into solving some challenging LeetCode questions, a sudden notification alerted me of an email in my inbox. And I was shocked...
I can't believe my eyes. I pinched my self and read the mail again thrice. The mail state that I was selected for MLH Fellowship 🎉🎉🎉🎉🎊.
For Fellowship my task was to work on Glean.
Glean is a system for working with facts about source code. It is designed for collecting and storing detailed information about code structure, and providing access to the data to power tools and experiences from online IDE features to offline code analysis.
Glean could answer all the questions you'd expect your IDE to answer, accurately and efficiently on a large-scale codebase. Things like:
- Where is the definition of this method?
- Where are all the callers of this function?
- Who inherits from this class?
- What are all the declarations in this file?
Understanding the working of Glean
I am quite new to Glean so, I tried my best to understand it and I am trying my best to present what I actually understand
So let's understand it with a real-life scenario.
Imagine you join your dream company let it be Meta.
So now, on the ferry first day, your manager.
asked you to update a Function ( let it be A) and report me. So now you really don't understand the codebase and it might take a month because it is a meta Code base with a billion line of Code.
You are trying hard 
Now the Glean came into picture. It will help you by answering all these question
- Where is the definition of this method?
- Where are all the callers of this function?
- Who inherits from this class?
- What are all the declarations in this file?
So, let understand How actually the Glean helped you.
So, lets Imange you are working a python project. The IDE ( let VS Code) along with whole python env setup, created a abstract syntax tree for you.
Now the Glean Uses this abstract syntax tree, to understand the Code with the help of the binary it uses.
It lists up all the Facts and store them in database, RockDB. to get answer you queries you need Angle which is a DQL for Glean.
The Glean store each fact once and when you made some changes, it will update the corresponding facts, instead of all the facts.
My Contributions to Glean
For the Fellowship My task was to add support for Scip-Python Indexer and Dotnet Indexer.
Scip-Python might sounds new but it is just a normal python indexer, scip is just a better code indexing format than LSIF. You can understand more about it on on SourceGraph page.
Steps for adding any indexer can be divided into 4 steps
- Updating the CI
- Added wrapper for that Indexer
- Writing Test Files
- Updatin the Docs
There is one more steps which is related to Glass, a Glean's client.
- Updating the Glass Schema
- Writing Test files for Glass test
If you are really excited to work on Glean, please checkout my blog on making Local Glean setup, because for the first time, Glean takes around 7-8+ hrs for build. It took for around 3-4 days to make my local setup working :)
Working on my First Indexer
So now, I know what you have to do.
My first step was to look for python binary, for this I reachedout to sourcegraph and looked for scip-python binary.
I updated all the 3 CI files, here you can check the code
Updating the CI for Scip-Python
Now my next task was to add a wrapper for scip-python. The wrapper code is in Haskell so you need a good knowledge to understand and write the Haskell code.
At my time a took a course and tried a lot really to get it.
Here is an image for wrapper code
you can checkout this commit for scip-python wrapper. In this I made a number of other changes, so please have a look.
Writing test cases for scip-python indexer.
This is the most exciting part, here I need to add an example Python project. For this, first I was confused what to add and what should be the complexity of that project. At last, I added a snake game that uses the Python-tkinter module.
Top comments (0)