Table of Contents
My Workflow
Motivation behind the Workflow
Let's all agree that we, as Human Beings, prefer laziness most of the time. We love snoozing the alarm for an extra five minutes every morning, we prefer texting our siblings who are just sitting in the next room, sometimes we don't even run through our exam paper before submitting it just because we are too lazy to do so.
Besides laziness, we usually get bored so easily, from tedious tasks at work right through to waiting in line at the supermarket. Our brains simply don't respond well to experiences that are too predictable and repetitive. This lack of amusement is what we call boredom.
That's why we invented Machines and enhanced the overall process of automation. Simply to make everything better, from increasing productivity to building more efficient and flawless products.
Well, in the Software Engineering field, we too have some tedious tasks to keep on doing daily to maintain the output product. Here comes GitHub, or more specifically GitHub Actions, to provide us a solution to automate any number of repetitive tasks and start the process with something just as simple as a mouse click! Impressive, right? Buckle up and get ready, because your mind is about to be blown away.
How did GitHub Actions help me?
Let me tell you my story from the 3rd-person point of view perspective. So, Shehab is an Open-Source enthusiast who created a GitHub repository called DiffCpp almost a year ago. At this moment in time, DiffCpp has a lot of users, forks, and stars (well, this is not the case for now, but hopefully it might be shortly). Anyway, this GitHub repo currently requires regular maintenance and updates to be alive. Since Shehab is a slothful person who avoids doing monotonous tasks, he decided to use existing GitHub Actions to build a perfect workflow to automate any kind of work needed to ensure that the codebase will not be broken in whatever way before releasing it.
So, he decided to brainstorm and list out all of the possible sets of tasks needed to get the job done. He found out that these tasks can be categorized as follows:
-
Documentation
- Check the Formatting of the Codebase
- Since this is a C++ project, Clang-Format will be used.
- Generate Documentation
- Since this is a C++ project, Doxygen will be used.
- Generate Release Notes to notify the user with changes.
- Check the Formatting of the Codebase
-
Testing
- Check for a Successful Build on top of:
- Linux (Ubuntu)
- Windows
- Use Google Test for:
- Unit Testing: Test the logic of each API on its own.
- Block Testing: Test the logic of the package as a whole.
- Check the Benchmark of the package APIs using Google Benchmark.
- Check for a Successful Build on top of:
Well obviously, this is much to do, but cautious! SPOILER ALERT is on the way, and let me just tell you not only that GitHub allowed me to do all of the previously listed tasks with ZERO effort but it also provided me with this charming pipeline visualization at the end of the process.
How I created this workflow?
For simplicity, letโs consider the following scenario. I woke up this morning, grabbed a cup of coffee, and checked out my GitHub account just to find out that someone created an issue in the DiffCpp repo mentioning the following:
Input = ๐ต
Expected Output = ๐ด
Actual Output = โช
This needs an immediate fix. So, I decided to offer $10,000,000 (I wish I had this amount of money in real life) for the first contributor who solves the issue. Well, for the first thought, you may think that this will solve the issue but have you imagined how many pull requests would I get per minute?
Apart from the number of pull requests, I wouldn't even be able to check all of them to find out which ones get the job done, this would take me forever! Thanks to Google Test that allowed me to add a new test with something just as simple as these 2 lines of code:
Output = Input(๐ต);
EXPECT_EQ(Output, ๐ด);
So, here comes GitHub Actions (in collaboration with Google Test) to solve the problem. So, I decided to build a workflow to check each submitted pull request for the following:
-
Successful Build ๐ง
YAML snippet (Click to Expand)
build-linux: name: Linux runs-on: ubuntu-latest steps: - name: Checkout Repository uses: actions/checkout@v2 - name: Build run: cmake . - name: Make run: make - name: Upload Linux Artifact uses: actions/upload-artifact@v2 with: name: DiffCpp-Linux path: ./DiffCpp
For windows build, the same steps would be used using windows-specific CMake commands.Actions Used:
-
actions/checkout@v2 is used to checkout my GitHub repo.
Here is a request from the GitHub Team
- actions/upload-artifact@v2 is used to allow the user to download the built binaries later.
-
actions/checkout@v2 is used to checkout my GitHub repo.
-
Code Formatting ๐
YAML snippet (Click to Expand)
clang-format-check: name: Format Check runs-on: ubuntu-latest steps: - name: Checkout Repository uses: actions/checkout@v2 - name: Check Codebase Format uses: RafikFarhad/clang-format-github-action@v1.0.1 with: sources: "src/*.cpp,include/*.h,test/*.cpp"
Actions Used:- RafikFarhad/clang-format-github-action@v1.0.1 is used to check the format of the submitted code using clang-format.
-
Unit and Block Testing ๐
YAML snippet (Click to Expand)
verification: name: Unit and Block Testing needs: - build runs-on: ubuntu-latest steps: - name: Checkout Repository uses: actions/checkout@v2 - name: Cache GoogleTest Package id: cached-gtest uses: actions/cache@v2 env: cache-name: cache-gtest-repo with: path: googletest key: ${{ runner.os }}-build-${{ env.cache-name }} - name: Install GoogleTest if: steps.cached-gtest.outputs.cache-hit != 'true' run: mkdir googletest && git clone https://github.com/google/googletest.git - name: Build run: cmake -DBUILD_TESTS=ON . - name: Make run: make - name: Run Unit Tests run: ./DiffCpp_tst
Actions Used:- actions/cache@v2 is used to cache the output of the specified step. This will allow GitHub to skip the specified action if it has been already done before.
Note: If you are using any verification method other than GoogleTest, you can simply use the uploaded artifact from the build job and pass it to this one. GitHub is great isn't it? Anyway, for C++ projects, GoogleTest is perfect for verification. You will even have the ability to check the test results from the GitHub Actions log as it is shown right here ๐
When these checks pass successfully, the following will be shown in the pull request itself to facilitate direct merge.
Moreover, you can feast your eyes on the charming visualization of this pipeline showing the complete flow of the code review right here ๐
We are not done yet! Additionally, GitHub can even notify me when someone submits a valid pull request that solves the issue. So, I will be able to successfully check over millions of pull requests effortlessly.
Well, now it is time to generate a release, but again I have neither intention nor energy to do any other work (as if I have done any). Thus, I decided to build another workflow, based on the previous one, that will be in charge of generating the release from A to Z and can be triggered with a simple mouse click.
First things first, my strategy will be based on DRY (Don't Repeat Yourself). Thus, I decided to make use of what is called Reusable Workflows. Yes! it is exactly as fascinating as you read, GitHub has just introduced a new feature of reusing workflows, many thanks to @blackgirlbytes for her awesome blog post about the feature.
GitHub Actions: You Can Build Reusable Workflows!
Rizรจl Scarlett for GitHub ใป Nov 17 '21
So, I needed to add the following tasks:
-
Generate Documentation for the Codebase ๐
YAML snippet (Click to Expand)
doxygen-doc: environment: name: Doxygen Documentation url: https://shehab7osny.github.io/DiffCpp/include/ name: Generate Documentation needs: - codebase runs-on: ubuntu-latest steps: - name: Checkout Repository uses: actions/checkout@v2 - name: Run Doxygen uses: mattnotmitt/doxygen-action@v1 with: working-directory: 'include/' - name: Deploy to GitHub Pages uses: peaceiris/actions-gh-pages@v3 with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_dir: ./include/html destination_dir: ./include
Actions Used:-
mattnotmitt/doxygen-action@v1 is used to build doxygen docs.
Currently documentation is only available for the include directory but hopefully I will expand it later.
- peaceiris/actions-gh-pages@v3 is used to deploy html output of doxygen into GitHub pages.
The environment specified here has a url of Doxygen output displayed just right in the workflow visualization graph itself.
This job has a single dependency โก๏ธ codebase formatting
The generated URL will consist of detailed code documentation as follows:
-
mattnotmitt/doxygen-action@v1 is used to build doxygen docs.
-
Compute the Benchmark ๐
For those of you who don't know, Benchmarking means measuring the performance of a certain program with respect to time.
YAML snippet (Click to Expand)
benchmarking: environment: name: Benchmark Results url: https://shehab7osny.github.io/DiffCpp/dev/bench name: Benchmark (Performance) needs: - verification runs-on: ubuntu-latest steps: - name: Checkout Repository uses: actions/checkout@v2 - name: Cache Google Benchmark Package id: cached-benchmark uses: actions/cache@v2 env: cache-name: cache-gbench-repo with: path: benchmark key: ${{ runner.os }}-build-${{ env.cache-name }} - name: Install Google Benchmark if: steps.cached-benchmark.outputs.cache-hit != 'true' run: mkdir benchmark && git clone https://github.com/google/benchmark.git benchmark - name: Build run: cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON . - name: Make run: make - name: Run Benchmark run: ls && ./DiffCpp_BENCH --benchmark_format=json | tee benchmark_result.json - name: Store benchmark result uses: benchmark-action/github-action-benchmark@v1 with: tool: 'googlecpp' output-file-path: benchmark_result.json github-token: ${{ secrets.GITHUB_TOKEN }} fail-on-alert: true comment-on-alert: true auto-push: true
Actions Used:- benchmark-action/github-action-benchmark@v1 is used to convert the JSON results from Google Benchmark into a statistical graph.
- actions/cache@v2 is used to cache the output of the specified step. In this case the step of checking out Google Benchmark repo is cached.
-
Create a new Release (with Release Notes) ๐ญ
YAML snippet (Click to Expand)
release-project: name: Release Project needs: - benchmarking - doxygen-doc runs-on: ubuntu-latest steps: - name: Download Linux Artifact uses: actions/download-artifact@v2 with: name: DiffCpp-Linux - name: Download Windows Artifact uses: actions/download-artifact@v2 with: name: DiffCpp-Windows - name: Create GitHub Release id: create-new-release uses: actions/github-script@v4.0.2 with: github-token: ${{secrets.GITHUB_TOKEN}} script: | await github.request(`POST /repos/${{ github.repository }}/releases`, { tag_name: "${{ github.event.inputs.release_ver }}", generate_release_notes: true });
Actions Used:- actions/download-artifact@v2 is used to download the previously uploaded artifact in case it is needed to be packed in the release.
- actions/github-script@v4.0.2 is used to create a new release and generate an Automated Release Notes along with it.
This job has two dependencies
โก๏ธ benchmarking
โก๏ธ doxygen-docIn case you are new to Auto Generate Release Notes, don't hesitate to check out this charming video created by @mishmanners , many thanks for her. Anyway, I decided to take this feature to a new level and include it in an automated action itself. So, I'm literally automating an automated task!
Here is a snapshot of the auto-generated release notes:
Besides, the built artifacts will be ready for download (both for Linux and Windows)
This job has a single dependency โก๏ธ verification
The generated URL will consist of Benchmark Graph as follows:
Finally, after the completion of all these tasks, we can have a look on this charming visualization of the pipeline showing the complete flow of the release process right here ๐
Submission Category:
Maintainer Must Haves
In conclusion, I just want to point out that relating this workflow with laziness is for nothing but simplicity of illustration. However, in real life, developers commonly tend to avoid such monotonous tasks so that they can keep focused on the development itself rather than dealing with numerous amount of tasks for verification and release purposes. So, from my point of view, using this workflow in any project is a must, and will help developers to focus more on introducing new features to the project while making sure that the codebase is not broken.
Yaml File or Link to Code
- Reusable Workflows
- Main Workflows:
Here is the DiffCpp repo
Shehab7osny / DiffCpp
Diff tool for Windows/Linux implemented in C++
DiffCpp
This package is used to compare two files line by line and show the differences.
Build and Installation
Linux
mkdir build
cd build
cmake ..
make
Windows
mkdir build
cd build
cmake ..
cmake --build .
cd Debug/
Usage
Linux
./DiffCpp File_1_Path File_2_Path [/N] [/A] [/W]
Windows
DiffCpp File_1_Path File_2_Path [/N] [/A] [/W]
Main Arguments
Argument | Description |
---|---|
File_1_Path | This represents the old version of the file |
File_2_Path | This represents the new version of the file |
Optional Arguments
Argument | Description |
---|---|
/N | Option to add line number to the displayed output |
/A | Option to display the updates only and ignore common matches |
/W | Option to ignore whitespaces difference while comparing files |
Sample Use Case
File 1 (recursion_Old.py)
def tri_recursion(k):
if(k > 0):
result = k + tri_recursion(k - 1)
else:
result = 1
return result
File 2 (recursion_New.py)
def tri_recursion(
โฆAdditional Resources
Open Source projects using (hosting) the workflow โก๏ธ DiffCpp
-
Open Source projects used in the workflow:
-
Existing Actions that have been used in the workflow:
This project is licensed under the terms of the MIT license.
-
Honorable mention
Many thanks to @MylesBorins and @bdougieyo for their ultimate useful info about GitHub's new feature Automated Release Notes that has been applied in this project. You can check their video right here ๐
-
Some enhancements requested from the GitHub team:
- Allow some sort of caching the step of checking out the repository to avoid applying the same step several times.
- Allow reusable workflows (YAML files) to be called from a child directory under the workflows directory itself.
Top comments (21)
This was very useful for me
keep going man
Thank you a lot!
Glad it helped ๐
I like the style it is so good and the idea is way way more better than I expected, keep going
Much appreciated!
Glad you liked the idea ๐
Great work!, Good presentation. Great effort keep going
Thank you a lot!
Glad you liked it ๐
This was quite helpful and interesting topic. Way to go
More upcoming topics are on the way ๐๐
Stay tuned!
That's great ! Keep going โบ๏ธ
Thank you โบ๏ธ
Thats a very inspiring article. Its super relatable XD.
Procrastination is a real epidemic and I think your way of tackling it is very intuitive.
Thank you a lot! โบ๏ธ
Yup, you are right... procrastination is a real epidemic
Great Work, You have a very impressive way in presenting a very useful tool that can help any one not only developers. keep going
Many Thanks! Much appreciated ๐
This is great , very helpful and interesting.
Way to go.
Thank you!
Glad it was useful ๐
I love the motivation for such work, very relatable ๐
Neatly done as well ๐๐ผ keep it up
Very insightful shehab, keep going.
Thank you! โบ๏ธ