DEV Community

Cover image for Multi-component CMake-based project with Git X-Modules
Dmitry Linov
Dmitry Linov

Posted on • Originally published at support.tmatesoft.com

Multi-component CMake-based project with Git X-Modules

#cpp #git #submodules #xmodules

How to build a multi-component project without installing all libraries.

At the time of writing this, there's no default solution for dependency management in the C world. There're several competing tools like Basel, Meson, conan, etc but none of them is recognized as standard de facto like Maven in Java world.

In this post, we will describe a CMake-based setup of a project consisting of several modules, which:

  • can be built and installed independently;
  • can be build together, so all the app and library sources can be edited, debugged, and committed simultaneously.

This setup resolves the pain of "fix and build the library" -> "install the library" -> "build the app with the library" -> "test the fix from the app" cycle.

As an example, we will create an app that computes SHA-1 checksum of its argument. To compute SHA-1 checksum we will use an external library by Steve Reid.

Project structure.
Currently, the SHA-1 library in question has only a Makefile to build the project. We want to add a CMake-related build file (CMakeLists.txt) to this library. As we don't have write access to this repository we will fork this project to our Git server. For that, we will use Atlassian Bitbucket Server/Data Center as one of the most popular self-hosting Git solutions. It's also convenient because there's a special Git X-Modules App for it. If you are on any other Git Server, you may still use Git X-Modules as a command-line tool.

The SHA-1 library component will be called 'sha-1'. Then we will create a 'checksum' component that uses 'sha-1' library and produces an executable file. One should be able to build this component independently assuming 'sha-1' library is installed (into /usr/lib and /usr/include or any other OS-specific directory for libraries).

Finally, we will create a 'checksum-project' that includes both 'sha-1' and 'checksum' components and can be built without having to install 'sha-1' library to the system. This will allow us to open, edit, and build all components in an IDE.

22-structure|434x264

From the Git perspective, one can use Git submodules for that. But this solution causes a lot of problems. For example, the Projectile package of Emacs doesn't currently work well with submodules. We will use a better solution: Git X-Modules.

Step 1.
Create 'sha-1' repository with Atlassian Bitbucket Server/Data Center interface.

01-create-sha-1-repository|449x500

We want now to fork https://github.com/clibs/sha1 project there. To do that, clone the sha1 project from GitHub and push it to our self-hosted Bitbucket Server repository (we assume it's running on example.org domain):

git clone https://github.com/clibs/sha1.git sha-1
cd sha-1
git remote set-url origin http://example.org/scm/checksum/sha-1.git
git push origin master
Enter fullscreen mode Exit fullscreen mode

Step 2. Add our custom CMakeLists.txt to 'sha-1' project with the following content:

cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
project(sha1)

add_library(sha1 sha1.c)
target_include_directories(sha1 PUBLIC .)
set_target_properties(sha1 PROPERTIES PUBLIC_HEADER "sha1.h")
install(TARGETS sha1 LIBRARY DESTINATION lib ARCHIVE DESTINATION lib PUBLIC_HEADER DESTINATION include)
Enter fullscreen mode Exit fullscreen mode

The install() command tells CMake to install the library to /usr/local/lib and the header to /usr/local/include, so it can be easily included by the app.

The target_include_directories() call is not necessary to build this project but will be useful later for the multi-module project structure.

Commit and push the change:

git add CMakeLists.txt
git commit -m "CMakeLists.txt added."
git push origin master
Enter fullscreen mode Exit fullscreen mode

02-sha-1-repository-toc|615x500
03-sha-1-repository-history|690x180

Make sure CMake can build the project. We can use these old-fashioned commands:

mkdir build
cd build
cmake ..
make
Enter fullscreen mode Exit fullscreen mode

To install the library into the system one can use "sudo make install" command but later we will describe how to create a multi-component structure that doesn't need installation. So we do not install the library.

Step 3. Create a Git repository for our app.

Create a 'checksum' repository using Atlassian Bitbucket Server/Data Center interface.
04-create-checksum-repository|446x500

Clone and "cd" into the repository:

git clone http://example.org/scm/checksum/checksum.git checksum
cd checksum
Enter fullscreen mode Exit fullscreen mode

Create the main.c file with the following content:

#include <sha1.h>
#include <stdio.h>
#include <string.h>

#define SHA1_SIZE_IN_BYTES 20

void print_usage(char* command) {
  fprintf(stderr, "Usage: %s STRING\n", command);
}

void print_hex(char* buffer, int buffer_size) {
  for(int i = 0; i < buffer_size; i++) {
    printf("%02x", buffer[i] & 0xff);
  }
}

int main(int argc, char** argv) {
  if (argc != 2) {
    print_usage(argv[0]);
    return 1;
  }
  char sha1[SHA1_SIZE_IN_BYTES + 1];
  char* str = argv[1];

  SHA1(sha1, str, strlen(str));
  sha1[SHA1_SIZE_IN_BYTES] = '\0';

  print_hex(sha1, SHA1_SIZE_IN_BYTES);
  printf("\n");

  return 0;
}
Enter fullscreen mode Exit fullscreen mode

Then create CMakeLists.txt file with this content:

cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
project(checksum)

add_executable(checksum main.c)

target_include_directories(checksum PRIVATE sha1)
target_link_libraries(checksum PRIVATE sha1)
Enter fullscreen mode Exit fullscreen mode

The main.c is simple: it just calls SHA1() function from 'sha-1' library and CMakeLists.txt adds a dependency on 'sha-1' library. Commit and push the change:

git add main.c CMakeLists.txt
git commit -m "Initial commit."
git push origin master
Enter fullscreen mode Exit fullscreen mode

05-checksum-repository-toc|690x358
06-checksum-repository-history|653x316

If 'sha-1' is installed on the system with "sudo make install" command of "Step 2", our checksum app can be build using CMake. But if 'sha-1' is not installed, CMake build command will fail because it can't find the header file for the library:

mkdir build
cd build
cmake ..
make
Enter fullscreen mode Exit fullscreen mode 
Scanning dependencies of target checksum
[ 50%] Building C object CMakeFiles/checksum.dir/main.c.o
/home/dmit10/work/blog/02-cmake-post/checksum/main.c:1:10: fatal error: sha1.h: No such file or directory
 #include <sha1.h>
          ^~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/checksum.dir/build.make:63: CMakeFiles/checksum.dir/main.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/checksum.dir/all] Error 2
make: *** [Makefile:84: all] Error 2
Enter fullscreen mode Exit fullscreen mode

Step 4. Multi-module structure.
Now we want to create a multi-module repository 'checksum-project' with the following structure:

CMakeLists.txt
checksum/    <-- 'checksum' module insertion point
libs/sha-1/  <-- 'sha-1' library insertion point
Enter fullscreen mode Exit fullscreen mode

If you don't have the Git X-Modules app for Bitbucket installed, install it now. To do so, go to Administration | Find new apps | Search the Marketplace and type "X-Modules".
07-install-x-modules-app|690x203

Now create a new Git repository and name it 'checksum-project'.

08-create-checksum-project-repository|449x500

Once the Git X-Modules app is installed, there will be an X-Modules button on the Git repository page. Click it.

09-x-modules-button|690x181

As the repository is empty, click "Create Default Branch".
10-x-modules-create-default-branch|690x285

Then click "Add Module" to add the 'sha-1' library.

11-x-modules-add-first-module|690x275

Choose the 'sha-1' repository.
12-x-modules-choose-sha-1-repository|690x426

Choose the 'master' branch in the 'sha-1' repository.
13-x-modules-choose-sha-1-branch|690x420

Make sure the 'sha-1' module is inserted into 'libs/sha-1' so "This Repository Path" should be 'libs/sha-1'.
14-x-modules-choose-sha-1-module-path|690x452

Click "Add Module". Do not "Apply Changes" yet.

Now add another module to add 'checksum' app to our multi-module structure. To do that click 'Add Module' again:
15-x-modules-add-second-module|690x428

Then choose the 'checksum' repository.
16-x-modules-choose-checksum-repository|690x462

Choose the 'master' branch in it.
17-x-modules-choose-checksum-branch|690x425

Make sure the 'checksum' repository is inserted into the 'checksum' directory, so "This Repository Path" should be 'checksum'. The directory name is arbitrary (as well as "libs/sha-1"), we will specify it later in add_subdirectory() call.
18-x-modules-choose-checksum-module-path|690x460

Click 'Add Module'

19-x-modules-apply-changes|690x460

Apply the changes.

Now 'checksum-project' repository contains 2 directories: 'checksum' with 'checksum' module and 'libs/sha-1' with 'sha-1' library. They are inserted into 'checksum-project' as normal directories in Git.
20-checksum-project-repository-toc|677x500

Step 5. Create a multi-module CMake configuration in 'checksum-project'.

Clone and 'cd' into the 'checksum-project':

git clone http://example.org/scm/checksum/checksum-project.git checksum-project/
cd checksum-project/
Enter fullscreen mode Exit fullscreen mode

Create CMakeLists.txt with the following content:

cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
project(checksum-project)

add_subdirectory(libs/sha-1)
add_subdirectory(checksum)
Enter fullscreen mode Exit fullscreen mode

Commit and push the file:

git add CMakeLists.txt
git commit -m "CMakeLists.txt configuration added."
git push origin master
Enter fullscreen mode Exit fullscreen mode

That's all!

The project can now be built without installation:

mkdir build
cd build
cmake ..
make
Enter fullscreen mode Exit fullscreen mode 
[ 25%] Building C object libs/sha-1/CMakeFiles/sha1.dir/sha1.c.o
[ 50%] Linking C static library libsha1.a
[ 50%] Built target sha1
[ 75%] Building C object checksum/CMakeFiles/checksum.dir/main.c.o
[100%] Linking C executable checksum
[100%] Built target checksum
Enter fullscreen mode Exit fullscreen mode

Now let's try our app:

checksum/checksum foobar
8843d7f92416211de9ebb963ff4ce28125932878
Enter fullscreen mode Exit fullscreen mode

and compare it with standard 'sha1sum' utility:

echo -n foobar | sha1sum
8843d7f92416211de9ebb963ff4ce28125932878  -
Enter fullscreen mode Exit fullscreen mode

All sources are in the same repository as normal Git directories. Now one can modify 'libs/sha-1' and 'checksum/' inside the 'checksum-project' directly and these changes will be synchronized with corresponding 'sha-1' and 'checksum' repositories. And vice versa.
21-structure|600x395
Have fun!

Top comments (0)

Read next

nozibul_islam_113b1d5334f profile image

Git Mastery: Essential Questions and Answers for Developers 🚀

Nozibul Islam -

arsenikavalchuk profile image

Manual Memory Management and Garbage Collection in Kotlin Multiplatform Native Shared Libraries

Arseni Kavalchuk -

hardy_mervana profile image

Top 15 Git Commands That Every Developer Should Know

Hardik Mervana -

developedbyviv profile image

I Passed the GitHub Foundations Certification: My Honest Experience

Vivek Singh -