Under the hood of Apache AGE Makefile

Hola hola,

Today I'm going to explain the Makefile of Apache AGE and why it does look like that looking.

Firstly let's take a view of it in short view:

# License ....
MODULE_big = age

OBJS = src/backend/age.o \
       .... # some other objects
       src/backend/utils/name_validation.o

EXTENSION = age

DATA = age--1.3.0.sql

# sorted in dependency order
REGRESS = scan \
          ... all regression file names \
          name_validation \
          drop

srcdir=`pwd`

ag_regress_dir = $(srcdir)/regress
REGRESS_OPTS = --load-extension=age --inputdir=$(ag_regress_dir) --outputdir=$(ag_regress_dir) --temp-instance=$(ag_regress_dir)/instance --port=61958 --encoding=UTF-8

ag_regress_out = instance/ log/ results/ regression.*
EXTRA_CLEAN = $(addprefix $(ag_regress_dir)/, $(ag_regress_out)) src/backend/parser/cypher_gram.c src/include/parser/cypher_gram_def.h src/include/parser/cypher_kwlist_d.h

GEN_KEYWORDLIST = $(PERL) -I ./tools/ ./tools/gen_keywordlist.pl
GEN_KEYWORDLIST_DEPS = ./tools/gen_keywordlist.pl tools/PerfectHash.pm

ag_include_dir = $(srcdir)/src/include
PG_CPPFLAGS = -I$(ag_include_dir) -I$(ag_include_dir)/parser

PG_CONFIG ?= pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)

src/backend/parser/cypher_keywords.o: src/include/parser/cypher_kwlist_d.h

src/include/parser/cypher_kwlist_d.h: src/include/parser/cypher_kwlist.h $(GEN_KEYWORDLIST_DEPS)
    $(GEN_KEYWORDLIST) --extern --varname CypherKeyword --output src/include/parser $<

src/include/parser/cypher_gram_def.h: src/backend/parser/cypher_gram.c

src/backend/parser/cypher_gram.c: BISONFLAGS += --defines=src/include/parser/cypher_gram_def.h

src/backend/parser/cypher_parser.o: src/backend/parser/cypher_gram.c
src/backend/parser/cypher_keywords.o: src/backend/parser/cypher_gram.c

src/backend/parser/ag_scanner.c: FLEX_NO_BACKUP=yes

installcheck: export LC_COLLATE=C

Let's break it down

Apache License

"The Apache License is a permissive free software license written by the Apache Software Foundation (ASF).[4] It allows users to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software under the terms of the license, without concern for royalties. The ASF and its projects release their software products under the Apache License. The license is also used by many non-ASF projects. " wiki said.

MODULE_big & others

Returning to PostgreSQL "Extension Building Infrastructure"
at chapter 38. Extending SQL, the explanation of adding an extension to their code-base is well explaining, it is mentioned that: "If you are thinking about distributing your PostgreSQL extension modules, setting up a portable build system for them can be fairly difficult. Therefore the PostgreSQL installation provides a build infrastructure for extensions, called PGXS, so that simple extension modules can be built simply against an already installed server. PGXS is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that PGXS is not intended to be a universal build system framework that can be used to build any software interfacing to PostgreSQL; it simply automates common build rules for simple server extension modules. For more complicated packages, you might need to write your own build system." PostgreSQL documentation said.

In conclusion they have mentioned that to use the PGXS infrastructure for your extension. you must write a Makefile in a specific format that sets some variables will be expected and used on the side of PostgreSQL,
In the makefile, you need to set some variables and include the global PGXS makefile, and also they provide a simple example of a Makefile that builds an extension module named isbn_issn

MODULES = isbn_issn
EXTENSION = isbn_issn
DATA = isbn_issn--1.0.sql
DOCS = README.isbn_issn
HEADERS_isbn_issn = isbn_issn.h

PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)

Why MODULE_big?

Setting one of these three variables to specify what is built alongside PostgreSQL and obviously AGE is a single shared library so the second one was our selection

• MODULES: list of shared-library objects to be built from source files with same stem (do not include library suffixes in this list)

• MODULE_big: a shared library to build from multiple source files (list object files in OBJS)

• PROGRAM: an executable program to build (list object files in OBJS)

Others

Variables can also be set

• EXTENSION: extension name(s); for each name you must provide an extension.control file, which will be installed into prefix/share/extension
• MODULEDIR: subdirectory of prefix/share into which DATA and DOCS files should be installed (if not set, default is extension if EXTENSION is set, or contrib if not)
• DATA: random files to install into prefix/share/$MODULEDIR
• DATA_built: random files to install into prefix/share/$MODULEDIR, which need to be built first
• DATA_TSEARCH: random files to install under prefix/share/tsearch_data
• DOCS: random files to install under prefix/doc/$MODULEDIR
• HEADERS: Files to (optionally build and) install under prefix/include/server/$MODULEDIR/$MODULE_big.
• HEADERS_built: Unlike DATA_built, files in HEADERS_built are not removed by the clean target; if you want them removed, also add them to EXTRA_CLEAN or add your own rules to do it.

• HEADERS_$MODULE:
  Files to install (after building if specified) under prefix/include/server/$MODULEDIR/$MODULE, where $MODULE must be a module name used in MODULES or MODULE_big.

• HEADERS_built_$MODULE: Unlike DATA_built, files in HEADERS_built_$MODULE are not removed by the clean target; if you want them removed, also add them to EXTRA_CLEAN or add your own rules to do it.

  It is legal to use both variables for the same module, or any combination, unless you have two module names in the MODULES list that differ only by the presence of a prefix built_, which would cause ambiguity. In that (hopefully unlikely) case, you should use only the HEADERS_built_$MODULE variables.

• SCRIPTS:
  script files (not binaries) to install into prefix/bin

• SCRIPTS_built:
  script files (not binaries) to install into prefix/bin, which need to be built first

• REGRESS:
  list of regression test cases (without suffix), see below

• REGRESS_OPTS:
  additional switches to pass to pg_regress

• ISOLATION:
  list of isolation test cases, see below for more details

• ISOLATION_OPTS:
  additional switches to pass to pg_isolation_regress

• TAP_TESTS:
  switch defining if TAP tests need to be run, see below

• NO_INSTALL:
  don't define an install target, useful for test modules that don't need their build products to be installed

• NO_INSTALLCHECK:
  don't define an installcheck target, useful e.g., if tests require special configuration, or don't use pg_regress

• EXTRA_CLEAN:
  extra files to remove in make clean

• PG_CPPFLAGS:
  will be prepended to CPPFLAGS

• PG_CFLAGS:
  will be appended to CFLAGS

• PG_CXXFLAGS:
  will be appended to CXXFLAGS

• PG_LDFLAGS:
  will be prepended to LDFLAGS

• PG_LIBS:
  will be added to PROGRAM link line

• SHLIB_LINK:
  will be added to MODULE_big link line

• PG_CONFIG:
  path to pg_config program for the PostgreSQL installation to build against (typically just pg_config to use the first one in your PATH)

Conclusion

It was a quick overview about the Makefile of AGE and what it is written in that form, I hope it give a some sort of new information you get out from that article.

References:

• https://www.postgresql.org/docs/current/extend-pgxs.html
• https://age.apache.org/
• https://github.com/apache/age