This article appeared first on https://www.pascallandau.com/ at Set up PHP QA tools and control them via make [Tutorial Part 5]
In the fifth part of this tutorial series on developing PHP on Docker we will setup some PHP code quality tools and provide a convenient way to control them via GNU make.
FYI: Originally I wanted this tutorial to be a part of
Create a CI pipeline for dockerized PHP Apps
because QA checks are imho vital part of a CI setup - but it kinda grew "too big" and took a way too much space from, well, actually setting up the CI pipelines :)
All code samples are publicly available in my Docker PHP Tutorial repository on Github. You find the branch with the final result of this tutorial at part-5-php-qa-tools-make-docker.
All published parts of the Docker PHP Tutorial are collected under a dedicated page at Docker PHP Tutorial. The previous part was Run Laravel 9 on Docker in 2022 and the following one is Use git-secret to encrypt secrets in the repository.
If you want to follow along, please subscribe to the RSS feed or via email to get automatic notifications when the next part comes out :)
Table of contents
Introduction
Code quality tools ensure a baseline of code quality by automatically checking certain rules, e.g. code style definitions, proper usage of types, proper declaration of dependencies, etc. When run regularly they are a great way to enforce better code and are thus a
perfect fit for a CI pipeline. For this tutorial, I'm going to setup the following tools:
- Style Checker: phpcs
- Static Analyzer: phpstan
- Code Linter: php-parallel-lint
- Dependency Checker: composer-require-checker
and provide convenient access through a qa
make target. The end result will look like this:
FYI: When we started out with using code quality tools in general, we have used GrumPHP - and I would still recommend it. We have only transitioned away from it because make
gives us a little more flexibility and control.
You can find the "final" makefile at .make/01-02-application-qa.mk
.
CAUTION: The Makefile
is build on top of the setup that I introduced in Docker from scratch for PHP 8.1 Applications in 2022, so please refer to that tutorial if anything is not clear.
##@ [Application: QA]
# variables
CORES?=$(shell (nproc || sysctl -n hw.ncpu) 2> /dev/null)
# constants
## files
ALL_FILES=./
APP_FILES=app/
TEST_FILES=tests/
## bash colors
RED:=\033[0;31m
GREEN:=\033[0;32m
YELLOW:=\033[0;33m
NO_COLOR:=\033[0m
# Tool CLI config
PHPUNIT_CMD=php vendor/bin/phpunit
PHPUNIT_ARGS= -c phpunit.xml
PHPUNIT_FILES=
PHPSTAN_CMD=php vendor/bin/phpstan analyse
PHPSTAN_ARGS=--level=9
PHPSTAN_FILES=$(APP_FILES) $(TEST_FILES)
PHPCS_CMD=php vendor/bin/phpcs
PHPCS_ARGS=--parallel=$(CORES) --standard=psr12
PHPCS_FILES=$(APP_FILES)
PHPCBF_CMD=php vendor/bin/phpcbf
PHPCBF_ARGS=$(PHPCS_ARGS)
PHPCBF_FILES=$(PHPCS_FILES)
PARALLEL_LINT_CMD=php vendor/bin/parallel-lint
PARALLEL_LINT_ARGS=-j 4 --exclude vendor/ --exclude .docker --exclude .git
PARALLEL_LINT_FILES=$(ALL_FILES)
COMPOSER_REQUIRE_CHECKER_CMD=php vendor/bin/composer-require-checker
COMPOSER_REQUIRE_CHECKER_ARGS=--ignore-parse-errors
# call with NO_PROGRESS=true to hide tool progress (makes sense when invoking multiple tools together)
NO_PROGRESS?=false
ifeq ($(NO_PROGRESS),true)
PHPSTAN_ARGS+= --no-progress
PARALLEL_LINT_ARGS+= --no-progress
else
PHPCS_ARGS+= -p
PHPCBF_ARGS+= -p
endif
# Use NO_PROGRESS=false when running individual tools.
# On NO_PROGRESS=true the corresponding tool has no output on success
# apart from its runtime but it will still print
# any errors that occured.
define execute
if [ "$(NO_PROGRESS)" = "false" ]; then \
eval "$(EXECUTE_IN_APPLICATION_CONTAINER) $(1) $(2) $(3) $(4)"; \
else \
START=$$(date +%s); \
printf "%-35s" "$@"; \
if OUTPUT=$$(eval "$(EXECUTE_IN_APPLICATION_CONTAINER) $(1) $(2) $(3) $(4)" 2>&1); then \
printf " $(GREEN)%-6s$(NO_COLOR)" "done"; \
END=$$(date +%s); \
RUNTIME=$$((END-START)) ;\
printf " took $(YELLOW)$${RUNTIME}s$(NO_COLOR)\n"; \
else \
printf " $(RED)%-6s$(NO_COLOR)" "fail"; \
END=$$(date +%s); \
RUNTIME=$$((END-START)) ;\
printf " took $(YELLOW)$${RUNTIME}s$(NO_COLOR)\n"; \
echo "$$OUTPUT"; \
printf "\n"; \
exit 1; \
fi; \
fi
endef
.PHONY: test
test: ## Run all tests
@$(EXECUTE_IN_APPLICATION_CONTAINER) $(PHPUNIT_CMD) $(PHPUNIT_ARGS) $(ARGS)
.PHONY: phplint
phplint: ## Run phplint on all files
@$(call execute,$(PARALLEL_LINT_CMD),$(PARALLEL_LINT_ARGS),$(PARALLEL_LINT_FILES), $(ARGS))
.PHONY: phpcs
phpcs: ## Run style check on all application files
@$(call execute,$(PHPCS_CMD),$(PHPCS_ARGS),$(PHPCS_FILES), $(ARGS))
.PHONY: phpcbf
phpcbf: ## Run style fixer on all application files
@$(call execute,$(PHPCBF_CMD),$(PHPCBF_ARGS),$(PHPCBF_FILES), $(ARGS))
.PHONY: phpstan
phpstan: ## Run static analyzer on all application and test files
@$(call execute,$(PHPSTAN_CMD),$(PHPSTAN_ARGS),$(PHPSTAN_FILES), $(ARGS))
.PHONY: composer-require-checker
composer-require-checker: ## Run dependency checker
@$(call execute,$(COMPOSER_REQUIRE_CHECKER_CMD),$(COMPOSER_REQUIRE_CHECKER_ARGS),"", $(ARGS))
.PHONY: qa
qa: ## Run code quality tools on all files
@"$(MAKE)" -j $(CORES) -k --no-print-directory --output-sync=target qa-exec NO_PROGRESS=true
.PHONY: qa-exec
qa-exec: phpstan \
phplint \
composer-require-checker \
phpcs
The QA tools
phpcs and phpcbf
phpcs
is the CLI tool of the style checker squizlabs/PHP_CodeSniffer. It also comes with phpcbf
- a tool to automatically fix style errors.
Installation via composer:
make composer ARGS="require --dev squizlabs/php_codesniffer"
For now we will simply use the pre-configured ruleset for PSR-12: Extended Coding Style. When run in the application
container for the first time on the app
directory via
vendor/bin/phpcs --standard=PSR12 --parallel=4 -p app
i.e.
--standard=PSR12 => use the PSR12 ruleset
--parallel=4 => run with 4 parallel processes
-p => show the progress
we get the following result:
root:/var/www/app# vendor/bin/phpcs --standard=PSR12 --parallel=4 -p app
FILE: /var/www/app/app/Console/Kernel.php
----------------------------------------------------------------------
FOUND 2 ERRORS AFFECTING 1 LINE
----------------------------------------------------------------------
28 | ERROR | [x] Expected at least 1 space before "."; 0 found
28 | ERROR | [x] Expected at least 1 space after "."; 0 found
----------------------------------------------------------------------
PHPCBF CAN FIX THE 2 MARKED SNIFF VIOLATIONS AUTOMATICALLY
----------------------------------------------------------------------
FILE: /var/www/app/app/Http/Controllers/HomeController.php
----------------------------------------------------------------------
FOUND 4 ERRORS AFFECTING 2 LINES
----------------------------------------------------------------------
37 | ERROR | [x] Expected at least 1 space before "."; 0 found
37 | ERROR | [x] Expected at least 1 space after "."; 0 found
45 | ERROR | [x] Expected at least 1 space before "."; 0 found
45 | ERROR | [x] Expected at least 1 space after "."; 0 found
----------------------------------------------------------------------
PHPCBF CAN FIX THE 4 MARKED SNIFF VIOLATIONS AUTOMATICALLY
----------------------------------------------------------------------
FILE: /var/www/app/app/Jobs/InsertInDbJob.php
-------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-------------------------------------------------------------------------------
13 | ERROR | [x] Each imported trait must have its own "use" import statement
-------------------------------------------------------------------------------
PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
-------------------------------------------------------------------------------
FILE: /var/www/app/app/Models/User.php
-------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-------------------------------------------------------------------------------
13 | ERROR | [x] Each imported trait must have its own "use" import statement
-------------------------------------------------------------------------------
PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
-------------------------------------------------------------------------------
All errors can be fixed automatically with phpcbf
:
root:/var/www/app# vendor/bin/phpcbf --standard=PSR12 --parallel=4 -p app
PHPCBF RESULT SUMMARY
-------------------------------------------------------------------------
FILE FIXED REMAINING
-------------------------------------------------------------------------
/var/www/app/app/Console/Kernel.php 2 0
/var/www/app/app/Http/Controllers/HomeController.php 4 0
/var/www/app/app/Jobs/InsertInDbJob.php 1 0
/var/www/app/app/Models/User.php 1 0
-------------------------------------------------------------------------
A TOTAL OF 8 ERRORS WERE FIXED IN 4 FILES
-------------------------------------------------------------------------
Time: 411ms; Memory: 8MB
and a follow-up run of phpcs
doesn't show any more errors:
root:/var/www/app# vendor/bin/phpcs --standard=PSR12 --parallel=4 -p app
.................... 20 / 20 (100%)
Time: 289ms; Memory: 8MB
phpstan
phpstan
is the CLI tool of the static code analyzer phpstan/phpstan (see also the full PHPStan documentation). It provides some default "levels" of increasing strictness to report potential bugs based on the AST of the analyzed PHP code.
Installation via composer:
make composer ARGS="require --dev phpstan/phpstan"
Since this is a "fresh" codebase with very little code let's go for the highest level 9 (as of 2022-04-24) and run it in the application
container on the app
and tests
directories via:
vendor/bin/phpstan analyse app tests --level=9
--level=9 => use level 9
root:/var/www/app# vendor/bin/phpstan analyse app tests --level=9
25/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%
------ ------------------------------------------------------------------------------------------------------------------
Line app/Commands/SetupDbCommand.php
------ ------------------------------------------------------------------------------------------------------------------
22 Method App\Commands\SetupDbCommand::getOptions() return type has no value type specified in iterable type array.
� See: https://phpstan.org/blog/solving-phpstan-no-value-type-specified-in-iterable-type
34 Method App\Commands\SetupDbCommand::handle() has no return type specified.
------ ------------------------------------------------------------------------------------------------------------------
------ -------------------------------------------------------------------------------------------------------
Line app/Http/Controllers/HomeController.php
------ -------------------------------------------------------------------------------------------------------
22 Parameter #1 $jobId of class App\Jobs\InsertInDbJob constructor expects string, mixed given.
25 Part $jobId (mixed) of encapsed string cannot be cast to string.
35 Call to an undefined method Illuminate\Redis\Connections\Connection::lRange().
62 Call to an undefined method Illuminate\Contracts\View\Factory|Illuminate\Contracts\View\View::with().
------ -------------------------------------------------------------------------------------------------------
------ ------------------------------------------------------------------------------------------------------------------
Line app/Http/Middleware/Authenticate.php
------ ------------------------------------------------------------------------------------------------------------------
17 Method App\Http\Middleware\Authenticate::redirectTo() should return string|null but return statement is missing.
------ ------------------------------------------------------------------------------------------------------------------
------ ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Line app/Http/Middleware/RedirectIfAuthenticated.php
------ ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
26 Method App\Http\Middleware\RedirectIfAuthenticated::handle() should return Illuminate\Http\RedirectResponse|Illuminate\Http\Response but returns Illuminate\Http\RedirectResponse|Illuminate\Routing\Redirector.
------ ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------ -----------------------------------------------------------------------
Line app/Jobs/InsertInDbJob.php
------ -----------------------------------------------------------------------
22 Method App\Jobs\InsertInDbJob::handle() has no return type specified.
------ -----------------------------------------------------------------------
------ -------------------------------------------------
Line app/Providers/RouteServiceProvider.php
------ -------------------------------------------------
36 PHPDoc tag @var above a method has no effect.
36 PHPDoc tag @var does not specify variable name.
60 Cannot access property $id on mixed.
------ -------------------------------------------------
------ ----------------------------------------------------------------------------------------------------------------------------------------------------------
Line tests/Feature/App/Http/Controllers/HomeControllerTest.php
------ ----------------------------------------------------------------------------------------------------------------------------------------------------------
24 Method Tests\Feature\App\Http\Controllers\HomeControllerTest::test___invoke() has parameter $params with no value type specified in iterable type array.
� See: https://phpstan.org/blog/solving-phpstan-no-value-type-specified-in-iterable-type
38 Method Tests\Feature\App\Http\Controllers\HomeControllerTest::__invoke_dataProvider() return type has no value type specified in iterable type array.
� See: https://phpstan.org/blog/solving-phpstan-no-value-type-specified-in-iterable-type
------ ----------------------------------------------------------------------------------------------------------------------------------------------------------
------ ---------------------------------------------------------------------------------------------------------------------
Line tests/TestCase.php
------ ---------------------------------------------------------------------------------------------------------------------
68 Cannot access offset 'database' on mixed.
71 Parameter #1 $config of method Illuminate\Database\Connectors\MySqlConnector::connect() expects array, mixed given.
------ ---------------------------------------------------------------------------------------------------------------------
[ERROR] Found 16 errors
After fixing (or ignoring :P) all errors, we now get
root:/var/www/app# vendor/bin/phpstan analyse app tests --level=9
25/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%
[OK] No errors
php-parallel-lint
php-parallel-lint
is the CLI tool of the PHP code linter php-parallel-lint/PHP-Parallel-Lint. It ensures that all PHP files are syntactically correct.
Installation via composer:
make composer ARGS="require --dev php-parallel-lint/php-parallel-lint"
"Parallel" is already in the name, so we run it on the full codebase ./
with 4 parallel processes and exclude the .git
and vendor
directories to speed up the execution via
vendor/bin/parallel-lint -j 4 --exclude .git --exclude vendor ./
i.e.
-j 4 => use 4 parallel processes
--exclude .git --exclude vendor => ignore the .git/ and vendor/ directories
we get
root:/var/www/app# vendor/bin/parallel-lint -j 4 --exclude .git --exclude vendor ./
PHP 8.1.1 | 4 parallel jobs
............................................................ 60/61 (98 %)
. 61/61 (100 %)
Checked 61 files in 0.2 seconds
No syntax error found
No further TODOs here.
composer-require-checker
composer-require-checker
is the CLI tool of the dependency checker maglnet/ComposerRequireChecker. The tool ensures that the composer.json
file contains all dependencies that are used in the codebase.
Installation via composer:
make composer ARGS="require --dev maglnet/composer-require-checker"
Run it via
vendor/bin/composer-require-checker check
root:/var/www/app# vendor/bin/composer-require-checker check
ComposerRequireChecker 4.0.0@baa11a4e9e5072117e3d180ef16c07036cafa4a2
The following 1 unknown symbols were found:
+---------------------------------------------+--------------------+
| Unknown Symbol | Guessed Dependency |
+---------------------------------------------+--------------------+
| Symfony\Component\Console\Input\InputOption | |
+---------------------------------------------+--------------------+
What's going on here?
We use Symfony\Component\Console\Input\InputOption
in our \App\Commands\SetupDbCommand
and the code doesn't "fail" because InputOption
is defined in thesymfony/console
package that is a
transitive dependency of laravel/framework
, see the laravel/framework composer.json
file.
I.e. the symfony/console
package does actually exist in our vendor
directory - but since we also use it as a first-party-dependency directly in our code, we must declare the dependency explicitly. Otherwise, Laravel might at some point decide to drop symfony/console
and we would be left with broken code.
To fix this, I run
make composer ARGS="require symfony/console"
which will update the composer.json
file and add the dependency. Running composer-require-checker
again will now yield no further errors.
root:/var/www/app# vendor/bin/composer-require-checker check
ComposerRequireChecker 4.0.0@baa11a4e9e5072117e3d180ef16c07036cafa4a2
There were no unknown symbols found.
Additional tools (out of scope)
In general, I'm a huge fan of code quality tools and we use them quite extensively. At some point I'll probably dedicate a whole article to go over them in detail - but for now I'm just gonna leave a list for inspiration:
-
brianium/paratest
- Running PhpUnit tests in parallel
- malukenho/mcbumpface
- Update the versions in the
composer.json
file after an update - qossmic/deptrac-shim
- A shim for qossmic/deptrac: A tool to define dependency layers based on e.g. namespaces
- icanhazstring/composer-unused
- Show dependencies in the
composer.json
that are not used in the codebase - roave/security-advisories
- Gives a warning when packages with known vulnerabilities are used
- Alternative: local-php-security-checker
QA make targets
You might have noticed that all tools have their own configuration options. Instead of remembering each of them, I'll define corresponding make targets in .make/01-02-application-qa.mk
. The easiest way to do so would be to "hard-code" the exact commands that I ran previously, e.g.
.PHONY: phpstan
phpstan: ## Run static analyzer on all application and test files
@$(EXECUTE_IN_APPLICATION_CONTAINER) vendor/bin/phpstan analyse app tests --level=9
(Please refer to the Run commands in the docker containers section in the previous tutorial for an explanation of the EXECUTE_IN_APPLICATION_CONTAINER
variable).
However, this implementation is quite inflexible: What if we want to check a single file or try out other options? So let's create some variables instead:
PHPSTAN_CMD=php vendor/bin/phpstan analyse
PHPSTAN_ARGS=--level=9
PHPSTAN_FILES=$(APP_FILES) $(TEST_FILES)
.PHONY: phpstan
phpstan: ## Run static analyzer on all application and test files
@$(EXECUTE_IN_APPLICATION_CONTAINER) $(PHPSTAN_CMD) $(PHPSTAN_ARGS) $(PHPSTAN_FILES)
This target allows me to override the defaults and e.g. check only the file app/Commands/SetupDbCommand.php
with --level=1
make phpstan PHPSTAN_FILES=app/Commands/SetupDbCommand.php PHPSTAN_ARGS="--level=1"
$ make phpstan PHPSTAN_FILES=app/Commands/SetupDbCommand.php PHPSTAN_ARGS="--level=1"
1/1 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%
[OK] No errors
The remaining tool variables can be configured in the exact same way:
# constants
## files
ALL_FILES=./
APP_FILES=app/
TEST_FILES=tests/
# Tool CLI config
PHPUNIT_CMD=php vendor/bin/phpunit
PHPUNIT_ARGS= -c phpunit.xml
PHPUNIT_FILES=
PHPSTAN_CMD=php vendor/bin/phpstan analyse
PHPSTAN_ARGS=--level=9
PHPSTAN_FILES=$(APP_FILES) $(TEST_FILES)
PHPCS_CMD=php vendor/bin/phpcs
PHPCS_ARGS=--parallel=$(CORES) --standard=psr12
PHPCS_FILES=$(APP_FILES)
PHPCBF_CMD=php vendor/bin/phpcbf
PHPCBF_ARGS=$(PHPCS_ARGS)
PHPCBF_FILES=$(PHPCS_FILES)
PARALLEL_LINT_CMD=php vendor/bin/parallel-lint
PARALLEL_LINT_ARGS=-j 4 --exclude vendor/ --exclude .docker --exclude .git
PARALLEL_LINT_FILES=$(ALL_FILES)
COMPOSER_REQUIRE_CHECKER_CMD=php vendor/bin/composer-require-checker
COMPOSER_REQUIRE_CHECKER_ARGS=--ignore-parse-errors
The qa
target
From a workflow perspective I usually want to run all configured qa tools instead of each one individually (being able to run individually is still great if a tool fails, though).
A trivial approach would be a new target that uses all individual tool targets as preconditions:
.PHONY: qa
qa: phpstan \
phplint \
composer-require-checker \
phpcs
But we can do better, because this target produces quite a noisy output:
$ make qa
25/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%
[OK] No errors
PHP 8.1.1 | 4 parallel jobs
............................................................ 60/61 (98 %)
. 61/61 (100 %)
Checked 61 files in 0.3 seconds
No syntax error found
ComposerRequireChecker 4.0.0@baa11a4e9e5072117e3d180ef16c07036cafa4a2
There were no unknown symbols found.
.................... 20 / 20 (100%)
Time: 576ms; Memory: 8MB
I'd rather have something like this:
$ make qa
phplint done took 1s
phpcs done took 1s
phpstan done took 3s
composer-require-checker done took 6s
The execute
"function"
We'll make this work by suppressing the tool output and using a user-defined execute
make function to format all targets nicely.
Though "function" isn't quite correct here, because it's rather a multiline variable defined via define ... endef
that is then "invoked" via the call function.
# File: 01-02-application-qa.mk
# call with NO_PROGRESS=true to hide tool progress (makes sense when invoking multiple tools together)
NO_PROGRESS?=false
ifeq ($(NO_PROGRESS),true)
PHPSTAN_ARGS+= --no-progress
endif
# Use NO_PROGRESS=false when running individual tools.
# On NO_PROGRESS=true the corresponding tool has no output on success
# apart from its runtime but it will still print
# any errors that occured.
define execute
if [ "$(NO_PROGRESS)" = "false" ]; then \
eval "$(EXECUTE_IN_APPLICATION_CONTAINER) $(1) $(2) $(3) $(4)"; \
else \
START=$$(date +%s); \
printf "%-35s" "$@"; \
if OUTPUT=$$(eval "$(EXECUTE_IN_APPLICATION_CONTAINER) $(1) $(2) $(3) $(4)" 2>&1); then \
printf " %-6s" "done"; \
END=$$(date +%s); \
RUNTIME=$$((END-START)) ;\
printf " took $${RUNTIME}s\n"; \
else \
printf " %-6s" "fail"; \
END=$$(date +%s); \
RUNTIME=$$((END-START)) ;\
printf " took $${RUNTIME}s\n"; \
echo "$$OUTPUT"; \
printf "\n"; \
exit 1; \
fi; \
fi
endef
- the
NO_PROGRESS
variable is set tofalse
by default and will cause a target to be invoked as before, showing all its output immediately- if the variable is set to
true
, the target is instead invoked viaeval
and the output is captured in theOUTPUT
bash variable that will only be printed if the result of the invocation is faulty
- if the variable is set to
The tool targets are then adjusted to use the new function.
.PHONY: phpstan
phpstan: ## Run static analyzer on all application and test files
@$(call execute,$(PHPSTAN_CMD),$(PHPSTAN_ARGS),$(PHPSTAN_FILES),$(ARGS))
We can now call the phpstan
target with NO_PROGRESS=true
like so:
$ make phpstan NO_PROGRESS=true
phpstan done took 4s
An "error" would look likes this:
$ make phpstan NO_PROGRESS=true
phpstan fail took 9s
------ ----------------------------------------
Line app/Providers/RouteServiceProvider.php
------ ----------------------------------------
49 Cannot access property $id on mixed.
------ ----------------------------------------
Parallel execution and a helper target
Technically, this also already works with the qa
target and we can even speed up the process by
running the tools in parallel with the -j flag for "Parallel Execution"
$ make -j 4 qa NO_PROGRESS=true
phpstan phplint composer-require-checker phpcs done took 5s
done took 5s
done took 7s
done took 10s
Well... not quite what we wanted. We also need to use --output-sync=target
to controll the "Output During Parallel Execution"
$ make -j 4 --output-sync=target qa NO_PROGRESS=true
phpstan done took 3s
phplint done took 1s
composer-require-checker done took 5s
phpcs done took 1s
Since this is quite a mouthful to type, we'll use a helper target qa-exec
for running the tools and put all the inconvenient-to-type options in the final qa
target.
# File: 01-02-application-qa.mk
#...
# variables
CORES?=$(shell (nproc || sysctl -n hw.ncpu) 2> /dev/null)
.PHONY: qa
qa: ## Run code quality tools on all files
@"$(MAKE)" -j $(CORES) -k --no-print-directory --output-sync=target qa-exec NO_PROGRESS=true
.PHONY: qa-exec
qa-exec: phpstan \
phplint \
composer-require-checker \
phpcs
For the number of parallel processes I use nproc
(works on Linux and Windows) resp. sysctl -n hw.ncpu
(works on Mac) to determine the number of available cores. If you dedicate less resources to docker you might want to hard-code this setting to a lower value (e.g. by adding a CORES
variable in the .make/.env
file).
Sprinkle some color on top
The final piece for getting to the output mentioned in the Introduction is the bash-coloring:
To make this work, we need to understand first how colors work in bash:
This [coloring] can be accomplished by adding a
\e
[or\033
] at the beginning to form an
escape sequence. The escape sequence for specifying color codes is\e[COLORm
(COLOR
represents our (numeric) color code in this case).
(via Adding colors to Bash scripts)
E.g. the following script will print a green text:
printf "\033[0;32mThis text is green\033[0m"
So we define the required colors as variables and use them in the corresponding places in the execute
function:
## bash colors
RED:=\033[0;31m
GREEN:=\033[0;32m
YELLOW:=\033[0;33m
NO_COLOR:=\033[0m
# ...
# Use NO_PROGRESS=false when running individual tools.
# On NO_PROGRESS=true the corresponding tool has no output on success
# apart from its runtime but it will still print
# any errors that occured.
define execute
if [ "$(NO_PROGRESS)" = "false" ]; then \
eval "$(EXECUTE_IN_APPLICATION_CONTAINER) $(1) $(2) $(3) $(4)"; \
else \
START=$$(date +%s); \
printf "%-35s" "$@"; \
if OUTPUT=$$(eval "$(EXECUTE_IN_APPLICATION_CONTAINER) $(1) $(2) $(3) $(4)" 2>&1); then \
printf " $(GREEN)%-6s$(NO_COLOR)" "done"; \
END=$$(date +%s); \
RUNTIME=$$((END-START)) ;\
printf " took $(YELLOW)$${RUNTIME}s$(NO_COLOR)\n"; \
else \
printf " $(RED)%-6s$(NO_COLOR)" "fail"; \
END=$$(date +%s); \
RUNTIME=$$((END-START)) ;\
printf " took $(YELLOW)$${RUNTIME}s$(NO_COLOR)\n"; \
echo "$$OUTPUT"; \
printf "\n"; \
exit 1; \
fi; \
fi
endef
Please note, that i did not include the tests in the qa
target. I like to run those separately, because our tests usually take a lot longer to execute. So in my day-to-day work I would run make qa
and make test
to ensure that code quality and tests are passing:
$ make qa
phplint done took 1s
phpcs done took 1s
composer-require-checker done took 14s
phpstan done took 16s
$ make test
PHPUnit 9.5.19 #StandWithUkraine
....... 7 / 7 (100%)
Time: 00:03.772, Memory: 28.00 MB
OK (7 tests, 13 assertions)
Further updates in the codebase
I've also cleaned up the codebase a little in branch part-5-php-qa-tools-make-docker and even though those changes have nothing to todo with "QA tools" I didn't want to leave them unnoticed:
-
removing unnecessary files (
.styleci.yml
,package.json
,webpack.mix.js
)- removing unused values from the
.env.example
file - run a
composer update
to get the latest Laravel version - add a
show-help
script to thescripts
section of thecomposer.json
file that references theMakefile
(see also this discussion on Twitter)
{ "scripts": { "show-help": [ "make" ] }, "scripts-descriptions": { "show-help": "Display available 'make' commands (we use make instead of composer scripts)." } }
- replace
docker-compose
withdocker compose
to use compose v2
- removing unused values from the
For some reason, the last point caused some trouble because Linux and Docker Desktop for Windows require a -T
flag for the exec
command to disable a TTY allocation in some cases. Whereas on Docker Desktop for Mac the missing TTY lead to a cluttered output ("staircase effect").
Thus I modified the Makefile
to populate a DOCKER_COMPOSE_EXEC_OPTIONS
variable based on the OS
# Add the -T options to "docker compose exec" to avoid the
# "panic: the handle is invalid"
# error on Windows and Linux
# @see https://stackoverflow.com/a/70856332/413531
DOCKER_COMPOSE_EXEC_OPTIONS=-T
# OS is defined for WIN systems, so "uname" will not be executed
OS?=$(shell uname)
ifeq ($(OS),Windows_NT)
# Windows requires the .exe extension, otherwise the entry is ignored
# @see https://stackoverflow.com/a/60318554/413531
SHELL := bash.exe
else ifeq ($(OS),Darwin)
# On Mac, the -T must be omitted to avoid cluttered output
# @see https://github.com/moby/moby/issues/37366#issuecomment-401157643
DOCKER_COMPOSE_EXEC_OPTIONS=
endif
And use the variable when defining EXECUTE_IN_*_CONTAINER
in .make/02-00-docker.mk
ifeq ($(EXECUTE_IN_CONTAINER),true)
EXECUTE_IN_ANY_CONTAINER:=$(DOCKER_COMPOSE) exec $(DOCKER_COMPOSE_EXEC_OPTIONS) --user $(APP_USER_NAME) $(DOCKER_SERVICE_NAME)
EXECUTE_IN_APPLICATION_CONTAINER:=$(DOCKER_COMPOSE) exec $(DOCKER_COMPOSE_EXEC_OPTIONS) --user $(APP_USER_NAME) $(DOCKER_SERVICE_NAME_APPLICATION)
EXECUTE_IN_WORKER_CONTAINER:=$(DOCKER_COMPOSE) exec $(DOCKER_COMPOSE_EXEC_OPTIONS) --user $(APP_USER_NAME) $(DOCKER_SERVICE_NAME_PHP_WORKER)
endif
Wrapping up
Congratulations, you made it! If some things are not completely clear by now, don't hesitate to leave a comment. You should now have a blueprint for adding code quality tools for your dockerized application and way to conveniently control them through a Makefile.
In the next part of this tutorial, we will set up git secret to encrypt secret values and store them directly in the git repository.
Please subscribe to the RSS feed or via email to get automatic notifications when this next part comes out :)
Top comments (4)
Feedback appreciated :)
If you are a macOS user, ServBay.dev is worth to try. You don't need to spend some time or couple of days to setup anything. Just download it and you can use it immediately. You can run multiple PHP versions simultaneously and switch between them effortlessly.
Honestly, this tool has greatly simplified my PHP development and is definitely worth trying!
Does it work with Mac?
Yes, as long as you have Docker installed 👍