A collection of practical tips and techniques that simplify your developer experience of Maestro E2E framework!
1. Installing Maestro
Let's add a new script to the package.json file:
{
"scripts": {
"install-maestro": "MAESTRO_VERSION=1.36.0 curl -Ls 'https://get.maestro.mobile.dev' | bash"
}
}
I define the version we now support using the environment variable MAESTRO_VERSION. The use of the same version by our team and CI is guaranteed.
docs: https://maestro.mobile.dev/getting-started/installing-maestro
2. Identify the app as being run by Maestro.
When an app is run by e2e, we occasionally need to configure the behavior of the app:
- Disable animation / pause video;
- Stop sending analytics;
- Turn off React Native LogBox
LogBox.ignoreAllLogs().
For this case you need to add arguments to a launchApp command
- launchApp:
appId: "com.example.app"
arguments:
isE2E: "true"
And then on JS side using react-native-launch-arguments package you can access that arguments
import { LaunchArguments } from "react-native-launch-arguments";
LaunchArguments.value().isE2E // true
3. Dot env file support .env
You need to create .env file with next text:
-e UNIVERSAL_USERNAME=user-e2e@test.com
-e UNIVERSAL_PASSWORD=123qwe
After that you can use that file as argument for test or record commands:
{
"scripts": {
"record": "$HOME/.maestro/bin/maestro record @.env"
"test": "$HOME/.maestro/bin/maestro test @.env"
}
}
Additionally, you can combine various env files (maestro test @.env @.env.prod), in which case all variables from the most recent file will replace, extend, or reset env variables from earlier files.
"test": "$HOME/.maestro/bin/maestro test @.env"
"test-prod": "yarn test @.env.prod"
"test-staging": "yarn test @.env.staging"
Example:
# .env file:
-e BACKEND=staging
-e USERNAME=test@test.com
-e PASSWORD=123qwe
#-----------------
# .env.prod
-e BACKEND= # unset variable
-e USERNAME=myprod@test.com # edit existing
-e SKIP_UNBOARDING=true # add a new variable
The following values will be available in the Maestro environment:
${BACKEND} // null
${USERNAME} // "myprod@test.com"
${PASSWORD} // "123qwe"
${SKIP_UNBOARDING} // "true"
4. Different appId between platforms or environments
It's common practice to differentiate production apps from non-production (staging/beta) apps using a Bundle ID (for example: com.example.myapp & com.example.myapp.staging)
# .maestro/my-test.yaml
appId: ${APP_ID}
name: My test name
---
- launchApp # used an `appId` specified above
Then using -e MY_VAR=myVal you can pass APP_ID in each test:
{
"scripts": {
"test": "$HOME/.maestro/bin/maestro test"
"test-android-staging": "yarn test -e APP_ID=com.example.myapp.staging"
"test-android-prod": "yarn test -e APP_ID=com.example.myapp"
}
}
and finally, an example of a terminal command:
yarn test-android-staging .maestro/my-test.yaml
# ^^^ Run test using staging app
yarn test-android-prod .maestro/my-test.yaml
# ^^^ Run test using prod app
5. Filtering tests to run using tags
You have the ability to run a particular set of tests using tags in Maestro.
Your nightly builds, for instance, frequently verify the following crucial features:
appId: com.example.myapp
name: Sign-in email+password
tags:
- "on:pre_release"
- "on:pull_request"
- "on:nightly_build"
- "feature:auth"
- "backend:prod"
- "backend:pre_prod"
- "backend:staging"
---
- launchApp
# ...
You can use --include-tags="on:nightly_build" and path to your directory with yaml files to run all test that include an on:nightly_build tag:
{
"scripts": {
"test": "$HOME/.maestro/bin/maestro test"
"nightly-test": "yarn test --include-tags='on:nightly_build' ./path_to_flows"
"pr-test": "yarn test --include-tags='on:pull_request' ./path_to_flows"
}
}
6. Complicated waitFor logic
I had the issue that Maestro couldn't wait for the appearance of the "A" or "B" elements.
But using a repeat & runFlow we can simulate this logic:
- evalScript: ${output.myStatus = 'unknown'}
- evalScript: ${output.attemptsCount = 0}
# ^^^ specify initial parameters
- repeat:
while:
true: ${output.myStatus === 'unknown'}
# run until the status changes
commands:
- runFlow:
when:
visible: "Booking failed"
commands:
- evalScript: ${output.myStatus = 'error'}
# ^^^ first `runFlow` wait for an error case
- runFlow:
when:
true: ${output.myStatus === 'unknown'}
commands:
- runFlow:
when:
visible: "Booking success"
commands:
- evalScript: ${output.myStatus = 'success'}
# ^^^ second nested `runFlow` will wait for a success case
# The remaining logic deals with a timeout case
- runFlow:
when:
true: ${output.attemptsCount > 10} # Check attempt limit
commands:
- evalScript: ${output.myStatus = 'timeout'}
- evalScript: ${output.attemptsCount = output.attemptsCount + 1} # Incerunmen an attempt counter
# ^^^ After that you can run any logic based on `output.myStatus`
# for example throw an error if status isn't `success`
- assertTrue: ${output.myStatus === 'success'}
7. Useful bash commands
Android Debug (required to run metro in background)
./android/gradlew assembleDebug -p ./android # build debug apk
find ./android -type f -name "*.apk" # find apk file
yarn start # run metro bundler
adb reverse tcp:8081 tcp:8081 # open port
adb install "<path_to_apk_file>" # `path_to_apk_file` - result of `find` command above
Android Release (bundle JS with apk)
./android/gradlew assembleRelease -p ./android # build debug apk
find ./android -type f -name "*.apk" # find apk file
adb install "<path_to_apk_file>" # `path_to_apk_file` - result of `find` command above
8. Run on CI using GitHub Actions
⚠️ If you want to run Maestro test using GitHub Actions on Android you can't use default ubuntu runners as nested virtualization is disabled (that required by Android Emulator).
See run-on: * table:
| Runners | Android | iOS | Price (min) | Spec |
|---|---|---|---|---|
| large ubuntu | ✅ | ⛔ | from $0.016 to $0.256 | 4-64CPU 15-256GB RAM |
ubuntu-* |
⛔ | ⛔ | $0.008 | 2CPU 7GB RAM |
custom buildjet |
✅ | ⛔ | from $0.004 to $0.048 | 2-34CPU 8-64GB RAM |
macos-*-xl |
✅ | ✅ | 0.32$ | 12CPU ??GB RAM |
macos-* |
✅ | ✅ | 0.08$ | 3CPU 14GB RAM |
Simple Github Actions workflow .github/workflows/mobile-e2e.yml to run Android & iOS e2e test using Maestro:
- no caching
- no versionlock (xcode,java,nodejs,cocoapods)
name: Maestro E2E
on: [push]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true # auto cancel prev. run
env:
MAESTRO_VERSION: 1.27.0
ANDROID_ARCH: x86_64
jobs:
ios_e2e:
name: iOS
runs-on: macos-12 # or macos-12-xl
steps:
- uses: actions/checkout@v3
- name: Installing Maestro
run: curl -Ls "https://get.maestro.mobile.dev" | bash # will use `MAESTRO_VERSION` from env
- name: Installing Maestro dependencies
run: |
brew tap facebook/fb
brew install facebook/fb/idb-companion
- name: Install node_modules
run: yarn install --frozen-lockfile
- name: Install Pods
working-directory: ios
run: pod install
- name: Build app for simulator
working-directory: ios
env:
DERIVED_DATA_PATH: my_build
run: |
xcrun xcodebuild \
-scheme "Myapp" \
-workspace "Myapp.xcworkspace" \
-configuration "Release" \
-sdk "iphonesimulator" \
-destination "generic/platform=iOS Simulator" \
-derivedDataPath "${{ env.DERIVED_DATA_PATH }}"
echo "Print path to *.app file"
find "${{ env.DERIVED_DATA_PATH }}" -type d -name "*.app"
# ^^^ Path to *.app file (based on derivedDataPath + working-directory):
# ./ios/my_build/Build/Products/Release-iphonesimulator/Myapp.app
- name: Run e2e tests
env:
APP_PATH: "./ios/my_build/Build/Products/Release-iphonesimulator/Myapp.app"
# ^^^ change this path to your *.app file
run: |
echo "Launching iOS Simulator"
xcrun simctl boot "iPhone 14 Pro"
echo "Installing app on Simulator"
xcrun simctl install booted "${{ env.APP_PATH }}"
echo "Start video record"
xcrun simctl io booted recordVideo video_record.mov & echo $! > video_record.pid
echo "Running tests with Maestro"
$HOME/.maestro/bin/maestro test .maestro/ --format junit
- name: Stop video record
if: always()
run: kill -SIGINT $(cat video_record.pid)
- name: Store video record
if: always()
uses: actions/upload-artifact@v3
with:
name: e2e_ios_report
path: |
video_record.mov
report.xml
android_e2e:
name: Android
runs-on: macos-12 # or buildjet-4vcpu-ubuntu-2204, ubuntu-22.04-4core, macos-12-xl
steps:
- uses: actions/checkout@v3
- name: Installing Maestro
run: curl -Ls "https://get.maestro.mobile.dev" | bash # will use `MAESTRO_VERSION` from env
- name: Install node_modules
run: yarn install --frozen-lockfile
- name: Build apk for emulator
working-directory: android
run: |
./gradlew assembleRelease --no-daemon -PreactNativeArchitectures=${{ env.ANDROID_ARCH }}
echo "Print path to *.apk file"
find . -type f -name "*.apk"
- name: Install Maestro and run e2e tests
uses: reactivecircus/android-emulator-runner@v2
env:
APK_PATH: ./android/app/build/outputs/apk/release/app-release.apk
# ^^^ change this path to your *.apk file
with:
api-level: 33 # Android 13
arch: ${{ env.ANDROID_ARCH }}
script: |
adb install "${{ env.APK_PATH }}"
$HOME/.maestro/bin/maestro test .maestro/ --format junit
- name: Store tests result
uses: actions/upload-artifact@v3
with:
name: e2e_android_report
path: |
report.xml
Top comments (2)
Very useful. Thanks a lot for the article 🙏
Thanks for the tip on passing environment variables from a .env, super helpful!