As tech person whom adopted flutter in tech portfolio (got much friction around this). The most question I've got is testing automation engineer did not know dart and appium not officialy support flutter, how can they write test for it, how can they do automate test with it? So I will share context about how we can test flutter application with appium-flutter-driver with a practical example.
Project setup
- Since Flutter did not use native conponents, it's mean you need to define keys in all of your widget to made your widgets visible to appium-flutter-driver (also for normal flutter testing).
Be aware! your testing apk need to be "debug" or "profiling" package.
Example
Text(
'Welcome to Home Page',
key: Key('homeGreetingLbl'),
)
- Add "enableFlutterDriverExtension()" before "runApp()"
void main() {
enableFlutterDriverExtension();
runApp(MyApp());
}
- Add "test" and "flutter-driver" to pubspec.yml
dev_dependencies:
test: any
flutter_test:
sdk: flutter
flutter_driver:
sdk: flutter
My tutorial was extended from TruongSinh Tran-Nguyen's tutorial, Big thank you to him. Installation can follow with his tutorial as below.
appium
/
appium-flutter-driver
Appium Flutter Driver is a test automation tool for Flutter apps on multiple platforms/OSes. Appium Flutter Driver is part of the Appium mobile test automation tool maintained by community
Appium Flutter Driver
Appium Flutter Driver is a test automation tool for Flutter apps on multiple platforms/OSes. Appium Flutter Driver is part of the Appium mobile test automation tool maintained by the community. Feel free to create PRs to fix issues/improve this driver.
Flutter Driver vs Appium Flutter Driver
Even though Flutter comes with superb integration test support, Flutter Driver, it does not fit some specific use cases, such as
- Writing tests in other languages than Dart
- Running integration test for Flutter app with embedded webview or native view, or existing native app with embedded Flutter view
- Running tests on multiple devices simultaneously
- Running integration tests on device farms that offer Appium support (Please contact the availability for each vendor)
Under the hood, Appium Flutter Driver uses the Dart VM Service Protocol with extension ext.flutter.driver, similar to Flutter Driver, to control the Flutter app-under-test (AUT).
Appium drivers for
…Let's test our app with appium-flutter-driver!
I've created demo application for our appium testing in github you can checkout and try
Appium-flutter-driver demo
Flutter example project for appium-flutter-driver.
Project has flutter app project and "appium" directory for testing. In "appium" directory consist of
- package.json - Package which is required by appium-flutter-driver.
{
"name": "appium",
"version": "1.0.0",
"description": "",
"main": "index.js",
"dependencies": {
"appium-flutter-finder": "0.0.13",
"webdriverio": "^5.15.0"
},
"scripts": {
"start": "node test.js",
"ios": "APPIUM_OS=ios npm start",
"android": "APPIUM_OS=android npm start",
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "TruongSinh Tran-Nguyen <i@truongsinh.pro",
"private": true,
"license": "MIT"
}
- test.js - This is main testing file.
const wdio = require('webdriverio');
const assert = require('assert');
const find = require('appium-flutter-finder');
const osSpecificOps = process.env.APPIUM_OS === 'android' ? {
platformName: 'Android',
deviceName: 'emulator-5554',
// @todo support non-unix style path
app: __dirname + '/../build/app/outputs/apk/debug/app-debug.apk',
}: process.env.APPIUM_OS === 'ios' ? {
platformName: 'iOS',
platformVersion: '12.2',
deviceName: 'iPhone X',
noReset: true,
app: __dirname + '/../ios/Runner.zip',
} : {};
const opts = {
port: 4723,
capabilities: {
...osSpecificOps,
automationName: 'Flutter'
}
};
(async () => {
console.log('Initial app testing')
const driver = await wdio.remote(opts);
assert.strictEqual(await driver.execute('flutter:checkHealth'), 'ok');
await driver.execute('flutter:clearTimeline');
await driver.execute('flutter:forceGC');
//Enter login page
await driver.execute('flutter:waitFor', find.byValueKey('loginBtn'));
await driver.elementSendKeys(find.byValueKey('emailTxt'), 'test@gmail.com')
await driver.elementSendKeys(find.byValueKey('passwordTxt'), '123456')
await driver.elementClick(find.byValueKey('loginBtn'));
//Enter home page
await driver.execute('flutter:waitFor', find.byValueKey('homeGreetingLbl'));
assert.strictEqual(await driver.getElementText(find.byValueKey('homeGreetingLbl')), 'Welcome to Home Page');
//Enter Page1
await driver.elementClick(find.byValueKey('page1Btn'));
await driver.execute('flutter:waitFor', find.byValueKey('page1GreetingLbl'));
assert.strictEqual(await driver.getElementText(find.byValueKey('page1GreetingLbl')), 'Page1');
await driver.elementClick(find.byValueKey('page1BackBtn'));
//Enter Page2
await driver.elementClick(find.byValueKey('page2Btn'));
await driver.execute('flutter:waitFor', find.byValueKey('page2GreetingLbl'));
assert.strictEqual(await driver.getElementText(find.byValueKey('page2GreetingLbl')), 'Page2');
await driver.switchContext('NATIVE_APP');
await driver.back();
await driver.switchContext('FLUTTER');
//Logout application
await driver.elementClick(find.byValueKey('logoutBtn'));
driver.deleteSession();
})();
Run Appium (Android)
- Change directory to "appium" folder.
- Start appium server with command "appium" in console.
- Start Android emulator (check your emulator address with "adb devices
Taweechais-MBP:~ taweechaimaklay$ adb devices
List of devices attached
emulator-5554 device
install required nodejs package
npm install
and change Android configuration in test.js") and run command
APPIUM_OS=android npm start
to start appium automation.
Testing result
Automated running application with appium-flutter-driver
Console output as below
Others Appium's command which I've tested.
//Enter value to tartget TextInput
await driver.elementSendKeys(find.byValueKey('telphoneEditTxt'), '1234567890')
//Wait for element visible
await driver.execute('flutter:waitFor', find.byValueKey('page2Btn'));
//Screen scrolling
await driver.execute('flutter:scrollIntoView', find.byValueKey('bottomSubmitBtn'), {alignment: 0.1});
//Using Native command and switch context back to flutter
await driver.switchContext('NATIVE_APP');
driver.back();
await driver.switchContext('FLUTTER');
//Tap on widget element
await driver.elementClick(find.byValueKey('backBtn'));
Conclusion
I did not have strong opinion about appium-flutter-driver is the best way to do flutter app automate testing. But it help QA or automation tester to get familiar with flutter app testing since they can reuse their Appium which already mature to start automate flutter app testing and they can lower their guard about flutter which is very new platform and not much test tooling for it. By the way for flutter developer I think flutter-driver is enough for automate testing (since they already know dart :)). That all from me, comment, question and suggestion is welcome.
Top comments (20)
Is there any ways to automate google Authorization using appium ?
How can i test webview using this. library .
Thanks in Advance\
I also encountered this problem. I want to test a hybrid application of flutter and webview. Have you found a good solution?
Hello Taweechai,
How are you.
Thanks for the blog.
Am trying to set up the test automation for flutter application using Appium,
am new to Appium & flutter, am looking at setting up the testing project , preferably in javascript.
Thanks in advance
(node:74846) UnhandledPromiseRejectionWarning: Error: An unknown server-side error occurred while processing the command. Original error: Cannot execute command waitFor, server reponse {
"isError": true,
"response": "Uncaught extension error while executing waitFor: 'package:flutter_driver/src/extension/extension.dart': Failed assertion: line 373 pos 14: 'WidgetsBinding.instance!.isRootWidgetAttached || !command.requiresRootWidgetAttached': No root widget is attached; have you remembered to call runApp()?\n#0 _AssertionError._doThrowNew (dart:core-patch/errors_patch.dart:46:39)\n#1 _AssertionError._throwNew (dart:core-patch/errors_patch.dart:36:5)\n#2 FlutterDriverExtension.call (package:flutter_driver/src/extension/extension.dart:373:14)\n#3 BindingBase.registerServiceExtension. (package:flutter/src/foundation/binding.dart:597:32)\n\n",
"type": "_extensionType",
"method": "ext.flutter.driver"
}
Getting below error can some one help here
Hey Taweechai, thanks for your tutorial. But, it still a bit difficult for me as a new user to understand the procedure for it.
Can you please create a demo video?
Thank you
Hi Taweechai,
Hope you are doing good.
I'm getting the below issue while running nodejs project.
Node Version: 12.13.1
appium: 1.18.1
2020-09-02T19:43:13.369Z ERROR webdriver: Request failed due to Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
(node:29568) UnhandledPromiseRejectionWarning: Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
(node:29568) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by
rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:29568) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js pro
cess with a non-zero exit code.
Hi ,
Am also facing the same issue..
Did u get the solution for this issue
Can you explain this statement, but in a way a test engineer who does NOT write DART/Flutter code can go check and understand this? My team uses a framework that generates a lot of code, so the explanation assumes a but much of me. Giving controls a key or ID changes the kind of control or is the key "metadata" that the special driver hooks into?
"Since Flutter did not use native conponents, it's mean you need to define keys in all of your widget to made your widgets visible to appium-flutter-driver (also for normal flutter testing)."
I can't catch id, accessibility id, xpath after switch to NATIVE even though runInbackground worked my code await driver.switchContext('NATIVE_APP');
await driver.saveScreenshot('./native-screenshot.png');
driver.background(10)
// await driver.back();
await driver.pause(3000)
await (await await driver.$('//android.widget.ImageView[@content-desc="Project\n Tab 2 of 4"]')).click();
Hi Taweechai,
It was a great explanation. As I am a beginner, I wanted to check can we implement this using java?
Thank you
Sharath
Take a look at the following:
github.com/truongsinh/appium-flutt...
And this article which describes how to use Appium Flutter Finder and Java (use Google Translate browser plugin to convert the language):
cnblogs.com/hbolin/p/12510967.html
I've found JavaScript only for now.
Hey Taweechai, still I am confused how to set up the flutter driver over eclipse, can you please create a demo video of Flutter app automation?
I think you don't need any IDE to do testing setup.
Hey Taweechai, thanks for your tutorial. i couldn't understand if Appium fluuter driver works on real IOS device , not just on emulator or simulator.
is it possible?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.