Cypress is built on top of Mocha and so it gets the mocha's bdd syntax, hooks and mocha reports. In this post we will be discussing about Mocha Reporters. Mocha provides a whole bunch of reporters to choose from. Some of the mocha built-in reporters are spec, dot matrix, nyan and json. These are good but not as awesome as the one we are going to discuss in this post: Mochawesome Reporter.
Mochawesome reporter is a custom reporter which generates a standalone HTML/CSS report to help visualize your test runs. It has simple, clean, and modern design. The report has filters to display only the tests you want and shows stack trace for failed tests.
Challenge in integrating Mochawesome Reporter with Cypress
Starting Cypress v.3.0.0 Cypress executes each spec in isolation and hence a separate test report is generated for each spec. This is a problem because we need one single report for the complete run (which include multiple specs).
Solution: mochawesome-merge can be used to merge these reports and then generate one HTML report for all your cypress tests.
We will be needing the following npm packages and so let's see what each of them does:
cypress-multi-reporters
Generate multiple mocha reports in a single mocha execution.
mochawesome
Mochawesome is a custom reporter for use with the Javascript testing framework, mocha. It runs on Node.js (>=8) and works in conjunction with mochawesome-report-generator to generate a standalone HTML/CSS report to help visualize your test runs.
mochawesome-merge
Merge several Mochawesome JSON reports.
mochawesome-report-generator (marge)
marge (moch*awesome-report-ge*nerator) is the counterpart to mochawesome, a custom reporter for use with the Javascript testing framework, mocha. Marge takes the JSON output from mochawesome and generates a full fledged HTML/CSS report that helps visualize your test suites.
Setup
Step 1: Installation
-
Install Mocha
npm install mocha --save-dev -
Install cypress-multi-reporters
npm install cypress-multi-reporters --save-dev -
Install mochawesome
npm install mochawesome --save-dev -
Install mochawesome-merge
npm install mochawesome-merge --save-dev -
Install mochawesome-report-generator
npm install mochawesome-report-generator --save-dev
Step 2: Add reporter settings in cypress.json
"reporter": "cypress-multi-reporters",
"reporterOptions": {
"reporterEnabled": "mochawesome",
"mochawesomeReporterOptions": {
"reportDir": "cypress/reports/mocha",
"quite": true,
"overwrite": false,
"html": false,
"json": true
}
}
Step 3: Add scripts in package.json file
For Windows:
"scripts": {
"clean:reports": "rmdir /S /Q cypress\\reports && mkdir cypress\\reports
&& mkdir cypress\\reports\\mochareports",
"pretest": "npm run clean:reports",
"scripts": "cypress run",
"combine-reports": "mochawesome-merge
cypress/reports/mocha/*.json > cypress/reports/mochareports/report.json",
"generate-report": "marge cypress/reports/mochareports/
report.json -f report -o cypress/reports/mochareports",
"posttest": "npm run combine-reports && npm run generate-report",
"test" : "npm run scripts || npm run posttest"
}
For Mac:
"scripts": {
"clean:reports": "rm -R -f cypress/reports && mkdir cypress/reports
&& mkdir cypress/reports/mochareports",
"pretest": "npm run clean:reports",
"scripts": "cypress run",
"combine-reports": "mochawesome-merge
cypress/reports/mocha/*.json > cypress/reports/mochareports/report.json",
"generate-report": "marge cypress/reports/mochareports/
report.json -f report -o cypress/reports/mochareports",
"posttest": "npm run combine-reports && npm run generate-report",
"test" : "npm run scripts || npm run posttest"
}
pretest script would create the report folders and clear them if they already exist
test script would do the following:
a. run your test suite
b. create 'mocha' folder under 'cypress/reports'
c. create .json files (one for each spec executed) in the 'mocha' folder
posttest script would combine all the .json files present in 'cypress/reports/mocha' folder and place the combined file 'report.json' in 'cypress/reports/mochareports' and create a beautiful html report.
Execution
It's time to run test suites and view report.
npm run test
Pre and post scripts are automatically executed before and after the main script execution.
You can find a sample project here: https://github.com/bushralam/Cypress
Oldest comments (46)
Updated with cy-report-setup-helper : dev.to/cssoldiervoif/cypress-mocha... :)
This is an overkill. There is a simple and easy one step solution to solve this problem.
Its not working for windows with your updated code for windows
Getting error for code: rmdir /S /Q cypress\reports
(This is no more valid in windows 10)
Second error:
ERROR: Failed to merge reports
Error: Pattern --reportDir matched no report files
at C:\Users\anandteerth.onkar\Cypress-Training\node_modules\mochawesome-merge\lib\index.js:14:11
at Array.map ()
at C:\Users\anandteerth.onkar\Cypress-Training\node_modules\mochawesome-merge\lib\utils.js:3:46
at merge (C:\Users\anandteerth.onkar\Cypress-Training\node_modules\mochawesome-merge\lib\index.js:59:17)
at Object. (C:\Users\anandteerth.onkar\Cypress-Training\node_modules\mochawesome-merge\bin\mochawesome-merge.js:5:1)
at Module._compile (internal/modules/cjs/loader.js:955:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
at Module.load (internal/modules/cjs/loader.js:811:32)
at Function.Module._load (internal/modules/cjs/loader.js:723:14)
at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
It would be great if you could make an updated vid tut on the mochawesome reporter.
After resolving problem 1, I faced some other issues, mentioned in link:
github.com/cypress-io/cypress/issu...
There's some issue with clean:reports for windows (not sure about mac). If the directory is empty or not present there, the script fails.
For windows, please instead of using this below,
"clean:reports": "rmdir /S /Q cypress\reports && mkdir cypress\reports
&& mkdir cypress\reports\mochareports",
use this instead:
"clean:reports": "IF EXIST cypress\reports (rmdir /S /Q cypress\reports && mkdir cypress\reports && mkdir cypress\reports\mocha && mkdir cypress\reports\mochareports\) ELSE (mkdir cypress\reports && mkdir cypress\reports\mocha && mkdir cypress\reports\mochareports\)",
So what does it do?
It checks for the dir Reports. IF its there it deletes it with all contents and then creates a new one with sub folders. If its not there (ELSE), then it creates a new one with sub folders.
clean: reports is throwing an error for me
npm run clean:reports
The system cannot find the file specified.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! cypress-test@1.0.0 clean:reports:
rmdir /S /Q cypress\reports && mkdir cypress\reports && mkdir cypress\reports\mochareportsnpm ERR! Exit status 2
npm ERR! Failed at the cypress-test@1.0.0 clean:reports script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\mehtap\AppData\Roaming\npm-cache_logs\2020-05-09T07_53_17_751Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! cypress-test@1.0.0 pretest:
npm run clean:reportsnpm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the cypress-test@1.0.0 pretest script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\mehtap\AppData\Roaming\npm-cache_logs\2020-05-09T07_53_17_825Z-debug.log
Thanks for writing this article. In my case, after executing cypress run, it prompts an npm err! and exits which doesn't allow the scripts after cypress run. So, I am unable to proceed with combining and merging reports. Did you come across such an error or do you have any workaround for such a scenario??
Hi Mitesh,
What error do you get? Is it same as mentioned here: dev.to/turner749/comment/oj8a
If so.. please follow the solution mentioned.
Hi Bushra,
Thanks for responding. I was able to create and merge reports as per the suggestion you mentioned above. But after execution of command cypress run, it generates the test results but exits with npm err.
As I want the scripts for combine and merge reports to execute directly after test execution which is not happening in my case.
So, I am following this post github.com/cypress-io/cypress/issu... which talks about configuring cypress run. While doing that I need to target "after:test:run" event as mentioned in official docs but the event doesnot trigger after all tests are executed. Now I am blocked at this point. Did you come accross such an issue? I would appreciate if you could suggest me on this.
Hi Bushra,
I was able to resolve the npm err issue by writing a custom cypress runner and successfully combine and generate html report. I followed this links:
github.com/nottyo/cypress-e2e-code...
github.com/Antontelesh/mochawesome...
Anyone who is stuck with the npm error post ** cypress run ** can follow the above links.
Thanks for the excellent article. But I have a problem, when my test fails (an error occurs) the report is not generated, how could I adjust this problem? And already taking advantage, how would you do to attach the screenshots in the report?
Could you please share the error screenshot ?
Thank you for this wonderful article. just a quick question, How does pretest getting executed? As posttest script is referenced in "test" script, but pretest not referenced anywhere, but still its cleaning the report folder and creating. if you could clarify how its getting executed, will be a great help.
Thanks in advance.
Hi
Well.. that's the advantage of using pre and post scripts.. they are called automatically before and after the main script. So when we run the script 'test'.. the script 'pretest' would automatically run before it and the script 'posttest' would automatically run after script 'test' has finished execution.
I have also explained that in the video that is embedded in the post.
Hello people, for those having issues with the posttest step and the combine-reports npm script I have a fix.
It looks like the newer versions of mochawesome-merge removed the --reportDIr.
I had to replace the combine-reports script slightly.
Inside package.json:
"combine-reports": "mochawesome-merge ./cypress/reports/mocha/*.json > cypress/reports/mochareports/report.json",
Notice the reportDir is gone and instead its a file directory starting with "." and ending with "*.json"
Loved the tutorial it was super helpful, but I did get caught up on this issue for awhile. Hopefully you'll be able to update this because this guide is incredibly helpful! Thanks!
Thank you Alexander.. I have updated the post.
I am sorry.. I should have done that long ago.
Thanks again.
@bushraalam You missed the
./in your update. I should bemochawesome-merge ./cypress, notmochawesome-merge cypressHello Bushra Alam,
Thank you sharing this wonderful know i have added all required things which you mention in this blog but i get following error message after running :npm run test Please kindly suggest regarding this issue;
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\shahmed\AppData\Roaming\npm-cache_logs\2020-05-08T11_46_56_279Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 145
npm ERR! cypress24@1.0.0 pretest:
npm run clean:reportsnpm ERR! Exit status 145
npm ERR!
npm ERR! Failed at the cypress24@1.0.0 pretest script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\shahmed\AppData\Roaming\npm-cache_logs\2020-05-08T11_46_56_370Z-debug.log
Hi Shakeel,
Could you try one of the solutions mentioned here: stackoverflow.com/q/22948189
Let me know what worked for you and I will update the post accordingly.
Thanks.
I added my solution to this issue. It works for me 100% Hopefully it works for everyone else. Thanks for the helpful article.
Hi Bushra
My mocha report was generated But it generated report for only 1 spec despite have about 10 specs within my integration folder.
I executed this command
"test" : "npm run scripts || npm run posttest"
My test execution shows 10 specs passed but only 1 spec report generated.
How can this be resolved.
Thank you Bushra for this amazing post! It really is excellently put together and saved me a ton of time. Much appreciated!
Nice Article. It saved lot of time. Thank You!
Hi,
I'm getting this error when trying to generate reports. Any help appreciated. Thank you.
ERROR: Failed to merge reports
Error: Directory C:\repo\mochawesome-report does not exist
at fse.readdir.catch (C:\repos\drug-data-pipeline-testing\node_modules\mochawesome-merge\lib\index.js:33:11)
I've followed this discussion with interest. I too had issues whereby I was getting errors when generating mochawesome reports. I also wanted to archive generated reports, labelling these into folders with current date and time.
I have ended up with this package.json config. Feel free to take and use as required :
Thanks for the scripts. I am using above scripts to run and I was able to run my tests successfully but after the test execution I am getting below error at combine-reports. I see there is only reports/mocha/mochawesome.json created.
The system cannot find the path specified.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! combine-reports:
mochawesome-merge cypress/reports/mocha/*.json > cypress/reports/mochareports/report.jsonnpm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the combine-reports script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
@Srihari Dama I got the same error.. could you able to fix it?
ERROR: Failed to merge reports - Mochawesome
I've installed the Mochawesome reporter and all the needed plugins. Whenever I try to run a folder it keeps giving me an ERROR that Cypress failed to merge the reports. It looks like that the script does not create a file in the folder mocha.
It only creates a report.json in Mochareports.
image
this is a part of my package.json.
{
"scripts": {
"cypress:open": "npm install && cypress open",
"cypress:open:local": "npm install && cypress open --env configFile=local",
"cypress:open:dev": "npm install && cypress open --env configFile=dev",
"cypress:open:tst": "npm install && cypress open --env configFile=tst",
"cypress:open:acc": "npm install && cypress open --env configFile=acc",
"cypress:run": "npm install && cypress run",
"cypress:run:local": "npm install && cypress run --browser chrome --headless --env configFile=local",
"cypress:run:tst": "npm install && cypress run --browser chrome --headless --env configFile=tst",
"cypress:run:acc": "npm install && cypress run --browser chrome --headless --env configFile=acc",
"cypress:run:acc:teampark": "npm install && cypress run --browser chrome --headless --env configFile=acc --spec cypress/integration/team_park.js --config video=false",
"clean:reports": "(if exist cypress\reports (rmdir /S /Q cypress\reports)) && mkdir cypress\reports && mkdir cypress\reports\mocha && mkdir cypress\reports\mochareports",
"pretest": "npm run clean:reports",
"scripts": "cypress run --spec cypress/integration/website/schooltrip/**/",
"combine-reports": "mochawesome-merge cypress/reports/mocha/.json > cypress/reports/report.json",
"generate-report": "marge cypress/reports/mochareports/report.json -f report -o cypress/reports/mochareports",
"posttest": "npm run combine-reports && npm run generate-report",
"test": "npm run scripts || npm run posttest"
Desired behavior:
What cypress needs to do according to all online explanation is to create a file in the Mocha and Mochareports and merge these files in order to get an HTML report.
Error given after a run:
@ posttest C:\Users\iessadeq\Desktop***l-cypress-tests-master
npm run combine-reports && npm run generate-report
@ combine-reports C:\Users\iessadeq\Desktop*******-cypress-tests-master
mochawesome-merge cypress/reports/mocha/.json > cypress/reports/report.json
ERROR: Failed to merge reports
Error: Pattern cypress/reports/mocha/*.json matched no report files
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ combine-reports: mochawesome-merge cypress/reports/mocha/*.json > cypress/reports/report.json
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ combine-reports script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\iessadeq\AppData\Roaming\npm-cache_logs\2020-07-10T13_54_21_097Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ posttest: npm run combine-reports && npm run generate-report
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ posttest script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\iessadeq\AppData\Roaming\npm-cache_logs\2020-07-10T13_54_21_243Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ test: npm run scripts || npm run posttest
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\iessadeq\AppData\Roaming\npm-cache_logs\2020-07-10T13_54_21_386Z-debug.log
Test code to reproduce
"reporter": "cypress-multi-reporters",
"reporterOptions": {
"reporterEnabled": "mochawesome",
"mochawesomeReporterOptions": {
"reportDir": "cypress/reports/mocha",
"quite": true,
"overwrite": false,
"html": false,
"json": true
}
I've tried several things without any succes.. Could anyone help please?
Versions
"devDependencies": {
"chance": "^1.1.0",
"cypress": "^4.7.0",
"cypress-file-upload": "^3.5.3",
"cypress-multi-reporters": "^1.4.0",
"lodash": "^4.17.15",
"mocha": "^8.0.1",
"mocha-junit-reporter": "^1.23.3",
"mochawesome": "^6.1.1",
"mochawesome-merge": "^4.1.0",
"mochawesome-report-generator": "^5.1.0"
Thanks for the excellent article. But I have a problem, when my test fails (an error occurs) the report is not generated, how could I adjust this problem? And already taking advantage, how would you do to attach the screenshots in the report?
So for anyone having trouble with the Windows install. Once you complete the installation as detailed in Bushra's post, you will notice you don't have the report folder in your directory which is causing all those issues. Please follow these steps.
After your test run in browser it will generate the reports folder and any mocha dependencies you are missing.
You can now run the "npm test run" and it will work as expected.
In closing, there are some dependencies missing initially in the windows set up that need to be executed at least once via in the browser. Once this happens the expected directories/dependencies will be available. -Cheers
Some comments may only be visible to logged-in visitors. Sign in to view all comments.