We frequently encounters the following situation
- The test runs and the XML result file says it's ok (Passed)
- The report in CT shows the result as failed, on steps that are apparenlty not part of his code, and the test is marked as failed in report/reporter, like this one : https://continuoustesting.bnpparibas.com/reporter/html-report/index.html?test_id=229
This poses an issue as many times, we open the access to ContinuousTesting to non-technical people (Business Analysts, for example), and they see the tests as failed when they aren't
Even if I tell them repeatedly that only the Junit/testNG/whatever XML result file is to be taken into consideration, I still have this kind of questions.
Here it happened on a step that exists only in Continuoustesting apparently, but it also happens when we use webdriverwait. on ContinuousTesting, It fails on finding the object once, then finds it and continue.
Is there something in the works regarding this, or any way to mitigate this kind of issues ?
by: Olivier B. | 20 days ago | Tests Execution
Comments
Hi Olivier,
Thank you for your feedback.
We will look into the issue and update you.
Marking it as a candidate at this point.
Michael Abramovich, Product Manager