Subscribe Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security. Subscribe

How to add test result for javascript?

well i am new in testrail,
we are using cypress framework, i am trying to figure it out how i can update test result in testrail ?
i am able to see the test has been ran , but the result is not updated,
http://docs.gurock.com/testrail-api2/start#add_result_for_case , not sure how can i use in my framework,

thanks

Hi @nik,
what have you already done to integrate your framework with TestRail? I guess you do not expect to get TestRail updated automatically by magic.

Usually you define what you want to test in TestRail (TestPlan and/or TestRuns) for given test cases and enter your results manually or use the API to upload/update results with a reporter or e.g. a Jenkins-job.

Within the forum you will find some posts about the integration of different automation tools/frameworks.

E.g. for Cypres you can check here. Just found by via google search. :innocent:

Don’t hesitate to ask for further help…

Karsten

i am using https://www.npmjs.com/package/cypress-testrail-reporter, and i can see test run is updating in test rail,
the problem i am facing is - it’s not updating test results (it’s always untested).
as you mentioned API to upload/update with the reporter, can you guide me a bit how can i upload the result?
i was expecting reporter is enough for updating the result on testrail as well,
of course, the test result will not update in testrail by magic :slightly_smiling_face:

@nik,
I’m not using Cypress so I’m not really in touch with the reporter itself.
I took a quick look at the code and I’m not excited. As far as I understand, it will create a new run for every single test result, what is a real bad idea (OK, IMHO). Some posts on Stackoverflow also discuss this point, there is also another reporter mentioned.

So, if you get a new testrun for every test execution posted via the reporter, it is working as expected. It won’t update existing test results in existing runs.

If you even don’t get new runs, there is an issue with your config or usage.

Let’s check some basics, please excuse if they may sound silly…

  • Do you provide the correct combination of case id and suite to publish the result?
  • Did you check the callback function is used to indicate an error? (Not sure how this looks like.)
  • Did you create the test run and select the cases manually?

For the integration of test automation you will usually add results and never update results.
Either you have a defined set of tests (selected test cases from same same suite as a kind of instance) as a test run OR you create a run with tests on the fly. For both you add results to the tests later on.

So, I hope you get some ideas…

Karsten

thank you for the prompt reply,
so test execution posted via the reporter.
what actually i am trying to do is
i want to run the automation test, and want to see the result on testrail, it seems it is not possible with the reporter.
do i need some other tool to integrate?

Hi @nik,
every solution/tool is based on the usage of the TestRail API.
AS mentioned above there is a second reporter, which is using another paramter to get the test run passed. I’m not sure if there are other public solutions available for the integration with cypress.

But first you have to make your requirements clear. The discussed reporting will bring the results to TestRail, so your requirement is fulfilled, be happy. :grinning:

But I guess, this won’t help you. Let’s assume you have a daily test execution of 100 tests. This might lead into 365x100 test runs having one test with one result in. Usually you get the request to post the automation results to TestRail for reporting reasons, try to report across these 36.500 runs. :innocent:

Basic: You need to have a case for every automated test in TestRail.
Do you plan to structure them in different suites?

Where do you define which cases have to be executed (or at least needs to have a result in TestRail)? You need to have a test run with selected cases to add a result to. Who is in charge to create this run and the selection of the test cases (you can do so before execution or probably on the fly with a special logic).

How do you want to structure your test execution?

  • Having a run per single test execution (like with the reporter).
  • Having a run for the daily execution of your collection of tests. Created automatically or manual, as part of a testplan (indicating a structure like releases or versions) or not.

Do you plan to add more information to the results than the status? How do you plan to a solution to add additional information of e.g. custom fields…?

Not sure if there is a solution available for your needs beeding flexible enough, I bet it isn’t. :thinking:

I wasn’t able to find a public available solution for my (well, our) needs to integrate the results from a JUnit&Selenium based test framework as part of a Jenkins pipe. So I implemented my own solution for it, using a Java implementation using the TestRail API.
[No, it’s not public.] :yum: