Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

TestRail API not updating all the test cases results running thru automation


#1

I have been using the API with my automation to publish the test results of the run in testrail. But lately I am seeing that all the test cases results are not being published in test rail. There are test cases which are executed but in test rail they are still marked as “Untested”.

Can someone please point me to a direction where I can investigate more on this? This is very crucial of the QA in my org for reporting.

Thanks.


#2

are you sure to use the right test case ids and run ids ?

I was using the api to update my test plans and this was working fine


#3

Hi!

I can also recommend reviewing the test/case and test run IDs as well as the response of the API requests. If the IDs are not/no longer valid, the API requests will also return a meaningful error message.

I hope this helps!

Cheers,
Tobias


#4

I checked and the runId and test case id are correct but I do see this error in the stacktrace when I execute the test cases-

[main] ERROR _ org.springframework.test.context.TestContextManager - Caught exception while allowing TestExecutionListener
[org.springframework.test.context.support.DependencyInjectionTestExecutionListener@70e889e9] to prepare test instance
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:94)
at org.springframework.test.context.DefaultTestContext.getApplicationContext(DefaultTestContext.java:72)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:212)


#5

Thanks for the additional details! This is actually unrelated to TestRail and looks like a Spring/Java issue. I found the following references and it looks like a configuration issue:

I hope this helps!

Cheers,
Tobias


#6

Thanks I solved the error msg but not the testrail problem.
Can writing the testrail annotation after the junit test annotation make any difference in the result updating?


#7

The annotation order shouldn’t make any difference I believe but I don’t know which tools/libraries or TestRail integration you are using exactly. You can also look into TestRail’s application log if you are sure the case/run etc. IDs are correct and TestRail would log invalid API requests (e.g. invalid arguments) in its application log.

Cheers,
Tobias