Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Update Test Progress via the API


#1

Hi

One of the teams we’re migrating to TestRail have many long running automated tests and they need to report progress within the test (Time Elapsed or % of complete) when they are running.

I don’t see a way in the api to update the elapsed time in a test without adding a new result.

I could add an In Progress result which is added multiple times with the elapsed time updated but this is going to create 1000’s of results per run. (if I had a project option to enable if current state is In Progress and we log In Progress via the API you simply update the existing result with new values, that would be really useful - maybe there is a use case this would break but I can’t think of one in our environment)

Do you have a recommended way to add progress while using the api so that progress within a test can be displayed on the Test Run view of Tests? Or somewhere else in TestRail?

Cheers

JG


#2

Hi John,

Thanks for your posting! Yes, you would need to add a result to update a test with elapsed times. How long do your tests run? Do you submit all results in one go or add a result per automated test?

Cheers,
Tobias


#3

Hi Tobias,

Some tests are 72 hours long today. In the future we may have tests that run to failure i.e. the test time is unbounded.

I can probably work around this by using one In Progress result as a start time and make a custom report which uses current time to report time running. Not the most elegant solution as I don’t get the benefits of the TestRail interactive ui but better than swamping TestRail with results…I can’t add a custom field that is derived from another field, right? (MySql has date functions I can use to calculate a RunTime column from Current Timestamp - Start Time column but custom fields don’t appear to have this capability)

An update TestResult feature in the API or a flag to say update last in the add Result method would be much cleaner.

Cheers

JG


#4

Hi John,

Thanks for your additional feedback! We will think about ways to make this easier with a future version and an In Progress status or similar as you mentioned should also work in the meantime. Dependent custom fields are currently not possible unfortunately and we would recommend using regular fields instead (or the built-in elapsed field once your test is completed).

Cheers,
Tobias


#5

Thanks Tobias - I realized over the weekend that In Progress was a state we had added not a built in one so it wasn’t going to have the features I was asking about!!!


#6

Hi John,

Yes, it’s a custom status and TestRail has a similar, built-in feature to mark tests as in-progress via the Start Progress button. This is currently only available via the UI and not the API but we are happy to look into this for a future update.

Thanks again for your feedback!

Cheers,
Tobias


#7

I’d like to +1 on that feature request. We always want to use this feature to increase the accuracy of our forecasts. And with a large automated test suite it would be most helpful to be able to use the in-progress feature via the TestRail API


#8

Thanks for your feedback, Dmitry! It might also make sense to support starting a test run with a status other than Untested for the tests to use a different default (without adding a result first) and we are happy to look into this.

Cheers,
Tobias


#9

I am interested in this as well. I’ll currently use an “In Progress” custom status as suggested, but it would be great to have this supported natively via the API.


#10

Thanks for your feedback, Gavrie!

Cheers,
Tobias


#11

This is a very useful feature for practical test execution.