Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Custom field default value


#1

I added a custom field that is a dropbox with 4 options having option 1 as the default. I set the field to be mandatory assuming that all existing test cases would get the default value. But that’s not what I see. All existing test cases have no value. Is there a way to set the value on all the existing test cases without editing them 1 by 1?

I’m using TestRail v3.1.2.3142


#2

Hello Gary,

Thanks for your posting! The default value is only for new test cases and is mostly used to pre-select a value for the Add Case form. Newer TestRail versions (starting with 4.0) have a bulk-edit feature, so setting the value for your custom field for many cases would be just a few clicks. Do you maybe have the option to update to a newer TestRail version?

Cheers,
Tobias


#3

Thanks Tobias. I’m the new to the company but know we have plans to upgrade to 4.0 sometime in January or February.

In the mean time, can I use the API to perform a bulk update? Will the API I pulled down work with older versions? It looks like it would be fairly easy to write a simple python script to perform a mass update using the API.


#4

Hello Gary,

Yes, that’s possible. You would need to call the same API method per test case but that would still be faster than doing this manually in the UI of course. The relevant API method would be update_case and here’s the documentation:

http://docs.gurock.com/testrail-api2/reference-cases

The API comes with bindings for various programming languages which are easy to use but please let me know in case you have any questions on how to use the API or the specific API method:

http://docs.gurock.com/testrail-api2/accessing
http://docs.gurock.com/testrail-api2/bindings-python

Please note that we no longer recommend using the old API which you likely downloaded and TestRail >= 3.0 comes with its own integrated API which is easier to use and more powerful (see links above).

Cheers,
Tobias


#5

I’ve written a small script to mass update all test cases so they get the default value. However I’m seeing that the api is really slow. I can access our TestRail site without and delay but API calls in the code take a long time to complete.

>>> start = time.time()
>>> case = client.send_get('get_case/22688')
>>> end = time.time()
>>> print end - start
126.572767973
>>> 

That’s 2 minutes to do 1 get_case! Is that normal? I know it’s impossible to really judge based on how we have TestRails installed, but if the web access is decent, is there any reason the API should be so slow?

I’ve not timed updates or doing a mass get ‘get_cases’ because the singleton get is so slow.


#6

Hello Gary,

Thanks for your posting. The API is usually very fast and you can expect the same or even better response times compared to the UI. This might be an issue independent of TestRail (e.g. a DNS resolution issue). Do you have the option to step in the send_get method or add a few log statements to find out where the time is actually spent?

Cheers,
Tobias


#7

The delay is in the call to ‘urllib2.urlopen’.

I added print statements to testrail.py and tried a few commands check network connectivity. I worried about DNS so I used the machine’s ip address.

traceroute looks good:

gsekinger@ubuntu-gsekinger:~/testrail$ traceroute 192.168.220.6
traceroute to 192.168.220.6 (192.168.220.6), 64 hops max
  1   192.168.10.1  13.120ms  0.896ms  0.475ms 
  2   192.168.220.0  23.520ms  26.475ms  23.872ms 
  3   192.168.220.6  45.937ms  54.951ms  27.396ms 

ping is good too:

gsekinger@ubuntu-gsekinger:~/testrail$ ping -c 10 192.168.220.6
PING 192.168.220.6 (192.168.220.6) 56(84) bytes of data.
64 bytes from 192.168.220.6: icmp_seq=1 ttl=62 time=20.6 ms
64 bytes from 192.168.220.6: icmp_seq=2 ttl=62 time=19.4 ms
64 bytes from 192.168.220.6: icmp_seq=3 ttl=62 time=20.5 ms
64 bytes from 192.168.220.6: icmp_seq=4 ttl=62 time=20.1 ms
64 bytes from 192.168.220.6: icmp_seq=5 ttl=62 time=20.1 ms
64 bytes from 192.168.220.6: icmp_seq=6 ttl=62 time=19.4 ms
64 bytes from 192.168.220.6: icmp_seq=7 ttl=62 time=20.0 ms
64 bytes from 192.168.220.6: icmp_seq=8 ttl=62 time=19.9 ms
64 bytes from 192.168.220.6: icmp_seq=9 ttl=62 time=18.8 ms
64 bytes from 192.168.220.6: icmp_seq=10 ttl=62 time=19.7 ms

--- 192.168.220.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9014ms
rtt min/avg/max/mdev = 18.880/19.899/20.635/0.538 ms
gsekinger@ubuntu-gsekinger:~/testrail$

Improved log messages.

gsekinger@ubuntu-gsekinger:~/testrail$ p api_time.py 
getting cases
2015-12-16 14:56:22.596555: in __send_request
2015-12-16 14:56:22.596635: building request for url(http://192.168.220.6/index.php?/api/v2/get_cases/4&suite_id=27)
2015-12-16 14:56:22.596887: request built
2015-12-16 14:56:22.596979: trying urllib2.urlopen
2015-12-16 14:59:30.366378: urllib2.urlopen back
2015-12-16 14:59:30.366547: trying json.loads
2015-12-16 14:59:30.375349: json.loads back
"get_cases/4&suite_id=27" took 187.778981924 seconds, and returned 173 test cases.
2015-12-16 14:59:30.375620: in __send_request
2015-12-16 14:59:30.375718: building request for url(http://192.168.220.6/index.php?/api/v2/get_case/22656)
2015-12-16 14:59:30.375881: request built
2015-12-16 14:59:30.375961: trying urllib2.urlopen
2015-12-16 15:01:36.926497: urllib2.urlopen back
2015-12-16 15:01:36.926534: trying json.loads
2015-12-16 15:01:36.926639: json.loads back
"get_case/22656"" took 126.551042795 seconds
2015-12-16 15:01:36.926665: in __send_request
2015-12-16 15:01:36.926672: building request for url(http://192.168.220.6/index.php?/api/v2/get_case/23259)
2015-12-16 15:01:36.926730: request built
2015-12-16 15:01:36.926739: trying urllib2.urlopen
2015-12-16 15:02:43.282675: urllib2.urlopen back
2015-12-16 15:02:43.282752: trying json.loads
2015-12-16 15:02:43.282823: json.loads back
"get_case/23259"" took 66.3561849594 seconds
2015-12-16 15:02:43.282863: in __send_request
2015-12-16 15:02:43.282874: building request for url(http://192.168.220.6/index.php?/api/v2/get_case/15739)
2015-12-16 15:02:43.282958: request built
2015-12-16 15:02:43.282970: trying urllib2.urlopen
2015-12-16 15:03:49.291267: urllib2.urlopen back
2015-12-16 15:03:49.291341: trying json.loads
2015-12-16 15:03:49.291438: json.loads back
"get_case/15739"" took 66.0086009502 seconds
2015-12-16 15:03:49.291476: in __send_request
2015-12-16 15:03:49.291486: building request fo url(http://192.168.220.6/index.php?/api/v2/get_case/22657)
2015-12-16 15:03:49.291604: request built
2015-12-16 15:03:49.291620: trying urllib2.urlopen
2015-12-16 15:05:55.993241: urllib2.urlopen back
2015-12-16 15:05:55.993401: trying json.loads
2015-12-16 15:05:55.993656: json.loads back
"get_case/22657"" took 126.702311993 seconds

any suggestion on how I can debug any further?


#8

Hello Gary,

Thanks for the additional details! Could you please check if you see a different behavior with a raw curl request instead of using Python? What happens if you run curl/python from a different machine? The curl instructions can be found here:

http://docs.gurock.com/testrail-api2/accessing#example_request

If you see the same behavior with curl, we can also try to enable debug logging in TestRail to see if the time is spent on TestRail’s side and you can configure debug logging as follows:

http://docs.gurock.com/testrail-faq/system-debug

Cheers,
Tobias


#9

The results are the same. I’ll turn on Testrail debugging to see what that uncovers.

Here’s the curl command I ran:

gsekinger@ubuntu-gsekinger:~/git/test-automation/src/1off$ curl -w "@curl-format.txt" -o /dev/null -H "Content-Type: application/json" -u "*******:******" 'http://192.168.220.6/index.php?/api/v2/get_case/22656' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3604 100 3604 0 0 54 0 0:01:06 0:01:06 --:--:-- 823 time_namelookup: 0.000 time_connect: 0.026 time_appconnect: 0.000 time_pretransfer: 0.026 time_redirect: 0.000 time_starttransfer: 66.642 ---------- time_total: 66.642 gsekinger@ubuntu-gsekinger:~/git/test-automation/src/1off$

Here’s the CURL format file I used:

gsekinger@ubuntu-gsekinger:~/git/test-automation/src/1off$ cat curl-format.txt time_namelookup: %{time_namelookup}\n time_connect: %{time_connect}\n time_appconnect: %{time_appconnect}\n time_pretransfer: %{time_pretransfer}\n time_redirect: %{time_redirect}\n time_starttransfer: %{time_starttransfer}\n ----------\n time_total: %{time_total}\n gsekinger@ubuntu-gsekinger:~/git/test-automation/src/1off$


#10

I know this is an unrelated question but we are running an old version of Testrail and have plans to upgrade to V5 this January. Is it easy to upgrade from TestRail v3.1.2.3142 to V5? I’m wondering if I should just skip all the debugging and wait on V5.


#11

Hi Gary,

Have you already had the chance to look into enabling TestRail’s debug logging as well? It would be great if you could enable debug logging, send a request via the API and then either review the log or send it to contact@gurock.com for us to review.

A few more questions:

  • Do you see the same behavior with other API methods as well (e.g. get_projects)?

  • Do you maybe use LDAP/Active Directory authentication that may be slow or times out? You can review this under Administration > Site Settings > Login. You wouldn’t notice a slow LDAP/Active Directory authentication with regular UI/browser requests because TestRail would only authenticate a user once when you log in and then work with a session cookie. This is different for the API which would run the full authentication for each request. TestRail 4.2 introduced API keys which you can use for authenticating API requests instead: https://blog.gurock.com/testrail-4-2-shortcuts-field-options-api-keys/

Regarding the update: yes, it’s easy to upgrade from 3.x to 5.x and the upgrade documentation can be found here:

http://docs.gurock.com/testrail-admin/installation-upgrading

Updating TestRail basically consists of copying the files from TestRail’s ZIP into your existing TestRail installation directory and then running through the upgrade wizard when you access TestRail with your browser again (which updates the database).

Cheers,
Tobias


#12

Hi Tobias,

I wanted to bring closure to this topic. I installed V5 last weekend and just tried my timed test again. It’s sooo much faster. Where each round trip on V3 was over 60seconds, it’s now sub-second for me on V5.

gsekinger@vb-gsekinger:~/git/test-automation/src/1off$ python api_time.py
getting cases
2016-01-26 09:58:37.358294: in __send_request
2016-01-26 09:58:37.358321: building request for url(https://192.168.80.41/index.php?/api/v2/get_cases/4&suite_id=27)
2016-01-26 09:58:37.358517: request built
2016-01-26 09:58:37.358534: trying urllib2.urlopen
2016-01-26 09:58:37.744748: urllib2.urlopen back
2016-01-26 09:58:37.744889: trying json.loads
2016-01-26 09:58:37.763856: json.loads back
"get_cases/4&suite_id=27" took 0.405694007874 seconds, and returned 180 test cases.
2016-01-26 09:58:37.764025: in __send_request
2016-01-26 09:58:37.764046: building request for url(https://192.168.80.41/index.php?/api/v2/get_case/22656)
2016-01-26 09:58:37.764210: request built
2016-01-26 09:58:37.764231: trying urllib2.urlopen
2016-01-26 09:58:37.970659: urllib2.urlopen back
2016-01-26 09:58:37.970717: trying json.loads
2016-01-26 09:58:37.970862: json.loads back
"get_case/22656"" took 0.206866025925 seconds

Thanks for all the help!

gary


#13

Hello Gary,

Great to hear that :slight_smile: That’s a bit unexpected to be honest because they are no significant performance related changes to the API between 3.x and 5.x (the API has always been fast :wink:) , but this could be caused by something else as a side effect outside of TestRail’s control. Anyway, great to hear that you are now happy with the performance!

Cheers,
Tobias