Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Jenkins is not uploading correct results to TestRail


#1

Hi All,

We are running TestScripts in ranorex from jenkins and I am trying to upload TestResults from Jenkins to TestRail but Jenkins is not uploading correct results. TestResults are uploaded to TestRail as “Passed” even though TestCase fails in Jenkins.
Jenkins Job log

[2019/03/08 03:02:34.633][Failure][Test]: Test Case ‘C166355_Verify_Query_Store_option_is_NOT_selected_by_default_for_2017_and_above_versions’ completed with status ‘Failed’.
[2019/03/08 03:02:34.635][Failure][Test]: Test Suite ‘SQLDMTestAutomation’ completed with status ‘Failed’.
[2019/03/08 03:02:34.745][Debug ][Logger]: Console logger stopping.
Build step ‘Run a Ranorex test suite’ changed build result to FAILURE
Recording test results
Test Cases:
SQLDMTestAutomation: C164220_Verify_default_value_for_Select_top_and_Plans_by
SQLDMTestAutomation: C164221_Verify_invalid_data_in_collect_query_plans_count
SQLDMTestAutomation: C164873_Verify_query_plan_count_options_are_not_enabled
SQLDMTestAutomation: C164222_Select_SQL_Trace_and_verify_query_plans_options_are_not_enabled
SQLDMTestAutomation: C164874_Select_two_2008_version_instances_for_monitoring_and_verify_query_plan_options_are_enabled
SQLDMTestAutomation: C164330_Verify_minimum_and_maximum_limit_of_query_plan_count
SQLDMTestAutomation: C165705_Verify_new_columns_are_added_in_‘MonitoredSQLServers’table_of_SQLdmRepository
SQLDMTestAutomation: C164356_Select_Query_Store_option_for_2017_instance_and_verify_query_wait_data_is_collected
SQLDMTestAutomation: C166356_Select
’Query_Store’_option_for_2017_instance_and_verify_query_wait_data_is_collected
SQLDMTestAutomation: C166355_Verify_Query_Store_option_is_NOT_selected_by_default_for_2017_and_above_versions
SQLDMTestAutomation: C166353_Verify_query_data_default_collection_option_for_different_servers
SQLDMTestAutomation: C164843_verify_Estimated_Query_plan_is_collected_when_globe_icon_of_a_batch_type_SQL_statement_is_se
SQLDMTestAutomation: C170476_verify_Estimated_Query_plan_is_collected_when_globe_icon_of_a_stored_procedure_type_SQL_stat
SQLDMTestAutomation: C170477_verify_Estimated_Query_plan_is_collected_when_globe_icon_of_a_SQL_statement_is_selected
SQLDMTestAutomation: C170478_verify_Estimated_Query_plan_is_NOT_collected_when_globe_icon_of_a_SQL_statement_is_selected
Sample_Flow_for_app_not_opened_closed_every_time_by_TestCases: C164220_Verify_default_value_for_Select_top_and_Plans_by2
Munging test result files.
Scanning C:\Users\ADMINI~1.SIM\AppData\Local\Temp\jenkins7697883967912174327
processing jenkins7697883967912174327
processing current-report.rxlog.junit.xml
Uploading results to TestRail.
Successfully uploaded test results.
Finished: FAILURE

But in TestRail the TestRun result is updated as below

Passed
3/8/2019 5:03 AM
Elapsed1m 7s

This test was marked as ‘Passed’.

Can you guys please help me with this issue.


#2

Hi Naresh,

Thanks for the post! If a test case is being marked as ‘Passed’ this would indicate your automation script is passing a status_id of 1 for the test.

Could you provide the body of the request along with the method used by Jenkins so we can look into this further? If you prefer, you can send this information to contact@gurock.com instead.

Thanks!
Jon


#3

Hi Joy,

Thanks for the update.
I will share details to contact mail, appreciate your help in it.

Regards,
Naresh.