QMetry Test Management Plug-in for Jenkins

Robot Framework

Robot Framework result files can be of multiple hierarchy structures. The structure of these files depends upon the libraries that are used to generate the automation results.

Supported Fields

User Defined Fields

Test Case Fields with UDFs in JSON format

{ 
"component":["com1", "com2"],
"priority":"High",
"testCaseState":"Open",
"testCaseType":"Manual",
"testcaseOwner":"lizamathew",
"estimatedTime":60,
"description":"Sample Description", 
"userDefinedFields" : 
 { 
 "reviewedDate": "08-30-2019",
 "executionTime": 40,
 "tagName": "Sample test asset.",
 "additionalComment": "This is a sample test asset"
 "country": ["United States"]"langaugePreference": ["English",Portuguese"] 
 }
}

Test Suite Fields with UDFs in JSON format

{ 
"testSuiteState":"Open",
"testsuiteOwner":"lizamathew",
"description":"Testing 8.5.4 API Changes", 
"userDefinedFields" :
  { 
  "reviewedDate": "08-30-2019",
  "executionTime": 40,
  "tagName": "Sample test asset.",
  "additionalComment": "this is a sample test asset"
  "country": ["United States"]"langaugePreference": ["English",Portuguese"]
  }
}

The following table depicts probable import result according to the scenario.

On successful completion of the process, you can see the console output. It displays success message for uploading the result file.

QTM Cloud

QTM Server

Finally, check test execution page in QMetry and review your test run results

1. On the Add post-build Actions drop-down, select the "Publish test result to QMetry" option.

2. Fill in the required data. You can get more information regarding fields by clicking on the Help icon for respective fields. It explains what to enter in that particular field.

   Parameter	Type	Required	Description
   QMetry URL	string	Yes	QMetry Test Management URL
   Automation API Key	string	Yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
   Proxy URL	string	No	Enter Proxy URL if you have configured any Proxy.
   Automation Framework	string	Yes	Format of result file to be imported. Supported formats: Cucumber TestNG JUnit QAS HP UFT Robot
   Automation Hierarchy	string	No	Hierarchy which will be used to parse test result files on QTM. Supported formats are :- JUnit 1 - Use current Hierarchy in which JUnit Testcase is treated as TestStep and Testsuite is treated as Testcase (default) JUnit File Type 1: Wherein testsuite "name" matches the test case "classname". Test Suite will have test cases as per the <testsuite> tag with steps as per the <testcase> tags. Only one version of test cases will be created with multiple test steps. JUnit File Type 2: Wherein testsuite "name" does not match the test case "classname". Test Suite will have multiple versions of the same test case linked as per the <testsuite> tag. The number of versions will match the occurrences of the <testcase> tag. For each test case version, there will be one test step. 2 - Use Junit Testcase as Testcase and link all those (from all testsuites and all files of Junit) to one Testsuite 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites TestNG 1 - Use class 'name' as Testcase and test-method as TestStep (default) 2 - Use test-method as Testcase 3 - Use test 'name' as Testcase and test-method as TestStep
   Result File(s) Path/Directory	string	Yes	Path wherein your result file to be uploaded. Supported file extensions: .json, .xml and .zip (zip file must contain files of the format given in the 'Automation Framework' param). If your automation generates multiple result files of the supported format in a folder then provide the absolute path till the folder name(It should not be the path till workspace). QMetry plugin will zip all supported format files and upload them to QMetry.
   Test Suite ID	string	No	Key of the test suite exists in QMetry. In the absence of Key, a new test suite will be created. Note: If for Junit - 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then the parameter Test Suite ID will be ignored.
   Test Suite Name	string	No	If existing "testsuiteId" is not being used and auto-generated name for the new test suite is not desired, then a custom test suite name can be entered for this parameter. Note: If for Junit - 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then this parameter will be ignored.
   Test Suite Folder Path	string	No	New test suites will be created in a specified test suite folder while importing automated test results. If the folder does not exist, it gets created. This parameter will be ignored if the test suite is being reused. This parameter is not supported for server edition of QTM now and if specified, it will be ignored.
   Project	string	Yes	ID, Key, Name of the Project within which the test suite will be created.
   Release	string	*No	ID, Name of the Release in which the test execution will be created. If Release is not mentioned, then Default Release will be considered.
   Cycle	string	No	ID, Name of the Cycle in which the test execution will be created. If Cycle is not mentioned, then Default Cycle will be considered.
   Build	string	No	Build Name/Id. In the absence of a value, it will create the default build.
   Platform	string	No	Name of Platform on which test run to be executed.
   Test Case Fields	JSON	No	Mention system defined fields and user defined fields for test case as shown in  Test Case JSON format below. All the mandatory fields other than Summary should be mentioned in this parameter.
   Test Suite Fields	JSON	No	Mention system defined fields and user defined fields for test case as shown in Test Suite JSON format below. All the mandatory fields other than Summary should be mentioned in this parameter. This parameter will be ignored if existing Test suite Id is used to upload results.
   Skip Warning	string	No	Skip Warning while uploading test result file(s) to QMetry. Supported values are: 0 - Test Case Import will be failed if the Test Case summary is more than 255 characters. (default) 1 - Test Cases can be imported by ignoring the warning about summary length. If the test case summary is longer, it will be truncated to 255 characters.
   Is Matching Required	string	No	Create a new test case version or reuse the already created version while uploading test result file(s) to QMetry.  1 - Create a new test case or test case version if the test summary and steps of any of the test case versions do not match with the test case in the result file. (default) 0 - Reuse already linked test case version in test suite or auto link the existing latest version of test case to test suite, if the test case entity key OR test case summary matches with the test case in the result file. The execution status of the linked test cases will be updated. This parameter is not supported for server edition of QTM now and if specified, it will be ignored.

1. To create a new Jenkins job, click New Item option on the Jenkins menu.

   

2. Provide a name for your job.

   Select Pipeline on the screen and click OK.

3. Next, the configuration page appears where you can configure your job by various categories: General, Build Triggers, Advanced Project Options, and Pipeline.

4. Optionally, you can set Build Triggers.

   Select the Build Triggers tab. Let’s just add a periodic trigger which triggers build every minute – every hour – every day (Schedule: * * * * *).

5. Advanced Project Options: This is optional. You can give Display Name which will replace the name you gave earlier to the Pipeline project.

6. Jenkins works as a bridge between QMetry Test Management and automation frameworks. To publish results in QMetry, provide Pipeline script.

   Enter Pipeline script to define your code repository, generate test result and upload test results into QMetry Test Management.

   stage('Upload Result to QTM'){step([$class: 'QTMReportPublisher', disableaction:false, qtmUrl: 'https://newuiqa.qmetry.com/', automationFramework: 'JUNIT', automationHierarchy: '2', testResultFilePath: '/cucumber', qtmAutomationApiKey: '4FR2Oo1g42vUnb5NtDgi3JIjzrFOvUUV8QJBk38D', project : '7242', release: 'Release 1', cycle: 'Cycle 1', buildName: 'testBuild1', platformName: 'Jenkins', testSName: 'junit pipeline', proxyUrl:'', testSuiteName: 'TC2-TS-116', tsFolderPath: 'Fitness Tracker\Track1', testcaseFields: '{"description":"Automated Test case", "testCaseType":"Automated", "testCaseState":"Open", "component":["UI"], "priority":"Blocker", "testcaseOwner":"liza.mathew", "estimatedTime":"143", "userDefinedFields" : {"Integrate" : "Custom Field Testcase"}}',testsuiteFields: '{"description":"Automated Test suite", "testsuiteOwner": "liza.mathew", "testSuiteState": "New", "userDefinedFields": {"Integrate" : "Custom Field Testsuite"}}', skipWarning: '0', isMatchingRequired: '1'])}

   Note

   testSName is "Test Suite Name" and testSuiteName is "Test Suite ID" of the test suite.

   Syntax of automationFramework:

   • CUCUMBER

   • TESTNG

   • JUNIT

   • QAS

   • HPUFT

   • ROBOT

   Parameters

   Refer to the Parameters section for Freestyle Project above.

7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file.

   QTM Cloud

   

   QTM Server

   

8. Finally check test execution page in QMetry and review your test run results.