QMetry Bamboo Integration Instructions
QMetry Bamboo Integration add-on is designed for QMetry Test Management v8.x. QMetry Bamboo Integration is a free add-on. The automation results can also be uploaded from a Bamboo remote agent server.
Download Link: QMetry Bamboo Integration add-on
Once you are done with its installation, follow these steps to set up a simple build Plan.
Note
Release is mandatory if Cycle is mentioned.
Setting Up a Bamboo Build Plan
Open the Run drop-down menu at the top right and select the Run Plan option.
You can see the success message in the logs for the job.
Open the test execution page in QMetry and review your test run results.
Launch Bamboo in a web browser.
Open the Create drop-down menu and select Create a new Plan. If you are using Bamboo for the first time, click Create your first build plan.
The Configure plan page opens.
Provide Project name, Plan name and other details related to the plan. You can even link a code repository with the plan.
Click Configure plan.
Note
Bamboo follows a hierarchical structure consisting of a Plan, Stage, Job, and Task.
This task is associated with the default Job, and the default Job is associated with the default Stage.
Set up a repository for the newly created test plan. Add Source Code Checkout task that uses the repository you created earlier.
Add a Builder Task type. Maven 3.x task is added here.
Click Add task to add more tasks to the plan.
Select task QMetry Bamboo Integration from the All tab.
Configure the QMetry Bamboo Integration task.
If you are importing an XML automation result file using a Custom Automation Import Template, select Custom Template under Automation Framework and enter the template name.
Using a Custom Automation Import Template
In addition to the built-in automation frameworks, the QMetry Bamboo Integration task supports XML automation results using a Custom Automation Template defined in QMetry.
A Custom Automation Template is a reusable, instance-level mapping (stored as JSON) that defines how elements and tags in an XML result file are converted into QMetry test cases, steps, test suites, execution statuses, and other fields. This is useful for custom or unsupported report formats and for mixed (JUnit-style) XML files.
Pre-requisites
Create the template in QMetry under Integration in Custom Automation Template, and note its exact name.
Ensure you have a valid Automation API Key for the project (the same key used for standard automation imports).
Ensure your result file is in XML format.
Configuring the Task to use a Custom Template
In the QMetry Bamboo Integration task configuration, under Automation Framework, select Custom Template.
In the Custom Template Name text box that appears, enter the template name exactly as defined in QMetry.
Provide the required parameters (QMetry Base URL, Automation API Key, Test Result File or Directory Path, Project, and Release.
Save the task and run the plan.
On execution, the plugin validates that the custom template exists, then uploads results using the field mappings defined in the template’s JSON.
Note
The template name must exactly match a template defined in Custom Automation Templates under Integration.
The uploaded file must be in XML format only.
Custom Template selection is supported for both Freestyle and Pipeline projects.
What changes when you use a custom template
When Custom Template is selected, the plugin sets the underlying Automation API entityType to XMLCUSTOM and passes the template name through the customTemplateName property (accepts template name or ID). The following parameter behaviors change:
Parameter | Behavior with a Custom Template |
|---|---|
Automation Hierarchy | Ignored. The template's JSON already defines the parsing hierarchy and field mapping. |
Format | Only XML formats are valid. |
Test Suite Name | Auto-generated if not provided. Ignored if the template maps Test Suite Summary. |
Test Suite Id | Can still be used to link results to an existing Test Suite. |
Parameters
Parameter | Required | Description |
|---|---|---|
Base URL | Yes | QMetry Test Management URL |
Automation API Key | Yes | QMetry API Key. The result is imported for the project associated with the API Key. Go to Integrations. Select Automation API. Click Generate. |
Test Result File Path or Directory Path | Yes | Result file to be uploaded. Example : The Bamboo plugin is expecting the file in BambooHome/xml-data/build-dir/<Project Name> If your project name is TestNGProject, then you will find a folder name with TestNGProject "Key Name" in bamboo home BambooHome/xml-data/build-dir/<Project Name> Your result file should be under BambooHome/xml-data/build-dir/<Project Name> The path that should be set in plugin configuration is "/Testng-result.xml" |
Format | Yes | Supported Formats: Cucumber, TestNG, JUnit, QAS, HP UFT, Robot, XMLCustom. |
customTemplateName | Required only when Format is XMLCUSTOM. | Exact name of the Custom Automation Template defined under Custom Automation Templates. |
Automation Hierarchy (ignored when you use a Custom Automation Template) | No | Hierarchy which used to parse test result files on QTM. Supported formats are: JUnit 1 - Use result file's 'testcase' tag as TestStep and 'testsuite' tag as Testcase (default) JUnit File Type 1: Wherein testsuite "name" matches the test case "classname".
JUnit File Type 2: Wherein testsuite "name" does not match the test case "classname".
2 - Create Single Testsuite and link all Testcases to that Testsuite ('testcase' tag will be treated as Testcase) 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites ('testcase' tag will be treated as Testcase) TestNG 1 - Use result file's 'class' tag as Testcase and 'test-method' tag as TestStep (default) 2 - Use result files 'test-method' tag as Testcase 3 - Use test 'name' as Testcase and test-method as TestStep |
Project | Yes | ID/Key/Name of the Project within which the test suite will be created as mentioned in the Test Suite parameter below. |
Release | *No | ID/Name of the Release in which the test execution will be created. If Release is not mentioned, then Default Release will be considered. |
Cycle | No | ID/Name of the Cycle in which the test execution will be created. If Cycle is not mentioned, then Default Cycle will be considered. |
Build | No | ID/Name of the Build that provides reference to the test run which is to be executed. |
Test Suite | No | Key/ID of the test suite exists in QMetry. In the absence of Key, a new test suite will be created in the following syntax: Automation-<Format>-<Date & Time> e.g. Automation-JUNIT-Mon Mar 12 07:25:22 UTC 2018 Note: If for Junit, Option 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then the parameter Test Suite ID will be ignored. |
Test Suite Name | No | Custom name for a new test suite. Ignored when reusing an existing test suite. For custom templates, ignored when the template maps Test Suite Summary. |
Test Suite Folder Path | No | New test suites will be created in a specified test suite folder while importing automated test results. If the folder does not exist, it gets created. This parameter will be ignored if the test suite is being reused. This parameter is not supported for server edition of QTM now and if specified, it will be ignored. |
Platform | No | Name/ID of Platform on which test run to be executed. If Platform is not mentioned, then the test run will be executed on the default platform named “No Platform”. |
Test Case Fields (JSON Format) | No | Mention system defined fields and user defined fields for test case as shown in Test Case JSON format below. All the mandatory fields other than Summary should be mentioned in this parameter. |
Test Suite Fields (JSON Format) | No | Mention system defined fields and user defined fields for test case as shown in Test Suite JSON format below. All the mandatory fields other than Summary should be mentioned in this parameter. This parameter will be ignored if existing Test suite Id is used to upload results. |
| No | User-defined fields for the test case execution in JSON format. Applied on every import. All mandatory execution UDFs must be provided or the import fails. See Test case execution UDF fields in JSON format below. |
Skip Warning | No | Skip Warning while uploading test result file(s) to QTM. Supported values are: 0 - Test Case Import will be failed if the Test Case summary is more than 255 characters. (default) 1 - Test Cases can be imported by ignoring the warning about summary length. If the test case summary is longer, it will be truncated to 255 characters. |
Is Matching Required | No | Create a new test case version or reuse the already created version while uploading test result file(s) to QMetry. Supported values are: 1 - Create a new test case or test case version if the test summary and steps of any of the test case versions do not match with the test case in the result file. (default) 0 - Reuse already linked test case version in test suite or auto link the existing latest version of test case to test suite, if the test case entity key OR test case summary matches with the test case in the result file. The execution status of the linked test cases will be updated. This parameter is not supported for server edition of QTM now and if specified, it will be ignored. |
Robot Framework
Robot Framework result files can be of multiple hierarchy structures. The structure of these files depends upon the libraries that are used to generate the automation results.
Robot File Tag - Structure & Hierarchy | Tag that is created as QMetry Test Suite | Tag that is created as QMetry Test Case | Tag that is created as QMetry Test Case Step |
|---|---|---|---|
suite>test case>steps | <suite> tag will be created as Test Suite. | <test> tag under the <suite> tag will be considered as Test Case. | <kw> tag under the<test> tag will be considered as Test Case Step. |
suite>(sub/inner) suite>test case>steps | <(sub/inner) suite> tag under the <suite> tag will be considered as single Test Suite. Multiple test suites will be created each (sub/inner) suites. | <test> tag under the <(sub/inner) suite> tag will be considered as Test Case. | <kw> tag under the <test> tag will be considered as Test Case Step. |
suite>test case>steps>sub steps | <suite> tag will be considered as Test Suite | <test> tag under the <suite> tag will be considered as Test Case. | <kw> tag under the <test> tag will be considered as Test Case Step. <kw> tag under the <kw> tag (i.e. sub steps) will be ignored and will not be imported into QMetry. |
Supported Fields for Test Case Fields and Test Suite Fields
Field Name as mentioned in QMetry > Customization > Manage Fields > Field Name.
For Look up list and multi lookup list fields:
If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.
If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it.
If a new value is added to Multi Look Up List, the older value will remain unchanged and the new value will be added to the list.
List values are added to the same version of the test case.
System Fields - Test Case and Test Suite
The system fields are listed in the following table.
Field | Example | Test Case | Test Suite | Notes |
|---|---|---|---|---|
"component" | "component" : ["Label1, "Label2"] | Yes | No | Old values will persist and new value will get set e.g., L1 and L2 are set and L3 is assigned, then the updated values for component will be L1, L2 and L3. |
"priority" | "priority" : "Minor" | Yes | No | As mentioned for Look Up and Multi Look Up lists above. |
"testCaseState" | "testCaseState":"Open" | Yes | No | |
"testCaseType" | "testCaseType":"UI" | Yes | No | |
"testcaseOwner" | "testCaseType":"John" | Yes | No | userAlias should be passed in request. Owner will not be set if the requested user is not found or the user doesn't have right for the Test Case module. |
"description" | "description":"Sample Description." | Yes | Yes | This will be set as requested. |
"estimatedTime" | "estimatedTime":6 | Yes | No | This will be set as requested (in minutes). |
"testSuiteState" | "testSuiteState":"Open" | No | Yes | A new Status value will be added if it does not exist in the project. |
"testsuiteOwner" | "testsuiteOwner":"lizamathew" | No | Yes | userAlias should be passed in request. Owner will not be set if the requested user is not found or the user doesn't have right for the Test Suite module. |
User Defined Fields
The user defined fields are listed in the following table:
Custom Field Type | Validation Rule | Example | Notes |
|---|---|---|---|
STRING | It accepts text in a single line | "tagName": "Sample test asset." | The field value will be set as requested. |
NUMBER | It accepts any number with decimal | "executionTime": 4 | The field value will be set as requested. |
MULTILOOKUPLIST | It accepts comma separated multiple values | "langaugePreference": ["English",Portuguese"] | Multi Lookup will be set as requested if its list value exists.
|
LOOKUPLIST | It accepts a single value on the list | "country": ["United States"] | Lookup type will be set as requested if its list value exists.
|
LARGETEXT | It accepts text in multiple lines | "additionalComment": "this is a sample test asset" | The field value will be set as requested. |
DATEPICKER | It accepts the Date format that has been set in user profile | "reviewedDate": "08-30-2019" | The field value will be set as requested. |
Test Case Fields with UDFs in JSON format
{
"component":["com1", "com2"],
"priority":"High",
"testCaseState":"Open",
"testCaseType":"Manual",
"testcaseOwner":"lizamathew",
"estimatedTime":60,
"description":"Sample Description",
"userDefinedFields" :
{
"reviewedDate": "08-30-2019",
"executionTime": 40,
"tagName": "Sample test asset.",
"additionalComment": "This is a sample test asset"
"country": ["United States"]
"langaugePreference": ["English",Portuguese"]
}
}Test Suite Fields with UDFs in JSON format
{
"testSuiteState":"Open",
"testsuiteOwner":"lizamathew",
"description":"Testing 8.5.4 API Changes",
"userDefinedFields" :
{
"reviewedDate": "08-30-2019",
"executionTime": 40,
"tagName": "Sample test asset.",
"additionalComment": "this is a sample test asset"
"country": ["United States"]
"langaugePreference": ["English",Portuguese"]
}
}Test Run UDF fields in JSON format
Use the testrunudf parameter to set user-defined field values for test case executions at the time of import. These values are applied on every import run. All field names must match the Field Name values configured under Customization > Manage Fields for the Test Case Execution module. Cascading fields are supported.
{
"userDefinedFields": {
"testcaseexecution_singleselectlist": "Open",
"testcaseexecution_largetext":
"Testing API changes.",
"testcaseexecution_string":
"Testing UDF API changes.",
"testcaseexecution_number": 40,
"testcaseexecution_multiselectlist": ["Hindi","English"],
"testcaseexecution_cascading":["US", "Nevada"],
"testcaseexecution_date":"08-30-2019"
}
}Import Behaviour for Test Case Execution UDF values
The following table depicts probable import results according to the scenario.
Is System Field/UDF Mandatory? | Is Auto-Create Enabled? | Does UDF Value exist? | Import Result |
|---|---|---|---|
No | No | No | Fail |
No | No | Yes | Success |
No | Yes | No | Success |
No | Yes | Yes | Success |
Yes | No | No | Fail |
Yes | No | Yes | Success |
Yes | Yes | No | Success |
Note
If you do not provide a value for an optional UDF, the existing value is retained. If you provide a value, the existing value is overwritten.
Open the Run drop-down and select Run plan option.
Log
 |
Review Test Run
Open the test execution page in QMetry and review your test run results.
 |