Test using TestNG

TestNG is a testing framework designed to simplify a broad range of testing needs, from unit testing (testing a class in isolation of the others) to integration testing (testing entire systems made of several classes, several packages and even several external frameworks, such as application servers).

Here is how you can specify the Test Cases and Test Steps which will be created in QMetry as a part of the result files. Automation Hierarchy to be passed as automationHierarchy parameter in Automation API.

Automation Hierarchy	Test Case	Test Step Tag
1 (default)	<class> tag under the <test> tag will be considered as Test Case. 'name' attribute of <class> tag will used as testcase summary.	<test-method> tag under <class> tag (with attribute "is-config" not set as true) will be considered as Test Case Step. 'name' attribute of <test-method> tag will be used as Step Description.
2	<test-method> tag under <class> tag (with attribute "is-config" not set as true) will be considered as Test Case. 'name' attribute of <test-method> tag will used as testcase summary.	Test Case Step will not be created.
3	<test> tag under the <suite> tag will be considered as Testcase. 'name' attribute of <test> tag will be used as testcase summary.	<test-method> tag under <class> tag (with attribute "is-config" not set as true) will be considered as Testcase step. 'name' attribute of <test-method> tag will be used as testcase step description.

Automation Hierarchy

Test Case

Test Step Tag

1 (default)

<class> tag under the <test> tag will be considered as Test Case.

'name' attribute of <class> tag will used as testcase summary.

<test-method> tag under <class> tag (with attribute "is-config" not set as true) will be considered as Test Case Step.

'name' attribute of <test-method> tag will be used as Step Description.

<test-method> tag under <class> tag (with attribute "is-config" not set as true) will be considered as Test Case.

'name' attribute of <test-method> tag will used as testcase summary.

Test Case Step will not be created.

<test> tag under the <suite> tag will be considered as Testcase.

'name' attribute of <test> tag will be used as testcase summary.

<test-method> tag under <class> tag (with attribute "is-config" not set as true) will be considered as Testcase step.

'name' attribute of <test-method> tag will be used as testcase step description.

Supported Version: 1.2.5

Supported file types : XML

Automation Hierarchy 1

Automation Hierarchy 2

Automation Hierarchy 3

Note

“Skip” status in the automation result file will be applied as "Not Applicable" instead of "Not Run" for automation test executions upload into QMetry.

Reuse Manual Test Cases

No.	Methods	Applicable to Automation Hierarchy	Description
1.	Summary	1,2,3 Download Sample File	If Test Case Summary in QMetry matches with Test Case Tag as per the Automation Hierarchy (1 to 3) described above, the same test case in QMetry will be updated. If the Test Case Summary matches, but Step Description in QMetry does not match with Test Step Tag as per the Automation Hierarchy (1 to 3) described above, then new version of the same test case will be created.
2.	Keyword @Test(groups = {"testEntityKey"})	1,2, 3 Download Sample File	When a valid entity key of the test case (i.e. testEntityKey) is passed through the result XML file and the executed test steps match with the steps and summary of any version of the existing test case, then the existing test case version will be reused. When a valid entity key of the test case is passed through the result XML file and the executed test steps do not match with the steps of any version of the existing test case, then a new version of the test case will be created. When an invalid/archived test case entity is passed through the result XML file: If the executed test steps match with the steps and summary of any version of the existing test case, then the existing test case version will be reused. If the executed test steps do not match with the steps and summary of any version of the existing test case, then a new test case will be created using that <class>. When the latest version of test case is archived and older versions are not archived: If the executed test steps match with the steps and summary of any unarchived version of the existing test case, then the existing unarchived test case version will be reused. If the executed test steps do not match with the steps and summary of any unarchived version of the existing test case, then a new test case will be created and linked to the test suite. Execution Status will also be updated. If a manual test case includes the manual steps, which are same as test-methods (using sharable test case), then on uploading test result using API, a new version will be created.

No.

Methods

Applicable to

Automation Hierarchy

Description

Summary

1,2,3

Download Sample File

If Test Case Summary in QMetry matches with Test Case Tag as per the Automation Hierarchy (1 to 3) described above, the same test case in QMetry will be updated.
If the Test Case Summary matches, but Step Description in QMetry does not match with Test Step Tag as per the Automation Hierarchy (1 to 3) described above, then new version of the same test case will be created.

Keyword

@Test(groups = {"testEntityKey"})

1,2, 3

Download Sample File

When a valid entity key of the test case (i.e. testEntityKey) is passed through the result XML file and the executed test steps match with the steps and summary of any version of the existing test case, then the existing test case version will be reused.
When a valid entity key of the test case is passed through the result XML file and the executed test steps do not match with the steps of any version of the existing test case, then a new version of the test case will be created.
When an invalid/archived test case entity is passed through the result XML file:
- If the executed test steps match with the steps and summary of any version of the existing test case, then the existing test case version will be reused.
- If the executed test steps do not match with the steps and summary of any version of the existing test case, then a new test case will be created using that <class>.
When the latest version of test case is archived and older versions are not archived:
- If the executed test steps match with the steps and summary of any unarchived version of the existing test case, then the existing unarchived test case version will be reused.
- If the executed test steps do not match with the steps and summary of any unarchived version of the existing test case, then a new test case will be created and linked to the test suite. Execution Status will also be updated.
If a manual test case includes the manual steps, which are same as test-methods (using sharable test case), then on uploading test result using API, a new version will be created.

Reuse Automation Test Cases

Here we anticipate that the file is imported in QMetry at least once. Now when the file is imported again -

If Test Case Summary and Step Description match: The same test case and its version will be reused and updated in QMetry.
If the Test Case Summary matches, but Step Description in QMetry does not match: A new version of test case will be created.

Link Requirements to Execution

Keyword	Applicable to Automation Hierarchy	Description
@Test(groups = {"requirementEntityKey"})	1, 2, 3 Download Sample File	When a valid requirement entity key (for internal/external requirement) is passed through the result xml file, the test case which is created using <class> is linked to the requirement. When an invalid/archived requirement entity key is passed through result xml file, no requirement will be linked to the test case which is created using <class>. When a requirement (latest version of which is archived but older version is not archived) entity key is passed through result xml file, the test case which is created using <class> is linked to the requirement's latest unarchived version. Multiple requirementEntityKey can be used in the same tag in the file to link multiple requirements to the test cases. e.g. <group name="requirementEntityKey:FIT-RQ-1 FIT-RQ-2"> If multiple test case keys are mentioned, then the multiple requirements will be linked individually with the test cases.

Keyword

Applicable to

Automation Hierarchy

Description

@Test(groups = {"requirementEntityKey"})

1, 2, 3

Download Sample File

When a valid requirement entity key (for internal/external requirement) is passed through the result xml file, the test case which is created using <class> is linked to the requirement.
When an invalid/archived requirement entity key is passed through result xml file, no requirement will be linked to the test case which is created using <class>.
When a requirement (latest version of which is archived but older version is not archived) entity key is passed through result xml file, the test case which is created using <class> is linked to the requirement's latest unarchived version.
Multiple requirementEntityKey can be used in the same tag in the file to link multiple requirements to the test cases. e.g.

<group name="requirementEntityKey:FIT-RQ-1 FIT-RQ-2">

If multiple test case keys are mentioned, then the multiple requirements will be linked individually with the test cases.

In this section:

Test using TestNG

Note

Reuse Manual Test Cases

Reuse Automation Test Cases

Link Requirements to Execution

Search results