Xray is a commercial test management application that runs directly inside Jira Data Center. It allows quality assurance teams to create test cases, organize testing cycles, record results, import automated test output, and connect failures to Jira defects.
Its biggest advantage is that testing remains inside Jira. Testers, developers, project managers, and product owners can use familiar Jira workflows, issue links, permissions, dashboards, filters, comments, and attachments without maintaining a separate test management platform.
In this blog post, we will review the primary capabilities of Xray Test Management for Jira Data Center and determine where it fits best.
- What Is Xray?
- Xray Issue Types
- Manual Testing
- Automated Testing
- Cucumber Support
- Test Repository and Test Sets
- Test Plans and Test Executions
- Test Runs and Environments
- Requirements Traceability
- Defect Management
- Reporting
- Jira Workflows and Permissions
- Data Center Administration
- Advantages
- Limitations
- Best Use Cases
- Conclusion
- Additional Resources
What Is Xray?
Xray extends Jira Data Center with dedicated issue types and features for managing software testing.
It can be used to:
- Create manual and automated tests.
- Organize tests into reusable collections.
- Plan release and regression testing.
- Execute tests and record results.
- Import results from automated frameworks.
- Link tests to requirements.
- Create or associate Jira defects.
- Track requirements coverage.
- Produce testing reports and dashboards.
Because Xray artifacts are stored as Jira issues, they can participate in the same project-management and security model as other Jira work items.
Xray Issue Types
Xray adds several issue types to Jira Data Center.
Test
A Test represents an individual test case. It may contain manual steps, an automated test definition, or a Cucumber scenario.
Precondition
A Precondition defines something that must be true before a test begins, such as an existing user account, configured test data, or an available service.
Test Set
A Test Set groups related tests. It is useful for organizing tests by feature, component, business process, or testing type.
Test Execution
A Test Execution represents an actual testing cycle performed against a particular build, release, environment, or configuration.
Test Plan
A Test Plan groups the tests and Test Executions associated with a release, sprint, project milestone, or regression effort.
These issue types create a clear structure for defining what must be tested, when it was tested, and what the outcome was.
Manual Testing
Manual Tests contain structured test steps.
Each step can include:
- Action
- Test data
- Expected result
- Attachments
During execution, testers can record:
- Passed or failed steps
- Actual results
- Comments
- Screenshots
- Supporting evidence
- Linked defects
- Overall test status
This is more structured than storing test cases in spreadsheets or documents. It also creates a complete Jira history showing who performed the test, when it was executed, and what happened.
Automated Testing
Xray can import automated test results from common testing frameworks.
Supported formats include:
- JUnit XML
- NUnit XML
- TestNG XML
- Robot Framework XML
- Cucumber JSON
- Xray JSON
Results can be imported through:
- Xray REST APIs
- CI/CD pipelines
- Jenkins jobs
- Build scripts
- Manual uploads
- Custom automation
The automated testing framework still performs the actual test. Xray stores the results and connects them to Jira requirements, Test Executions, Test Plans, environments, and defects.
This provides one location for reviewing both manual and automated testing activity.
Cucumber Support
Xray supports Behavior-Driven Development through Cucumber and Gherkin.
Teams can manage scenarios using terms such as:
- Given
- When
- Then
- And
- Examples
Cucumber scenarios can be associated with Jira requirements and exported for use by automation frameworks. Execution results can then be imported back into Xray.
This is useful for teams that want acceptance criteria, testing scenarios, and automated validation closely connected within Jira.
Test Repository and Test Sets
The Xray Test Repository provides a folder-based structure for organizing Tests and Preconditions within a Jira project.
Tests can be grouped by:
- Application
- Component
- Feature
- Release
- Business function
- Testing phase
Test Sets provide another reusable method for grouping related Tests.
The Test Repository is useful for browsing a large test library, while Test Sets are useful when the same logical group of tests must be referenced in several testing activities.
Test Plans and Test Executions
A Test Plan defines the scope of a testing effort.
For example, a Test Plan might represent:
- A software release
- A regression cycle
- A sprint
- A maintenance update
- A compliance review
A Test Execution represents one actual run of selected tests.
For example, separate Test Executions could be created for:
- Windows and Linux
- Chrome and Firefox
- Staging and production-like environments
- Different application builds
- Manual and automated regression runs
This separation allows teams to reuse the same Test definitions without duplicating test cases for every environment or release.
Test Runs and Environments
A Test Run is created when a Test is included in a Test Execution.
Each Test Run can record its own:
- Status
- Tester
- Execution date
- Step results
- Comments
- Evidence
- Defects
- Environment
Test Environments help distinguish where a test was performed.
Examples include:
- Development
- Staging
- Windows
- Linux
- Chrome
- Firefox
- Mobile
- API
A test may pass in one environment and fail in another. Xray preserves those results separately, providing better visibility into platform-specific or configuration-specific problems.
Requirements Traceability
Requirements traceability is one of Xray’s strongest capabilities.
Jira issue types such as Stories, Epics, Features, or custom requirements can be linked to Tests.
This creates a traceability path such as:
Requirement > Test > Test Run > Defect
Teams can then determine:
- Which requirements have tests.
- Which requirements remain untested.
- Which requirements have failing tests.
- Which defects are associated with failures.
- Whether a release meets its quality requirements.
This is particularly useful for regulated projects, formal release processes, and environments that require evidence showing how requirements were validated.
Defect Management
When a test fails, the tester can create a new Jira defect or link an existing issue.
The defect can include:
- Failed test information
- Test execution details
- Environment
- Comments
- Screenshots
- Supporting evidence
- Steps that failed
Because developers already work in Jira, they can investigate the defect without logging into another platform.
The test remains connected to the defect, making it easier to retest the fix and confirm whether the issue was resolved.
Reporting
Xray includes reports for reviewing testing progress, coverage, and traceability.
Common reporting capabilities include:
- Test execution results
- Test plan progress
- Requirements coverage
- Traceability
- Test runs by environment
- Test evolution
- Overall test status
- Defect associations
Xray information can also be displayed through Jira dashboards, saved filters, and JQL queries.
Reports are helpful, but producing a polished executive dashboard may require:
- Custom Jira filters
- Dashboard gadgets
- Consistent project configuration
- Standardized test statuses
- Additional reporting applications
The quality of the reports depends heavily on how consistently the team maintains its Tests, Test Plans, Executions, environments, and requirement links.
Jira Workflows and Permissions
Since Xray test artifacts are Jira issues, they can use Jira workflows and permissions.
Organizations can create workflows for:
- Draft test cases
- Peer review
- Approved tests
- Deprecated tests
- Planned executions
- Active testing
- Completed testing
Permissions can control who may:
- Create Tests
- Modify test steps
- Execute tests
- View sensitive test evidence
- Create defects
- Manage Test Plans
- Modify Xray settings
This flexibility is useful in enterprise environments, but excessive customization can make the platform more difficult to administer.
A simple, standardized workflow is usually easier to maintain.
Data Center Administration
Xray for Jira Data Center is installed and managed as a Jira application.
Administrators must consider:
- Jira and Xray version compatibility
- Data Center licensing
- Upgrade testing
- Database growth
- Search indexing
- Attachment storage
- Cluster performance
- Backup and recovery
- REST API usage
- Permissions
- Application logs
Large test environments can create significant amounts of data through:
- Test issues
- Test Runs
- Automated result imports
- Attachments
- Screenshots
- Execution history
- Defect links
Xray upgrades should be tested in a staging environment before production deployment, especially when Jira Data Center is also being upgraded.
Advantages
Xray provides several important benefits for Jira Data Center environments:
- Testing remains inside Jira.
- Manual and automated tests can be managed together.
- Requirements, tests, executions, and defects are connected.
- Jira permissions and workflows can be reused.
- Test cases have complete issue histories.
- Cucumber and common automation formats are supported.
- REST APIs support CI/CD integration.
- Tests can be reused across multiple executions.
- Test results can be separated by environment.
- Data remains within the self-managed Jira Data Center environment.
- Developers and testers work from the same platform.
The strongest value comes from reducing the separation between development work and quality assurance activity.
Limitations
Xray also has several drawbacks:
- It adds complexity to Jira administration.
- New users must learn several Xray issue types.
- Poorly designed workflows can create unnecessary overhead.
- Large automated imports can increase database and indexing activity.
- Reports may require additional configuration.
- Data quality depends on consistent team usage.
- Test repositories can become difficult to manage without standards.
- Licensing applies according to the Jira Data Center user tier.
- Major Jira or Xray upgrades require compatibility testing.
- It may be excessive for very small teams with simple testing needs.
Xray is most effective when the organization establishes clear standards for naming, workflows, environments, issue links, and test ownership.
Best Use Cases
Xray for Jira Data Center is a strong fit for organizations that:
- Already use Jira Data Center.
- Need centralized manual and automated testing.
- Require requirements traceability.
- Want test data hosted internally.
- Use formal release-management processes.
- Need testing evidence for audits or compliance.
- Integrate testing with Jenkins or other CI/CD systems.
- Manage multiple environments or product versions.
- Want developers and testers working within Jira.
It may be less suitable for teams that:
- Have only a small number of simple tests.
- Do not need traceability.
- Do not use Jira as the primary development platform.
- Want a lightweight standalone testing tool.
- Lack resources to administer Jira applications.
Conclusion
Xray transforms Jira Data Center from an issue-tracking platform into a more complete software test management system.
It supports manual testing, automated result imports, Cucumber scenarios, reusable test libraries, Test Plans, Test Executions, environment-specific results, requirements coverage, and defect traceability.
Its greatest benefit is keeping development and testing work in one self-managed Jira environment. Organizations that establish consistent workflows, permissions, naming standards, and reporting practices can gain significantly better visibility into software quality and release readiness.