CT-TAS — Test Automation Strategy glossary

175 terms the exam expects you to know, defined in plain language.

A

Alignment of Metrics with Test Automation Strategy: The principle that metrics should support the objectives of the test automation strategy, such as reducing manual effort or improving coverage.
Automated Test Maintenance(maintenance, automated test maintenance, test maintenance): The effort required to update automated tests when the application under test changes.
Automated test maintenance effort(maintenance effort): The effort required to update test scripts when the application under test changes.
Automation Architecture Flexibility(flexible automation architecture, modular automation architecture): The need for a modular automation design that adapts to different SDLC workflows, especially in hybrid models.
Automation Center of Excellence(center of excellence, automation team): A team or group that shares best practices and provides support for test automation across an organization.
Automation Coverage(automation coverage metric, automation coverage, test coverage, coverage, test automation coverage): A metric representing the percentage of tests automated within a test suite.
Automation Feasibility Assessment(feasibility assessment, technical feasibility): Evaluation of technical feasibility including testability, tool support, and team skills before committing to automation.
Automation Framework Validation(framework validation, validate automation framework): The process of confirming that the chosen automation framework meets project needs through pilot testing.
Automation Health Checks(automation health checks): Regular reviews of automation to identify bottlenecks, opportunities, and areas for improvement.
Automation Lifecycle(automation lifecycle, test automation lifecycle): The period over which automation costs and savings are evaluated, typically multiple releases.
Automation Lifecycle ROI Evaluation(lifecycle roi evaluation, lifecycle roi in test automation, lifecycle roi): The assessment of automation return on investment over the entire lifecycle, not just initial implementation.
Automation Metrics(automation metrics): Quantitative measures such as defect detection rate, execution time, maintenance effort, and ROI used to evaluate automation.
Automation Outcome Metrics(metrics that link automation outcomes to business value, automation metrics linked to business value, business value metrics, automation outcome metrics, outcome metrics): Measures linking automation results to business value, such as percentage of critical tests automated or reduction in manual effort.
Automation ROI(automation roi): The return on investment from test automation, used as a metric to evaluate effectiveness.
Automation Skills Upskilling(upskilling manual testers, training): Training manual testers in automation tools, scripting, and framework design to enable the transition.
Automation Strategy Alignment(automation strategy alignment): The principle that automation must support overall test strategy and business objectives to be effective.
Automation Transition Roadmap(transition roadmap, roadmap): A plan outlining phases, training, infrastructure, and incremental migration for adopting test automation.
Automation deployment risk assessment(identifying automation deployment risks, deployment risk assessment): Structured identification and prioritization of risks like insufficient test environments, poor data management, lack of skilled resources, and CI/CD integration issues.
Automation lifecycle context(lifecycle contexts, waterfall and agile): The stage of the software development lifecycle, such as waterfall or agile, in which automation is applied.

B

Balanced test automation suite(balanced test automation suite): A suite covering both functional and non-functional requirements to mitigate risk.
Big bang test automation approach(big bang approach): A transition strategy that shifts from manual to automated testing all at once.
Break-even Point(break-even point): The number of test executions required to recover the initial investment in test automation.
Business Drivers(organization's key business drivers, organizational business drivers): Key organizational forces such as revenue growth, customer satisfaction, or compliance that guide strategic priorities.
Business Value Metrics(business value metrics): Metrics measuring cost savings, time-to-market reduction, and ROI to evaluate financial and strategic benefits.

C

CI/CD Pipeline(ci/cd pipeline, continuous integration/continuous delivery pipeline, continuous integration pipeline, ci/cd pipeline test automation): An automated pipeline integrating continuous integration and continuous delivery, where test automation executes at multiple levels.
Centralized deployment model(centralized model): A model with a dedicated automation team that ensures high consistency but may create bottlenecks.
Commercial test automation tool(commercial tools, commercial test automation tools): A test automation tool with licensing fees that provides built-in features, technical support, and regular updates.
Common Test Automation Objectives(common automation objectives, common objectives of test automation, test automation objectives): Typical goals for test automation including reducing time-to-market, increasing coverage, improving defect detection, and lowering costs.
Common Test Automation Pitfalls(pitfalls, automation pitfalls): Mistakes such as automating unstable applications or rarely executed tests, leading to low ROI.
Configuration Management for Test Environments(configuration management): The process of managing configurations of test environment components to maintain consistency and reliability.
Conflicting Automation Goals(conflicting goals, conflict of automation objectives, automation objectives conflict, trade-offs in automation, business drivers and conflicting goals, conflicting automation goals): Situations where automation objectives, like maximizing coverage and minimizing maintenance, contradict each other.
Contingency Buffer(contingency buffer): An additional 20-30% of maintenance costs included in ROI calculations to account for frequent application changes.
Continuous Delivery(continuous delivery (cd), cd): A release strategy where tests are executed in staging environments before release to production.
Continuous Improvement of Test Automation(continuous improvement of test automation, test automation continuous improvement, continuous improvement): An ongoing process of analyzing metrics and feedback to identify and implement improvements in automation.
Continuous Integration(continuous integration (ci), ci, continuous integration): A release strategy where automated tests run on every code commit to provide rapid feedback.
Continuous Integration (CI)(ci): A release strategy where automated tests run on every code commit to provide rapid feedback.
Continuous Integration Pipeline(ci pipeline): Practice that prioritizes automation projects supporting frequent execution and integration in a CI pipeline.
Continuous Testing(continuous testing): A practice of executing automated tests throughout the software development lifecycle, often integrated with CI/CD pipelines.
Continuous Testing (CT)(ct): The practice of executing automated tests throughout the software development lifecycle, often integrated with CI/CD pipelines.
Continuous deployment: A release strategy where every change that passes tests is automatically deployed to production.
Continuous monitoring(continuous monitoring): A practice that automatically checks system health and performance on an ongoing basis.
Continuous monitoring of automation metrics(monitoring and continuous improvement): Tracking pass/fail rates, execution time, and maintenance effort to detect emerging risks and adjust strategies.
Cost per Test Execution(cost per test execution): The total cost incurred to run a single test case, including tooling, development, and maintenance.
Cost-Benefit Analysis(cost-benefit analysis): A comparison of expected benefits such as reduced time-to-market and increased coverage against costs of tools, infrastructure, and maintenance.
Cost-Benefit Analysis for Test Automation(cost-benefit analysis): An assessment comparing expected benefits of automation against costs of tools, infrastructure, and maintenance.
Cultural Change in Test Automation(cultural resistance, cultural change): The process of overcoming resistance and building buy-in from manual testers during automation adoption.
Custom-built test automation framework(custom-built solutions, custom-built frameworks, custom-built test automation framework): A test automation framework developed in-house for maximum flexibility but requiring the highest initial investment.

D

Data management in automation(data privacy and consistency, data management in automation): Ensuring data privacy and consistency across test environments to avoid failures and deployment issues.
Dead Code Risk: Risk of accumulating unused code behind toggles that are never removed.
Decentralized deployment model(decentralized model): A model where automation is embedded within each team, promoting agility but risking fragmentation.
Decouple Deployment from Release: Using feature toggles to separate the act of deploying code from enabling a feature for users.
Defect Detection Rate(defect detection rate): A metric measuring the number of defects found by automated tests relative to total tests executed.
Defect detection rate metric(defect detection rate): The proportion of defects found by automated tests, used to assess the value delivered by automation.
Deployment Risk Assessment(deployment risk assessment, deployment risks): A structured process to identify, prioritize, and mitigate risks such as insufficient environments, poor data management, and CI/CD integration issues.

E

Effectiveness Metrics(effectiveness metrics): Metrics measuring defect detection rate, false positive rate, and test coverage to assess quality improvement.
Efficiency Metrics(efficiency metrics): Metrics measuring automation execution time, tests per hour, and resource utilization to evaluate speed and resource usage.
Environment Provisioning Automation(environment provisioning should be automated, environment provisioning automation, environment provisioning, provisioning automation, automated environment provisioning): The automation of environment provisioning to avoid manual errors and delays in test environment setup.
Execution Time Savings(execution time savings): The reduction in time required to execute tests when moving from manual to automated testing.

F

Feature Toggle(feature toggles, feature toggle, feature flag): A technique to enable or disable features without deploying new code, used to mitigate risks when tests fail.
Feature Toggle Integration with CI/CD: Use of feature toggles in a CI/CD pipeline to bypass failing tests and allow deployment to proceed.
Feature Toggle Risk Mitigation: Strategy where feature toggles reduce pressure from test failures and prevent blocked pipelines.
Flaky Test Rate(flaky test rate): A metric indicating the percentage of automated tests that produce inconsistent pass/fail results across runs.
Flaky Test Refactoring(flaky test refactoring): A strategy to improve automation by rewriting or stabilizing tests that produce inconsistent results.
Flexible Automation Architecture(modular automation architecture, gtaa): A modular test automation design that decouples scripts, data, and environments to adapt to evolving workflows.
Frequency of Test Execution(test execution frequency): How often tests are run; a criterion for selecting projects with high execution frequency, such as regression tests.
Full Lifecycle ROI Evaluation(roi over the full lifecycle): An evaluation of return on investment that considers costs and benefits over the entire lifecycle, not just initial implementation.

G

Generic Test Automation Architecture(gtaa): A framework that defines the components and interactions of test automation, which may need adjustments as the product evolves.
Generic Test Automation Architecture (gTAA)(gtaa, generic test automation architecture): A standard architecture for test automation that supports automated deployment, environment provisioning, and maintainable scripts.
Goal Alignment(goal alignment): The practice of ensuring automation objectives directly support organizational business drivers and quality objectives.
Goal Alignment in Test Automation(goal alignment): Ensuring automation objectives directly support organizational business drivers and quality goals.

H

Hidden Costs in Test Automation ROI(hidden costs, hidden costs in roi): Additional costs such as test data management and environment provisioning beyond tool and script costs.
Hidden costs in test automation(hidden costs): Unexpected expenses from training, maintenance, and tool integration that can make the cheapest initial solution expensive long-term.
Hybrid deployment model(hybrid approach): A deployment model that combines elements of centralized and decentralized approaches.

I

ISTQB Definition of Test Automation(istqb definition, test automation definition): The use of software to perform or support test activities, emphasizing automation as a means to an end.
Incremental test automation approach(incremental approach): A gradual transition strategy that introduces automation in small steps, allowing flexibility and early feedback.
Intangible Benefits of Automation(intangible benefits): Non-monetary advantages such as improved quality and faster time-to-market from test automation.
Integration Tests(integration tests, integration testing): Automated tests run nightly to verify interactions between components.

K

Keyword-Driven Framework(keyword-driven frameworks): A test automation approach that uses action words to separate test logic from implementation for low maintenance.

L

Layered Test Automation Flexibility(layered flexibility, modular layers): The modular separation of concerns in gTAA allowing each layer to be modified independently for different SDLC workflows.
Leading and Lagging Indicators(leading indicators, lagging indicators): Leading indicators track test design progress; lagging indicators track defects found in production for forward-looking and historical analysis.
Licensing fees(licensing fees): Costs associated with using commercial test automation tools, typically per-user or per-module.
Licensing fees for test automation tools(licensing fees): Costs associated with using commercial test automation tools, typically per-user or per-module.
Lifecycle Outcome Metrics(outcome metrics): Metrics linking automation results to business value, such as percentage of critical tests automated and reduction in manual regression effort.

M

Manual Test Suite Assessment(assess manual test suite, test suite assessment): The evaluation of existing manual tests to identify high-value candidates for automation based on business risk and frequency.
Measurable Quality Objectives(measurable quality objectives, quality objectives, measurable objectives): Quantifiable targets derived from business drivers, such as reducing production defects by 20% or achieving 99.9% uptime.
Misaligned Automation Goals(misalignment of test automation): Automation goals defined without business context, often focusing on technical metrics rather than business outcomes.
Modular Automation Architecture(modular automation architecture): A design approach that structures automation into independent modules for flexibility and low maintenance.
Modular Test Automation Architecture(modular architecture): An automation framework design using independent, reusable components to enable adaptation to hybrid SDLC workflows.

O

Open-source test automation framework(open-source test automation frameworks, open-source solutions): A test automation framework with no licensing fees that requires internal expertise for setup and maintenance.
Operational risk(operational risks): Risk reduced by aligning test automation with the organization's release strategy.
Organizational Readiness for Test Automation(organizational readiness): The availability of skilled engineers, management support, and mature processes to support automation.
Organizational deployment model(centralized model, decentralized model, hybrid model): A structural approach to test automation that can be centralized, decentralized, or hybrid.
Outcome Metrics in Test Automation(outcome metrics): Metrics that measure the effectiveness and business value delivered by test automation efforts.
Over-automation risk(over-automation, over-automation risk): Danger of automating everything which can introduce more risk than benefit; prioritization based on risk and value is recommended.

P

Pass/Fail Rates(pass/fail rates): A metric showing the proportion of automated test executions that passed or failed, indicating test stability.
Percentage of Automated Test Cases(percentage of automated test cases, automation coverage metric, automation coverage): A metric showing the proportion of test cases that are automated, used to track automation progress.
Phased Transition Approach(phased approach, transition roadmap): A gradual method for moving from manual to automated testing, involving pilot, evaluation, and scaling phases.
Phased rollout(pilot projects, pilot project, phased rollout): Mitigation strategy deploying automation on a low-risk application first to validate the framework before full-scale deployment.
Pilot Automation Project(pilot project, pilot automation): A small-scale initial automation effort to validate the framework and tool selection before full adoption.
Pilot Implementation(pilot implementation): A validation phase where automation changes are tested on a small scale before full rollout.
Positive ROI Condition(positive roi): The requirement that savings from automation exceed costs within a reasonable timeframe to achieve positive ROI.

Q

Qualitative Feedback in Automation(qualitative feedback): Team input on usability, reliability, and trust in automation, complementing quantitative metrics.

R

ROI Analysis for Automation(roi analysis): Compares automation costs against savings to justify investment.
ROI Analysis for Test Automation(return on investment for test automation, roi calculation, automation roi, high roi, roi, test automation roi calculation, roi calculations, roi analysis for test automation, roi analysis, test automation roi): A calculation comparing automation costs against savings from reduced manual testing and faster feedback over the automation lifecycle.
ROI for Test Automation(roi): A measure of gain or loss generated by an investment relative to its cost, calculated as (benefits - costs) / costs.
Regression Testing(regression testing): Testing of existing stable features after changes to ensure no new defects are introduced, often automated in later lifecycle phases.
Regression Testing Automation(regression tests, regression test automation): Automation of regression tests to maintain velocity and catch regressions early, often prioritized in Agile and DevOps contexts.
Regression tests(regression tests, regression testing): Tests executed in later pipeline stages to verify that new changes do not break existing functionality.
Regular Alignment Review(revisiting alignment, continuous alignment): The ongoing process of revisiting automation goals as organizational priorities shift to maintain relevance.
Release strategy(release strategy): A plan for when and how automated tests are executed relative to software releases.
Repetitive and Data-Driven Tests(data-driven tests, repetitive tests): Tests that are repeated and use data sets; strong candidates for automation due to manual effort savings.
Return on Investment (ROI) of Test Automation(roi, return on investment): Ratio of benefits gained from automation to the cost invested, emphasizing long-term sustainability.
Reusability of Automation Scripts(script reusability): Ability to reuse automation across multiple releases or projects, improving ROI by amortizing upfront investment.
Risk Assessment in Test Automation(risk assessment): The identification and mitigation of risks related to environment, data, and skill gaps in automation projects.
Risk mitigation in test automation: The practice of automating tests for high-risk areas first to reduce potential failures.
Risk-Based Test Automation(risk-based testing, critical path automation): Focusing automation efforts on critical path and risk-based tests aligned with organizational reliability goals.
Risk-Based Test Prioritization(risk-based approach, risk-based testing, risk-based test prioritization, prioritization based on risk): An approach that prioritizes tests for critical functionalities and high-risk areas based on risk assessment.
Risk-Based Testing(risk-based tests, critical path and risk-based tests, risk-based testing in automation): A testing approach that prioritizes test efforts based on the risk of failure and business impact.
Risk-based test selection(risk-based test selection): A method to prioritize tests for regression suites based on risk assessment.
Risk-based testing in automation(risk-based prioritization, risk-based testing in automation, risk-based testing): Prioritizing automation efforts on high-risk areas to ensure critical business functions are covered despite deployment constraints.
Rollback Mechanism(rollback mechanisms, rollback mechanism): A deployment strategy feature that allows reverting to a previous version when automated tests fail in production.

S

Scalability in test automation(scalability): The ability of a test automation solution to handle growing test suites and team size without disproportionate cost increase.
Scalability of automation solution(scalability): The ability of an automation framework to handle growing test suites and team size without disproportionate cost increase.
Secure Toggle Configuration: Requirement that toggle settings must be secure and auditable to prevent unauthorized feature activation.
Service Virtualization(service virtualization, virtualized, stubbed): A technique to virtualize or stub dependencies such as external services, databases, and third-party APIs for test independence.
Shift-Left Testing(shift-left testing): A practice of integrating automated tests early in the development cycle to catch defects sooner.
Short-Lived Toggles: Best practice where toggles are kept temporary to limit complexity and maintenance overhead.
Smoke tests(smoke tests, smoke testing): Tests executed in early pipeline stages for build verification to ensure basic functionality.
Stability of Application Under Test(application stability, stable application): Criterion indicating whether the application's functionality is stable enough to warrant automation, avoiding high maintenance costs.
Stakeholder Agreement on Automation Goals(stakeholders from both business and technical sides agree, stakeholder agreement, stakeholder agreement on automation goals): The necessity for business and technical stakeholders to jointly define and approve automation objectives.
Static risk mitigation plans(static risk mitigation plan pitfall, static risk mitigation plans): Risk mitigation plans that become obsolete if not updated as automation scope evolves.

T

Tangible and Intangible Benefits of Automation(tangible benefits, intangible benefits): Benefits from test automation that include both quantifiable savings and qualitative improvements like team morale.
Tangible benefits of automation(tangible benefits): Quantifiable advantages from test automation such as cost savings and reduced time-to-market.
Technical Convenience vs Business Priority(technical convenience, automating the easiest tests, technical convenience vs business priority): A common misalignment where automation focuses on easy tests rather than high-business-value areas.
Technical convenience trap(technical convenience trap, technical convenience): The mistake of automating low-value tests because they are technically easy, rather than aligning with business priorities.
Test Adaptation Layer(test adaptation layer): A layer within the gTAA that can be swapped to support different environments (e.g., mobile vs. web) without rewriting test scripts.
Test Automation Alignment(automation alignment, aligning test automation goals): The practice of ensuring test automation goals directly support organizational business goals and quality objectives.
Test Automation Applicability(applicability, test automation applicability): The suitability of a test for automation based on stability, repeatability, and technical feasibility.
Test Automation Candidate Selection(selecting candidates, test candidates): The process of identifying manual tests suitable for automation based on repeatability, frequency, and ROI.
Test Automation Effectiveness Evaluation(automation effectiveness evaluation): A process of assessing automation using quantitative metrics and qualitative feedback to determine success.
Test Automation Environment(test automation environment): A stable, isolated, and version-controlled set of hardware, software, networks, and data required to execute automated tests.
Test Automation Goal Alignment(align test automation goals with organizational goals, alignment is not a one-time activity, test automation objectives alignment, aligning automation objectives with business goals, goal alignment): The process of ensuring automation goals directly support organizational business drivers and quality objectives.
Test Automation Metrics(automation metrics, metrics): Measurements used to evaluate automation effectiveness, such as execution time, defect detection, and maintenance cost.
Test Automation Objectives(objectives of test automation, automation objectives, test automation objectives): Specific automation goals that directly support quality objectives, like automating smoke tests to catch regressions early.
Test Automation ROI(automation roi, roi evaluation): Evaluation of automation costs and benefits over the entire lifecycle, including tool licenses, training, maintenance, and reduced manual effort.
Test Automation Reporting Tailored to Audience(reporting): Presenting metrics in a clear, actionable format appropriate for technical teams or management to support decision-making.
Test Automation Scope Evolution(automation scope evolution): The process of retiring obsolete tests and adding new ones as the product and its features change.
Test Automation Selection Criteria(selection criteria, project selection criteria): Key criteria used to evaluate and prioritize projects for automation, including stability, frequency, complexity, and resources.
Test Automation Viability(viability, automation viability): The degree to which test automation can be justified economically, technically, and organizationally.
Test Automation in Agile(automation in agile, test automation in agile, agile automation, agile test automation): An integrated test automation approach where scripts are developed alongside code, enabling continuous testing and rapid feedback per iteration.
Test Automation in DevOps(automation in devops, test automation in devops, devops automation, devops test automation): An approach where test automation is embedded in the CI/CD pipeline, executing at multiple levels to provide fast feedback for rapid releases.
Test Automation in Waterfall(automation in waterfall, test automation in a waterfall context, waterfall automation): A test automation approach where scripts are created after manual testing, focusing on regression testing of stable features late in the lifecycle.
Test Data Management for Automation(test data management, data management): The effort required to create and maintain test data for automated tests, a common cause of failure if underestimated.
Test Data Provisioning Service(data provisioning service): A service implemented within the test adaptation layer to manage test data and reduce environment setup time.
Test Execution Layer(execution layer): The gTAA layer that integrates with CI/CD pipelines to run automated tests.
Test Generation Layer(test generation layer): The gTAA layer responsible for creating test cases, usable for both manual and automated tests.
Test Management Layer(test management layer): The gTAA layer overseeing test planning, monitoring, and control activities.
Test Stability Assessment(test stability): The evaluation of a test's reliability before automating to prevent inflated maintenance costs and reduced ROI.
Test automation implementation sequence(implementation process, sequence): The prescribed steps for introducing test automation: assess current state, define strategy, plan transition, execute, monitor, improve.
Test automation viability assessment(test automation viability, readiness assessment, viability assessment): Evaluation of economic, technical, and organizational readiness to determine the feasibility of test automation.
Test data management(test data management): The practice of maintaining consistent and secure test data for automation.
Test environment management(test environment availability, test environment management, environment management): Ensuring robust availability and consistency of test environments to prevent automation deployment delays.
Test environment stability: Requirement that test environments and data be stable and available for reliable automation results.
Time-to-market goal(time-to-market goals): A business objective to reduce release cycles and deliver features faster.
Toggle Complexity Risk: Risk of increased code complexity and maintenance cost from a large number of active toggles.
Toggle Removal Best Practice: Practice of removing feature toggles once the associated feature is stable to avoid technical debt.
Toggle as Technical Debt: Viewing feature toggles as technical debt that must be tracked and resolved to avoid long-term maintenance issues.
Total Cost of Ownership(total cost of ownership, relative investment, tco): The complete cost of a test automation solution including acquisition, training, maintenance, and scalability.
Total Cost of Ownership (TCO)(total cost of ownership, tco): Comprehensive cost analysis including acquisition, training, maintenance, and scalability over the automation lifecycle.
Total cost of ownership for test automation(total cost of ownership, total cost of ownership in test automation, tco): The complete cost including acquisition, training, maintenance, and scalability over the automation lifecycle.
Traditional phased release: A release strategy with distinct phases, as opposed to continuous approaches.

V

Vendor Lock-in in Test Automation(vendor lock-in, lock-in): Dependency on a single commercial tool that increases long-term costs and reduces flexibility.
Vendor lock-in(vendor lock-in): Dependency on a single vendor's commercial tool that can increase long-term costs due to recurring subscriptions and lack of flexibility.
Version Control of Test Assets(version control of test assets, version control, version control for test assets): The management of test assets under version control to ensure reproducibility of test results.
Version-Controlled Environment Definition(version-controlled environment, environment version control): Environment definitions under version control to prevent drift and ensure reproducibility.