CT-PT — Performance Testing glossary

126 terms the exam expects you to know, defined in plain language.

#

90th Percentile Response Time(90th percentile, 90th percentile response time): A metric indicating the response time below which 90% of requests fall, sensitive to degradation.

A

Agent-Based Monitoring(agent-based monitoring): Monitoring requiring software agents on servers to collect CPU, memory, disk I/O, and network metrics.
Aggregation of Performance Data(aggregation): The process of combining individual measurements into summary statistics such as mean, median, and percentiles.
Answer Rationale(rationale document): A document that explains the reasoning behind correct and incorrect answers in the ISTQB sample exam.
Application Logs(application logs): Logs that record events and errors generated by software, providing detailed timing information at the code level.
Application Performance Management Tools(application performance management (apm) tools, apm tools): Tools providing continuous production monitoring to validate performance improvements.
Audience-tailored Communication(understandable by all stakeholders, tailor communication, audience-tailored communication): Adapting the level of detail and language in performance test reports to different stakeholder groups.

B

Baseline Thresholds(baseline thresholds, baselines): Predefined metrics used as a reference point to compare and detect performance changes over time.
Bottleneck Resource(bottleneck resource): A shared resource that becomes fully utilized, limiting system throughput and causing saturation.
Box Plot(boxplot): A visual representation showing quartiles, median, and potential outliers.
Business/Logic Layer Performance Risks(business/logic layer risks, business layer risks): Performance risks related to inefficient algorithms, concurrency issues, and resource management in the core application logic layer.

C

Connection Pool Exhaustion(connection pool exhaustion): A failure mode where database connections are depleted, leading to request queuing or rejection.
Constant Load(constant load): A workload model where the load remains steady over time during a performance test.
Content Delivery Network(cdn): A distributed network of servers that delivers web content to users based on their geographic location to reduce latency.
Continuous Performance Testing(continuous performance testing within a devops pipeline, continuous performance testing in devops, automated performance testing): The practice of running performance tests automatically and frequently within a DevOps pipeline.
Controller(controller, load test controller): The central component that orchestrates the test and distributes workload definitions to load generators.
Correlation vs Causation Warning(correlation vs causation): A caution not to confuse high error rate with a direct cause, as it may stem from a bottleneck elsewhere.
Cross-Layer Performance Risk Analysis(cross-layer dependencies, cross-layer analysis): The process of identifying root causes of performance bottlenecks by considering dependencies across presentation, business, and data layers.
Cumulative Distribution Function(cdf): A function showing the probability that a random variable is less than or equal to a given value.

D

Data Layer Performance Risks(data layer risks): Performance risks associated with database design, indexing, connection pooling, and storage performance in the data storage layer.
Data-Driven Testing(data-driven): A testing approach where scripts are driven by external data sources to cover multiple scenarios.
Descriptive Statistics(statistical representation): Statistics that summarize the distribution of performance data, including central tendency and dispersion.
Deterministic Workload Model(deterministic workload model): A workload model that uses fixed percentages for transaction mixes, commonly used for load testing.
Distractor Analysis(understanding why a distractor is wrong, distractor analysis): The process of understanding why each incorrect answer option is wrong, as emphasized in the sample exam rationale document.
Distributed Configuration(distributed configuration, distributed architecture): A deployment mode where tool components are spread across multiple machines to avoid resource contention.
Distributed Load Generators(distributed load generators): Load generation tools deployed across multiple machines to simulate users from various geographic locations.

E

Early Performance Test Planning(early planning of performance activities, early performance test planning): The practice of defining performance criteria and identifying risks during the requirements phase.
End-to-End Performance Testing(end-to-end performance tests at the system level): System-level performance testing that covers the entire application, not just small increments.
Endurance Testing(endurance testing, soak testing): A performance testing type that applies sustained load over an extended period to detect issues like memory leaks or resource exhaustion.
Error Rate(error rate, failed requests): The percentage of failed requests, such as HTTP 5xx errors, during a performance test, signaling system limits.
Executive Summary(executive summary): The section of a performance test report stating whether requirements were met and highlighting critical risks.

F

Findings-Impact-Recommendation Framework(observation, explain the impact, and propose a solution, findings-impact-recommendation framework): A structured approach presenting an observation, its impact, and a proposed solution.
Firewall and Proxy Handling(firewalls and proxies, firewall rules): A tool's ability to handle firewalls and proxies, either by recording/replaying through proxies or requiring direct access.

H

Histogram: A visual representation showing the distribution of data by grouping values into bins.

I

ISTQB Glossary(glossary): The official vocabulary resource for ISTQB terms, providing canonical definitions that should be used to evaluate answer choices.
ISTQB Sample Exam(sample exam): A representative practice exam provided by ISTQB to simulate the real exam format and identify areas needing further study.
ISTQB Syllabus Learning Objectives(syllabus learning objectives): Official objectives that define the depth of knowledge required for each topic in the ISTQB certification syllabus.
Inverse Relationship Between Response Time and Throughput(inversely related under load, inverse relationship, inverse response-time/throughput relationship): Under load, as throughput increases, response time typically increases, creating an inverse relationship.

L

Load Generation Tools(load generation tools): Tools that simulate virtual users applying workload to a system at protocol or UI level.
Load Generator(load generators, load generator): A distributed agent that simulates virtual users and generates actual load on the system under test.
Load Generator Placement(placement of load generators, load generator placement): The positioning of load generators as close to real user locations as possible to mimic realistic network conditions.
Load Ramp-Up(gradually ramping up): The gradual increase of load to the target level during test execution.
Load Testing(load testing): A performance testing type that simulates expected user loads to verify system meets performance targets under normal and peak conditions.
Load Testing Tools(load testing tools): Tools that generate synthetic user traffic to measure end-to-end response times, throughput, and error rates from the user's perspective.

M

Maximum System Capacity(maximum capacity, saturation point): The throughput level at which the system is saturated under a given workload.
Mean(average): The arithmetic average of a set of values, sensitive to outliers.
Memory Leak(memory leak): A defect where an application consumes increasing memory over time, eventually causing OutOfMemoryError.
Minimum and Maximum(range): The smallest and largest observed values, showing the range of data.
Monitoring Dashboard(dashboards): A visual interface that displays trends and alerts for key performance metrics during testing.
Monitoring Tool Configuration(monitoring tools configured): The setup of tools to capture required metrics such as CPU, memory, network, and response times.
Monitoring Tools(monitoring tools, agent-based tools, agentless tools): Tools that collect server-side metrics such as CPU utilization, memory usage, disk I/O, and network traffic from the system under test.
Monitoring and Analysis Tools(monitoring and analysis tools, monitoring tools, analysis tools): Tools that collect resource utilization data and process it to identify bottlenecks and correlate metrics.

N

Network Latency (Performance Testing)(network latency): The delay between load generator and system under test that affects response time accuracy and must be accounted for.

O

Observer Effect(observer effect): The phenomenon where monitoring tools and logging themselves consume resources and alter the performance of the system under test.
Operational Profile(operational profile): A quantitative characterization of how a system is used, identifying user types, transactions, and their relative frequencies.

P

Parameterization(parameterization): A capability of load generation tools that varies input data across virtual users.
Parameterized Scripts(parameterized scripts): Test scripts using parameters to vary input data for realistic user behavior simulation.
Percentile(percentile, p90, p95, p99, median, percentiles): A statistical measure summarizing response time distribution, e.g., p50, p90, p95, p99, indicating the percentage of responses faster than that value.
Percentile-based Metrics(percentiles (e.g., 90th percentile response time)): Use of statistical percentiles like 90th percentile to represent response times instead of averages.
Performance Regression Detection(early detection of performance regressions, performance regression detection): Early detection of performance degradations through incremental testing in iterative models.
Performance Risk Assessment(performance risk assessment): An assessment that identifies and prioritizes performance risks based on business impact and technical complexity.
Performance Test Automation(automated performance tests): Automating performance tests to run similar to functional regression tests in agile environments.
Performance Test Environment Preparation(test environment preparation): The process of ensuring the test environment is stable, instrumented, and isolated before execution.
Performance Test Environment Representativeness(test environment representative of production): The requirement that the test environment must be representative of production for valid results.
Performance Test Objectives(performance test objectives): Specific, measurable goals derived from business requirements and risks that guide the performance test.
Performance Test Plan(performance test plan): A document that defines the scope, objectives, approach, resources, schedule, and risks for performance testing activities.
Performance Test Report(performance test report): A document that includes executive summary, test objectives, environment details, workload model, results, analysis, and recommendations.
Performance Testing(performance testing): The testing process to determine the performance efficiency of a software product, using metrics like response time and throughput.
Performance Testing Integration with SDLC(performance testing integrated throughout the sdlc, continuous performance testing): A principle that performance testing should be planned and executed throughout all phases of the software development lifecycle, not just at the end.
Performance Testing Tool Architecture Components(tool architecture components, components of performance testing tools, performance testing tool architecture): The typical components of a performance testing tool: controller, load generators, monitors, and results analyzer.
Performance Testing Tool Categories(performance testing tool categories, tool categories): Groups including load generation, monitoring/analysis, profiling, APM, and virtualization tools.
Performance Testing as a Specialized Test Type(performance testing is a specialized test type, performance testing is a distinct test type): A distinct test type investigating non-functional attributes such as speed, scalability, and stability requiring specific skills and tools.
Pilot Test(pilot test): A preliminary test conducted to validate the test environment and tool setup before full execution.
Presentation Layer Performance Risks(presentation layer risks): Performance risks originating from client-side processing, network latency, and asset optimization in the user interface layer.
Probabilistic Workload Model(probabilistic workload model): A workload model that uses random distributions for transaction mixes, useful for simulating unpredictable spikes in stress testing.
Profiling Tools(profiling tools): Tools that identify performance issues at the code level, such as memory leaks or inefficient algorithms.
Proof-of-Concept for Tool Evaluation(proof-of-concept, poc): A representative test scenario used to verify a tool's suitability before commitment.

R

Ramp-Up Load(ramp-up load, ramp-up): A workload model where the load gradually increases to a target level over time.
Ramp-up/Ramp-down Scheduling(ramp-up/ramp-down scheduling): A load generation feature that gradually increases or decreases the number of virtual users over time.
Real User Monitoring (RUM)(real user monitoring, rum): A technique that captures performance data from actual users in production, providing insights into real-world user experience.
Real-Time Monitoring(real-time monitoring): Continuous tracking of metrics during test execution to detect anomalies early.
Realistic Workload Model(realistic workload models, realistic data, realistic workload model): A model using realistic data and usage patterns to simulate user behavior during performance testing.
Representative Test Environment(representative environment, production-like environment, representative test environment): A test environment that mirrors production hardware, network, and configuration to ensure valid performance test results.
Resource Contention (Performance Testing)(bottlenecks on the controller machine, resource contention): A situation where the controller machine becomes a bottleneck in single-machine setups, often requiring distributed architecture.
Resource Exhaustion(resource exhaustion): A failure mode caused by depletion of a finite resource, leading to system degradation or crash.
Resource Utilization(resource utilization, cpu usage, memory consumption, disk i/o, network bandwidth): Metrics including CPU usage, memory consumption, disk I/O, and network bandwidth that indicate system bottlenecks.
Response Time(response time): The time a system takes to respond to a user request, measured from request sent to complete response received.
Response Time Degradation(response time degradation): A failure mode where request processing time increases beyond acceptable thresholds as load increases.
Results Analyzer(results analyzer): A component that processes and presents the collected performance data for analysis.
Risk of Late Performance Issue Discovery(late discovery of critical issues): A risk in sequential models where performance issues are found late, making fixes costly and time-consuming.
Risk-Based Performance Testing(risk-based testing, risk-based approach, risk-based testing applied to performance, risk-based performance testing): Performance testing aligned with business priorities and focused on high-risk areas.

S

Scalability Testing(scalability testing): A performance testing type that determines how well a system scales up or out to handle increased load.
Scenario Design(scenario design): The creation of test scenarios representing typical, peak, and stress conditions based on the workload model.
Script Maintenance(script maintenance): Ongoing updating of test scripts to reflect application changes and ensure valid results.
Single-Machine Configuration(single-machine configuration): A deployment mode where all tool components run on one machine.
Smoke Test(smoke test): A low-load test executed to confirm that the test setup is working correctly before full performance testing.
Spike Test(spike test, spike pattern): A performance test that applies a sudden increase in load to assess system resilience.
Spike Testing(spike testing): A performance testing type that evaluates system response to sudden, large increases in load, focusing on rate of change.
Stress Testing(stress testing): A performance testing type that pushes a system beyond normal capacity to find its breaking point and observe failure behavior.
Synchronized Clocks(synchronized clocks): Practice of ensuring accurate, actionable results by synchronizing clocks in performance testing.

T

Technology Stack Alignment(technology stack alignment): Ensuring the testing tool matches the application's technology stack for accurate performance testing.
Test Abort Criteria(abort the test): Predefined conditions under which a performance test should be terminated to avoid invalid results.
Test Data Collection(collect all logs, metrics, and artifacts): The collection of logs, metrics, and artifacts after test completion for analysis.
Test Design Specification(test design specification): A document detailing test cases, preconditions, test data, expected results, and pass/fail criteria for performance tests.
Test Environment Isolation(isolated from other activities): Ensuring the test environment is separate from other activities to avoid interference.
Test Environment Plan(test environment plan): A plan describing the hardware, software, network configurations, and monitoring tools required for performance testing.
Test Environment Preparation(test environment preparation, environment preparation): Ensuring hardware, software, network, and monitoring tools are configured and isolated for valid performance tests.
Test Execution Validation(validate test execution): Verification that the test ran as intended, including checking for script errors and correct think times.
Test Repetition(repeat tests): Running a performance test multiple times to account for variability and confirm results.
Think Time Modeling(think time modeling): Simulation of user delays between actions in load generation tools.
Throughput(throughput): The number of transactions or requests a system can handle per unit of time, such as requests per second.
Throughput Saturation(throughput saturation): A failure mode where the system cannot process more transactions per unit time, even as load increases.
Time Synchronization (Load Testing)(time synchronization, time synchronization between load generators and the controller): The synchronization of clocks between load generators and controller for accurate timestamping of results.
Time Synchronization in Load Testing(time synchronization, clock synchronization): The process of aligning clocks across multiple load generators using NTP to ensure accurate timestamping of results.
Timely Communication(timely communication): The practice of reporting performance results quickly so stakeholders can act on findings.
Timely Communication of Performance Results(communicated in a timely manner): Delivering performance test results quickly to enable corrective actions.
Tool Evaluation Report(tool evaluation report): A report documenting tool selection criteria such as protocol support, scalability, and reporting capabilities.
Tool Overhead(tool overhead): Resource consumption by the testing tool that can skew results if not accounted for.
Tool Selection Criteria(tool selection criteria, selection criteria): Factors such as technology architecture, performance risks, and team skills that guide tool choice.
Tool Topology(topology, topology of a performance testing tool, tool topology): The networking and deployment of tool components relative to the system under test, including considerations like latency and placement.

V

Vendor Lock-In(vendor lock-in): Dependency on a specific vendor's tool due to proprietary features or data formats, limiting flexibility.
Virtual Users(virtual users, vus): Simulated users created by load generation tools to apply workload during performance testing.
Virtualization and Service Virtualization Tools(virtualization and service virtualization tools, service virtualization tools): Tools that simulate dependent components to enable testing in constrained environments.
Volume Testing(volume testing): A performance testing type that assesses system behavior when handling large amounts of data, such as database size or file storage.

W

Workload Model(workload model): A model specifying the distribution of virtual users, data volumes, and think times to simulate realistic production usage.
Workload Modeling(workload modeling): The process of defining user behavior, load patterns, and data volumes to simulate real-world usage.