ISTQB Foundation Level Chapter 1: Fundamentals of Testing

What this chapter covers

What is testing?

The syllabus opens by defining testing as a set of activities to discover defects and evaluate the quality of software work products. These work products, when being tested, are called test objects.

A common misconception is that testing only means executing tests, running the software and checking the results. The syllabus is explicit that testing includes other activities too, such as analysis and design, and that it must be aligned with the software development lifecycle.

Another common misconception is that testing is only about verification, checking whether the system meets specified requirements. Testing also involves validation, checking whether the system meets the needs of users and other stakeholders in its actual operating environment. Verification and validation are different questions: verification asks "did we build the system right?" and validation asks "did we build the right system?"

Testing can be dynamic or static. Dynamic testing involves executing the software. Static testing does not, and includes reviews and static analysis. Testing is also described as largely an intellectual activity that requires analytical thinking, critical thinking, and systems thinking, not just tool use.

Test objectives. The syllabus lists several typical reasons testing is done, including evaluating work products such as requirements and code, causing failures and finding defects, ensuring adequate coverage of a test object, reducing the risk of inadequate quality, verifying that requirements have been fulfilled, verifying compliance with contractual, legal, or regulatory requirements, giving stakeholders information to support decisions, building confidence in quality, and validating that the test object works as the stakeholders expect. Which of these objectives matters most depends on context, including the test level, the risks involved, and the development lifecycle being used.

Testing and debugging. Testing and debugging are separate activities. Dynamic testing can trigger a failure caused by a defect, or static testing can find a defect directly without any failure occurring. Debugging is the activity of finding the cause of a failure (diagnosis), and fixing it. After a fix, confirmation testing checks that the fix worked, ideally performed by the same person who ran the original test, and regression testing checks that the fix did not break anything else.

Why is testing necessary?

Testing is a form of quality control that helps a project meet its objectives within scope, time, quality, and budget constraints. The syllabus frames testing's value in a few specific ways.

Testing's contributions to success. Testing provides a cost-effective way of detecting defects, which can then be removed through debugging. It gives a way of directly evaluating the quality of a test object at different points in the lifecycle, which feeds into decisions like whether to release. It also gives users an indirect voice in the project, since testers represent user needs throughout development when involving real users directly is too costly. Testing may also be required to satisfy contractual, legal, or regulatory obligations.

Testing and quality assurance. Although people often use "testing" and "quality assurance" (QA) interchangeably, the syllabus treats them as different things. Testing is product-oriented and corrective, focused on activities that help achieve quality in the product itself. QA is process-oriented and preventive, focused on the processes used to build the product, on the logic that a good process tends to produce a good product. QA is everyone's responsibility, not just the test team's. The same test results feed both: testers use them to fix defects, while QA uses them as feedback on how well the development and test processes are working.

Errors, defects, failures, and root causes. People make errors (mistakes), which can produce defects (faults or bugs) in a work product, which can in turn cause failures when the software runs. Errors happen for reasons like time pressure, complexity, or lack of training. Defects can exist in any work product, including documentation, not just code, and a defect introduced early in the lifecycle can lead to further defects downstream if it goes undetected. Not every defect causes a failure, and not every failure is caused by a defect or error: environmental factors, such as radiation affecting firmware, can also cause failures. A root cause is the underlying reason a problem occurred, identified through root cause analysis, typically carried out after a failure or defect is found, with the goal of preventing similar problems in the future.

Testing principles

The syllabus describes seven general principles about how testing behaves. Because each principle has enough depth and exam relevance to deserve its own treatment, see the seven testing principles guide for a full explanation with examples of each one. In summary, the seven principles are: testing shows the presence of defects but cannot prove their absence; exhaustive testing is impossible except in trivial cases; early testing saves time and money; defects tend to cluster; repeating the same tests becomes less effective over time (tests wear out); testing must be adapted to its context; and it is a fallacy to assume that fixing every known defect ensures a successful system (the absence-of-defects fallacy).

Test activities, testware, and test roles

Test activities and tasks. The syllabus describes a test process made up of groups of activities. Although they can appear sequential, in practice they often overlap or repeat. The groups are:

• Test planning, which defines test objectives and the approach for achieving them within the constraints of the project.
• Test monitoring and test control, where monitoring is the ongoing check of test activities against the plan, and control is the action taken to keep testing on track toward its objectives.
• Test analysis, which examines the test basis to identify testable features and define and prioritize test conditions, answering the question "what to test?"
• Test design, which turns test conditions into test cases and other testware, including identifying coverage items, test data, and test environment needs, answering the question "how to test?"
• Test implementation, which creates or organizes the testware needed for execution, such as test procedures, scripts, and test suites, and prepares the test environment.
• Test execution, which runs the tests, compares actual results to expected results, logs results, and analyzes anomalies.
• Test completion, which typically happens at milestones such as a release or end of an iteration, and includes handling unresolved defects, archiving useful testware, shutting down the test environment, and producing a test completion report.

Test process in context. How this process is actually carried out depends on contextual factors such as the stakeholders involved, the skills and availability of the team, the business domain, technical factors like architecture and technology, project constraints (scope, time, budget), organizational factors, the software development lifecycle in use, and the tools available. These factors shape decisions like test strategy, which techniques are used, how much is automated, what level of coverage is needed, and how testing is reported.

Testware. Testware is the collective term for the work products created during testing. Examples include test plans and risk registers (from test planning), progress reports (from monitoring and control), test conditions and defect reports on the test basis (from analysis), test cases, test charters, and coverage items (from design), test procedures, scripts, and test suites (from implementation), test logs and defect reports (from execution), and completion reports, lessons learned, and change requests (from completion).

Traceability between the test basis and testware. Maintaining traceability between test basis elements (such as requirements), testware (such as test conditions and test cases), test results, and defects supports effective test monitoring and control. Good traceability allows coverage to be evaluated against measurable criteria, helps determine the impact of changes, supports audits, and helps communicate the status and meaning of testing to stakeholders in terms they understand.

Roles in testing. The syllabus describes two principal roles. The test management role takes overall responsibility for the test process, the test team, and leadership of test activities, and is mainly focused on test planning, monitoring, control, and completion. The testing role takes overall responsibility for the technical side of testing and is mainly focused on test analysis, design, implementation, and execution. The same person can hold both roles, and how the roles are distributed depends on context. In Agile teams, for example, some test management tasks may be shared across the team rather than held by a dedicated test manager.

Essential skills and good practices in testing

Generic skills required for testing. The syllabus lists skills that are particularly relevant for testers: testing knowledge to apply techniques effectively, thoroughness and attention to detail to find hard-to-spot defects, communication skills and active listening to work with stakeholders and report defects constructively, analytical and critical thinking and creativity, technical knowledge to use tools effectively, and domain knowledge to understand and communicate with end users and business representatives. Testers often deliver unwelcome news, and people can react defensively to that, sometimes due to confirmation bias. The syllabus emphasizes communicating defects and results constructively, focused on the product rather than the person who created it.

Whole team approach. Originating from Extreme Programming, the whole team approach means any team member with the right knowledge and skills can perform any task, and everyone shares responsibility for quality. Co-location, physical or virtual, supports this by making communication easier. Testers in this model work closely with business representatives on acceptance tests and with developers on test strategy and automation, and help transfer testing knowledge across the team. The approach is not always appropriate, for example in safety-critical contexts where a higher degree of independence may be needed.

Independence of testing. A degree of independence tends to make testers more effective at finding defects, because their cognitive biases differ from those of the work product's author, although independence does not replace the value of familiarity (a developer can often efficiently find defects in their own code). The syllabus describes a spectrum from no independence (the author tests their own work) through some independence (peers from the same team), high independence (testers outside the author's team but within the organization), to very high independence (testers from outside the organization). Most projects benefit from multiple levels of independence at different test levels. The benefit of independent testing is that independent testers are more likely to spot different kinds of defects and can challenge assumptions made during specification and implementation. The drawbacks include possible isolation from the development team, communication friction, an adversarial dynamic, developers feeling less ownership of quality, and independent testers being seen as a bottleneck or blamed for delays.

Chapter 1 at a glance

Minimum training time for this chapter: 180 minutes.