Boundary with Related Testing Techniques

← Back to Home

← Extended Background

View Extended Taxonomy Definitions →

This page clarifies how the survey distinguishes fuzzing from closely related testing techniques.

The purpose of this boundary is not to suggest that related techniques are unimportant. Instead, it explains when a study falls within the survey scope and when it is treated as outside the scope because it lacks an iterative fuzzing-style exploration workflow.

Operational Boundary Used in the Survey

A technique is considered fuzzing when it follows an iterative process that:

  1. generates, selects, or mutates test inputs;
  2. executes the system under test;
  3. observes the resulting behavior;
  4. checks for failures or suspicious outcomes; and
  5. uses feedback, novelty, coverage, or failure signals to guide continued exploration.

A study is generally excluded when it contributes only one isolated element, such as:

without integrating that element into an iterative fuzzing workflow.

Inclusion Decision at a Glance

Related technique Why it overlaps with fuzzing Included when Excluded when
Metamorphic testing Metamorphic relations can generate related tests and act as oracles when exact expected outputs are unavailable. The relations are embedded in an automated, iterative generation or mutation workflow with execution and failure checking. The study only defines, validates, or evaluates metamorphic relations.
Mutation testing Both mutation testing and fuzzing use mutation, but they mutate different artifacts. Mutation supports iterative exploration of test inputs for failure discovery. The study mutates programs, models, or tests only to evaluate test-suite adequacy.
Concolic or symbolic testing Path exploration and constraint solving can support input generation and target deeper execution states. Symbolic or concolic analysis is integrated with repeated generation or mutation, execution, feedback, and failure reporting. Symbolic execution or constraint solving is the complete testing method and no fuzzing loop is present.
Adversarial testing Adversarial input generation can search for model errors or robustness weaknesses. The adversarial method is used as a mutation operator or search strategy within a broader fuzzing workflow. The study is formulated only as an adversarial optimization or attack problem.
Coverage-based autonomous-vehicle testing Scenario generation and simulator execution may use coverage or behavioral feedback. Scenarios are iteratively generated or mutated, executed, evaluated, and guided by coverage or safety feedback. The study only generates scenarios, measures coverage, or constructs a benchmark.

Metamorphic Testing

Metamorphic testing is closely related to fuzzing because metamorphic relations can provide a practical oracle when the expected output is difficult or impossible to specify directly.

A metamorphic relation defines how the output should behave when the input is transformed.

For example, a transformation may be expected to preserve:

Included

A metamorphic-testing study is included when the relation is part of an automated process that repeatedly:

  1. generates or transforms inputs;
  2. executes the target;
  3. checks whether the relation holds; and
  4. continues exploration using the observed outcomes.

Excluded

A study is excluded when its main contribution is limited to:

The distinction is therefore based on the workflow, not on the presence of metamorphic relations alone.

Mutation Testing

Mutation testing and fuzzing both use the term mutation, but they usually modify different artifacts for different purposes.

Fuzzing

Fuzzing typically mutates:

The purpose is to expose failures in the original system under test.

Mutation Testing

Mutation testing typically modifies:

The purpose is to evaluate the effectiveness or adequacy of a test suite.

Included

A study is included when mutation is used to generate or evolve test inputs within an iterative fuzzing workflow.

Excluded

A study is excluded when mutation is used only to:

The key question is:

Is mutation being used to explore the input space and discover failures in the target, or to evaluate the effectiveness of a test suite?

Concolic and Symbolic Testing

Concolic and symbolic testing generate inputs by reasoning about program paths, constraints, and internal execution conditions.

These techniques overlap with fuzzing because they can improve:

Included

A study is included when concolic or symbolic analysis is one component of a broader workflow that also performs:

In such cases, symbolic reasoning supports the fuzzer rather than replacing the fuzzing loop.

Excluded

A study is excluded when:

The presence of constraint solving alone is therefore not sufficient for inclusion.

Adversarial Testing

Adversarial testing generates inputs designed to expose model errors, robustness weaknesses, or security failures.

It overlaps with fuzzing because both may:

Included

An adversarial-testing study is included when adversarial generation is used within a broader fuzzing workflow, for example as:

Excluded

A study is excluded when it is formulated only as:

The survey therefore does not equate all adversarial attacks with fuzzing.

Coverage-Based Autonomous-Vehicle Testing

Autonomous-vehicle testing often uses:

These features can resemble fuzzing, but scenario generation alone is not sufficient.

Included

A study is included when it repeatedly:

  1. generates or mutates driving scenarios;
  2. executes the autonomous-driving system or simulator;
  3. observes coverage, behavior, or safety outcomes;
  4. identifies failures or high-risk cases; and
  5. uses feedback to guide subsequent scenario exploration.

Excluded

A study is excluded when it focuses only on:

without feedback-guided mutation and failure discovery.

The central criterion is whether scenario generation participates in an iterative fuzzing loop.

Boundary Principle

The survey applies the following general rule:

A related technique is included when it contributes to an iterative, automated failure-discovery workflow that generates or mutates tests, executes the target, observes behavior, and continues exploration using feedback or failure signals.

This rule allows the survey to include hybrid approaches while excluding work whose contribution is limited to a standalone oracle, attack, adequacy metric, constraint solver, or scenario generator.