Property-Based Testing and Fuzzing for Java: Finding the Bugs Your Example Tests Miss
Every Java team knows this quiet problem. The suite is green, the coverage report looks healthy, and yet a bug still reaches production. The reason is simple. Example-based tests only check the inputs that someone thought of, while the bugs that hurt are usually the ones nobody thought of.
Why example tests keep missing the same bugs
Consider the famous midpoint bug. In 2006, Joshua Bloch explained that the binary search in the JDK’s own java.util.Arrays averaged two indexes with (low + high) / 2, which overflows for very large arrays. According to his account, the bug sat unnoticed for roughly nine years, even though the algorithm had been proven correct on paper (Bloch’s post). Careful authors and reviewers missed it. Therefore, a handful of hand-picked examples has little chance.
To make this concrete, I planted three small bugs in a plain-Java experiment. Each one passes the examples most of us would write first.
| Planted bug | Typical example inputs | What really breaks it |
|---|---|---|
A. Bucket index: Math.abs(hash) % 10 | 5, -7, 100 | Integer.MIN_VALUE, because its absolute value is still negative |
B. Midpoint: (a + b) / 2 | (2, 4), (10, 20) | Two large ints whose sum overflows |
| C. Leap year without the 400-year rule | 2024, 2023, 1900 | Any multiple of 400, such as 2000 |
Two tools, two kinds of search
jqwik: you state the rule, it generates the inputs
jqwik is a test engine for the JUnit platform. You write a property, which is a rule that must hold for all valid inputs, and you mark each parameter with @ForAll. By default, jqwik runs 1,000 tries per property, and it mixes in edge cases along the way. When a property fails, it also shrinks the failing sample to a simpler one and reports the random seed, so you can reproduce the run (see the jqwik user guide).
Jazzer: coverage feedback finds the hidden doors
Jazzer is a coverage-guided, in-process fuzzer for the JVM, developed by Code Intelligence and based on libFuzzer (project page). Instead of generating data blindly, it observes which branches each input reaches and keeps the inputs that reach something new. As a result, it can walk through nested checks that random data would almost never pass. In JUnit, you use the @FuzzTest annotation.
| jqwik | Jazzer | |
|---|---|---|
| Question | Does my rule hold for all valid inputs? | Can any input crash or break this code? |
| Input | Typed values with constraints | Raw bytes, or a FuzzedDataProvider |
| You write | The rule (the oracle) | A target that calls the code; exceptions and detected security issues are the signal |
| Typical use | Every build | Quick regression runs in CI, longer sessions on a schedule |
A small demo you can run
You need a recent JDK and Maven. First, add the three test dependencies. The versions below are the ones I confirmed in the projects’ documentation and release history; newer ones should work as well.
<dependencies>
<dependency>
<groupId>net.jqwik</groupId>
<artifactId>jqwik</artifactId>
<version>1.10.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.code-intelligence</groupId>
<artifactId>jazzer-junit</artifactId>
<version>0.30.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.14.4</version>
<scope>test</scope>
</dependency>
</dependencies>
A property for the leap-year bug
Next, we compare our method with the JDK’s trusted Year.isLeap. The class name ends in Tests so that Maven picks it up by default.
package demo;
import java.time.Year;
import net.jqwik.api.ForAll;
import net.jqwik.api.Property;
import net.jqwik.api.constraints.IntRange;
class LeapYearTests {
// code under test, with a planted bug: no 400-year rule
static boolean isLeap(int year) {
return year % 4 == 0 && year % 100 != 0;
}
@Property
boolean agreesWithTheJdk(
@ForAll @IntRange(min = 1, max = 3000) int year) {
return isLeap(year) == Year.isLeap(year);
}
}
The property fails only for the seven multiples of 400 between 1 and 3,000, namely 400, 800 and so on up to 2800. jqwik should report one of them as the failing sample. Meanwhile, three typical examples (2024, 2023, 1900) all pass.
A fuzz test for a gated input
Now the fuzzing side. The planted bug hides behind a four-byte header, which is exactly the kind of gate that blind data rarely passes.
package demo;
import com.code_intelligence.jazzer.junit.FuzzTest;
class DecoderFuzzTest {
// code under test, with a planted bug behind a 4-byte header
static int decode(byte[] in) {
if (in.length >= 4 && in[0] == 'F' && in[1] == 'U'
&& in[2] == 'Z' && in[3] == 'Z') {
throw new IllegalStateException("planted bug reached");
}
return in.length;
}
@FuzzTest
void decodeNeverThrows(byte[] data) {
decode(data);
}
}
Finally, run both. Without the environment variable, Jazzer only replays stored inputs, which keeps normal builds fast. With JAZZER_FUZZ=1, it actively searches (the command below is for Linux and macOS).
# jqwik runs like any other test mvn test -Dtest=LeapYearTests # Jazzer fuzzes only when JAZZER_FUZZ is set JAZZER_FUZZ=1 mvn test -Dtest=DecoderFuzzTest
One caution before adopting Jazzer: its licence terms have changed over time, so read the licence of the exact release you choose on the project page.
What the numbers say
To measure the difference, I wrote a library-free Java harness and ran each setting 2,000 times with different random seeds. It is a toy model of the two ideas, not jqwik or Jazzer themselves. First, the three bugs with a budget of 1,000 tries, which is jqwik’s default.

The examples caught none of the three bugs. Plain random data found bug B almost instantly (median of 2 tries) and bug C in about nine runs out of ten. However, it never found bug A, because MIN_VALUE is one value out of roughly 4.3 billion. Mixing in boundary values fixed that: all 2,000 runs found it, with a median of 51 tries in my toy. This is why edge-case injection matters so much. Budget matters too. For bug C, random search had found it in about 21% of runs after 100 tries, 51% after 300, and 91% after 1,000.
Next, the gated input. A blind generator has a one in 2564 chance per try of producing the right four bytes, which is about one in 4.3 billion. I compared it with a tiny feedback-guided loop that keeps any input reaching a new branch.

The guided loop needed tens of thousands of executions, while the blind one found nothing in twenty million. In other words, feedback turns one impossible jump into four easy steps.
Quick checklist
- Write properties for rules such as round trips, ordering, and agreement with a trusted implementation.
- Point fuzzers at parsers and decoders that handle untrusted bytes.
- Keep every input a tool finds as a permanent regression test.
- Run jqwik on every build, and schedule longer fuzz sessions separately.
- Never turn off boundary values, because they find the bugs random data cannot.
What We Learned
Example tests verify what we already imagined, so they keep missing overflow, boundary and rare-rule bugs. Property-based testing with jqwik fixes this by checking a rule against many generated inputs, mixing in edge cases, and shrinking failures. Fuzzing with Jazzer goes one step further for byte-level code, because coverage feedback lets it pass nested checks that blind data cannot. In my small experiment, three example-passing bugs were found by random or boundary-aware generation, and a four-byte gate that defeated twenty million blind tries fell to feedback within about 59,000 executions. Therefore, use both tools: properties for rules, fuzzing for hostile input, and examples for documentation.

