chore: restructure the docs

Author: oetr

README.md

@@ -1,17 +1,4 @@
-## Advanced options
-
-* [Using Jazzer Standalone](#using-jazzer-standalone)
-* [Passing JVM Arguments](#passing-jvm-arguments)
-* [Coverage Instrumentation](#coverage-instrumentation)
-* [Trace Instrumentation](#trace-instrumentation)
-* [Value Profile](#value-profile)
-* [Custom Hooks](#custom-hooks)
-* [Keep Going](#keep-going)
-* [Export Coverage Information](#export-coverage-information)
-* [Native Libraries](#native-libraries)
-
-> [!NOTE]\
-> These settings apply to the old fuzzing approach using a `fuzzerTestOneInput` method and the native Jazzer binary. They don't work in the new JUnit integration.
+# Advanced options
 
 ## Using Jazzer Standalone
 There are two ways to use Jazzer standalone: by using the `jazzer` binary or by calling the Jazzer main class directly.
@@ -39,7 +26,44 @@ Please refer to [libFuzzer](https://llvm.org/docs/LibFuzzer.html) for documentat
 Various command line options are available to control the instrumentation and fuzzer execution.
 A full list of command-line flags can be printed with the `--help` flag.
 
-### Passing JVM Arguments
+
+## Reproducing a finding
+
+When Jazzer manages to find an input that causes an uncaught exception or a failed assertion, it prints a Java stack trace and creates two files that aid in reproducing the crash without Jazzer:
+
+* `crash-<sha1_of_input>` contains the raw bytes passed to the fuzz target (just as with libFuzzer C/C++ fuzz targets).
+  The crash can be reproduced with Jazzer by passing the path to the crash file as the only positional argument.
+* `Crash-<sha1_of_input>.java` contains a class with a `main` function that invokes the fuzz target with the crashing input.
+  This is especially useful if using `FuzzedDataProvider` as the raw bytes of the input do not directly correspond to the values consumed by the fuzz target.
+  The `.java` file can be compiled with just the fuzz target and its dependencies in the classpath (plus `jazzer_standalone.jar` or `com.code-intelligence:jazzer-api:<version>` if using `FuzzedDataProvider`).
+
+## Minimizing a crashing input
+
+With the following argument you can minimize a crashing input to find the smallest input that reproduces the same "bug":
+
+```bash
+-minimize_crash=1 <path/to/crashing_input>
+```
+
+## Parallel execution
+
+libFuzzer offers the `-fork=N` and `-jobs=N` flags for parallel fuzzing, both of which are also supported by Jazzer.
+
+
+## Recommended JVM Options
+
+The following JVM settings are recommended for running Jazzer:
+
+* `-XX:-OmitStackTraceInFastThrow`: Ensures that stack traces are emitted even on hot code paths.
+  This may hurt performance if your Fuzz Test frequently throws and catches exceptions, but also helps find flaky bugs.
+* `-XX:+UseParallelGC`: Optimizes garbage collection for high throughput rather than low latency.
+* `-XX:+CriticalJNINatives`: Is supported with JDK 17 and earlier and improves the runtime performance of Jazzer's
+  instrumentation.
+* `-XX:+EnableDynamicAgentLoading`: Silences a warning with JDK 21 and later triggered by the Java agent that Jazzer
+  attaches to instrument the fuzzed code.
+
+
+## Passing JVM Arguments
 
 When Jazzer is started using the `jazzer` binary, it starts a JVM in which it executes the fuzz target.
 Arguments for this JVM can be provided via the `JAVA_OPTS` environment variable.
@@ -73,7 +97,7 @@ Both flags take a list of glob patterns for the java class name separated by col
 By default, JVM-internal classes and Java as well as Kotlin standard library classes are not instrumented,
 so these do not need to be excluded manually.
 
-### Trace Instrumentation
+## Trace Instrumentation
 
 The agent adds additional hooks for tracing compares, integer divisions, switch statements and array indices.
 These hooks correspond to [clang's data flow hooks](https://clang.llvm.org/docs/SanitizerCoverage.html#tracing-data-flow).
@@ -88,35 +112,22 @@ The particular instrumentation types to apply can be specified using the `--trac
 
 Multiple instrumentation types can be combined with a colon (Linux, macOS) or a semicolon (Windows).
 
-### Value Profile
+## Value Profile
 
 The run-time flag `-use_value_profile=1` enables [libFuzzer's value profiling mode](https://llvm.org/docs/LibFuzzer.html#value-profile).
 When running with this flag, the feedback about compares and constants received from Jazzer's trace instrumentation is associated with the particular bytecode location and used to provide additional coverage instrumentation.
 See [ExampleValueProfileFuzzer.java](../examples/src/main/java/com/example/ExampleValueProfileFuzzer.java) for a fuzz target that would be very hard to fuzz without value profile.
 
-### Custom hooks
-
-In order to obtain information about data passed into functions such as `String.equals` or `String.startsWith`, Jazzer hooks invocations to these methods.
-This functionality is also available to fuzz targets, where it can be used to implement custom sanitizers or stub out methods that block the fuzzer from progressing (e.g. checksum verifications or random number generation).
-See [ExampleFuzzerHooks.java](../examples/src/main/java/com/example/ExampleFuzzerHooks.java) for an example of such a hook.
-An example for a sanitizer can be found in [ExamplePathTraversalFuzzerHooks.java](../examples/src/main/java/com/example/ExamplePathTraversalFuzzerHooks.java).
-
-Method hooks can be declared using the `@MethodHook` annotation defined in the `com.code_intelligence.jazzer.api` package, which is contained in `jazzer_standalone.jar` (binary release) or in the Maven artifact [`com.code-intelligence:jazzer-api`](https://search.maven.org/search?q=g:com.code-intelligence%20a:jazzer-api).
-See the [javadocs of the `@MethodHook` API](https://codeintelligencetesting.github.io/jazzer-docs/jazzer-api/com/code_intelligence/jazzer/api/MethodHook.html) for more details.
-
-To use the compiled method hooks, they have to be available on the classpath provided by `--cp` and can then be loaded by providing the flag `--custom_hooks`, which takes a colon-separated list of names of classes to load hooks from.
-Hooks have to be loaded from separate JAR files so that Jazzer can [add it to the bootstrap class loader search](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html#appendToBootstrapClassLoaderSearch-java.util.jar.JarFile-).
-The list of custom hooks can alternatively be specified via the `Jazzer-Hook-Classes` attribute in the fuzz target JAR's manifest.
 
-### Keep Going
+## Keep Going
 
 With the flag `--keep_going=N` Jazzer continues fuzzing until `N` unique stack traces have been encountered.
 Specifically `--keep-going=0` will keep the fuzzer running until another stop condition (e.g. maximum runtime) is met.
 
 Particular stack traces can also be ignored based on their `DEDUP_TOKEN` by passing a comma-separated list of tokens via
 `--ignore=<token_1>,<token2>`.
 
-### Export Coverage Information
+## Export Coverage Information
 
 > [!WARNING]\
 > This feature is deprecated. The standalone JaCoCo agent should be used to generate coverage reports.
@@ -143,7 +154,7 @@ java -jar path/to/jacococli.jar report coverage.exec \
   --name FuzzCoverageReport
 ```
 
-### Native Libraries
+## Native Libraries
 
 Jazzer supports fuzzing of native libraries loaded by the JVM, for example via `System.load()`.
 For the fuzzer to get coverage feedback, these libraries have to be compiled with `-fsanitize=fuzzer-no-link`.

docs/advanced.md

@@ -0,0 +1,122 @@
+## Implementation
+
+Jazzer's JUnit integration starts from
+the [`FuzzTest`](../src/main/java/com/code_intelligence/jazzer/junit/FuzzTest.java) annotation. As mentioned in the
+annotation's javadoc, our integration runs in one of two modes: fuzzing and regression. Fuzzing mode will generate new
+inputs to feed into the tests to find new issues and regression mode will run the tests against previous findings, no
+fuzzing is done. The main entrypoints for the actual integration code are found in two of the annotations
+on `FuzzTest`: `@ArgumentsSource(FuzzTestArgumentsProvider.class)` and `@ExtendsWith(FuzzTestExtensions.class)`.
+
+Because these same files and functions are involved in two mostly separate sets of functionality, this will look at the
+flow of the different methods involved in integrating with JUnit in fuzzing mode (when `JAZZER_FUZZ` is set to a truthy 
+value (`true`, `1`, `yes`)) and in regression mode (when `JAZZER_FUZZ` is not set) separately.
+
+### Fuzzing Flow
+
+JUnit will call the following methods for each test marked with `FuzzTest`.
+
+> [!NOTE]\
+> In fuzzing mode, only a single fuzz test is executed per test run
+> (see [#599](https://github.com/CodeIntelligenceTesting/jazzer/issues/599) for details).
+
+#### `evaluateExecutionCondition`
+
+The first call to this test will determine if the test should be run at all. In fuzzing mode, we only allow one test to
+be run due to global state in libfuzzer that would mean multiple tests would interfere with each other. Jazzer will
+accept the first fuzz test that is checked as the test to be run. It will cache which test it has seen first and
+return that test as enabled.
+
+If this returns that a test is disabled, JUnit will not run the rest of these methods for this test and instead skip
+to the next one.
+
+#### `provideArguments`
+
+This will configure the fuzzing agent to set up code instrumentation, instantiate a `FuzzTestExecutor` and put it into
+JUnit's `extensionContext`, then create a stream of a single empty argument set. As the comment mentions, this is so
+that JUnit will actually execute the test but the argument will not be used.
+
+#### `evaluateExecutionCondition`
+
+This will be called for each argument set for the current test. In fuzzing mode, there will only be the single
+empty argument set which will be enabled.
+
+#### `interceptTestTemplateMethod`
+
+This will call `invocation.skip()` which prevents invoking the test function with the default set of
+arguments `provideArguments` created. It will instead extract the `FuzzTestExecutor` instance from
+the `extensionContext` and then calls `FuzzTestExecutor#execute` which creates a `FuzzTargetRunner` to run the actual
+fuzzing.
+
+Crashes are saved in `resources/<package>/<test file name>Inputs/<test method name>` and results that are interesting to
+libfuzzer are saved in `.cifuzz-corpus`.
+
+### Regression Flow
+
+Similar to fuzzing mode, JUnit will call these methods for each test marked with `FuzzTest`.
+
+#### `evaluateExecutionCondition`
+
+This checks if the given test should be run at all. In regression mode, all tests are run so this will always return
+enabled.
+
+#### `provideArguments`
+
+This will configure the fuzzing agent as in fuzzing mode, then gather test cases to run from the following sources:
+
+1. A default argument set of just an empty input
+2. A set of arguments from the associated resources directory
+3. If a `.cifuzz-corpus` directory exists, relevant entries from that are added as well
+
+Prior to returning, the stream of test cases is put through `adaptInputsForFuzzTest` to turn the raw bytes from the
+files into the actual types to be given to the tested function.
+
+##### Resources Tests
+
+The tests from the resources directory are gathered by `walkInputs`. This will look for inputs in two places:
+
+- `resources/<package>/<test class name>Inputs` - files found directly within this directory will be used as inputs for
+  any tests within this class. This allows for easy sharing of corpus entries. Jazzer does not automatically put entries
+  here, instead a human will need to decide a finding should be shared and manually move it.
+- `resources/<package>/<test class name>Inputs/<test method name>` - files found in this directory and any directory
+  under it are used as inputs for only the test of the same name.
+
+JUnit will use the file's name as the name of the test case for its reporting. It also accepts .jar files where it will
+search with the given directory in the jar.
+
+> [!IMPORTANT]\
+> When adding these inputs files to a Git repository, consider creating a `.gitattributes` file containing
+> `src/test/resources/** binary` or similar to explicitly mark the files as binary files. Otherwise Git might
+> erroneously consider them text files, misinterpret bytes as line terminators and convert them on checkout.
+> This would change the content of the inputs files and can lead to unexpected behavior.
+
+##### Corpus
+
+The corpus kept in `.cifuzz-corpus/<test class name>/<test method name>` holds any inputs that libfuzzer found worth
+saving and not necessarily just inputs that caused a crash. Jazzer is able to set the directory but the contents of
+these directories are managed entirely by libfuzzer. Unlike with the resources test inputs above, this will not look
+in `.cifuzz-corpus/<test class name>` for shared test cases. This is a limitation of libfuzzer.
+
+#### `evaluateExecutionCondition`
+
+This will run once per argument set returned by `provideArguments` for this test. All argument sets will return as
+enabled.
+
+#### `interceptTestTemplateMethod`
+
+This will run for each individual test case for each fuzz test and will mostly just allow the test function to proceed
+with the provided arguments. Prior to the call to the test, it will enable the agent's hooks and then disable them
+afterward. It will also check for and report any findings from Jazzer to JUnit.
+
+### Diagrams
+
+Below are two sequence diagrams for how JUnit calls `evaluateExecutionConditions` and `provideArguments` in fuzzing and
+regression mode. These diagrams ignore `interceptTestTemplateMethod` for brevity as its behavior and place in the
+sequence is more clear.
+
+#### Fuzzing
+
+![created on sequencediagram.org, load the svg in the editor to edit](./images/fuzzing-flow.svg)
+
+#### Regression
+
+![created on sequencediagram.org, load the svg in the editor to edit](./images/regression-flow.svg)