This PR introduces structured fuzzing with [libFuzzer](https://llvm.org/docs/LibFuzzer.html). The expression target implementation is not complete, but already found a compatibility issue. More fuzzing targets should be moved from `tests/fuzz` to `fuzz` and benefit from more advanced fuzzing techniques. - [x] Add fuzzing guide to `README.md` - Install `cargo-fuzz`. - Use the nightly version of cargo with the `fuzz` dev shell or use rustup to switch versions. - Run `cargo fuzz run ...` - [x] Add all binary operations. # 🐞 Bugs Compatibility issue found when trying to `select ?` with a `NaN` value. Sqlite returns `NULL`, while Limbo returns `NaN` (reasonable, but incompatible). ``` thread '<unnamed>' panicked at fuzz_targets/expression.rs:130:5: assertion `left == right` failed left: Null right: Float(NaN) note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ==59288== ERROR: libFuzzer: deadly signal #0 0x00010564c0f0 in __sanitizer_print_stack_trace+0x28 (librustc-nightly_rt.asan.dylib:arm64+0x5c0f0) #1 0x0001024e7b64 in fuzzer::PrintStackTrace()+0x30 (expression:arm64+0x101c53b64) #2 0x0001024da650 in fuzzer::Fuzzer::CrashCallback()+0x60 (expression:arm64+0x101c46650) #3 0x000195fa6de0 in _sigtramp+0x34 (libsystem_platform.dylib:arm64+0x3de0) #4 0x000195f6ff6c in pthread_kill+0x11c (libsystem_pthread.dylib:arm64+0x6f6c) #5 0x000195e7c904 in abort+0x7c (libsystem_c.dylib:arm64+0x79904) #6 0x000102580990 in std::sys::pal::unix::abort_internal::hd275d720c474f43c+0x8 (expression:arm64+0x101cec990) #7 0x000102621604 in std::process::abort::h62d9ecef2f17e944+0x8 (expression:arm64+0x101d8d604) #8 0x0001024d93bc in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h3b4b43a8f9432830+0xb8 (expression:arm64+0x101c453bc) #9 0x000102577de0 in std::panicking::rust_panic_with_hook::h19683f6fd94fb24c+0x2b8 (expression:arm64+0x101ce3de0) #10 0x000102577970 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h4e98e5e8777eac5e+0x8c (expression:arm64+0x101ce3970) #11 0x0001025754e0 in std::sys::backtrace::__rust_end_short_backtrace::h12a2d70ebc9128b2+0x8 (expression:arm64+0x101ce14e0) #12 0x000102577628 in rust_begin_unwind+0x1c (expression:arm64+0x101ce3628) #13 0x000102623340 in core::panicking::panic_fmt::h8c4d74b8e5179d60+0x1c (expression:arm64+0x101d8f340) #14 0x0001026236cc in core::panicking::assert_failed_inner::he8fd1f85d57f866a+0x104 (expression:arm64+0x101d8f6cc) #15 0x0001025c73dc in core::panicking::assert_failed::h3e7590b91d46bff9 panicking.rs:364 #16 0x000100930910 in expression::do_fuzz::hfcf5c5e5fde1a31c expression.rs:130 #17 0x0001009373fc in rust_fuzzer_test_input lib.rs:359 #18 0x0001024d2f34 in std::panicking::try::do_call::hce6ebc856827ae8b+0xc4 (expression:arm64+0x101c3ef34) #19 0x0001024d9624 in __rust_try+0x18 (expression:arm64+0x101c45624) #20 0x0001024d896c in LLVMFuzzerTestOneInput+0x16c (expression:arm64+0x101c4496c) #21 0x0001024dc3cc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long)+0x148 (expression:arm64+0x101c483cc) #22 0x0001024db8dc in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*)+0x58 (expression:arm64+0x101c478dc) #23 0x0001024dd920 in fuzzer::Fuzzer::MutateAndTestOne()+0x258 (expression:arm64+0x101c49920) #24 0x0001024de908 in fuzzer::Fuzzer::Loop(std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>&)+0x38c (expression:arm64+0x101c4a908) #25 0x0001024fd120 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1bac (expression:arm64+0x101c69120) #26 0x00010250d884 in main+0x34 (expression:arm64+0x101c79884) #27 0x000195bf0270 (<unknown module>) NOTE: libFuzzer has rudimentary signal handlers. Combine libFuzzer with AddressSanitizer or similar for better crash reports. SUMMARY: libFuzzer: deadly signal MS: 3 CopyPart-ShuffleBytes-CrossOver-; base unit: a36928cfe783d55be82d526168a2da57372fdfdc 0xff,0xfd,0xff,0x3f,0x87,0x0,0x6e,0x6f,0x77,0x48,0x48,0x48,0xff,0x48,0xff,0xff,0x5b,0xff,0x5b, \377\375\377?\207\000nowHHH\377H\377\377[\377[ artifact_prefix='/Users/levy/Documents/limbo/fuzz/artifacts/expression/'; Test unit written to /Users/levy/Documents/limbo/fuzz/artifacts/expression/crash-63bfc8813b82bd8b97c557650289a6bc2c055ca5 Base64: //3/P4cAbm93SEhI/0j//1v/Ww== ──────────────────────────────────────────────────────────────────────────────── Failing input: artifacts/expression/crash-63bfc8813b82bd8b97c557650289a6bc2c055ca5 Output of `std::fmt::Debug`: Value( Real( NaN, ), ) Reproduce with: cargo fuzz run expression artifacts/expression/crash-63bfc8813b82bd8b97c557650289a6bc2c055ca5 Minimize test case with: cargo fuzz tmin expression artifacts/expression/crash-63bfc8813b82bd8b97c557650289a6bc2c055ca5 ──────────────────────────────────────────────────────────────────────────────── ``` Reviewed-by: Preston Thorpe (@PThorpe92) Reviewed-by: Pere Diaz Bou <pere-altea@homail.com> Closes #1116

VM preemption? #20

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions