Add tsan sanitizer ?

[boxerab]

Hello!
I just ran the tsan sanitizer on my code and found a race condition. So, I think it would be a nice
addition to the existing fuzzer sanitzers.
Cheers,
Aaron

[jonathanmetzman]

Thanks for mentioning this. It's a good data point.
As #5266 mentions, we aren't planning on supporting TSAN in ClusterFuzz for now (I assume one reason is because the slowdown 5x-15x is enormous compared to other sanitizers).
Do you use CIFuzz? I could try to help you set up TSAN there if you'd like.

[boxerab]

thanks. Yes, I can see your point, it's a lot slower, and uses a lot more memory, with TSAN.
I am not using CIFuzz - wasn't aware of it until your post. But I would like to - great idea.

[boxerab]

Let me work on setting up CIFuzz, and then we can take a look at TSAN - thanks!

[boxerab]

@jonathanmetzman I've set up CIFuzz for my project:

https://github.com/google/oss-fuzz/tree/master/projects/grok
https://github.com/GrokImageCompression/grok/actions

Unfortunately, the seed corpus images that I clone (working correctly in ozz-fuzz) are not found in CIFuzz, so zipped corpus is not created and CIFuzz fails.
It looks like the two repos that are cloned in the oss-fuzz Dockerfile (one is for the code and one is for the seed corpus) are not available to the CIFuzz image, which does a second clone of the code and builds from there (but seed images are missing).

[boxerab]

I will close this issue, and open a new one re CIFuzz

[jonathanmetzman]

I'll help you figure this out. But my first guess is this is easy to fix.
basically, CIFuzz clobbers your source repo (because it needs to checkout something other than the default branch, it does a new clone of the repo (to avoid complexity with running code in multiple docker containers)). So if your build depends on a modification done to the repo directory (a modification that isn't in git history) CIFuzz breaks.
So if you change this line to

RUN git clone --depth 1 https://github.com/GrokImageCompression/grok-test-data.git grok-data

And this file: https://github.com/GrokImageCompression/grok/blob/master/tests/fuzzers/build_seed_corpus.sh#L1 to use grok-data, things should work.
Sorry, I really should document this (or maybe fix this behavior).

[jonathanmetzman]

FYI, I've never tested TSAN in CIFuzz before. Doing this now here

[jonathanmetzman]

I actually just tried fuzzing with tsan and it looks like more support is needed since libFuzzer isn't thread safe according to TSAN.
@kcc do you think it's worth making libFuzzer work with TSAN (including writing crashes when it finds them)? Or would you prefer we link against a special driver that executes the corpus instead of actually fuzzing?

[kcc]

My preference has always been to run tsan on the corpus using the plain test driver (no special driver needed)

[kcc]

To explain:

• tsan is much slower (and takes more RAM) than asan/msan/ubsan, and thus fuzzing with tsan will be more expensive per one mutant.
• tsan+libFuzzer requires additional work (e.g. the coverage counters are racy and tsan complains about them, IIRC, will need to be fixed). This is doable, and not even too much work, but some.
• The majority of our fuzz targets are single-threaded, so the ROI will be relatively small.
• I expect that the amount of races we can find this way is not much greater than the races we can find by running tsan on the corpus, w/o fuzzing. The only way to see if my expectation is right is to try. It's a good experiment to do, just not on top of my TODO list.