Hibernate ORM enhanced field access sometimes (randomly?) returning `null`

[yrodiere]

One test in the Hibernate ORM extension is failing from time to time and being marked as flaky: io.quarkus.hibernate.orm.applicationfieldaccess.PublicFieldAccessInheritanceTest.testFieldAccess

We get this error:

java.lang.AssertionError: 

Expecting actual not to be null
	at io.quarkus.hibernate.orm.applicationfieldaccess.PublicFieldAccessInheritanceTest$FieldAccessEnhancedDelegate$1.assertValue(PublicFieldAccessInheritanceTest.java:132)
	at io.quarkus.hibernate.orm.applicationfieldaccess.PublicFieldAccessInheritanceTest.doTestFieldAccess(PublicFieldAccessInheritanceTest.java:91)
	at io.quarkus.hibernate.orm.applicationfieldaccess.PublicFieldAccessInheritanceTest.testFieldAccess(PublicFieldAccessInheritanceTest.java:52)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at io.quarkus.test.QuarkusUnitTest.runExtensionMethod(QuarkusUnitTest.java:499)

It seems it's been flaky since at least April 16, see the test history (loading will take a while): https://ge.quarkus.io/scans/tests?search.relativeStartTime=P900D&search.timeZoneId=Europe%2FParis&tests.container=io.quarkus.hibernate.orm.applicationfieldaccess.PublicFieldAccessInheritanceTest&tests.test=testFieldAccess#

The first PR to see this test as flaky is #40329 (formerly #40160), and that PR seems way too related to the test (bytecode enhancement) for this to be a coincidence... @Sanne this may be a bug caused by the changes in that PR?

Originally mentioned here:

          @yrodiere @marko-bekhta the flaky test is a bit concerning here ^

Originally posted by @gsmet in #42455 (comment)

[quarkus-bot]

/cc @gsmet (hibernate-orm)

[yrodiere]

The first PR to see this test as flaky is #40329 (formerly #40160), and that PR seems way too related to the test (bytecode enhancement) for this to be a coincidence... @Sanne this may be a bug caused by the changes in that PR?

Actually after that PR was merged, the test wasn't flaky until August 1st...

So I think it's more likely that @Sanne had a problem in his PR that caused flakiness, but he fixed it before merging, and then sometime before August 1st another change happened that caused this flakiness to reappear, probably for a different reason...

What change, you ask? I don't know, but it's got to be something shortly before #42264

[Sanne]

This will be tricky unfortunately.
When working on that patch I remember having to figure out other, often unrelatd, race conditions, as making things faster sometimes highlights latent issues in other areas. I suppose this could be triggered by other optimisations, or potentially other fixes in combination with that one.

I'd not be keen to add "slowness" here and there :) and I doubt the issue is caused by semantic changes in my PR (although that's not entirely to be ruled out either); I'm afraid someone will need dig deep and find the deep reason.

[yuhaibohotmail]

Is there any progress on this bug? One of my projects uses Hibernate inheritance. Is it unsafe to upgrade to 3.14?

[yrodiere]

There is no progress at the moment. As this has not been investigated yet, I cannot provide guidance.
What's for sure is this only affects applications whose entities contain public fields -- if you're using getters and setters, you will be fine.

[yuhaibohotmail]

Thanks a lot!
Unfortunately, the project uses public fields with Panache, it seems I still need to wait for your investigation.

[FroMage]

Unfortunately, the project uses public fields with Panache, it seems I still need to wait for your investigation.

Actually, ORM with Panache does not use the ORM public fields system, so you should be safe with Panache. Under the hood, Panache will turn all public fields into protected fields with accessors. It's transparent to you, but this ensures that Hibernate-generated accessors are used (for lazy loading and dirty tracking).

[yrodiere]

@FroMage the failure happens in Quarkus, and nothing in Quarkus uses the ORM public fields system.

This bug happens precisely in a test that relies on Panache's own public fields system -- though Panache may not be at fault, there's probably some wrong interaction going on.

[FroMage]

@FroMage the failure happens in Quarkus, and nothing in Quarkus uses the ORM public fields system.

Oh, I was misled by the title mentioning Hibernate ORM enhanced field access, sorry.

Damn.

[gsmet]

@yuhaibohotmail are you able to reproduce this issue reliably?

[yrodiere]

@gsmet I don't think so, they were simply worried about the bug and thus didn't want to upgrade.

As far as I know, this problem was only ever encountered on our CI; nobody else reported anything similar. It's entirely possible this is related to our test setup.

Anyway... I'm currently trying to reproduce the problem with this: https://github.com/quarkusio/quarkus/compare/main...yrodiere:quarkus:i42479-reproducer?expand=1
And I must say, so far I'm failing.

Which is not surprising if you consider that even on CI, over the last 30 days it passed 751 times and was flaky just 7 times... This fails less than 1% of the time even in a consistent environment :(