Hibernate batch-fetching seems to be broken in some situations where it works with pure Hibernate

[lbilger]

Describe the bug

It seems that any database query executed after the one that loads the entity makes
the entity no longer eligible for batch fetching of lazy associations or element collections.

Expected behavior

Batch fetching should apply regardless of other queries being issued.

Actual behavior

Batch fetching is not applied if a query - even an unrelated one - is issued after the entity is loaded.

How to Reproduce?

Clone https://github.com/lbilger/quarkus-hibernate-batch-fetch-reproducer and run ./mvnw test.

Watch the console output. The ExampleEntityQuarkusTest will fetch
the anElementCollection of each ExampleEntity individually, while the ExampleEntityStandaloneTest will
fetch all in one batch.

Output of uname -a or ver

Darwin xyz 22.6.0 Darwin Kernel Version 22.6.0: Tue Nov 7 21:42:24 PST 2023; root:xnu-8796.141.3.702.9~2/RELEASE_ARM64_T6020 arm64

Output of java -version

openjdk version "21.0.2" 2024-01-16 OpenJDK Runtime Environment GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30) OpenJDK 64-Bit Server VM GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30, mixed mode, sharing)

Quarkus version or git rev

3.11.1

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) Maven home: /Users/xyz/.m2/wrapper/dists/apache-maven-3.9.6-bin/3311e1d4/apache-maven-3.9.6 Java version: 21.0.2, vendor: GraalVM Community, runtime: /Library/Java/JavaVirtualMachines/graalvm-community-openjdk-21/Contents/Home Default locale: de_DE, platform encoding: UTF-8 OS name: "mac os x", version: "13.6.3", arch: "aarch64", family: "mac"

Additional information

This is not related to native build. I'm not sure if the problem is in Quarkus or in Hibernate itself, but as the test shows, a pure Hibernate setup works. The only difference I can see except for Quarkus being involved is the transaction management (JTA in Quarkus vs. resource-local in the pure Hibernate setup).

[quarkus-bot]

/cc @gsmet (hibernate-orm), @yrodiere (hibernate-orm)

[yrodiere]

Hey,

Thanks for reporting.

but as the test shows, a pure Hibernate setup works

One thing you could try is to base your reproducer on https://github.com/hibernate/hibernate-test-case-templates/blob/main/orm/hibernate-orm-6/src/test/java/org/hibernate/bugs/QuarkusLikeORMUnitTestCase.java . If it fails there, then it's definitely a problem in Hibernate ORM, but possibly in a feature Quarkus enables by default, like bytecode enhancement. That would justify reporting directly to Hibernate: https://hibernate.atlassian.net/browse/HHH

[lbilger]

Hi Yoann, thanks for the suggestion! I have added a test based on QuarkusLikeORMUnitTestCase (see https://github.com/lbilger/quarkus-hibernate-batch-fetch-reproducer/tree/main/hibernate-testing). The result is the same as in the ExampleEntityStandaloneTest, i.e. the batch fetching works.

[yrodiere]

Okay that's concerning, we'll need to have closer look. Thanks for checking.

[yrodiere]

I had a look at the reproducer, and it turns out:

1. Executing a query may trigger an auto-flush, depending on a few things.
2. An auto-flush will invalidate all candidates for batch-loading.
3. Quarkus explicitly requires that auto-flush always happens, for some reason:
   

   quarkus/extensions/hibernate-orm/runtime/src/main/java/io/quarkus/hibernate/orm/runtime/session/JTASessionOpener.java

   Line 35 in b2cddcc

   .flushMode(FlushMode.ALWAYS);

This makes me think this probably isn't a new problem.

@lbilger can you confirm you're not reporting this as a regression, i.e. you don't have a precise version of Quarkus in mind where this used to work as you expect?

I tracked down this choice of using FlushMode.ALWAYS to... an apparently unrelated commit by some idiot (me): d9ed7bc#diff-3c3d11a887a43713907202d00e1d26f8d664c00c408de92e21904f76f10d7b84R35
I'm really not sure why we would force FlushMode.ALWAYS... I'll inquire.

EDIT: I started a discussion on the Hibernate chat: https://hibernate.zulipchat.com/#narrow/stream/132094-hibernate-orm-dev/topic/FlushMode.2EALWAYS

[lbilger]

Hi @yrodiere, thanks for investigating and fixing this issue! Yes, I can confirm this is not a regression. We have tested it in several Quarkus versions and the result was always the same.

As we are using Red Hat build of Quarkus in the real-world project where this issue occurred, it might take a while until we can benefit from the fix. Is there a way to override the flush mode configuration manually?

[yrodiere]

Yes, I can confirm this is not a regression. We have tested it in several Quarkus versions and the result was always the same.

Thanks.

As we are using Red Hat build of Quarkus in the real-world project where this issue occurred

I would suggest going through Red Hat support if you have a RHBQ subscription, as you will at least have a guarantee of timely answers -- I can't personally guarantee timely answers on all GitHub issues, unfortunately.
Feel free to open a GitHub issue with the full details, and just link to it in your support request.

it might take a while until we can benefit from the fix.

True. Backporting this change is probably not an option, as we have to balance potential improvements against the risk of regression to other users.

Is there a way to override the flush mode configuration manually?

You can call session.setHibernateFlushMode(FlushMode.AUTO) on a session as a workaround. Make sure to call it as early as possible, before you run queries. That should cover specific code where you saw the problem happen.

Now I suppose you'd want a more "global" workaround that would automatically set the right flush mode on all sessions... I can't see any Hibernate event that would allow you to do this, but assuming you always use transactions for Hibernate sessions, and always use Hibernate sessions in transactions, you could consider registering a CDI event listener:

    // Put this in an `@ApplicationScoped` bean somewhere.
    void onBeginTransaction(@Observes @Initialized(TransactionScoped.class) Object event, Session session) {
        session.setHibernateFlushMode(FlushMode.AUTO);
    }

I didn't test this, so I suggest you try it first, and carefully check it solves your problem. But in theory, it should.