[Bug]: <Detect Flaky Tests in FirestoreConvertersTest>

[LucaDai]

Related Template(s)

v2/googlecloud-to-googlecloud

Template Version

N/A

What happened?

Detected flaky behavior in unit test under NonDex randomization.

Test class: com.google.cloud.teleport.v2.utils.FirestoreConvertersTest
Test method: testCheckNoKeyAllInvalid

NonDex detected inconsistent test results caused by nondeterministic JSON string ordering when serializing and comparing error messages.

Under certain random seeds, the order of JSON fields in the serialized error string differs between runs.

Expected

Stable comparison behavior independent of JSON field order.

Environment

Java: 17.x
Maven: 3.9.x
Build module

mvn clean install -DskipTests -pl v2/googlecloud-to-googlecloud -am

Run a single test (no shuffling)

mvn test -pl v2/googlecloud-to-googlecloud \
  -Dtest=com.google.cloud.teleport.v2.utils.FirestoreConvertersTest \
  -Dmaven.test.redirectTestOutputToFile=false

Run with NonDex (shuffling)

mvn edu.illinois:nondex-maven-plugin:2.2.1:nondex \
  -pl v2/googlecloud-to-googlecloud \
  -Dtest=com.google.cloud.teleport.v2.utils.FirestoreConvertersTest \
  -DnondexRuns=5

Relevant log output

[ERROR] Tests run: 8, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 5.23 s <<< FAILURE! - in com.google.cloud.teleport.v2.utils.FirestoreConvertersTest
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   FirestoreConvertersTest.testCheckNoKeyAllInvalid:280 RemoveNoKeys/DetectInvalidEntities.failures: 
Expected: iterable with items ["{\"message\":\"Datastore Entity Without Key\",\"stacktrace\":null,\"data\":\"{\\\"properties\\\":{\\\"street\\\":{\\\"stringValue\\\":\\\"Some street\\\"},\\\"number\\\":{\\\"integerValue\\\":\\\"1\\\"}}}\"}", "{\"data\":\"{\\\"properties\\\":{\\\"street\\\":{\\\"stringValue\\\":\\\"Some street\\\"},\\\"number\\\":{\\\"integerValue\\\":\\\"1\\\"}}}\",\"message\":\"Datastore Entity Without Key\",\"stacktrace\":null}"] in any order                                                                                                                                                         
     but: not matched: "{\"data\":\"{\\\"properties\\\":{\\\"street\\\":{\\\"stringValue\\\":\\\"Some street\\\"},\\\"number\\\":{\\\"integerValue\\\":\\\"1\\\"}}}\",\"message\":\"Datastore Entity Without Key\",\"stacktrace\":null}"                                                                                                                                                                                    
[ERROR]   FirestoreConvertersTest.testCheckNoKeyBothCorrectAndInvalid:325 RemoveNoKeys/DetectInvalidEntities.failures: 
Expected: iterable with items ["{\"data\":\"{\\\"properties\\\":{\\\"street\\\":{\\\"stringValue\\\":\\\"Some street\\\"},\\\"number\\\":{\\\"integerValue\\\":\\\"0\\\"}}}\",\"message\":\"Datastore Entity Without Key\",\"stacktrace\":null}", "{\"message\":\"Datastore Entity Without Key\",\"stacktrace\":null,\"data\":\"{\\\"properties\\\":{\\\"street\\\":{\\\"stringValue\\\":\\\"Some street\\\"},\\\"number\\\":{\\\"integerValue\\\":\\\"1\\\"}}}\"}"] in any order                                                                                                                                                         
     but: not matched: "{\"data\":\"{\\\"properties\\\":{\\\"street\\\":{\\\"stringValue\\\":\\\"Some street\\\"},\\\"number\\\":{\\\"integerValue\\\":\\\"1\\\"}}}\",\"message\":\"Datastore Entity Without Key\",\"stacktrace\":null}"                                                                                                                                                                                    
[ERROR]   FirestoreConvertersTest.testCheckSameKey:209 RemoveDupKeys/ErrorOnDuplicateKeys.errors: 
Expected: iterable with items ["{\"stacktrace\":null,\"message\":\"Duplicate Datastore Key\",\"data\":\"{\\\"key\\\":{\\\"partitionId\\\":{\\\"projectId\\\":\\\"my-project\\\",\\\"namespaceId\\\":\\\"some-namespace\\\"},\\\"path\\\":[{\\\"kind\\\":\\\"monkey\\\",\\\"id\\\":\\\"1234\\\"}]},\\\"properties\\\":{\\\"someString\\\":{\\\"stringValue\\\":\\\"someValue\\\"}}}\"}", "{\"data\":\"{\\\"key\\\":{\\\"partitionId\\\":{\\\"projectId\\\":\\\"my-project\\\",\\\"namespaceId\\\":\\\"some-namespace\\\"},\\\"path\\\":[{\\\"kind\\\":\\\"monkey\\\",\\\"id\\\":\\\"1234\\\"}]},\\\"properties\\\":{\\\"SomeBSProp\\\":{\\\"stringValue\\\":\\\"Some BS Value\\\"}}}\",\"message\":\"Duplicate Datastore Key\",\"stacktrace\":null}"] in any order                                                                                       
     but: not matched: "{\"stacktrace\":null,\"message\":\"Duplicate Datastore Key\",\"data\":\"{\\\"key\\\":{\\\"partitionId\\\":{\\\"projectId\\\":\\\"my-project\\\",\\\"namespaceId\\\":\\\"some-namespace\\\"},\\\"path\\\":[{\\\"kind\\\":\\\"monkey\\\",\\\"id\\\":\\\"1234\\\"}]},\\\"properties\\\":{\\\"SomeBSProp\\\":{\\\"stringValue\\\":\\\"Some BS Value\\\"}}}\"}"