compare-version-with-baselines fails for files that should be ignored

[stephan-herrmann]

JDT work in branch BETA_JAVA24 (e.g., eclipse-jdt/eclipse.jdt.core#3388) is blocked by

[2024-12-03T23:53:56.514Z] [ERROR] Failed to execute goal org.eclipse.tycho.extras:tycho-p2-extras-plugin:4.0.10:compare-version-with-baselines (compare-attached-artifacts-with-release) on project org.eclipse.jdt.core.tests.builder: Baseline and reactor have the same fully qualified version, but different content
[2024-12-03T23:53:56.514Z] [ERROR] different
[2024-12-03T23:53:56.514Z] [ERROR]    META-INF/ECLIPSE_.RSA: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]    META-INF/ECLIPSE_.SF: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]    resources/autoValueSnippet/libs/auto-value-1.6.5.jar: different
[2024-12-03T23:53:56.514Z] [ERROR]       META-INF/ECLIPSE_.RSA: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]       META-INF/ECLIPSE_.SF: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]    resources/autoValueSnippet/libs/auto-value-annotations-1.6.5.jar: different
[2024-12-03T23:53:56.514Z] [ERROR]       META-INF/ECLIPSE_.RSA: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]       META-INF/ECLIPSE_.SF: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]    workspace/Test571363.jar: different
[2024-12-03T23:53:56.514Z] [ERROR]       META-INF/ECLIPSE_.RSA: present in baseline only
[2024-12-03T23:53:56.514Z] [ERROR]       META-INF/ECLIPSE_.SF: present in baseline only

As per

eclipse.platform.releng.aggregator/eclipse-platform-parent/pom.xml

Lines 523 to 528 in e673a52

	<ignoredPatterns>
	<pattern>META-INF/ECLIPSE_.RSA</pattern>
	<pattern>META-INF/ECLIPSE_.SF</pattern>
	<pattern>META-INF/CODESIGN.RSA</pattern>
	<pattern>META-INF/CODESIGN.SF</pattern>
	</ignoredPatterns>

these files should be ignored, but they are not.

For investigation I ran with mvn -X with results in https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Github/job/PR-3388/5

We can see that the desired <ignoredPatterns> are set correctly, but this appears only in the configuration for org.eclipse.tycho:tycho-p2-plugin:4.0.10:p2-metadata-default (default-p2-metadata-default) whereas the error is raised during org.eclipse.tycho.extras:tycho-p2-extras-plugin:4.0.10:compare-version-with-baselines (compare-attached-artifacts-with-release)

So either the parent pom fails to set the ignore-filter on the necessary plugin(s) or tycho fails to evaluate it where it should.

The first guess was that this was some misconfiguration of the branch build, but:

• the few relevant differences between branches don't seem to give a hint
• recently the same failure occurred also on master

[laeubi]

@stephan-herrmann for me it looks like baseline replace do not work and as mentioned before suspect that the baseline repo is not set correctly for the branch.

So what should happen is

• metadata-default compares current build result of org.eclipse.jdt.core.tests.builder and sees nothing has changed (beside the ignored files)
• the actual org.eclipse.jdt.core.tests.builder is replaced with the baseline version
• compare-version-with-baselines will see no difference

recently the same failure occurred also on master

If you look at the PR it currently has conflicts with the master branch so you most likely need to rebase it to fix the issue.

[stephan-herrmann]

@stephan-herrmann for me it looks like baseline replace do not work and as mentioned before suspect that the baseline repo is not set correctly for the branch.

Generically blaming the branch doesn't help. I showed all relevant configuration diffs between master and BETA_JAVA24 in eclipse-jdt/eclipse.jdt.core#3388 (comment) - if any of that looks wrong please be more specific. Also the error doesn't say it failed to find files for comparison, quite to the contrary ...

So what should happen is

• metadata-default compares current build result of org.eclipse.jdt.core.tests.builder and sees nothing has changed (beside the ignored files)

And that's exactly the problem: it finds differences concerning to-be-ignored files. So ignoring doesn't seem to work.

[stephan-herrmann]

Perhaps the bug consists of several parts? Failing the build due to changes on files that should be ignored surely looks like a tycho bug. But perhaps those changes are not the actual problem and only the error message is off?

In eclipse-jdt/eclipse.jdt.core#3446 @laeubi suggested to rebase the change and indeed that same error was gone. Scanning the configuration changes brought in by that rebase, there's exactly one potentially relevant change: a test plugin was bumped up from 1.4.600.qualifier to 1.4.700.qualifier (nothing akin to a change of development streams).

Perhaps the error was trying to tell us that the current version of this test plugin was lower than something found in some baseline? Which let's me ask: What exactly is the baseline:

• is all comparison made against the latest release?
  In that case the recent micro change in master should not have had any impact
• is comparison made against the previous successful build?

If all this does not fit for a branch other than master, can we define a different baseline for BETA_JAVA24? If so, how?

[laeubi]

Failing the build due to changes on files that should be ignored surely looks like a tycho bug. But perhaps those changes are not the actual problem and only the error message is off?

I have explained it already in my previous comment, the ignore is only relevant for baseline-replace, that is if you build a new artifact that is semantically (regarding some rules and ignores) equivalent to something you have build before (aka i-build) so it is replaced.

The we have compare where the artifact is compared to the previous release and if the version is not different it must be the same.

there's exactly one potentially relevant change: a test plugin was bumped up from 1.4.600.qualifier to 1.4.700.qualifier (nothing akin to a change of development streams).

So this is not "a relevant change" it is "the relevant change" and it cause by the new development stream requiring a version bump.

Which let's me ask: What exactly is the baseline

We have two baselines:

1. eclipse.platform.releng.aggregator/eclipse-platform-parent/pom.xml

   Line 85 in 4ce13e4

   <comparator.repo>https://download.eclipse.org/eclipse/updates/4.35-I-builds</comparator.repo>

   (used by p2-metadata)

2. eclipse.platform.releng.aggregator/eclipse-platform-parent/pom.xml

   Line 129 in 4ce13e4

   <previous-release.baseline>https://download.eclipse.org/eclipse/updates/4.34/R-4.34-202411201800/</previous-release.baseline>

   (used by compare-attached-artifacts-with-release)

If all this does not fit for a branch other than master, can we define a different baseline for BETA_JAVA24? If so, how?

You need an own I-build, an own deployment and reference an own version of the parent or modify the build scripts to specify your "beta 24 i build repository" ... if all that seems easier than simply sync the JAVA_24 branch mit master it could be a solution.

[laeubi]

A concrete example was elaborated here:

• [BUG] [JSR-199] ECJ cannot resolve JPMS modules if using a user-provided file manager eclipse-jdt/eclipse.jdt.core#3446 (comment)

for me it looks like BETA_24 is missing some version bumps.

To summarize:

• ignores are only considered on baseline-replace
• if branch falls behind version increments it do not do baseline-replace anymore and therefore ignores can't take effect.
• it then later on fails due to baseline-compare finds it is different but has no version increments compared to the last release

If you still see failures after making sure version increments applied to BETA_24 please reopen.

[stephan-herrmann]

Thanks, @laeubi , with explanations here and in eclipse-jdt/eclipse.jdt.core#3446 (comment) we should be able to resolve this issue now and in the future. This is much appreciated.

Some random thoughts from a distance:

• perhaps it would be more consistent if all baseline related tasks would indeed ignore META-INF/ECLIPSE_.RSA and friends?
• if failure to perform baseline-replace can produce unexpected results downstream, would it make sense to make this problem more prominent, perhaps even let it cause the build to fail, rather than distracting users with secondary errors?