Feature Test Summary

[tjwatson]

Test Strategy

Most of the testing for the feature is done the Open Liberty fat project com.ibm.ws.classloading_classpath_fat which already has a large number of library JARs for testing classpath order and content. Within this project there is a server configuration called overrideLibWarTest which sets up an application <classloader/> configuration with the following references:

1. Two overrideLibrary references for libraries A and B that are expected to override classes contained in the application.
2. Two privateLibrary references for library C and D that have duplicate classes from the application, but are NOT expected to override the application classes
3. One classProvider reference to a resourceAdaptor which also has a classloader configuration with an overrideLibrary to library E which is expected to override classes contained in the resourceAdator

Positive tests verify

1. That libraries A, B and E correctly override the expected classes.
2. That all libraries correctly contribute resources to the class loaders

Negative tests verify

1. That privateLibrary references cannot override resources or classes from the application's class path.

Additional tests are privided for CDI bean visibility and JDBC driver discover from privateLibrary references in the following fat projects

• com.ibm.ws.cdi.visibility_fat - verifies beans in override libraries are visible and that code in override libraries have visibility to beans in the application as expected
• com.ibm.ws.jdbc_fat - verifies that jdbc libraries can be discovered from privateLibrary references.

List of FAT projects affected

• com.ibm.ws.classloading_classpath_fat
• com.ibm.ws.cdi.visibility_fat

Confidence Level

Collectively as a team you need to assess your confidence in the testing delivered based on the values below. This should be done as a team and not an individual to ensure more eyes are on it and that pressures to deliver quickly are absorbed by the team as a whole.

Please indicate your confidence in the testing (up to and including FAT) delivered with this feature by selecting one of these values:

0 - No automated testing delivered

1 - We have minimal automated coverage of the feature including golden paths. There is a relatively high risk that defects or issues could be found in this feature.

2 - We have delivered a reasonable automated coverage of the golden paths of this feature but are aware of gaps and extra testing that could be done here. Error/outlying scenarios are not really covered. There are likely risks that issues may exist in the golden paths

3 - We have delivered all automated testing we believe is needed for the golden paths of this feature and minimal coverage of the error/outlying scenarios. There is a risk when the feature is used outside the golden paths however we are confident on the golden path. Note: This may still be a valid end state for a feature... things like Beta features may well suffice at this level.

4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.

5 - We have delivered all automated testing we believe is needed for this feature. The testing covers all golden path cases as well as all the error/outlying scenarios that make sense. We are not aware of any gaps in the testing at this time. No manual testing is required to verify this feature.

Based on your answer above, for any answer other than a 4 or 5 please provide details of what drove your answer. Please be aware, it may be perfectly reasonable in some scenarios to deliver with any value above. We may accept no automated testing is needed for some features, we may be happy with low levels of testing on samples for instance so please don't feel the need to drive to a 5. We need your honest assessment as a team and the reasoning for why you believe shipping at that level is valid. What are the gaps, what is the risk etc. Please also provide links to the follow on work that is needed to close the gaps (should you deem it needed)

Confidence level is 4

[dave-waddling]

@tjwatson: Thanks for completing this FTS, I appreciate the clarity. Having reviewed, the test coverage makes sense and seems perfectly sufficient. There's no outstanding technical debt of defect work, and so I'll hope to approve on Monday once we've triaged the SOE. To that end, please can you link the final PR that was merged so I can be sure your changes were in the run?

[tjwatson]

Here is the last PR #32777
This is currently in the release branch so I hope that means it will be in the SOE.

[dave-waddling]

@tjwatson: Many thanks! The good news is that that PR made it in by the skin of its teeth and that the testing in question is passing cleanly. I'll add the approval to #32477 now.