More robust loading of agent launchers

[CharliePoole]

Pluggable agents generally contain two assemblies, the launcher and the agent itself. TestAgency loads the launcher in it's own context and launcher starts the separate agent executable as a separate process. This means that the agent is independent of the engine, as intended. It uses whatever versions of nunit.engine.api, nunit.agent.core, etc. with which it was initially built.

It was initially thought that loading the launcher in the context of the engine and runner wouldn't cause problems. But that isn't the case. See issue #1727 for a very simple change that breaks all our launchers.

It would be a substantial improvement to the design if we were able to use launchers in their own context, with the same dependencies with which they were built. Ideally, we could do this by changing the engine alone but if necessary we could modify the existing agents and re-release them if necessary, before we create any more of them.

[CharliePoole]

TASKS

• Modify ExtensionManager and ExtensionNode to create .NET Framework extensions derived from MarshalByRefObject in a separate AppDomain. Modify launcher base class, LocalProcessAgentLauncher, to derive from MarshalByRefObject. Do a pre-release so agents can use this build.
• Modify each agent launcher to derive from MarshalByRefObject and release new versions.
  • Net462PluggableAgent
  • Net80PluggableAgent
  • Net90PluggableAgent
• Update NUnit.ConsoleRunner package to reference new agent versions

[CharliePoole]

I'll explain here what the problem is and why it needs some discussion.

THE PROBLEM

The agents themselves are completely independent of the runner and engine that uses them. They run in a separate process and use whatever version of nunit.engine.api and other NUnit asemblies they were built with. This works very well.

Because launchers for each agent are loaded in-process with the engine, they need to all use the compatible versions of any dependencies. In addition, these dependencies must be compatible with the engine itself. Currently, this applies to nunit.engine.api and nunit.agent.core`.

ORIGINAL SOLUTION

The intent of the original issue was to resolve the problem by running launchers in a separate AppDomain. This seemed reasonable since launchers currently target the .NET Framework. Problems...

1. It essentially requires all extensions targeting .NET Framework to be MarshalByReference, whether they need it or not or for us to determine which extensions are MarshalByReference _without loading them~. This is non-trivial.
2. It requires a lot of classes passed across the marshaling boundary to be MarshalByReference or Serializable, introducing a great deal of communication overhead to do a few very simple operations.
3. Even if we make it work, we introduce a great deal more dependency on Windows Remoting, which we would like to get away from.

WHAT WE ACTUALLY NEED

1. We need a solution that just applies to agent launchers, not all extensions.
2. The solution should continue to work when we start using .NET 8.0 as a console platform to launch agents.
3. The solution should use as much declarative information, such as ExtensionProperties, as possible.
4. Changing the IAgentLauncher interface is definitely an option, since V4 is not yet released.

WORKAROUND FOR NOW

In the original implementation, each launcher had a small amount of duplicated code. I centralized some of it in the nunit.agent.core assembly, which made it a lot more likely for a conflict to arise. I'll duplicate that code in each of the three launchers to avoid the problem of versioning conflicts.

@nunit/engine-team @nunit/core-team Looking for alternative ideas here.

[CharliePoole]

@nunit/engine-team

The workaround is now in place for the latest 4.1.0-alpha builds of all three agent launchers. They now depend only on NUnit.Engine.Api and NUnit.Extensibility.Api. While it's possible that those could still change before their final 4.0 release, they are expected to be very stable afterwards. When they do change, it's necessary to make changes in parallel in all three launchers as I have just done.

We should still continue to think about a more robust interface definition but it's not an urgent matter at this point.