Unloading and Loading DirectRunner causes crash

[CharliePoole]

I discovered this while experimenting various ways to handle reloading assemblies in the GUI. The engine supports two approaches: (1) Reload and (2) Unload followed by Load.

If you do an Unload and then Load the same assembly, the calls are allowed. In the case of tests being run in a separate process it works. But if the tests are in the same process an exception is thrown because of trying to access an unloaded AppDomain.

Looking at the code, I think it is because the AppDomain is created in LoadPackage and unloaded in UnloadPackage. However, the creation in LoadPackage is conditioned on no package being already loaded.

In principle, the various runners could work either way. That is, they could allow reuse or not allow it at all. But whichever is chosen, the runners should all work the same regardless of whether the tests are being run in or out of process.

For my own project I'm now trying a different approach but I think this should be fixed.

[ChrisMaddock]

This sounds similar to #325 - which is fixed in the latest framework master, but not in a release yet. What do you think @CharliePoole - worth trying?

If it is that, I wonder if we can come up with some sort of workaround for the engine too. I'm not sure if it'll be possible. 🤔

[CharliePoole]

@ChrisMaddock This is similar but I think it's a somewhat different issue. The engine actually gets rid of the domain itself when you call Unload. That makes the domain runner unusable but there are still references to it.

It looks like the engine Reload call does what I want - at first I didn't think it was working. So there is really no reason to unload and then load the runner with the same files. I think a good error message would do the job here. Then if problems come up with Reload - because we have never really exercised it with the console runner - we can fix them.

Basically, when you get a runner from the engine for a testpackage, an entire structure of subordinate runners is created that match the contents of the package. If you were allowed to reuse a runner once it was unloaded, then you might give it a completely different test package that didn't map to the pre-existing runner hierarchy at all! So I think that a runner, once unloaded, has to be unusable. Maybe we should dispose of it at that point and rely on the .NET message from usiing a disposed object.