testcentric-gui and extensions

[cowinnie]

Hello,
I have a question about installing extensions in Testcenric. I followed everything exactly as it can be found in the instructions for NUnit3 and testcentric. I wrote an extension with the interface IResultWriter. I put this dll in an addins folder and specified the relative path to it in the .addins file. Everything in the Nunit directory (unzipped zip file version 1.6.1). When I start Testcentric's exe and find the dll, the GUI does not open. If no dll is found, the GUI opens. Can I look up where the problem is, e.g. in a log file? I use version 3.13.0 for NUnit. The actual unit tests are also running, but I don't have any extensions to run. What I simply want is an output of the unit test results as xml.
Can anyone help me further?

[cowinnie]

The code for the extension dll looks like this:

[Extension]
[ExtensionProperty("Format", "custom")]
public class ResultWriter : IResultWriter
{
public void CheckWritability(string outputPath)
{
//...
}

public void WriteResultFile(XmlNode resultNode, string outputPath)
{
//...
}

public void WriteResultFile(XmlNode resultNode, TextWriter writer)
{
//...
}
}

[TypeExtensionPoint(Description = "Supplies a writer to write the result of a test to a file using a specific format.")]
public interface IResultWriter
{
void CheckWritability(string outputPath);
void WriteResultFile(XmlNode resultNode, string outputPath);
void WriteResultFile(XmlNode resultNode, TextWriter writer);
}

[CharliePoole]

The outline of your definition of the extension looks OK to me.

However, you should not be defining IResultWriter, because that's already defined in the nunit.engine.api assembly. By defining it, your code is implementing your own version of IResultWriter, not NUnit's.

[cowinnie]

You were right. that was the mistake. Thank you very very much. I looked for my mistake for three days. I am so happy now. Thanks alot.

[cowinnie]

Unfortunately, I am already facing the next problem. I also wanted to add an EventListener as an addin. Did this exactly as with the result writer. Just use the example from the NUnit page:

[Extension(Description = "Test Reporter Extension", EngineVersion = "3.4")]
public class MyTestEventListener : ITestEventListener
{
public void OnTestEvent(string report)
{
Console.WriteLine(report);
}
}

What am I doing wrong here? I also changed the version to 3.11, 3.11.1 and 3.11.1.0, but the extension is not displayed in the Gui.

[cowinnie]

The next problem is that the ResultWriter extension is not running. Do I have to specify a value for the property values as in the description for Nunit?
nunit-console test.dll --result=CustomResult.xml;format=custom
If I start my call like this ...
start "" "NUnit\bin\testcentric.exe" UnitTests.dll --result=CustomResult.xml;format=custom
I get an error displayed and the GUI does not start. The error says the following: Invalid argument: --result=CustomResult.xml;format=custom

[CharliePoole]

There is no such option to the GUI. Use --help to see a list of the command-line options or check the documentation at https://test-centric.org/testcentric-gui/command-line

It seems that the dialog used for saving results needs to include a format dropdown, populated with the available result formats based on what extensions are installed. That's a new feature, but not terribly complicated.

Shall we rename this issue to track that change or shall I create a new one while we keep using this to discuss your overall problems with extensions?

[CharliePoole]

Re your test writer extension: Why are you specifying a minimum engine version at all?

[cowinnie]

You are welcome to open a new case where we can discuss everything. That would be very nice.

[cowinnie]

The problem with the EventListener has been resolved. I thought I had to bring in this trait. I had only used the basic example from the NUnit page. But if I leave the properties out it works and my code is even executed :-)
Thank you also here for your help,

[CharliePoole]

I'm making a note to test setting the minimum engine version. However, it could be that your problem resulted from simply not specifying the correct engine version, which is not related to the NUnit framework version. The minimum version should only be specified if you require some feature not present in earlier engine versions. I doubt that's the case here.

[CharliePoole]

I opened a new issue to track setting the format and labeled this issue as a Question - or series of Qs. 😄

[cowinnie]

Yes, you are right. I don't need the version. Without specifying the version, everything works as I want it to. Many thanks for the help.
I have a project where everything worked with NUnit2. The output of the test results was also implemented differently. With the switch to Nunit3 everything should work as before. Thanks to your help, it does now :-)

[cowinnie]

A question about extensions using IResultWriter:
How can you use such an extension? I implemented this just like an extension with ITestEventListener. The extension is also recognized in the user interface of testcentic, but apparently not called. I just implemented a folder creation there, but it won't be created. A TestResult.xml is always created in the folder from which I run my cmd, which opens my tests with testcenric. But that has nothing to do with the extension.

When you need to specify the location and format [ExtensionProperty ("Format", "custom")] how and where can this be specified? As I said, I call up the tests via a cmd file and you can't pass this specification there.
The NUnit homepage (https://docs.nunit.org/) only says the following:
"An ExtensionPropertyAttribute must be provided giving the name of the format you support. Users would access the format from the NUnit Console command-line by using that name in a result specification, such as:
nunit-console test.dll --result = CustomResult.xml; format = custom"

Info:
Since I am now using ITestEventListener to create my own result outputs, I don't need IResultWriter, but I would still be interested in how to get such an extension to work.

[CharliePoole]

@cowinnie That's why I created issue #671

For the result writer, the user needs to be able to specify that the format should be "custom". In the GUI, the natural place to do that is when specifying the location to save the XML. If you have further questions or comments about that issue, we should discuss them there.

Each type of extension is called at a particular point. The ITestEventListener is the only one that called during the running of the tests. It's not suitable for things that are done before the tests start or after the test run is complete. Saving the result file only happens after the test run is over. You could have some sort of a log that you created while the test was running, but won't have any effect on the NUnit result file.

Note that the description of how to create a ResultWriter extension on nunit.org is only referring to how a batch command-line runner like nunit3-console could specify the format to be used. The NUnit project, of course, doesn't have a GUI runner. The current version 1 GUI isn't really intended to be used as a batch runner in a CI environment. I plan to expand its capabilities for version 2, in which case a command-line option may make more sense.

[cowinnie]

@cowinnie That's why I created issue #671 --> Thank you :-)

[cowinnie]

One other question. I hope the last one ;-)

Maybe I am asking in the wrong place, but maybe you also have an idea?
My tests are running to test the executability of my code in another program. This program writes out a log file. In this log file there is often the line:

Resolve failed: nunit.framework, Version = 3.13.0.0, Culture = neutral, PublicKeyToken = 2638cd05610744eb

This line makes up more than half of the entire log file.
Any idea why that could be? Is there a possibility in the NUnit settings to prevent such a flood of lines?

[CharliePoole]

Since this is appearing in your own log file, you would have to control it somehow in your own code. My guess is that you are using an assembly resolve hook in the AppDomain where the test is running. So it's a question of how your code interacts with NUnit.

And of course, the nunit framework has nothing to do with TestCentric.

I'll mark this question closed for now, but if you need to follow-up, I'll still see it here.