Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No test events generated for Project elements #601

Open
CharliePoole opened this issue Apr 3, 2019 · 0 comments
Open

No test events generated for Project elements #601

CharliePoole opened this issue Apr 3, 2019 · 0 comments

Comments

@CharliePoole
Copy link
Member

PR #596 (currently awaiting merge) creates Project test-suite elements (i.e. tests) for each project being run, with one or more Assembly test-suites as children of the project.

This solves the problem of the absence of project elements in the result XML and in displays constructed by any GUI that shows the entire test tree (e.g. TestCentric) but creates one additional, albeit minor, problem.

Virtually all test events are generated by the framework itself. Currently, the only exception is the start of run and end of run events, which are generated by the engine. In principle, any test nodes created by the engine should have test start and end events corresponding to them.

The lack of these events is only visible in a GUI that shows the entire tree of tests. In TestCentric GUI, for example, the tree node representing the project is never flagged as passed, failed or even run because no event is ever received. The framework, of course, can't generate such events, since it knows nothing about projects. The highest level test for which the framework generates start and finish events is the assembly.

This could be handled in the MasterTestRunner, which is currently the only runner that knows about projects, in one of two ways:

  1. Special handling for project nodes.
  2. Special handling for Engine-created nodes (currently only test-run and project)

The latter is more general and may require a new level of abstraction to deal with engine-created tests.

In either case, what needs to be done is

  1. Note which projects contain which assemblies. This info is available in both the TestPackage and the constructed tree of tests. It may need to be made more accessible through some sort of assembly to containing project lookup.

  2. Intercept and examine events from the framework before passing them on to the listeners.

  3. When the first assembly for a project starts, issue the project start event before passing on that for the assembly.

  4. When the last assembly for a project finishes, issue the project end event after passing on that for the assembly.

This could have been done as part of #596 but seems a bit complicated such a small problem, so I decided to make it a separate issue. Currently, the only side effect is that the project tree nodes remain grey rather than changing color and it's probably possible to build a workaround into the GUI until this is fixed in the engine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant