Tracking issue: Spin out a core library for JavaScript runners to build on

[davidjgoss]

Background

The direction of travel for Cucumber in the JavaScript ecosystem is to not ship our own test runner, but instead provide as nice a way as possible to extend existing test runners with Cucumber functionality.

There were already examples of this in the wild:

• https://github.com/badeball/cypress-cucumber-preprocessor
• https://github.com/vitalets/playwright-bdd
• https://github.com/portabletext/editor/tree/main/packages/racejar

And we've now released an implementation that extends the Node.js test runner: https://github.com/cucumber/cucumber-node

Whilst all of these implementations use the low-level libraries (gherkin, cucumber-expressions, tag-expressions etc) to good effect, they are all doing roughly the same thing in terms of pickles + code = test cases. We could do with a core library to provide this, reducing the maintenance burden of the existing projects and lowering the barrier to entry for new ones.

Requirements

This core should not be specific to any JavaScript runtime - it should work in the browser, on Node.js, Deno and Bun with just language features and as few dependencies as possible.

We should agree early on what is in and out of scope of this core.

In scope

• Registration and validation of support code (steps, hooks, parameter types)
• Assembly of executable test cases from pickles plus the support code
• DataTable - it would be good to handle this consistently across implementations, it makes sense to live here as it doesn't quite warrant its own polyglot library
• Methods to produce Cucumber Messages for the support code and test cases

Out of scope

• Sourcing and filtering of pickles, because this is going to vary between runner implementations e.g. cucumber-node using a module loader vs racejar that accepts text vs cucumber-js that streams from the file system. The gherkin library makes it pretty easy to go from source to pickles in a few lines of code so this is not a big concern as far as duplication goes.
• Injecting context to support code functions including the "world", because again this will vary between implementations e.g. cucumber-node injecting context as the first argument vs cucumber-js binding world to this - the test case assembly part should have a generic affordance to enable this, but nothing more
• Snippets for the same reasons as above
• Projecting the test cases onto the test runner for actual execution
• Doing anything with the Cucumber Messages, because we should not assume the implementation can or will use them

Proposal

The parts of cucumber-node that deal with these "core" capabilities are already mostly decoupled. A good first pass would be to spin out a core module in-place as a POC, and sense check that against the other existing implementations and see where it falls down.

[vitalets]

I totally support this initiative.
One thing that worries me - with so different test-runners, will not it become a very complex module to fulfill all the requirements?

For example, let's take the support code. In playwright-bdd we added tags support on step definitions level. I can write the following:

import { createBdd } from 'playwright-bdd';

const { Given, When, Then } = createBdd();

Given('a step', { tags: '@foo' }, async () => {
  // ...
});

How can it be supported with the common module?

[davidjgoss]

Thanks @vitalets.

With that specific example of tag-filtered step definitions, those aren't a first-class Cucumber concept, so I'd be a little hesitant to add them to the core library. An extension point might be an option though. I'll think about it some more.

[vitalets]

those aren't a first-class Cucumber concept, so I'd be a little hesitant to add them to the core library. An extension point might be an option though.

Agree. I just want to showcase that. It may help to shape the contract with core module.