Support Cucumber reporters

[vitalets]

Cucumber has several built-in reporters (also called formatters). It would be useful to output results of Playwright test runner to these reporters (especially html-reporter or messages reporter).
There are different approaches:

1. Write custom Playwright reporter that will create Cucumber report
2. Use Playwright json report and convert it into Cucumber messages, and then convert to html

Ideally, final usage should allow to use any Cucumber report:

import { defineConfig, devices } from '@playwright/test';
import { cucumberReporter } from 'playwright-bdd';

export default defineConfig({
  // ...
  reporter: cucumberReporter('json:cucumber-report.json'),
});

Known problem - reporting Scenario Outline.
In Cucumber it is possible to have the same test title with different data. In Playwright each test must have unique title (it was also discussed in #3 (comment)). Example of Cucumber html report for Scenario Outline:

I would appreciate more ideas on that from your reporting usage.

[evix1101]

Not elegant, and my needs are currently minimal. I'm supporting tests against multiple environments at the project level via extending PlaywrightTestProject

import { defineConfig, devices, PlaywrightTestProject } from '@playwright/test';

export interface ExtendedPlaywrightTestProject extends PlaywrightTestProject {
  testEnv?: string;
}

const config = defineConfig({ 
  testDir: './.features-gen',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: process.env.CI ? [ ['html'], ['junit', { outputFile: 'test-results/results.xml' }] ] : [ ['html'],['list']],
  use: {
    trace: 'retain-on-failure',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'prod-chromium',
      use: {
        ...devices['Desktop Chrome'],
        baseURL: 'https://www.google.com/',
      },
      testEnv: 'prod',
    } as ExtendedPlaywrightTestProject, 
    {
      name: 'dev-chromium',
      use: {
        ...devices['Desktop Chrome'],
        baseURL: 'https://dev.google.com/',
      },
      testEnv: 'dev',
    } as ExtendedPlaywrightTestProject,
  ],
});

export default config;

My thought was there might be a slick way to make this more dynamic to support the scenario outlines.

[Jag46]

import { defineConfig, devices } from '@playwright/test';
import { cucumberReporter } from 'playwright-bdd';

export default defineConfig({
// ...
reporter: cucumberReporter('json:cucumber-report.json'),
});

Hi @vitalets , when can we expect cucumberReporter function? It would be great if get this up running soon..Thank you..!

[vitalets]

@Jag46 can't give concrete date but will increase the priority for that.
Btw are you interested in particular json reporter or html?

[Jag46]

Thanks @vitalets , we are looking for a cucumber html report. That would great addition if you can help us with this blocker. Thanks

[harikishanl]

We are looking for a cucumber style html report as well, the current report is poorly formatted in playwright style and is not really making use of the BDD style reporting. If we can bring this enhancement soon, it will help our team use the html report without much hassle.

[vitalets]

We are looking for a cucumber style html report as well, the current report is poorly formatted in playwright style and is not really making use of the BDD style reporting. If we can bring this enhancement soon, it will help our team use the html report without much hassle.

Ok, got it. Could you clarify what parts of cucumber html reporter are the most important for you? E.g. formatting Scenario Outlines or something else? As there are a lot of events in both playwright and cucumber test run flows.

[harikishanl]

We are looking for a cucumber style html report as well, the current report is poorly formatted in playwright style and is not really making use of the BDD style reporting. If we can bring this enhancement soon, it will help our team use the html report without much hassle.

Ok, got it. Could you clarify what parts of cucumber html reporter are the most important for you? E.g. formatting Scenario Outlines or something else? As there are a lot of events in both playwright and cucumber test run flows.

Hi @vitalets - It would be helpful if we can cover: 'Feature, Scenario Outline, Scenario, Given, When, Then & Examples' in the report

[vitalets]

Cucumber messages can be validated with https://github.com/cucumber/compatibility-kit