Forum Discussion

vincent-psarga's avatar
vincent-psarga
SmartBear Alumni (Retired)
5 years ago

Cucumber-Ruby 4.0.0 has been released

This is the first stable Cucumber release of the 4.x series. It's the first stable release for 2 years. Upgrading from 3.x should not require any changes, but there are some deprecations (see below). We also have some exciting new features!

 

Added new Gherkin syntax: Rules and Examples

 

One of the most visible change in the new release is the support of the Gherkin 6+ syntax. This is still backward compatible with your existing features of course, but here are the changes introduced:

  • the Rule keyword acts as a grouping of scenarios inside a feature
  • the Scenario Outline keyword is not needed anymore when using Examples with a Scenario
  • the Scenario keyword now has synonym: Example

If you are familiar with example mapping, you will see how easy it is to translate your cards from an example mapping session into a feature file.

 

support of the cucumber-messages protocol

 

cucumber-messages is the new format that is used by Cucumber to represent the various information emitted during an execution.

This library satisfies multiple interests:

  • same output for all cucumber and related implementation (cucumber-ruby, cucumber-jvm, cucumber-jss, SpecFlow ...)
  • it relies on protobuf, so it is fairly easy to generate a consumer in any language supported by protobuf.
  • we have published libraries to consume this format for ruby, Java, JavaScript and C#
  • it is easier to generate tools consuming this output

To generate the new messages, you can use the message reporter.

Example:

bundle exec cucumber --format message

 

sending results to a server

 

You now have the option to send the results of the execution directly to a web server. You can achieve this by specifying an http or http URL to the --out parameter.

Example:

bundle exec cucumber --format messages --out "http://example.com/report-service"

 

HTML reporter

 

We've added a brand new HTML reporter in this release. Everything is embedded in a single file (no CSS or JS file), so you can easily share the generated page.

 

Example:

bundle exec cucumber --format html --out report.html

 

Deprecated JSON formatter

 

With the introduction of the message formatter, we have decided to deprecate the JSON formatter. It will be removed in the 5.0.0 release. After the removal, you will still have the possibility to export to JSON using the json-formatter which takes cucumber-messages as input.

 

puts

 

When using puts in your step definitions, the data you output is caught by cucumber and then processed by the formatter (for example, when using the json formatter, the data provided to puts will appear in the output field of the step). This solution is now deprecated in favor of the log method. In cucumber-ruby 5, calls to puts will not be caught by the formatter and the default Ruby behavior will be triggered.

 

embed

 

The embed method to add attachments is also deprecated in favor of the attach method. This is part of task to make the different implementations of cucumber more uniform.

Cucumber-Ruby
productUpdates
  • dmaisells's avatar
    dmaisells
    Occasional Contributor
    5 years ago

    I am using the new --format html --out somefile.html

     

    What happened to the pass/fail summary of the tests at the top; that was rather nice

    I use log "string".red   and the colorize gem in my scripts yet the html does not translate the colored text

     

    Also I used to be able to specify just tags at a root directory level and it would search all folders and sub-folders for those tags for scripts to run.  Now I must specify the exact file where the tags are located or up to one parent folder above.  Why does the HTML output contain all the scripts that weren't selected via tags (true they are all grayed out but makes it hard to look for one scripts let's say of a file of 20 scenarios)

     

    • vincent-psarga's avatar
      vincent-psarga
      SmartBear Alumni (Retired)
      5 years ago

      Thank for the feedback 🙂

       

      So, in order:

       - global pass/fail summary on the top: definitely a great idea to put back. I've create an issue on out tracker about that

       - when using string.red, it will produce characters recognized by the terminal but not the browser used for rendering the text. I also made another issue about that.

       - we are currently working on having filtering/searching capabilities in the report, meaning you should be able to use tag expression to filter out the not-run scenarios (or other ways to filter to output, maybe simply not show at all those that have not been ran).

       

      Once again, thank you for the feedback 🙂

      • dmaisells's avatar
        dmaisells
        Occasional Contributor
        5 years ago

        Vincent

        Thanks a lot for following up.  Seems like a lot of functionality was lost and being a QA engineer of 27 years, I am like "What the heck happened?"  Is there a QA team there which checks new releases of cucumber or is that left to community at large?   I would love to be on a team that evaluates a beta version of a release since I love Cucumber and have not been fully convinced to jump ship to the rspec train.

    • dmaisells's avatar
      dmaisells
      Occasional Contributor
      5 years ago

      The other thing I noticed about the .html file in Cucumber 4.0.0. is it looks like the file is NOT written until ALL scenarios are complete.  With Cucumber 3.0.0 and using the command syntax of  ...--format html > somefile.html   the html file used to be written as the script executed so a tester could follow along the progress of scripts, halting their execution if enough failures dictated that.   This was especially important for a several hour suite to be able review what passed and failed already while the scripts continued to execute.

      • vincent-psarga's avatar
        vincent-psarga
        SmartBear Alumni (Retired)
        5 years ago

        This is still possible using the message formatter and the stand-alone HTML reporter.

         

        gem install cucumber
gem install cucumber-html-formatter

// Run the tests and output the results as the new message format:
cucumber --format message > output.ndjson

// When checking for the current execution status
cat output.ndjson | cucumber-html-formatter > report.html

         

  • KLangendorf's avatar
    KLangendorf
    Frequent Visitor
    5 years ago

    Hi 

    I am trying to follow the github instructions https://github.com/cucumber/cucumber/tree/master/json-formatter#cucumber-json-formatter on how to convert the --format message to a json file containing the test results.

    Described my problem here:

    https://stackoverflow.com/questions/63466645/cucumber-json-formatter-for-ruby-2-6-how-to-convert-from-message-to-json?stw=2

     

    Do I need the Go command line tool ? ...how to get that tool to use the formatter?

     

    What am I doing wrong/missing?

     

    kr Kirsten