Cucumber-Ruby 4.0.0 has been released

[vincent-psarga]

This is the first stable Cucumber release of the 4.x series. It's the first stable release for 2 years. Upgrading from 3.x should not require any changes, but there are some deprecations (see below). We also have some exciting new features!

Added new Gherkin syntax: Rules and Examples

One of the most visible change in the new release is the support of the Gherkin 6+ syntax. This is still backward compatible with your existing features of course, but here are the changes introduced:

• the Rule keyword acts as a grouping of scenarios inside a feature
• the Scenario Outline keyword is not needed anymore when using Examples with a Scenario
• the Scenario keyword now has synonym: Example

If you are familiar with example mapping, you will see how easy it is to translate your cards from an example mapping session into a feature file.

support of the cucumber-messages protocol

cucumber-messages is the new format that is used by Cucumber to represent the various information emitted during an execution.

This library satisfies multiple interests:

• same output for all cucumber and related implementation (cucumber-ruby, cucumber-jvm, cucumber-jss, SpecFlow ...)
• it relies on protobuf, so it is fairly easy to generate a consumer in any language supported by protobuf.
• we have published libraries to consume this format for ruby, Java, JavaScript and C#
• it is easier to generate tools consuming this output

To generate the new messages, you can use the message reporter.

Example:

bundle exec cucumber --format message

sending results to a server

You now have the option to send the results of the execution directly to a web server. You can achieve this by specifying an http or http URL to the --out parameter.

Example:

bundle exec cucumber --format messages --out "http://example.com/report-service" 

HTML reporter

We've added a brand new HTML reporter in this release. Everything is embedded in a single file (no CSS or JS file), so you can easily share the generated page.

Example:

bundle exec cucumber --format html --out report.html

Deprecated JSON formatter

With the introduction of the message formatter, we have decided to deprecate the JSON formatter. It will be removed in the 5.0.0 release. After the removal, you will still have the possibility to export to JSON using the json-formatter which takes cucumber-messages as input.

puts

When using puts in your step definitions, the data you output is caught by cucumber and then processed by the formatter (for example, when using the json formatter, the data provided to puts will appear in the output field of the step). This solution is now deprecated in favor of the log method. In cucumber-ruby 5, calls to puts will not be caught by the formatter and the default Ruby behavior will be triggered.

embed

The embed method to add attachments is also deprecated in favor of the attach method. This is part of task to make the different implementations of cucumber more uniform.

[dmaisells]

I am using the new --format html --out somefile.html

What happened to the pass/fail summary of the tests at the top; that was rather nice

I use log "string".red   and the colorize gem in my scripts yet the html does not translate the colored text

Also I used to be able to specify just tags at a root directory level and it would search all folders and sub-folders for those tags for scripts to run.  Now I must specify the exact file where the tags are located or up to one parent folder above.  Why does the HTML output contain all the scripts that weren't selected via tags (true they are all grayed out but makes it hard to look for one scripts let's say of a file of 20 scenarios)

[vincent-psarga]

Thank for the feedback 🙂

So, in order:

 - global pass/fail summary on the top: definitely a great idea to put back. I've create an issue on out tracker about that

 - when using string.red, it will produce characters recognized by the terminal but not the browser used for rendering the text. I also made another issue about that.

 - we are currently working on having filtering/searching capabilities in the report, meaning you should be able to use tag expression to filter out the not-run scenarios (or other ways to filter to output, maybe simply not show at all those that have not been ran).

Once again, thank you for the feedback 🙂

[dmaisells]

Vincent

Thanks a lot for following up.  Seems like a lot of functionality was lost and being a QA engineer of 27 years, I am like "What the heck happened?"  Is there a QA team there which checks new releases of cucumber or is that left to community at large?   I would love to be on a team that evaluates a beta version of a release since I love Cucumber and have not been fully convinced to jump ship to the rspec train.

[vincent-psarga]

We do not have a QA team dedicated to the changes in Cucumber open. What we generally do is provide "release candidate" (which are beta version of the new major release to come).

For this new release, we've had 6 release candidates before cutting the 4.0 which were available for anyone to try out. In fact, we've decided that prior to any release of a new major version, we'd keep a full month between the last release candidate and the final release to ensure that people have enough time to try it out and provide useful feedback. We generally announce the release candidates on Cucumber blog and Slack channel.

Any feedback is more than welcome, so don't hesitate to help us (as you've already done with your previous message).

To provide a bit more context on the changes in the HTML formatter, we've started a new formatter from scratch, which is compatible and consistent accross all Cucumbers (Java, Javascript and Ruby). This explains that some of the functionnalities may have disappeared (but they'll be back, thanks to feedbacks like yours).

[dmaisells]

The other thing I noticed about the .html file in Cucumber 4.0.0. is it looks like the file is NOT written until ALL scenarios are complete.  With Cucumber 3.0.0 and using the command syntax of  ...--format html > somefile.html   the html file used to be written as the script executed so a tester could follow along the progress of scripts, halting their execution if enough failures dictated that.   This was especially important for a several hour suite to be able review what passed and failed already while the scripts continued to execute.

[vincent-psarga]

This is still possible using the message formatter and the stand-alone HTML reporter.

gem install cucumber
gem install cucumber-html-formatter

// Run the tests and output the results as the new message format:
cucumber --format message > output.ndjson

// When checking for the current execution status
cat output.ndjson | cucumber-html-formatter > report.html

[dmaisells]

Thank you, but...

cat output.ndjson | cucumber-html-formatter > report.html
/Users/douglas.maisells/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/optparse.rb:1614:in `[]=': wrong number of arguments (given 3, expected 2) (ArgumentError)
from /Users/douglas.maisells/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/optparse.rb:1614:in `permute!'
from /Users/douglas.maisells/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/optparse.rb:1635:in `parse!'
from /Users/douglas.maisells/.rvm/gems/ruby-2.3.0/gems/cucumber-html-formatter-6.0.2/bin/cucumber-html-formatter:18:in `<top (required)>'
from /Users/douglas.maisells/.rvm/gems/ruby-2.3.0/bin/cucumber-html-formatter:23:in `load'
from /Users/douglas.maisells/.rvm/gems/ruby-2.3.0/bin/cucumber-html-formatter:23:in `<main>'
from /Users/douglas.maisells/.rvm/gems/ruby-2.3.0/bin/ruby_executable_hooks:24:in `eval'
from /Users/douglas.maisells/.rvm/gems/ruby-2.3.0/bin/ruby_executable_hooks:24:in `<main>'

[mpkorstanje]

Comes to mind that the html formatter probably isn't the right tool to follow progress. I would recon a plain text based output like the `pretty` formatter would be more reliable for this..

[dmaisells]

Can you please provide instructions for 'pretty'.  The following used to work for me   --format html > somefile.html.  and I could monitor progress of the scripts by opening the html and refreshing it in my browser (with cucumber 3.0.0)