Custom filters
Hello, community!
In my team we have identified the need to be able to skip whole scenarios in a way more flexible than the current tag filtering.
One use case, for instance, would be being able to automatically skip features/scenarios tagged as @ignore-until-20210601 until 1st of June of 2021, but resuming executing them after that date. There are some situations in our context where that could come in handy.
Another use case would be deciding if a scenario should be skipped or not depending on its group of tags, not just on the presence of one of them. Example: We'd like to have scenario outlines with "examples" tagged with "@staging-data" and/or "@production-data" (among other environments names). When running a suite for production we'd like to skip examples tagged with "@staging-data" but not with "@production-data", but we'd like to run normally scenarios without any "@something-data" tag.
We tried using hooks, but it was not ideal. The result of this approach would be having, in the test results, all the steps of a skipped scenario shown as skipped, instead of not having the scenario listed at all. That turned out to be a problem, as some mechanisms in our pipeline would consider those as signs of failures, and behave in undesirable ways.
After coming to the conclusion that what we wanted was to have the scenarios filtered out before execution, pretty much like the tags processing work, I came up with a solution that involves allowing cucumber users to set their own filtering class in cucumber options, and then chaining that filter to the other filters (tag, name and line filters) during configuration. This is my solution's branch. It accepts a class reference in CucumberOptions. That class is supposed to implement Predicate<Pickle>, so it can decide whether to skip or not a pickle based on any accessible information (all pickle's, but also system params or external services, for instance).
Having created the above branch I learned that cucumber-ruby supports @AfterConfiguration hooks that would allow that same thing without changes to Cucumber, but a quick look at the code told me cucumber-jvm doesn't support that by design, and changing that would be no trivial task.
So, summing up:
- Is there any other possibility that cucumber-jvm already supports and I haven't explored yet?
- If not, what are your thoughts about the changes proposed in my branch? Could it be a new feature? Any changes need anticipated?
- I know this is coming, so: What may I have got wrong about Cucumber and the usage I'm giving it? 😅
Thank you!
