Runner.Stop(true) clears variables and ignores further breakpoints
My project currently has the following test items:
Testcase B //this one fails intentionally
Log results to excel file
The logger keeps track of the testcases that have been run in a global array variable. It then loops over the testcases in this array, finds the result of each testcase in the log file, and then writes the result to my own excel file.
I also want to stop a test item from continuing further when an error occurs, but I still want the following test items to be run. To do that, I have disabled 'Stop execution on error' for the project. And I created an event OnLogError that calls Runner.Stop(true). According to the documentation, this should only stop the execution of the current test (which it does). However, I am noticing some unwanted side effects when I use this function.
After I call Runner.Stop(true), the global variable that holds the testcases has been emptied. I made this conclusion when I did the following:
Running test A and C results in both results being written.
Running test A and B results in no results being written.
Running test B and C results in only C being written.
Running all tests results in only C being written.
So when B fails, it continues running, but I lose the contents of the array.
It was difficult to debug this, because I also noticed that after calling Runner.Stop(true), all breakpoints are no longer triggered.
Is this intended behaviour?
I do have exactly the same problem.
It seems to me like the Runner object holds all references to variables and breakpoints. And when calling Runner.Stop this object gets disposed and a new Runner object is created which starts with the next test item in queue, but has no idea of any variables or breakpoints.
I did not yet check this with Project variables, just with a global script unit which holds the information I need throughout the tests.
Kind of makes me feel like my test architecture is not correct, because like this the Runner.Stop method is completely unusable.
Anyone else having any experience with this?
Project Variables do in fact keep their value after a call of Runner.Stop (true)
I do use this runner.stop to end the current test item and proceed with the next test in line.
The GeneralOnLogError script code is in "MySubs" unit.
Running a Project starts TestItems one after the other, if one test item fails, the GeneralOnLogError event is triggered. In this logEvent, I call the runner.stop.
My understanding is "MySubs" is a child of MyProject and MyTestItem. So with this in mind, my global variables are defined in the MyProject so they do not die with MyTestItem ending.