There are things you can do to make the log smaller if you want to try that.
Make sure you aren't saving screenshots for everything and you can also turn off the event logging (all the little messages about this button was clicked etc.) and that will save a bunch of space.
I do have visualizer turned off. And there were only 8 errors, so there were only a few images for those errors. As for the events, I had considered that, but I hesitate to remove useful debug info like that should something go wrong. In conjunction with testing a large number of individual test cases we are also using the long test to test performance and memory leaks over time, so having the events in the log has proved useful, and given the runtime of the test, we don't necessarily have the resources to simply turn the events back on and rerun when something does fail. And only saving the last x number of events before an error hasn't always proved sufficient either.
But thank you for the response.
cshukusky wrote:
As for the events, I had considered that, but I hesitate to remove useful debug info like that should something go wrong. I
This is why the following option exists under Tools -> Current Project Properties -> Playback:
Basically, what this does is will "lock" the events on the project so they won't write out to the log for every click or keystroke. However, whenever a warning or error is logged, the last X number of events preceeding that warning or error are logged. This provides the desired debug info to determine why the warning/error appeared. If there are no warnings or errors, there are no events in the log.
I turned of the jUnit Style log and it had no real impact.
So I looked into using the store last x number of events option and I discovered 2 things.
First, it doesn't really work for a lot of our tests because we use warnings as messages of higher importance, such as functionality unsupported for certain test cases so it will be skipped or alterations or skipping of test cases due to bugs, as a couple of examples. I set it to save the last 50 events based on the size of one of the larger of the test cases and the result was very few events were actually removed, such that the size of the log folder only decreased by about 1%.
Now I could investigate further into different number of events stored, but while looking into that I discovered the size of the log does not appear to have much correlation, let alone causation, with the memory usage of Jenkins. Using jconsole I monitored the heap memory usage of my master during 3 sets of tests. Two tests with store last x events set to 0 and one with it set to 50 (producing the 1% smaller log).
The master consistently fluctuates back and forth between 150MB and 2GB of heap memory usage every few hours during the test run as memory builds and garbage collection kicks in. When the log returned in the first test with the full log, the memory spiked to the max 6GB I gave it and crashed. In the second test with the full log, the memory only went to 2GB causing no issue. And in the 50 event test case, with the 1% smaller log, the memory went to 4GB. In all cases the memory was below 1GB immediately before the log was returned, so it wasn't just an issue of being at the memory peak prior to garbage collection.
As such I am guessing this is more likely an issue with Jenkins/Java than with TestComplete or its logs.
I know you were looking for different suggestions, but I did flinch at 10000 test cases and 24 hours. That's an awfully big chunk of code to try to manage. I would wish for something smaller.
Are there problems with the logs if you run the tests outside Jenkins?
