cancel
Showing results for 
Search instead for 
Did you mean: 

Display TestExecute/TestComplete results on TeamCity build log

SOLVED
Highlighted
Frequent Contributor

Display TestExecute/TestComplete results on TeamCity build log

Hello,

 

I am running my TestComplete tests using the TestExecute command line on TeamCity using batch files. The tests are running ok on TeamCity, however, I do not see any information regarding the passed/failed tests on the TeamCity build log section. On TeamCity it always shows a success irrespective of my tests failing or passing. TestComplete is generating the test results in an html files after running the tests. But I am not sure how to show the test results on the TeamCity build log. I read about the TeamCity service messages here but I am not sure how to use them. Any suggestions? For eg: ##teamcity[testStarted name='testname']

How do I use these messages and where do I enter them?

 

https://confluence.jetbrains.com/display/TCD65/Build+Script+Interaction+with+TeamCity#BuildScriptInt...

 

Thank you.

Regards,

Sameer

1 ACCEPTED SOLUTION

Accepted Solutions
Community Manager

Re: Display TestExecute/TestComplete results on TeamCity build log

Hi @sameerjade,

 

Have you tried adding the captureStandardOutput parameter to your test as VladYatskovskiy suggested? The following help article contains the info how to do this:

https://www.jetbrains.com/help/teamcity/build-script-interaction-with-teamcity.html

---------
Tanya Gorbunova
SmartBear Community Manager

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️

View solution in original post

8 REPLIES 8
Highlighted

Re: Display TestExecute/TestComplete results on TeamCity build log

Hello. I'm not familiar with TeamCity, but it seems that combining " captureStandardOutput='true' " flag of TeamCity test, and handling TC\TE exit code in your batch files as it described here:
https://support.smartbear.com/testcomplete/docs/working-with/automating/command-line-and-exit-codes/...
may help you.

Highlighted
Occasional Contributor

Re: Display TestExecute/TestComplete results on TeamCity build log

My teammate and I are working on better TeamCity intergration.   Right now what we have built is a custom API and a Python script its a bit run around,  and is still loaded with bugs.  Once we have it more refined we will be releasing it to the community,  I have no ETA on our plans to release it.  

 

Our product is a web application,  we use a git flow methodolgy and TeamCity builds and deploys to a test enviornment for the branch. 

 

  1. TeamCity's python script tells our custom-API to start tests and the environment.  (http://tcapidev/start?=http://webdev521:52319/ )
  2. Our custom API triggers a bat file with "psexec" to launch TestExecute (I would like to switch over to using TE's built in TestRunner API instead of this psexec bat so we have better control to remotely stop the tests).
  3. Our TestExecute/TeamCity project has an OnStartTest event,  determins if it's the first test gets the targetURL, then OnLogEvent's to store errors etc,  and then OnStopTest to post the final state and isPassing of recorded logEvents to our custom API.

    (http://tcapidev/post?sessionID=123&testresult=

    {'testname':'welcome page',

    'isPassing': 'true',

    'log':'{'type':'checkpoint',

    'message':'loaded welcomepage'}

    '}  )

  4. TeamCity's python script peridocialy checks our in house API for these status messages that TestExecute has posted and gives that to the TamCity build log.

I wish TestComplete's built in API gave this information,  and our Python script could just poll for that instead of having to post it to our own custom API.

Highlighted
Frequent Contributor

Re: Display TestExecute/TestComplete results on TeamCity build log

@VladYatskovskiy Thank you for your reply. Where is the " captureStandardOutput='true' " flag in TeamCity? I could find this flag option in TeamCity UI. 

Thank you.

Highlighted
Frequent Contributor

Re: Display TestExecute/TestComplete results on TeamCity build log

@garyschechter Thank you for your reply. I am hoping that there is an in-built function in TeamCity which can do this and get test results and display it on the build log. Based on this answer, it is possible using the TeamCity service messages feature but its not clear how to use it.

https://stackoverflow.com/questions/9584699/how-to-export-testexecute-testcomplete-results-to-teamci...

Highlighted
Community Manager

Re: Display TestExecute/TestComplete results on TeamCity build log

Thanks for your suggestions, VladYatskovskiy, garyschechter!

 

@sameerjade, did you make TestComplete and TeamCity work together?

 

 

---------
Tanya Gorbunova
SmartBear Community Manager

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️
Highlighted
Frequent Contributor

Re: Display TestExecute/TestComplete results on TeamCity build log

Hello @TanyaGorbunova ,

I am still trying to find a solution for this. I am able to run my TestComplete tests from TeamCity but I cannot get my test results (pass/fail/error) to show on the TeamCity build log page. On TeamCity, the tests always show as passed as long as the script ran ok. Thank you.

Regards,

Sameer

Highlighted
Frequent Contributor

Re: Display TestExecute/TestComplete results on TeamCity build log

I want to add the test run info on the TeamCity build log tab, for eg if my test failed because of a bug and gives an error, I want to show the error on the build log. I tried using TeamCity service messages for this but it is not working as I would expect. Maybe I am missing something or doing it incorrectly. Please advise. All suggestions are welcome. Thank you!

This is how I am using the service messages. I am entering the messages in my .bat file which I am running from TeamCity.

##teamcity[testStarted name='SmokeTest1']
run SmokeTest1.bat
##teamcity[testFailed name='SmokeTest1' message='error']
##teamcity[testFinished name='SmokeTest1']

The above code runs my test and always says that my test failed based on my 'testFailed' command. What I want is that it should check if my test actually failed and report if failed (maybe it can check for some keywords to check if my test failed). But it seems like what is happening is that I am already telling it that the test has failed with the message 'error'. I wouldn't know before running the test if it is going to fail. Am I missing something here?

Also, in the below example, are we supposed to enter the error even before running the test? So, if I use testStdErr  and enter an error, it will always report that error irrespective of if that error really occurred during my test run?

##teamcity[testStarted name='SmokeTest1']

run SmokeTest1.bat

##teamcity[testStdOut name='testname' out='text']

##teamcity[testStdErr name='testname' out='error text']

##teamcity[testFinished name='SmokeTest1']

Thank you!

Community Manager

Re: Display TestExecute/TestComplete results on TeamCity build log

Hi @sameerjade,

 

Have you tried adding the captureStandardOutput parameter to your test as VladYatskovskiy suggested? The following help article contains the info how to do this:

https://www.jetbrains.com/help/teamcity/build-script-interaction-with-teamcity.html

---------
Tanya Gorbunova
SmartBear Community Manager

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️

View solution in original post

New Here?
Join us and watch the welcome video:
Announcements
Top Kudoed Authors