Forum Discussion

qa4ever's avatar
qa4ever
New Contributor
13 years ago

securitytestrunner, filter out assertions do not work

Hi,

after couple of scripts and some debugging that have given me gray hair, I've realized, that the -t automation/command line argument isn't working correctly.
It is parsed and read by the securitytestrunner.bat but settings in the settings file do not override those in the project.
Filter out [[Version x.y.z] Exposing version numbers gives] works within the SoapUI GUI, but not when automating. When running in the GUI, all tests are PASS with the soapui-settings.xml .

The problem is both setting/overriding the “Global Global Sensitive Information Tokens” while using SecurityTestRunner on command line, as well as starting the SecurityTestRunner from within SoapUI GUI.

The command line looks like this:
G:\123423412>"C:\Program Files\SmartBear\SoapUI-Pro-4.6.2\bin\securitytestrunner.bat" -ehttp://computer20:28136 -s"ServiceDirectoryBindingTypeTestSuite" -c"getServices ALL TestCase" -r -o -RHTML G:\123423412\wsServiceDirectory-soapui-project.xml -t G:\123423412\soapui-settings.xml
In the project and settings file I have removed certain [Version information] assertions in the “Global Global Sensitive Information Tokens”
Still when running the tests it reports: [Version x.y.z] on each and every method permutation call.
I've tested it in both 4.6.2 and 4.6.1 version.
I’m running on Windows 7 64 Bit as a local administrator.

Could you, please, investigate this?

Thank you,
Qa4Ever

3 Replies

  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi

    I have been debugging the security tests and my current hypothesis is that there might be a bug in the security tests. It looks like the host/endpoint/environment replacement isn't run as it should. If you set the endpoint on the test step inside SoapUI and save the project. Can you run the security tests from the command line then?

    Regards
    Joel Jonsson
    SmartBear Sweden
  • qa4ever's avatar
    qa4ever
    New Contributor
    13 years ago
    Hi Joel

    The project already have endpoint http://computer20:28136 set.
    I can send you the project for you to verify, separately.
    But there is no problem with the endpoint, if so we would not be able to get any response, right?
    The problem is with the security assertions.
    They assert [Version x.y.z] because the response (HTTP 200 ) contain version info.

    Which our aim to filter out with -t G:\123423412\soapui-settings.xml
    (actually the project have them removed too, so this might be a generic problem with the security tests, when automated from command line, with the securitytestrunner.bat/sh ONLY.)

    For example we have removed four [Version x.y.z] <con:property> sections from soapui-settings.xml:

      <con:property>
        <con:name>~(?s).*\w+/\d{1,2}(\.\d{1,3})+.*</con:name>
        <con:value>[Version x.y.z] Exposing version numbers gives unnecessary hints on your systems vulnerabilities</con:value>
      </con:property>


    Please could YOU examine this (try it out for yourself) and register a bug accordingly?

    In order to make it simpler for you, and if you do not have soap interface that returns version info, try add something similar to this to reproduce the issue at hand:


      <con:property>
        <con:name>~(?s).*(A|a)+.*</con:name>
        <con:value>[Version x.y.z] A response that contain a or A letter</con:value>
      </con:property>


    Thank you for your hard efforts,
    QA4Ever
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi,

    Defect SOAP-1145 has been opened for this issue.
    Thanks for reporting this.


    Regards,
    Marcus
    SmartBear Support