Forum Discussion

chris_selwyn's avatar
chris_selwyn
New Contributor
15 years ago

Problems with double encoding of URLs

I have a problem with URLs getting double url-escape-encoded.

My WSDL has a URL that has a %20 in it (representing a space).

However, when I use SoapUI 3.5.1 (free version) to post a SOAP message to the URL, the URL gets messed with and the message ends up being posted to the wrong URL.

I have found the setting called "Pre-encoded endpoints".

If I leave this setting unset then the URL that gets posted to has %252520 in it. i.e the % has been encoded twice.

If I set the "Pre-encoded endpoints" setting then the URL that gets posted to has %2520 in it. i.e. the % has been encoded once.

It seems that the "Pre-encoded endpoints" setting reduces the number of times that the URL gets encoded from twice to once. But I really need it not to be encoded any further than it is already.

I have tried replacing the %20 with a " " (space) in the URL and setting the "Pre-encoded endpoints" option, hoping that this will cause the " " to be encoded as %20.
However, all that happens then is that I get an error saying "host parameter is not null".

Does any have any clues as to how to avoid url-encoding?

Chris Selwyn
  • chris_selwyn's avatar
    chris_selwyn
    New Contributor
    15 years ago
    I have just found that SoapUI 3.5 works correctly as regards encoding of URLs. That is... if I set the "Pre-encoded URLs" flag then the URL is taken verbatim and not encoded at all.

    I narrowed the culprit piece of code to be a few lines that appeared in SoapUI 3.5.1 in the sendRequest() method in
    com.eviware.soapui.impl.wsdl.submit\transports.http.HttpClientRequestTransport.java.

    The following is an extract from that method...

    // dump file?
    httpMethod.setDumpFile( PathUtils.expandPath( httpRequest.getDumpFile(),
    ( AbstractWsdlModelItem )httpRequest, submitContext ) );

    // fix absolute URIs due to peculiarity in httpclient
    URI uri = ( URI )submitContext.getProperty( BaseHttpRequestTransport.REQUEST_URI );
    if( uri != null && uri.isAbsoluteURI() )
    {
    hostConfiguration.setHost( uri.getHost(), uri.getPort() );
    String str = uri.toString();
    int ix = str.indexOf( '/', str.indexOf( "//" ) + 2 );
    if( ix != -1 )
    {
    uri = new URI( str.substring( ix ) );
    httpMethod.setURI( uri );
    submitContext.setProperty( BaseHttpRequestTransport.REQUEST_URI, uri );
    }
    }

     // include request time?

    The lines of code in bold are not in SoapUI 3.5 and it is my belief that this is the reason why 3.5 works and 3.5.1 does not work.

    Chris Selwyn
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    15 years ago
    Hi!

    there is a small change in this code in the current 3.6 beta, namely the line

    uri = new URI( str.substring( ix ) );

    has been changed to

    uri = new URI( str.substring( ix ), true );

    where the second argument says that the uri is already escaped.. maybe that solves the issue? Can you give the beta a try?

    regards!

    /Ole
    eviware.com
  • JKowalczyk's avatar
    JKowalczyk
    Occasional Contributor
    13 years ago
    Hi,

    I'm having exactly the same problems with soapUI Pro 4.5.0 (Build [internal], Build Date 2012/03/26 16:19)
    I've tried many things to get the URL in such form: 
    POST /blababla/2012-08-01T12%3A45%3A03%2B0100

    but I always end up either with double URL encoding or without any encoding, ie:.
    POST /blababla/2012-08-01T12%253A45%253A03%252B0100
    POST /blababla/2012-08-01T12:45:03+0100


    I've tried tricks mentioned in this post: http://www.eviware.com/forum/viewtopic.php?f=13&t=13006
    and also manually encoding the parameter
    ${=java.net.URLEncoder.encode("2012-08-01T12:45:03+0100")}
    ${=java.net.URLEncoder.encode("2012-08-01T12:45:03+0100"),"UTF-8")}
    ${=java.net.URLEncoder.encode("2012-08-01T12:45:03+0100"),"UTF-16")}
    ${=java.net.URLEncoder.encode("2012-08-01T12:45:03+0100"),"US-ASCII")}


    but nothing works.

    Cheers,
    J
  • JKowalczyk's avatar
    JKowalczyk
    Occasional Contributor
    13 years ago
    OK, I found a really nasty way of overcoming this issue till its get solved. You can use the "Groovy Script" test step to invoke a foolproof cURL 


    //A string can be executed in the standard java way:
    // Create the String
    def command = """curl -ki -XPOST https://hostname/blablabla/2012-08-01T12%3A45%3A03%2B0100"""
    def proc = command.execute()                 // Call *execute* on the string
    proc.waitFor()                               // Wait for the command to finish

    // Obtain status and output
    log.info  "return code: ${ proc.exitValue()}"
    log.info  "stderr: ${proc.err.text}"
    log.info  "stdout: ${proc.in.text}" // *out* from the external program is *in* for groovy


    I found that script here: http://www.linuxquestions.org/questions/programming-9/running-bash-commands-in-groovy-or-beanshell-scripts-721162/