Forum Discussion

Bill_In_Irvine's avatar
Bill_In_Irvine
Contributor
7 years ago

A curl GET (with parameters) is accepted by the server but fails when done with Soap UI NG Pro

We have a REST API that I have made Soap UI NG Pro tests for. Unfortunately they are mostly GET requests with parameters. I am aware of the philosophy for avoiding parameters but I have no choice - I'm the tester! 

 

Our command runs fine in this form (I changed some of the characters to remove proprietary information) 

 

$ curl -X GET -k --cert certs/client.crt --key certs/client.key -v -d'myname="myvalue"' 'https://xxx.xx.xx.xxx:xxxxx/a/path'

 

It made a verbose output, much snipped but in essence here is the info:


> GET /a/path HTTP/1.1
> User-Agent: curl/7.27.0
> Host: xxx.xx.xx.xxx:xxxxx
> Accept: */*
> Content-Length: 16
> Content-Type: application/x-www-form-urlencoded
>


* upload completely sent off: 16 out of 16 bytes
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: application/json; charset=utf-8
< Content-Length: 101
< ETag: "611876272"
< Date: Fri, 21 Jul 2017 00:24:05 GMT
< Connection: keep-alive
<

So that works on the command line

 

Now the result of running the request with test runner is below, I get a error code 400 back from the server

 

Authorization: [Basic YWRtaW46cGFzc3dvcmQ=]
Connection: [Keep-Alive]
User-Agent: [Apache-HttpClient/4.5.2 (Java/1.8.0_111)]
Host: [xxx.xx.xx.xxx:xxxxx]
Accept-Encoding: [gzip,deflate]

GET https://xxx.xx.xx.xxx:xxxxx/a/path?myname="myvalue" HTTP/1.1
Accept-Encoding: gzip,deflate
Authorization: Basic YWRtaW46cGFzc3dvcmQ=
Host: xxx.xx.xx.xxx:xxxxx
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_111)


---------------- Response --------------------------
ETag: ["2124691785"]
Connection: [keep-alive]
#status#: [HTTP/1.1 400 Bad Request]
Content-Length: [141]

 

The request editor from the Ready! API Projects tab won't allow me to put a --data or -d for a curl data indicator in the section for name and value. I was exploring whether I needed to edit Content-Type and Content-Length for the headers and tried to force the length of the data or switch between application/json and application/x-www-form-urlencoded. I explored the requestFilter event handlers and noticed for my context and request interfaces when I do a "code completion" I am missing methods such as requestHeaders. I can only work with the request content but not set the headers. 

 

So my options are running out. 

 

I don't have the ability to change the REST API to use a POST with parameters - we don't want that.

 

Suggestions?

7 Replies

Sort By
  • sanj's avatar
    sanj
    Super Contributor
    7 years ago

    The header option is at the bottom

    I thought header can be modified 

    • Bill_In_Irvine's avatar
      Bill_In_Irvine
      Contributor
      7 years ago

      I tried the Headers at the top and haven't tried the headers tab at the bottom until just now. but I get the same error. It's as though the parameter part is still not seen. The raw request looks good and the only difference is the user agent is Apache-HttpClient/4.5.2 with Ready API.

       

      I notice that for PUT or POST, under the Projects tab you can check "POST QueryString" to send parameters in with the message body. That seems to be what Curl is doing. But Ready! API disallows this. 

      • Bill_In_Irvine's avatar
        Bill_In_Irvine
        Contributor
        7 years ago

        Just an update. I have been doing lots of researching on this and tried even using Groovy scripting in a requestFilter event handler to set the message body and try to trick the SoapUI NG to duplicate the curl command. The --data or -d in curl, according to the documentation, is like a POST:

         

        " -d: 

        (HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded."

         

        However when using a curl -X GET -k ... command and -d in combination, curl somehow converts it back to a GET request. I tried using wireshark to see exactly what is going on but our key exchange agreement is DH-based, so I cannot decrypt the packets. 

         

        I am at a loss at how something so basic like this has not been addressed.

  • nmrao's avatar
    nmrao
    Champion Level 3
    7 years ago

    Bill_In_Irvine,

     

    If you look at the both requests closely, two differences :

     

    • Content-Type: application/x-www-form-urlencoded - you can change it in soapui to the same text as value.
    • Accept-Encoding: [gzip,deflate] - is additional in your soapui request; but not in curl request. this can be changed in the preferences.

    Try to address the above and see if that helps.

     

    • Bill_In_Irvine's avatar
      Bill_In_Irvine
      Contributor
      7 years ago

      Still it is not working, but you made a good point! I got rid of that gzip / compressed part and Ready! API made me restart the tool. 

       

      I added a header "Accept: */*" and the "Content-Type: application/x-www-form-urlencoded"   - the odd thing is that if I added the Content-Length: header and a value of number of bytes in the name/value string, the sending process would go on forever. The square next to the green "send" triangle would stay red. 

       

      So I wonder if that is a clue as to what is going on?

       

      Here is my raw request that I tried sending from the Project tab:

      GET https://xx.xx.xx.xxx:xxxxx/a/proprietarypath?myname="myvalue" HTTP/1.1
      Authorization: Basic YWRtaW46cGFzc3dvcmQ=
      Accept: */*
      Content-Type: application/x-www-form-urlencoded
      Host: xx.xx.xx.xxx:xxxxx
      Connection: Keep-Alive
      User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_111)

       

      It looks closer to what the curl sent but the curl provided the Content-Length. 

       

      curl -X GET -k --cert certs/client.crt --key certs/client.key -v  -d'myname="myvalue"' 'https://xx.xx.xx.xxx:xxxxx/a/proprietarypath'

       

      here is the summary from the -v 


      > GET /a/proprietarypath HTTP/1.1
      > User-Agent: curl/7.27.0
      > Host: xx.xx.xx.xxx:xxxxx
      > Accept: */*
      > Content-Length: nn
      > Content-Type: application/x-www-form-urlencoded
      >

       

      And curl does not show the parameter name/value stuff.

       

      I noticed I can get this google api to work 

       

      curl -X GET -k -v 'http://maps.googleapis.com/maps/api/geocode/xml?address=1600+Amphitheatre+Parkway,+Mountain+view,+CA&sensor=false'

       

      but I cannot get it to respond successfully when trying -d or --data!

       

      curl -X GET -k -v 'http://maps.googleapis.com/maps/api/geocode/xml' -d'address=1600+Amphitheatre+Parkway,+Mountain+view,+CA&sensor=false'

       

       

       

      • nmrao's avatar
        nmrao
        Champion Level 3
        7 years ago
        You do not have to send content-length which would be calculated based on the request.