Send HTTP request without appending the end point

[Lightem]

This question was asked in a previous thread but their answer is not my issue and no other answer was posited.

I have a HTTP request I would like to add to a load test. It is like this

https://myhostname.net/internalservice-myhostnamestatus/v1/services/applicationstatus

When i run this in SoapUI i get an incomplete response. If I simply load this into a browser it works just fine.

It looks like SoapUI is actually sending

https://myhostname.net/internalservice-myhostnamestatus/v1/services/applicationstatus HTTP/1.1

Is there a way to submit the request without appending it with HTTP/1.1?

[richie]

Hi @Lightem 

The HTTP/1.1 isn't actually going across in the request string - it's not like it's an extra parameter - it's just declaring the version of HTTP it's using - which yes strictly is being sent - but not in the request string - it's like a header - although it's not actually a header as far as I'm aware.

Just had a look at a complete list of standard headers- nope - it's not a header

What makes you think you are passing this across when you submit the request?  is it just that you've see the string in the RAW tab indicates URI followed by HTTP/1.1?

I think that's steering you down the wrong path.  Ignore that for now and look at the differences between what you're submitting via the browser and your ReadyAPI!/SoapUI config - that's going to be the problem.

Also - can you clarify what you mean by 'incomplete response' please?

If you can provide the stacktrace for the request (ReadyAPI! log, response details etc.) that can only help people diagnose your issue, but I'd disregard the version of HTTP - that's not being added to your URI string.

Cheers,

richie

[Lightem]

Raw response to SoapUI request:

HTTP/1.1 200
Date: Thu, 28 Feb 2019 18:08:33 GMT
Content-Type: application/json;charset=ISO-8859-1
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d673ca4bba1ce54f6f2b056cb2b225aed1551377313; expires=Fri, 28-Feb-20 18:08:33 GMT; path=/; domain=.mailinator.com; HttpOnly
Set-Cookie: JSESSIONID=s3~60B72731C657FDA26FBAB97BAEAD9C94; Path=/; HttpOnly
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 4b04d6535a0f1ea7-SJC

{"messages":[],"to":"addressID"}

URI from SoapUI: https://api.mailinator.com/api/inbox?to=addressID%40mailinator.com&token=?&private_domain=true%20HTT...

Response when I copy the URI from SoapUI into browser:

{"messages":[{"fromfull":"source@source.com","subject":"email subject","from":"source@source.com","origfrom":"source@source.com","to":"addressID","id":"emailID","time":1551375385000,"seconds_ago":573}],"to":"emailID"}

It appears that the entirety of the "messages" is being ignored.

ReadyAPI log:

• Thu Feb 28 10:08:33 PST 2019: INFO: Got response for [TestStepName] in 247ms (63 bytes)

[Lightem]

Also - can you clarify what you mean by 'incomplete response' please?

By incomplete response, I am speaking to the issue of the missing message. 

Thank you for your reply to date @richie . I appreciate your help. 

[richie]

Hey @Lightem 

It took a bit of going through your message to work out the problem - for future reference if you try and lay out everything providing clear details - e.g. I am publishing a REST request (WHATEVER method - PUT, POST, GET) using this payload and this is the response etc.

Anyway - right - when you say  'incomplete response' you mean the message array in the response payload appears empty.

This is weird considering the response you get via your browser includes a populated message array in your .json response, but the response payload in ReadyAPI! is generated with an empty message array in the payload - so my first thought is you've got some eventhandler or something stripping the content in the response before it's presented - but you'd know about that if you'd gotten that setup - it's not like just hitting a checkbox by mistake!

Ok - we need a bit of help to assist diagnosing this.  Have you used Telerik's Fiddler before?

You need to download that - its a packet/network sniffer like WireShark (except a lot easier to use).

Download that - what you need to do is monitor the request when you try the submission in SoapUI and then compare with when you submit the request via the browser/postman or whatever.

Essentially you are proxying the requests to Fiddler before they get to their specified endpoint - Fiddler will indicate whats going on.

So follow the instructions here to setup Fiddler as ReadyAPI!/SoapUI  proxy

Once the Fiddler cert is setup and you're good to do - within Fiddler hit the 'Any Process' button to select the ReadyAPI! window as the target to proxy

Submit your request in ReadyAPI! and watch the magic happen in the traffic window

1. Within ReadyAPI!, disable the proxy (button on mainscreen below main toolbar)

2. De-select ReadyAPI! as the target process (via the Any Process button)

3. Have you browser ready and copy in the URI to be submitted.

4. <Alt>+<TAB> to Fiddler and click the 'Any Process' button and select the browser window

5. Within the browser - submit your request - traffic will be recorded

6. De-select browser as the target process (via the Any Process button) 

Please post the results so we can review - RAW, HEADERS and JSON results for both the ReadyAPI and Browser requests please!

Cheers,

richie

[Lightem]

Understood.

I managed to get Fiddler working and configured and captured both. 

The RAW header and response from Chrome:

GET https://api.mailinator.com/api/inbox?to=?%40mailinator.com&token=?&private_domain=true%20HTTP/1.1 HTTP/1.1

Host: api.mailinator.com

Connection: keep-alive

Cache-Control: max-age=0

Upgrade-Insecure-Requests: 1

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8

Accept-Encoding: gzip, deflate, br

Accept-Language: en-US,en;q=0.9

Cookie: __cfduid=d421be177b21122fcffa7164759d88f5c1547666655; JSESSIONID=s4~04C12A0D78A5976B550B2F01D68403D8

HTTP/1.1 200

Date: Fri, 01 Mar 2019 21:56:54 GMT

Content-Type: application/json;charset=ISO-8859-1

Connection: keep-alive

Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"

Server: cloudflare

CF-RAY: 4b0e623178026c4c-SJC

Content-Length: 344

{"messages":[{"fromfull":"leviosa@pivotal.io","subject":"Verify your account","from":"leviosa@pivotal.io","origfrom":"leviosa@pivotal.io","to”:”?”,”id”:”?”,”time":1551460931000,"seconds_ago":16483}],"to”:”?”}

The Raw header and response from ReadyAPI!:

GET https://api.mailinator.com/api/inbox?to=?%40mailinator.com&token=?&private_domain=true HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Length: 0
Host: api.mailinator.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_181)

HTTP/1.1 200
Date: Fri, 01 Mar 2019 22:06:41 GMT
Content-Type: application/json;charset=ISO-8859-1
Connection: keep-alive
Set-Cookie: __cfduid=d49121eb87460e82b00850caa8f6a751f1551478001; expires=Sat, 29-Feb-20 22:06:41 GMT; path=/; domain=.mailinator.com; HttpOnly
Set-Cookie: JSESSIONID=s4~B16B7590F17159CAAC26D55C4EAEC8BB; Path=/; HttpOnly
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 4b0e70841e9c6c22-SJC
Content-Length: 63

{"messages":[],"to":"?"}

[richie]

Hey @Lightem 

OK - can you do me a favour please?

Can you reconfirm the URI you are passing - according to the details below you are passing slightly different requests in browser compared wiht ReadyAPI!

Browser:

GET https://api.mailinator.com/api/inbox?to=?%40mailinator.com&token=?&private_domain=true%20HTTP/1.1

ReadyAPI!:

GET https://api.mailinator.com/api/inbox?to=?%40mailinator.com&token=?&private_domain=true

So  currently it actualy looks like you are posting 'HTTP/1.1' in the end of the request your are submitting in your browser.  I don't think it'sa typo - cos you wouldve just copied out of fiddler - but it's weird and different to ReadyAPI! so we need to clarify

Depending on your response to the above - I have an idea about your issue.

Can you submit the same request in POSTMAN please?  Rather than the browser - I want to see what gets generated in Fiddler from a REST client tool - rather than just a browser.

Essentially the proxying requests via Fiddler has confirmed that the issue wasn't ReadyAPI!s rendering of the response (cos ReadyAPI! can alter the response rendered in the results) - but Fiddler proved that the message array in your response payload is empty BEFORE it gets to ReadyAPI! - so ReadyAPI! is simply displaying the content it receives - I just needed to be sure.

Next - do the same again - but this time with a REST client like POSTMAN - I need to see what headers are actually being generated and the messages array in the payload for a REST client (other than ReadyAPI!)

Depending on the results - we might need to add in some headers to your ReadyAPI! request - I wouldn't expect the headers to alter the payload like this - but that's the only thing I can think of - emulate the request via the browser (or POSTMAN dependinng on the results) and go from there.

Have you spoken to your developers?  Have they confirmed if there are header parameters required on this GET?

Cheers!

richie

If you 

zx

[richie]

Hey @Lightem,

How's it going.....any progress fella?

Cheers,

richie

[Lightem]

"Have you spoken to your developers?  Have they confirmed if there are header parameters required on this GET?"

This is Mailinators' API

Thank you for your suggestions. Regardless of source, the only change in all of the manipulations to call that has made a difference is how the HTTP/1.1 is appended to the end of the call.

Logs from Fiddler show that the format that does work is to end the URL with:

...&private_domain=true%20HTTP/1.1 HTTP/1.1

The calls leaving the body of the message black end in 

...&private_domain=true HTTP/1.1

The problem I am having dealing with this knowledge is trying to manipulate ReadyAPI to behave in the same way. I have tried to stitch this "additional" HTTP non-header, header into the URL from ReadyAPI to no avail.

[richie]

Hey @Lightem 

So it's a 3rd party API?  Ok - we'll get to that

In all your posts (except the first one) you say you submit the following in chrome:

https://api.mailinator.com/api/inbox?to=?%40mailinator.com&token=?&private_domain=true%20HTTP/1.1

What made you append the ' HTTP/1.1' at the end of the query string when you submitted the request in Chrome?

Initially you stated in your first post that you wanted to NOT send the 'HTTP/1.1' at the end of your request? (confused!)

Anyway - yeah - what made you append the HTTP/1.1 at the end of the query string when you submitted via Chrome?

If you don't append the HTTP/1.1 at the end of the request in Chrome - what happens?  Do you still get a populated messages array in your response payload - or is it empty as well?  I don't see from the previous posts what you say about this - but this is important

I've had a look at the API detail at mailinators apidocs  and it does NOT mention appending the HTTP/1.1 at the end of the request at all for any of the api calls including your query of the private domain which you are exercising.  According to the detail - I would expect appending HTTP/1.1 to your query string to fail according to the api definitions in the apidocs.jsp page.

Also - It doesn't make sense - sticking HTTP/1.1 at the end of the request doesn't even comply to good RESTful API URI design which is probably one of the reasons you're struggling to configure ReadyAPI to support adding it into your query string

Can you please respond with the response you get when you submit the following URL into Chrome please? (is the messsages array in yoru response payload populated when you do this?)

https://api.mailinator.com/api/inbox?to=?%40mailinator.com&token=?&private_domain=true

Cheers,

richie