Using Ready API for sending messages to Amazon SQS Queue (AWS)

[digifredje]

Hi,

We are moving our application to an AWS cloud and meanwhile implementing eventing.

For that we should be able to POST (and GET) messages to an SQS queue and later also SNS.

Making a connection to an S3 bucket works fine with the built in AWS Signature as described here: https://support.smartbear.com/readyapi/docs/requests/auth/types/aws.html

But when we want to send a message to an SQS or SNS queue, we get the "SignatureDoesNotMatch" error with the "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details." message.

Looks like this is a problem with ReadyAPI, because in postman everything works fine.

We just invested in new licences for 3 years so we really need a solution (or at least a workaround) for this problem.

I can't believe we are the only one who have issues with this.

[nmrao]

1. You can log a support request for quick help or to check the supported feature.
2. If you can post the raw request with headers (need not be values be actuals, dummy will also do) from both Postman and SoapUI will help.

[digifredje]

Postman raw request:
********************

POST /?Action=SendMessage&MessageBody=test&QueueUrl=https://sqs.eu-west-1.amazonaws.com/724446437091/test-queue HTTP/1.1
Host: eu-west-1.queue.amazonaws.com
X-Amz-Security-Token: dummy_security_token
X-Amz-Date: 20200116T081726Z
Authorization: AWS4-HMAC-SHA256 Credential=dummy_credential/20200116/eu-west-1/sqs/aws4_request, SignedHeaders=host;x-amz-date;x-amz-security-token, Signature=dummy_signature_request
User-Agent: PostmanRuntime/7.21.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 2db1e0ad-0a1f-4ae6-9c40-195903878456
Accept-Encoding: gzip, deflate
Content-Length: 0
Connection: keep-alive

Postman raw response:
*********************

HTTP/1.1 200 OK
x-amzn-RequestId: f7bc9cc1-6cde-5459-9b4e-ca5b72865b0c
Date: Thu, 16 Jan 2020 08:17:28 GMT
Content-Type: text/xml
Content-Length: 378
<?xml version="1.0"?><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><SendMessageResult><MessageId>3fb09a97-1074-449a-ad27-d0da544e50e4</MessageId><MD5OfMessageBody>098f6bcd4621d373cade4e832627b4f6</MD5OfMessageBody></SendMessageResult><ResponseMetadata><RequestId>f7bc9cc1-6cde-5459-9b4e-ca5b72865b0c</RequestId></ResponseMetadata></SendMessageResponse>

readyAPI raw request:
*********************

Thu Jan 16 10:02:35 CET 2020: DEBUG: http-outgoing >>
POST /?Action=SendMessage&MessageBody=test&QueueUrl=https%3A%2F%2Fsqs.eu-west-1.amazonaws.com%2FF724446437091%2Ftest-queue HTTP/1.1
Accept-Encoding: gzip,deflate
x-amz-content-sha256: dummy_sha256
Authorization: AWS4-HMAC-SHA256 Credential=dummy_credential/20200116/eu-west-1/sqs/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=dummy_signature_request
x-amz-date: 20200116T090235Z
content-type: text/plain
x-amz-security-token: dummy_security_token
Content-Length: 0
Host: eu-west-1.queue.amazonaws.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_181)

readyAPI raw response:
**********************

Thu Jan 16 10:02:35 CET 2020: DEBUG: http-incoming <<
HTTP/1.1 403 Forbidden
x-amzn-RequestId: 5392c23c-a3a0-567e-b8b3-94a68b4dc5cd
Date: Thu, 16 Jan 2020 09:02:37 GMT
Connection: close
Content-Type: text/xml
Content-Length: 1500

<?xml version="1.0"?><ErrorResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><Error><Type>Sender</Type><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/
Action=SendMessage&amp;MessageBody=test&amp;QueueUrl=https%3A%2F%2Fsqs.eu-west-1.amazonaws.com%2FF724446437091%2Ftest-queue
content-type:text/plain
host:eu-west-1.queue.amazonaws.com
x-amz-content-sha256:dummy_sha256
x-amz-date:20200116T090235Z
x-amz-security-token:dummy_security_token

content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token
dummy_sha256'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20200116T090235Z
20200116/eu-west-1/sqs/aws4_request
dummy_signature_response'
</Message><Detail/></Error><RequestId>5392c23c-a3a0-567e-b8b3-94a68b4dc5cd</RequestId></ErrorResponse>

[nmrao]

Thank you.

There are header name differences (case sensitive). May be you want to try adding the same headers (including case) in readyapi and try if that helps?

[richie]

Hi @digifredje 

Just to reiterate what Rao already said - I'd do a fine comparison of the headers generated in the Postman  relative to what ReadyAPI! is generating and then manually make the appropriate changes.

I can see the inconsistent capitalisation that  Rao already mentioned - but you also have additional amz headers in the ReadyAPI! header set that aren't included in the Postman request.  Also you're Postman content type is text/xml whereas you have it set as text/plain in ReadyAPI!

Cheers,

rich

[sonya_m]

Thanks nmrao, thanks richie.

@digifredje have you tried the suggestions? Please mark the appropriate reply as a solution

[digifredje · Accepted solution]

If I overwrite de headers manually in ReadyAPI with the ones from Postman (and take the same time and signature) then I can also do the call with ReadyAPI.

But for automated tests this is not really a solution. The signature is not calculated correctly so we need a solution for that problem.

Ou develop team is now looking to work around it with API Gateway, but it would be nice if it worked "right-out-the-box".

[nmrao]

Ok. One step forward now.

Is it that you need to make a separate call to get the key?

[sonya_m]

@digifredje do you have any updates on this?

Feel free to fill us in on the details - the Community might be able to help you further.

[sonya_m]

Any updates @digifredje ?

I'll mark your reply as a temporary solution for this thread since it gives people general direction of what to do in a situation like this. 

In the meantime, please share your progress with us.