can’t match signature while trying to send http requests to aws

I have a SageMaker endpoint, which works just as expected from Postman. And I have tried to send the same HTTP POST request from python3, but always get some sort of error, while Postman seems to work fine. I would like to know what I am missing with signature in my Python code and what needs to be added.

Here’s how I have tried to send http post request to AWS SageMaker, with help from this aws-blog:

import datetime, hashlib, hmac 
import requests

method = 'POST'
service = 'sagemaker'
host = 'runtime.sagemaker.ap-south-1.amazonaws.com'
region = 'ap-south-1'
endpoint = 'https://runtime.sagemaker.ap-south-1.amazonaws.com/endpoints/del-this2/invocations'
# POST requests use a content type header. For DynamoDB,
# the content is JSON.
content_type = 'application/json'



# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = '<my-access-key-id>'
secret_key = '<my-secret-access-key>'

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


canonical_uri = '/'

request_parameters =  '{'
request_parameters +=  '"longitude": [-121.31, -122.37, -120.43, -119.81, -119.82],'
request_parameters +=  '"latitude": [36.42, 37.93, 34.97, 36.83, 36.76],'
request_parameters +=  ' "housing_median_age": [21.0, 37.0, 28.0, 19.0, 46.0],'
request_parameters +=  '"total_rooms": [2740.0, 709.0, 1433.0, 6789.0, 2194.0],'
request_parameters +=  '"total_bedrooms": [615.0, 190.0, 270.0, 1200.0, 563.0],'
request_parameters +=  '"population": [2630.0, 644.0, 1001.0, 2325.0, 924.0],'
request_parameters +=  '"households": [564.0, 174.0, 278.0, 1109.0, 542.0],'
request_parameters +=  '"median_income": [2.6629, 0.8641, 4.0125, 4.049, 1.4028]'
request_parameters +=  '}'


canonical_querystring = ''

# Step 4: Create the canonical headers. Header names must be trimmed
canonical_headers = f'content-type:{content_type}nhost:{host}nx-amz-date:{amz_date}'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers include those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'content-type;host;x-amz-date'

def sign(key, msg):
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()

def getSignatureKey(key, date_stamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), date_stamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Step 6: Create payload hash. In this example, the payload (body of
# the request) contains the request parameters.
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()

# Step 7: Combine elements to create canonical request
# canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
canonical_request = f'{method}n{canonical_uri}n{canonical_querystring}n{canonical_headers}n{signed_headers}n{payload_hash}'


# ************* TASK 2: CREATE THE STRING TO SIGN*************
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = f'{date_stamp}/{region}/{service}/aws4_request'
# string_to_sign = algorithm + 'n' +  amz_date + 'n' +  credential_scope + 'n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
string_to_sign = f"{algorithm}n{amz_date}n{credential_scope}n{hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()}"


# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, date_stamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()

# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

headers = {'Content-Type':content_type,
           'X-Amz-Date':amz_date,
           'Authorization':authorization_header}


# ************* SEND THE REQUEST *************
print('nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + endpoint)

r = requests.post(endpoint, json=request_parameters, headers=headers)

print('nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %dn' % r.status_code)
print(r.text)
print(r.headers)

and this got printed, throwing me the signature mismatch error.


BEGIN REQUEST++++++++++++++++++++++++++++++++++++
Request URL = https://runtime.sagemaker.ap-south-1.amazonaws.com/endpoints/<end-point-name>/invocations

RESPONSE++++++++++++++++++++++++++++++++++++
Response code: 403

{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.nnThe Canonical String for this request should have beenn'POSTn/endpoints/<end-point-name>/invocationsnncontent-type:application/jsonnhost:runtime.sagemaker.ap-south-1.amazonaws.comnx-amz-date:20210831T173545Znncontent-type;host;x-amz-daten9f25b83b64690911a5f87dcfd3bdc969c857ff13c09e451b6612f9c8c2b5179e'nnThe String-to-Sign should have beenn'AWS4-HMAC-SHA256n20210831T173545Zn20210831/ap-south-1/sagemaker/aws4_requestna1ad468d304da39c19f34ef8f2a891af4285c3d5f79717bb977b0995cc82d1c6'n"}
{'x-amzn-RequestId': '<request-id>', 'x-amzn-ErrorType': 'InvalidSignatureException:http://internal.amazon.com/coral/com.amazon.coral.service/', 'Date': 'Tue, 31 Aug 2021 17:35:46 GMT', 'Content-Type': 'application/json', 'Content-Length': '690'}

So, What have I missed and how do I fix this signature mis-match error that I receive.
And also, I can’t use boto3 or any additional python library for that matter.

Source: Python Questions

LEAVE A COMMENT