Problem Statement

Let’s consider a scenario where a third-party application wants to access s3 objects (i.e s3 files & folders), but there are a few serious security reasons and you should not be accessing s3 objects via API.

By the way, API needs access key & access token to connect to S3 bucket through S3 SDK API. So, your application needs permanent access key’s. And having permanent key’s is a huge security concern any any organization. That is the reason, API key’s are temporary and you have to update application configuration regularly.

Solution

One of the possible solution is to use AWS Lambda functions. AWS Lambda is a way to write simple code which gets executed with AWS environment and it has full access control. You need to configure AWS Lambda & Cloud Front Trigger to execute AWS Lambda function.

Let’s say one of the http calls in your application loads list of files from AWS bucket. Once request is initiated and reach to CloudFront, CloudFront checks the rule & behaviour of the request and if it is mapped to AWS Lambda trigger configured at the CloudFront, it will forward request to Lambda Functions. AWS Lambda function return appropriate response.

Here is the sample python code to fetch list of s3 objects.

import logging
import boto3
from botocore.exceptions import ClientError

logger = logging.getLogger()
logger.setLevel('INFO')

""" Bucket Name """
bucket_name = "sandbox-bucket-name"

def lambda_handler(event, context):
    objects = list_bucket_objects(bucket_name)
    objectMap = {}
    if objects is not None:
        # List the object names
        logging.info(f'Objects in {bucket_name}')
        count = 0
        for obj in objects:
            objectMap.update({count : obj["Key"]})
            count = count + 1

    logging.info(objectMap)
    return objectMap
  
def list_bucket_objects(bucket_name):
    """List the objects in an Amazon S3 bucket
    :param bucket_name: string
    :return: List of bucket objects. If error, return None.
    """
    # Retrieve the list of bucket objects
    s3 = boto3.client('s3')
    try:
        response = s3.list_objects_v2(Bucket=bucket_name)
    except ClientError as e:
        # AllAccessDisabled error == bucket not found
        logging.error(e)
        return None
    return response['Contents']

Reads more in

Python Code Samples for Amazon S3 https://docs.aws.amazon.com/code-samples/latest/catalog/code-catalog-python-example_code-s3.html

https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example-deployment-pkg.html#with-s3-example-deployment-pkg-python

https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html

Categories: AWS Solution, Python Solutions

Tags: aws, aws lambda, AWS Solution, Python Solutions, s3, S3 Objects