This content originally appeared on Level Up Coding - Medium and was authored by Lee James Gilmore
Serverless AWS S3 pre-signed URLs ?
A practical example of using AWS S3 pre-signed URLs within your serverless solutions along with supporting code samples and diagrams, using the Serverless Framework and TypeScript.
Introduction
There are often times when architecting solutions on AWS where you need to grant users temporary access to either download or upload files to your system which are private to them.
Examples could be employees downloading their payslips, uploading your bank statements to a mortgage broker, or downloading content which you have paid for such as course content.
This article discusses using AWS S3 pre-signed URLs to facilitate the requirements above, whilst also ensuring that we reduce the chances of malicious actors accessing the resources by making the urls expiry quickly for the private resources.
“You are essentially giving a user temporary access to your private resources without the need for them to have AWS security credentials or permissions.”
You can access the code repo here which has verbose comments for clarity.
? Please note this is the minimal code and architecture to demonstrate the use of pre-signed URLs in AWS S3, so this is not production ready and does not adhere to coding best practices.
What are pre-signed URLs?
We are using two separate AWS S3 pre-signed URL actions in this demo, putObject and getObject. As you could have guessed, putObject is used for uploading files, and getObject is used for downloading them.
“All objects and buckets are private by default. However, you can use a pre-signed URL to optionally share objects or enable your customers/users to upload objects to buckets without AWS security credentials or permissions.” — https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
As you can see from the code repo, we are using the aws-sdk to get the temporary URLs using the AWS.S3.getSignedUrl method, which takes the action (getObject and putObject), as well as the bucket name, key (filename) and the expiry in seconds.
What are we building?
The diagram below shows the architecture which we are building in the repo:
? When we say the pre-signed url is ‘returned to the user’, this is typically a frontend client or backend service such as a React frontend, so the user has no knowledge of this exchange of URLs happening i.e. its al done in code.
Get File endpoint
The get file endpoints works by:
- The user hits the get file endpoint for a particular image ID which is stored in a private bucket in S3 i.e. they don’t have access to it.
- API Gateway invokes a lambda which generates a pre-signed download URL for the image in the private bucket.
- The temporary pre-signed URL is returned to the user within the response.
- The user uses this URL to download the image content before the link expires.
? The example code only works for PNG images for simplicity of the demo so we don’t need to use other services such as DynamoDB to store the file keys.
Upload File endpoint
The upload file endpoints works by:
- The user hits the upload file endpoint to request uploading a specific image to the private S3 bucket.
- API Gateway invokes a lambda which generates a pre-signed upload URL for the private bucket.
- The temporary pre-signed URL is returned to the user within the response.
- The user uses the URL to upload the image content to the private bucket before the link expires.
? In a real World example you would have authentication on the two public endpoints which ensures that the user is authorised to make the requests for the pre-signed URLs. This may be authentication using Amazon Cognito, as well as checking a DynamoDB database that the person is part of a relevant group for example.
Deploying the solution! ??
? Note: Running the following commands will incur charges on your AWS account so change the config accordingly.
In the root of the folder run npm i and then npm run deploy:develop which will install all of the dependencies and then deploy to AWS.
? As part of the deploy we use the serverless-s3-sync plugin which automatically pushes any files in a local folder to S3. In our example repo this pushes the ‘1.png’ file to the S3 bucket so we can use the get endpoint straight away! The code for this is shown below:
custom:
s3Sync:
- bucketName: ${self:custom.bucketName}
localDir: src/assets
acl: private
The serverless deploy will generate the resources for you using the Serverless Framework i.e. the API, compute layer and s3 bucket. You will see similar output to this on success:
? Make a note of the main API url as you will need this for the following section when testing the endpoints with a Postman file.
Testing the API! ?
The following sections describe downloading and uploading images.
Testing an image download ⬇️
You can test the downloading of an image by using the Postman file in the ./postman folder, and hitting the ‘Get Image Download URL’ request with an ID of 1:
You can now copy the pre-signed url from the ‘url’ property in the response and open it in a browser, which should download the image as expected as shown below.
“Yep, it’s an image of a Fortnite ‘Giddy Up’ Funko Pop collectible on my desk.. I was struggling for inspiration” ?
Testing an image upload ⬆️
You can test the uploading of an image by using the Postman file in the ./postman folder, and hitting the ‘Put Image Upload URL’ request:
Now use the url property in the response (the pre-signed upload url) in the following curl command in your terminal (ensuring that you have an image named 2.png in the same directory your running the command from)
curl -X PUT -T 2.png -L "<the pre-signed upload url>"
You will see a similar response to the following:
If you now log into your AWS console and go to the S3 bucket you will see the uploaded image:
You can now use the original get image URL to download the image you have just uploaded using the correct key i.e. image ID of 2 in the example above.
Are there any security considerations about the URLs?
One thing to note is that once a url is used it is still active and not expired, so a malicious user getting access to the link can still use it before it expires. This is one reason to keep the url short lived, which in reality is typically a second in production from my experience when code is using the urls which are returned.
Wrapping up
I hope you found that useful as a light example of S3 pre-signed URLs and when you may use them in your own solutions.
I would love to connect with you on any of the following:
https://www.linkedin.com/in/lee-james-gilmore/
https://twitter.com/LeeJamesGilmore
If you found the articles inspiring or useful please feel free to support me with a virtual coffee https://www.buymeacoffee.com/leegilmore and either way lets connect and chat! ☕️
If you enjoyed the posts please follow my profile Lee James Gilmore for further posts/series, and don’t forget to connect and say Hi ?
Please also use the ‘clap’ feature at the bottom of the post if you enjoyed it! (You can clap more than once!!)
About me
“Hi, I’m Lee, an AWS certified technical architect and Lead Software Engineer based in the UK, currently working as a Technical Cloud Architect and Principal Serverless Developer, having worked primarily in full-stack JavaScript on AWS for the past 5 years.
I consider myself a serverless evangelist with a love of all things AWS, innovation, software architecture and technology.”
** The information provided are my own personal views and I accept no responsibility on the use of the information. ***
Serverless S3 Pre-signed URLs ? was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Lee James Gilmore
Lee James Gilmore | Sciencx (2021-08-13T02:53:56+00:00) Serverless S3 Pre-signed URLs. Retrieved from https://www.scien.cx/2021/08/13/serverless-s3-pre-signed-urls/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.