How to create & query images and files using GraphQL with AWS AppSync, AWS Amplify, and Amazon S3
Storing and querying for files like images and videos is a common requirement for most applications, but how do you do this using GraphQL?
One option would be to Base64 encode the image and send as a string in the mutation. This comes with disadvantages like the encoded file being larger than the original binary, the operation being computationally expensive, and the added complexity around encoding and decoding properly.
Another option is to have a separate server (or API) for uploading files. This is the preferred approach and the technique we will be covering in this tutorial.
To view or try out the final example project, click here.
How it all works
You typically would need a few things to make this work:
- A GraphQL API
- A storage service or database for saving your files
- A database to store the GraphQL data including a reference to the location of the file
Take for example the following schema for a product in an E-commerce app:
type Product {
id: ID!
name: String!
description: String
price: Int
image: ?
}
How could we use this image field and make it work with our app to store and reference an image? Let's take a look at how this might work with an image stored in Amazon S3.
Using Amazon S3 there are two main types of access: private and public.
Public access means anyone with the file url can view or download it at any time. In this use case, we could reference the image url as the image field in the GraphQL schema. Since the image url is public anyway, we don't care who can view the image.
Private access means that only users calling the API from your app can view or download the file. In this use case, we would only store a reference to the image key (i.e. images/mycoolimage.png) as the image field in the GraphQL schema. Using this key, we can fetch a temporary signed url to view this image on demand from S3 whenever we would like it to be viewed by someone.
In this tutorial, you'll learn how to do both.
Creating the client
In this tutorial I will be writing the client code in React, but you can use Vue, Angular, or any other JavaScript framework because the API calls the we will be writing are not React specific.
Create a new client project, change into the directory and install the amplify and uuid dependencies:
npx create-react-app gqlimages
cd gqlimages
npm install aws-amplify @aws-amplify/ui-react uuid
Public access
The first example we will create is a GraphQL API that has public image access.
The GraphQL type that we will be working with is a Product with an image field. We want this product's image to be public so it can be shared and visible to anyone viewing the app, regardless if they are signed in or not.
The GraphQL schema we will use is this:
type Product @model {
id: ID!
name: String!
description: String
price: Int
image: String
}
How could we implement the API for this?
For mutations
- Store the image in S3
- Send a mutation to create the Product in the GraphQL API using the image reference along with the other product data
For Queries
- Query the product data from the GraphQL API. Because the image url is public, we can just render the image field immediately.
Creating the services
To build this API, we need the following:
- S3 bucket to store the image
- GraphQL API to store the image reference and other data about the type
- Authentication service to authenticate users (only needed in order to upload files to S3)
The first thing we will want to do is create the authentication service. To do so, we'll initialize an Amplify project and add authentication.
If you have not yet installed & configured the Amplify CLI, click here to view a video walkthrough.
amplify init
amplify add auth
? Do you want to use the default authentication and security configuration? Default configuration
? How do you want users to be able to sign in when using your Cognito User Pool? Username
? What attributes are required for signing up? Email
Next, we'll create the storage service (Amazon S3):
amplify add storage
? Please select from one of the below mentioned services: Content (Images, audio, video, etc.)
? Please provide a friendly name for your resource that will be used to label this category in the project: gqls3
? Please provide bucket name: <YOUR_UNIQUE_BUCKET_NAME>
? Who should have access: Auth and guest users
? What kind of access do you want for Authenticated users?
❯◉ create/update
◉ read
◉ delete
? What kind of access do you want for Guest users?
◯ create/update
❯◉ read
◯ delete
? Do you want to add a Lambda Trigger for your S3 Bucket? N
Finally, we'll create the GraphQL API:
amplify add api
? Please select from one of the below mentioned services (Use arrow keys): GraphQL
? Provide API name: (gqls3)
? Choose an authorization type for the API: API key
? Do you have an annotated GraphQL schema? N
? Do you want a guided schema creation? Y
? What best describes your project: Single object with fields
? Do you want to edit the schema now? Y
When prompted, update the schema located at /amplify/backend/api/gqls3/schema.graphql with the following:
type Product @model {
id: ID!
name: String!
description: String
price: Int
image: String
}
Next, we can deploy the API using the following:
amplify push
? Do you want to generate code for your newly created GraphQL API Yes
? Choose the code generation language target javascript
? Enter the file name pattern of graphql queries, mutations and subscriptions src/graphql/**/*.js
? Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions Yes
? Enter maximum statement depth [increase from default if your schema is deeply nested] 2
Next, we'll configure index.js to recognize the Amplify app:
import Amplify from 'aws-amplify'
import config from './aws-exports'
Amplify.configure(config)
Now that the services have been deployed, we need to update the S3 bucket to have a public /images folder so that anything stored in the folder can be viewed by anyone.
Warning: when making an S3 folder public, you should make sure that you never store any sensitive or private information here as the folder is completely open for anyone to view it. In this case, we are simulating an E-commerce app where we have public images for our products that would go on the main site.
Open the S3 console at https://s3.console.aws.amazon.com and find the bucket that you created in the previous step.
Next, click on the Permissions tab to update the bucket policy.
Update the policy to the following. You need to update the Resource field to your bucket's resource name (i.e. the arn:aws:s3:::gqlimages6c6fev-dev needs to be replaced with the name for your bucket):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::gqlimages6c6fev-dev/public/images/*"
}
]
}
Interacting with the API from a client application
Now that the backend is created, how can we interact with it to upload and read images from it?
Here is the code that we could use to not only save files to our API, but also query and render them in the UI.
There are two main functions:
-
createProduct- uploads the product image to S3 and saves the product data to AppSync in a GraphQL mutation -
listProducts- queries the GraphQL API for all products
import React, { useEffect, useState } from 'react';
import { Storage, API, graphqlOperation } from 'aws-amplify'
import { v4 as uuid } from 'uuid'
import { withAuthenticator } from '@aws-amplify/ui-react'
import { createProduct as CreateProduct } from './graphql/mutations'
import { listProducts as ListProducts } from './graphql/queries'
import config from './aws-exports'
const {
aws_user_files_s3_bucket_region: region,
aws_user_files_s3_bucket: bucket
} = config
function App() {
const [file, updateFile] = useState(null)
const [productName, updateProductName] = useState('')
const [products, updateProducts] = useState([])
useEffect(() => {
listProducts()
}, [])
// Query the API and save them to the state
async function listProducts() {
const products = await API.graphql(graphqlOperation(ListProducts))
updateProducts(products.data.listProducts.items)
}
function handleChange(event) {
const { target: { value, files } } = event
const fileForUpload = files[0]
updateProductName(fileForUpload.name.split(".")[0])
updateFile(fileForUpload || value)
}
// upload the image to S3 and then save it in the GraphQL API
async function createProduct() {
if (file) {
const extension = file.name.split(".")[1]
const { type: mimeType } = file
const key = `images/${uuid()}${productName}.${extension}`
const url = `https://${bucket}.s3.${region}.amazonaws.com/public/${key}`
const inputData = { name: productName , image: url }
try {
await Storage.put(key, file, {
contentType: mimeType
})
await API.graphql(graphqlOperation(CreateProduct, { input: inputData }))
} catch (err) {
console.log('error: ', err)
}
}
}
return (
<div style={styles.container}>
<input
type="file"
onChange={handleChange}
style={{margin: '10px 0px'}}
/>
<input
placeholder='Product Name'
value={productName}
onChange={e => updateProductName(e.target.value)}
/>
<button
style={styles.button}
onClick={createProduct}>Create Product</button>
{
products.map((p, i) => (
<img
style={styles.image}
key={i}
src={p.image}
/>
))
}
</div>
);
}
const styles = {
container: {
width: 400,
margin: '0 auto'
},
image: {
width: 400
},
button: {
width: 200,
backgroundColor: '#ddd',
cursor: 'pointer',
height: 30,
margin: '0px 0px 8px'
}
}
export default withAuthenticator(App);
To launch the app, run npm start.
To see the completed project code, click here and open the
src/Products.jsfile.
Private Access
The next example we will create is a GraphQL API with a type that has a private image field.
This image can only be accessed by someone using our app. If someone tries to fetch this image directly, they will not be able to view it.
For the image field, we'll create a GraphQL type type that holds all of the information we need in order to create and read private files from an S3 bucket, including the bucket name and region as well as the key we'd like to read from the bucket.
The GraphQL type that we will be working with is a User with an avatar field. We want this avatar image to be private so it can be only be visible to someone signed in to the app.
The GraphQL schema we will use is this:
type User @model {
id: ID!
username: String!
avatar: S3Object
}
type S3Object {
bucket: String!
region: String!
key: String!
}
How could we implement the API to make this work?
For mutations
- Store the image in S3
- Send a mutation to create the User in the GraphQL API using the image reference along with the other user data
For Queries
- Query the user data from the API (including the image reference)
- Get a signed URL for the image from S3 in another API call
To build this app, we need the following:
- Authentication service to authenticate users
- S3 bucket to store image
- GraphQL API to store the image reference and other data about the type
Building the app
If you did not build the app in the previous example, go back and build the above project (create the authentication service, GraphQL API, and S3 bucket) in order to continue.
We can now update the schema located at /amplify/backend/api/gqls3/schema.graphql and add the following types:
type User @model {
id: ID!
username: String!
avatar: S3Object
}
type S3Object {
bucket: String!
region: String!
key: String!
}
Next, we can deploy the changes:
amplify push
? Do you want to update code for your updated GraphQL API Yes
? Do you want to generate GraphQL statements (queries, mutations and
subscription) based on your schema types? This will overwrite your cu
rrent graphql queries, mutations and subscriptions Yes
Interacting with the API from a client application
Now that the backend is created, how can we interact with it to upload and read images from it?
Here is the code that we could use to not only save files to our API, but also query and render them in the UI.
There are three main functions:
-
createUser- (uploads the user image to S3 and saves the user data to AppSync in a GraphQL mutation) -
fetchUsers- Queries the GraphQL API for all users -
fetchImage- Gets the signed S3 url for the image in order for us to render it and renders it in the UI.
import React, { useState, useReducer, useEffect } from 'react'
import { withAuthenticator } from 'aws-amplify-react'
import { Storage, API, graphqlOperation } from 'aws-amplify'
import { v4 as uuid } from 'uuid'
import { createUser as CreateUser } from './graphql/mutations'
import { listUsers } from './graphql/queries'
import { onCreateUser } from './graphql/subscriptions'
import config from './aws-exports'
const {
aws_user_files_s3_bucket_region: region,
aws_user_files_s3_bucket: bucket
} = config
const initialState = {
users: []
}
function reducer(state, action) {
switch(action.type) {
case 'SET_USERS':
return { ...state, users: action.users }
case 'ADD_USER':
return { ...state, users: [action.user, ...state.users] }
default:
return state
}
}
function App() {
const [file, updateFile] = useState(null)
const [username, updateUsername] = useState('')
const [state, dispatch] = useReducer(reducer, initialState)
const [avatarUrl, updateAvatarUrl] = useState('')
function handleChange(event) {
const { target: { value, files } } = event
const [image] = files || []
updateFile(image || value)
}
async function fetchImage(key) {
try {
const imageData = await Storage.get(key)
updateAvatarUrl(imageData)
} catch(err) {
console.log('error: ', err)
}
}
async function fetchUsers() {
try {
let users = await API.graphql(graphqlOperation(listUsers))
users = users.data.listUsers.items
dispatch({ type: 'SET_USERS', users })
} catch(err) {
console.log('error fetching users')
}
}
async function createUser() {
if (!username) return alert('please enter a username')
if (file && username) {
const { name: fileName, type: mimeType } = file
const key = `${uuid()}${fileName}`
const fileForUpload = {
bucket,
key,
region,
}
const inputData = { username, avatar: fileForUpload }
try {
await Storage.put(key, file, {
contentType: mimeType
})
await API.graphql(graphqlOperation(CreateUser, { input: inputData }))
updateUsername('')
console.log('successfully stored user data!')
} catch (err) {
console.log('error: ', err)
}
}
}
useEffect(() => {
fetchUsers()
const subscription = API.graphql(graphqlOperation(onCreateUser))
.subscribe({
next: async userData => {
const { onCreateUser } = userData.value.data
dispatch({ type: 'ADD_USER', user: onCreateUser })
}
})
return () => subscription.unsubscribe()
}, [])
return (
<div style={styles.container}>
<input
label="File to upload"
type="file"
onChange={handleChange}
style={{margin: '10px 0px'}}
/>
<input
placeholder='Username'
value={username}
onChange={e => updateUsername(e.target.value)}
/>
<button
style={styles.button}
onClick={createUser}>Save Image</button>
{
state.users.map((u, i) => {
return (
<div
key={i}
>
<p
style={styles.username}
onClick={() => fetchImage(u.avatar.key)}>{u.username}</p>
</div>
)
})
}
<img
src={avatarUrl}
style={{ width: 300 }}
/>
</div>
)
}
const styles = {
container: {
width: 300,
margin: '0 auto'
},
username: {
cursor: 'pointer',
border: '1px solid #ddd',
padding: '5px 25px'
},
button: {
width: 200,
backgroundColor: '#ddd',
cursor: 'pointer',
height: 30,
margin: '0px 0px 8px'
}
}
export default withAuthenticator(App)
To launch the app, run npm start.
To view or try out the final example project, click here.
My Name is Nader Dabit. I am a Developer Advocate at Amazon Web Services working with projects like AWS AppSync and AWS Amplify. I specialize in cross-platform & cloud-enabled application development.
Top comments (31)
Nader, when I set my S3 bucket to public per instruction I got a warning in AWS S3 console:
Does it mean that anyone can use this S3 bucket???
Please let me know
Thanks
Yes, if you set a bucket of folder in a bucket public, anyone can read from that bucket. I put a warning letting the readers know about this as it is not recommended by AWS security policy, but many people ask for or want this functionality so I showed how it could be done as well.
Can public users of the S3 bucket just read from it or write into it as well? How to make it more secure?
Ah, no they would only be able to read from it using the instructions here. To make it secure from reads as well, see the other example I provided in this tutorial.
Hi Nader, Thanks for this great tutorial. I got everything working except that the User uploaded images are not showing up (They are in S3 as I can tell) with "403 Fobidden" error in browser console even though I signed in. Can you tell what I missed?
Products images are shown without problem.
Same problem.
I have the same issue. Can't seem to figure it out. I wonder what we missed...
Just what I needed! Using multer with MongoDB simply did not cut it for me. Especially because you can only delete images/files locally and they are not deleted in S3! Looking forward to learning more about DynamoDB and what it can do for me especially when it comes to working with dynamic images NoSQL and graphql along with serverless. Thanks for this great post Nader!
Thanks Maria, glad you found it useful!
Absolutely! Since I got you here I'll ask a question I was going to do in person tonight, but asking here will leave me more time to ask other questions. As far as dynamically deleting an image or file from the client that results in its deletion on S3, would the Amplify S3Image component do the trick since it renders an Amazon S3 object key as an image, and therefore I as I understand it, would make it possible to identify the image in question for deletion? Because the other examples regarding deleting files only show the deletion of individual, hard coded file names. But if one were to use the key approach, files would be dynamically deleted, right? Or am I getting this all wrong? Thanks in advance!
So there are two parts to accessing the S3Object, one from the bucket itself and two from the actual API. Typically the best security practice is to leave all images secure and only access them using a signed URL. The example I gave with private images is typically the use case I recommend. If we use @auth rules for owner, only the user uploading them image would be able to view it but in reality we want it to be available to any user of the app. Sure, we could set queries to null and allow anyone to access the location that of the image, but either way we ideally only want users accessing the image directly from our app to be successful.
We actually have equal support for Angular & Vue. We now also have an advocate like me on our team who specializes in angular but does not write as much content, he's busy traveling around giving more workshops and talks. I think we see much more articles talking about React because I am very visible and active in that community, but in reality there is pretty much feature parity between the frameworks.
I don't know the answer to this. If this is a feature you'd like, I'd suggest submitting an issue in the GitHub repo and we can see about putting it on our roadmap.
1 No, the @auth rules only apply to the GraphQL API not the S3 bucket for storage. The rules you mentioned will allow anyone to read from the database, but the a user still needs to be authorized to read from the S3 bucket in some way, either signed in or not, via the Amplify SDK (sends a signed request, gets a signed url that is valid for a set period of time)
4 Yes, we support multi auth now (starting last week) from the CLI -> aws-amplify.github.io/docs/cli-too...
5 You can update the API key by changing the expiration date in the local settings and run
amplify pushto update -> aws-amplify.github.io/docs/cli-too...Yes you can combine authorization rules. See details here
Private access is built in to Amplify - See docs here referencing
privateaccessYes, the process of storing would be the same, the only difference is you would need to deal with standard streaming / buffering protocols on the client that are agnostic to Amplify.
I have seen some commants that say the upload feature is not working.
I also faced this issue but I resolved it due to configuring the amplify with
aws-exports.js.Hopefully, it's a reference to everyone.
Hi Nader,
For the Private Access part, I have 403 to get images both for :
Storage.put( this.key, file, {
level: 'public',
contentType: 'image/*'
} )
Storage.put( this.key, file, {
level: 'private',
contentType: 'image/*'
} )
How to fix it ?
I have found the solution here :
itnext.io/part-2-adding-authentica...
We must just store only the document key and for each access use :
await Storage.vault.get(key) as string;
Hi Nader, i have two questions
please i need your help ASAP Thank you.
Thanks for the tutorial! But if I'm not mistaken, you never actually explain how to get a signed URL to access the Image. And when I google it, the process seems pretty complex. Am I missing something?
Hi Nader,
Really interesting article. Its showing pretty much what I wish to do but in an Angular 10 environment. I have set up GraphQL using AWS Amplify.API and was hoping to have a n elegant way to upload documents to S3 and store with the S3Object. I thought the tools could do this uploading and seen some notes that suggest it may be possible.
Ideally I wish to;
My initial research seemed to indicate that Amplify.API should be able to do this but finding examples I can build upon appears to be very difficult.
You mentioned that there is an Angular specialist - Are there any links to any workshops, videos, example codes which could be shared.
Thanks,
Paul
It looks like its possible for the file upload to S3 to succeed, but the graphql Mutation to fail. How do you deal with the zombie files?
You should check out Object Expiration for S3. As part of the mutation, you could then remove the Object expiration, or copy the file to another "persistent" bucket.
Thanks Laurin