Introduction
We have explored quite a handful of the technologies we set up to learn. From extracting JSON data and session tokens from request objects in actix-web to working with forms, interactive UI elements and server-side rendering with SvelteKit. We still have some missing pieces, however. What about handling Multipart FormData in actix-web? How can we integrate the popular AWS S3 to our application to scalably manage its files (images, in this context)? These questions and many more will be addressed in this post. Apologies if it's lengthy.
Source code
The source code for this series is hosted on GitHub via:
rust-auth
A full-stack secure and performant authentication system using rust's Actix web and JavaScript's SvelteKit.
This application resulted from this series of articles and it's currently live here(I have disabled the backend from running live).
Run locally
You can run the application locally by first cloning it:
~/$ git clone https://github.com/Sirneij/rust-auth.git
After that, change directory into each subdirectory: backend and frontend in different terminals. Then following the instructions in each subdirectory to run them.
Implementation
You can get the overview of the code for this article on github. Some changes were also made directly to the repo's main branch. We will cover them.
Step 1: Install and configure AWS S3 dependencies
Amazon Simple Storage Service (Amazon S3) is a popular storage service, provided by the cloud giant, Amazon, to provide dependable, reliable, scalable and performant object storage. We'll be using this service to store our users' thumbnails. In rust, working with AWS S3 has been made seamless by Amazon's official SDK for rust, though still in Developer Preview mode. Let's install the package and it's mandatory aws-config. aws-config is important to properly configure the credentials necessary for interacting with AWS services. To work with multipart forms and data in actix web, actix-multipart is needed.
~/rust-auth/backend$ cargo add aws-config
~/rust-auth/backend$ cargo add aws-sdk-s3
~/rust-auth/backend$ cargo add actix-multipart
NOTE: Before proceeding, ensure you have prepared an S3 bucket with their crdentials, viz:
AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_REGION, andAWS_S3_BUCKET_NAME. Set the configurations in your.envor.env.developmentfile and ensure you do not push them to GitHub. If you haven't created a credential, kindly follow the **S3 Bucket* section of this guide.*
After the installation, create an uploads module in the backend and populate backend/src/uploads/client.rs with:
// backend/src/uploads/client.rs
use tokio::io::AsyncReadExt as _;
/// S3 client wrapper to expose semantic upload operations.
#[derive(Debug, Clone)]
pub struct Client {
s3: aws_sdk_s3::Client,
bucket_name: String,
}
impl Client {
/// Construct S3 client wrapper.
pub fn new(config: aws_sdk_s3::Config) -> Client {
Client {
s3: aws_sdk_s3::Client::from_conf(config),
bucket_name: std::env::var("AWS_S3_BUCKET_NAME").unwrap(),
}
}
pub fn url(&self, key: &str) -> String {
format!(
"https://{}.s3.{}.amazonaws.com/{key}",
std::env::var("AWS_S3_BUCKET_NAME").unwrap(),
std::env::var("AWS_REGION").unwrap(),
)
}
/// Facilitate the upload of file to s3.
pub async fn upload(
&self,
file: &actix_multipart::form::tempfile::TempFile,
key_prefix: &str,
) -> crate::types::UploadedFile {
let filename = file.file_name.as_deref().expect("TODO");
let key = format!("{key_prefix}{filename}");
let s3_url = self
.put_object_from_file(file.file.path().to_str().unwrap(), &key)
.await;
crate::types::UploadedFile::new(filename, key, s3_url)
}
/// Real upload of file to S3
async fn put_object_from_file(&self, local_path: &str, key: &str) -> String {
let mut file = tokio::fs::File::open(local_path).await.unwrap();
let size_estimate = file
.metadata()
.await
.map(|md| md.len())
.unwrap_or(1024)
.try_into()
.expect("file too big");
let mut contents = Vec::with_capacity(size_estimate);
file.read_to_end(&mut contents).await.unwrap();
let _res = self
.s3
.put_object()
.bucket(&self.bucket_name)
.key(key)
.body(aws_sdk_s3::primitives::ByteStream::from(contents))
.send()
.await
.expect("Failed to put object");
self.url(key)
}
/// Attempts to delete object from S3. Returns true if successful.
pub async fn delete_file(&self, key: &str) -> bool {
self.s3
.delete_object()
.bucket(&self.bucket_name)
.key(key)
.send()
.await
.is_ok()
}
}
Using aws-sdk-s3 requires that tokio is installed. The above code was drafted from actix forms with multipart and s3 example with few modifications. There is a Client wrapper with two main endpoints: upload and delete_file. upload uses put_object_from_file to upload files to S3 and returns the uploaded files' URLs while delete_file deletes a file. We also created some type in backend/src/types/upload.rs:
// backend/src/types/upload.rs
#[derive(Debug, serde::Serialize, Clone)]
pub struct UploadedFile {
filename: String,
s3_key: String,
pub s3_url: String,
}
impl UploadedFile {
/// Construct new uploaded file info container.
pub fn new(
filename: impl Into<String>,
s3_key: impl Into<String>,
s3_url: impl Into<String>,
) -> Self {
Self {
filename: filename.into(),
s3_key: s3_key.into(),
s3_url: s3_url.into(),
}
}
}
This type makes the uploaded file's URL available for public consumption. Now let's configure our credentials. Open up backend/src/startup.rs and add this function:
// backend/src/startup.rs
...
async fn configure_and_return_s3_client() -> crate::uploads::Client {
// S3 configuration and client
// Get id and secret key from the environment
let aws_key = std::env::var("AWS_ACCESS_KEY_ID").expect("Failed to get AWS key.");
let aws_key_secret =
std::env::var("AWS_SECRET_ACCESS_KEY").expect("Failed to get AWS secret key.");
// build the aws cred
let aws_cred = aws_sdk_s3::config::Credentials::new(
aws_key,
aws_key_secret,
None,
None,
"loaded-from-custom-env",
);
// build the aws client
let aws_region = aws_sdk_s3::config::Region::new(
std::env::var("AWS_REGION").unwrap_or("eu-west-2".to_string()),
);
let aws_config_builder = aws_sdk_s3::config::Builder::new()
.region(aws_region)
.credentials_provider(aws_cred);
let aws_config = aws_config_builder.build();
crate::uploads::Client::new(aws_config)
}
We got our credentials from environment variables and used the S3 configuration builder to build them and thereafter return the Client wrapper previously discussed. After that, we can opt to create an app data for it so that every handler can access it:
// backend/src/startup.rs
...
async fn run(
listener: std::net::TcpListener,
db_pool: sqlx::postgres::PgPool,
settings: crate::settings::Settings,
) -> Result<actix_web::dev::Server, std::io::Error> {
// For S3 client: create singleton S3 client
let s3_client = actix_web::web::Data::new(configure_and_return_s3_client().await);
...
// S3 client
.app_data(s3_client.clone())
...
}
...
That's it for the configuration. We will utilize it in the next subsection.
NOTE: Ensure you make all the modules connect. That will help you to understand how to move things around. If you get stuck, check out the repo.
Step 2: Write the user's profile update handler logic
In backend/src/routes/users/update_user.rs:
// backend/src/routes/users/update_user.rs
use sqlx::Row;
#[derive(actix_multipart::form::MultipartForm)]
pub struct UserForm {
first_name: Option<actix_multipart::form::text::Text<String>>,
last_name: Option<actix_multipart::form::text::Text<String>>,
#[multipart(limit = "1 MiB")]
thumbnail: Option<actix_multipart::form::tempfile::TempFile>,
phone_number: Option<actix_multipart::form::text::Text<String>>,
birth_date: Option<actix_multipart::form::text::Text<chrono::NaiveDate>>,
github_link: Option<actix_multipart::form::text::Text<String>>,
}
#[derive(serde::Deserialize, Debug)]
pub struct UpdateUser {
first_name: Option<String>,
thumbnail: Option<String>,
last_name: Option<String>,
}
#[derive(serde::Deserialize, Debug)]
pub struct UpdateUserProfile {
phone_number: Option<String>,
birth_date: Option<chrono::NaiveDate>,
github_link: Option<String>,
}
#[derive(serde::Deserialize)]
pub struct Thumbnail {
pub thumbnail: Option<String>,
}
#[tracing::instrument(name = "Updating an user", skip(pool, form, session))]
#[actix_web::patch("/update-user/")]
pub async fn update_users_details(
pool: actix_web::web::Data<sqlx::postgres::PgPool>,
form: actix_multipart::form::MultipartForm<UserForm>,
session: actix_session::Session,
s3_client: actix_web::web::Data<crate::uploads::Client>,
) -> actix_web::HttpResponse {
let session_uuid = match crate::routes::users::logout::session_user_id(&session).await {
Ok(id) => id,
Err(e) => {
tracing::event!(target: "session",tracing::Level::ERROR, "Failed to get user from session. User unauthorized: {}", e);
return actix_web::HttpResponse::Unauthorized().json(crate::types::ErrorResponse {
error: "You are not logged in. Kindly ensure you are logged in and try again"
.to_string(),
});
}
};
// Create a transaction object.
let mut transaction = match pool.begin().await {
Ok(transaction) => transaction,
Err(e) => {
tracing::event!(target: "backend", tracing::Level::ERROR, "Unable to begin DB transaction: {:#?}", e);
return actix_web::HttpResponse::InternalServerError().json(
crate::types::ErrorResponse {
error: "Something unexpected happend. Kindly try again.".to_string(),
},
);
}
};
// At first, set all fields to update to None except user_id
let mut updated_user = UpdateUser {
first_name: None,
last_name: None,
thumbnail: None,
};
let mut user_profile = UpdateUserProfile {
phone_number: None,
birth_date: None,
github_link: None,
};
// If thumbnail was included for update
if let Some(thumbnail) = &form.0.thumbnail {
// Get user's current thumbnail from the DB
let user_current_thumbnail = match sqlx::query("SELECT thumbnail FROM users WHERE id=$1")
.bind(session_uuid)
.map(|row: sqlx::postgres::PgRow| Thumbnail {
thumbnail: row.get("thumbnail"),
})
.fetch_one(&mut *transaction)
.await
{
Ok(image_url) => image_url.thumbnail,
Err(e) => {
tracing::event!(target: "sqlx",tracing::Level::ERROR, "Failed to get user thumbnail from the DB: {:#?}", e);
None
}
};
// If there is a current image, delete it
if let Some(url) = user_current_thumbnail {
let s3_image_key = &url[url.find("media").unwrap_or(url.len())..];
if !s3_client.delete_file(s3_image_key).await {
tracing::event!(target: "backend",tracing::Level::INFO, "We could not delete the current thumbnail of user with ID: {}", session_uuid);
}
}
// make key prefix (make sure it ends with a forward slash)
let s3_key_prefix = format!("media/rust-auth/{session_uuid}/");
// upload temp files to s3 and then remove them
let uploaded_file = s3_client.upload(thumbnail, &s3_key_prefix).await;
updated_user.thumbnail = Some(uploaded_file.s3_url);
}
// If first_name is updated
if let Some(f_name) = form.0.first_name {
updated_user.first_name = Some(f_name.0);
}
// If last_name is updated
if let Some(l_name) = form.0.last_name {
updated_user.last_name = Some(l_name.0);
}
// If phone_number is updated
if let Some(phone) = form.0.phone_number {
user_profile.phone_number = Some(phone.0);
}
// If birth_date is updated
if let Some(bd) = form.0.birth_date {
user_profile.birth_date = Some(bd.0);
}
// If github_link is updated
if let Some(gl) = form.0.github_link {
user_profile.github_link = Some(gl.0);
}
// Update a user in the DB
match update_user_in_db(&mut transaction, &updated_user, &user_profile, session_uuid).await {
Ok(u) => u,
Err(e) => {
tracing::event!(target: "sqlx",tracing::Level::ERROR, "Failed to update user in DB: {:#?}", e);
let error_message = crate::types::ErrorResponse {
error: format!("User could not be updated: {e}"),
};
return actix_web::HttpResponse::InternalServerError().json(error_message);
}
};
let updated_user = match crate::utils::get_active_user_from_db(
None,
Some(&mut transaction),
Some(session_uuid),
None,
)
.await
{
Ok(user) => {
tracing::event!(target: "backend", tracing::Level::INFO, "User retrieved from the DB.");
crate::types::UserVisible {
id: user.id,
email: user.email,
first_name: user.first_name,
last_name: user.last_name,
is_active: user.is_active,
is_staff: user.is_staff,
is_superuser: user.is_superuser,
date_joined: user.date_joined,
thumbnail: user.thumbnail,
profile: crate::types::UserProfile {
id: user.profile.id,
user_id: user.profile.user_id,
phone_number: user.profile.phone_number,
birth_date: user.profile.birth_date,
github_link: user.profile.github_link,
},
}
}
Err(e) => {
tracing::event!(target: "backend", tracing::Level::ERROR, "User cannot be retrieved from the DB: {:#?}", e);
let error_message = crate::types::ErrorResponse {
error: "User was not found".to_string(),
};
return actix_web::HttpResponse::NotFound().json(error_message);
}
};
if transaction.commit().await.is_err() {
return actix_web::HttpResponse::InternalServerError().finish();
}
tracing::event!(target: "backend", tracing::Level::INFO, "User updated successfully.");
actix_web::HttpResponse::Ok().json(updated_user)
}
#[tracing::instrument(name = "Updating user in DB.", skip(transaction))]
async fn update_user_in_db(
transaction: &mut sqlx::Transaction<'_, sqlx::Postgres>,
user_to_update: &UpdateUser,
user_profile: &UpdateUserProfile,
user_id: uuid::Uuid,
) -> Result<(), sqlx::Error> {
match sqlx::query(
"
UPDATE
users
SET
first_name = COALESCE($1, first_name),
last_name = COALESCE($2, last_name),
thumbnail = COALESCE($3, thumbnail)
WHERE
id = $4
AND is_active = true
AND (
$1 IS NOT NULL
AND $1 IS DISTINCT
FROM
first_name
OR $2 IS NOT NULL
AND $2 IS DISTINCT
FROM
last_name
OR $3 IS NOT NULL
AND $3 IS DISTINCT
FROM
thumbnail
)",
)
.bind(&user_to_update.first_name)
.bind(&user_to_update.last_name)
.bind(&user_to_update.thumbnail)
.bind(user_id)
.execute(&mut *transaction)
.await
{
Ok(r) => {
tracing::event!(target: "sqlx", tracing::Level::INFO, "User has been updated successfully: {:#?}", r);
}
Err(e) => {
tracing::event!(target: "sqlx",tracing::Level::ERROR, "Failed to update user into DB: {:#?}", e);
return Err(e);
}
}
match sqlx::query(
"
UPDATE
user_profile
SET
phone_number = COALESCE($1, phone_number),
birth_date = $2,
github_link = COALESCE($3, github_link)
WHERE
user_id = $4
AND (
$1 IS NOT NULL
AND $1 IS DISTINCT
FROM
phone_number
OR $2 IS NOT NULL
AND $2 IS DISTINCT
FROM
birth_date
OR $3 IS NOT NULL
AND $3 IS DISTINCT
FROM
github_link
)",
)
.bind(&user_profile.phone_number)
.bind(user_profile.birth_date)
.bind(&user_profile.github_link)
.bind(user_id)
.execute(&mut *transaction)
.await
{
Ok(r) => {
tracing::event!(target: "sqlx", tracing::Level::INFO, "User profile has been updated successfully: {:#?}", r);
}
Err(e) => {
tracing::event!(target: "sqlx",tracing::Level::ERROR, "Failed to update user profile into DB: {:#?}", e);
return Err(e);
}
}
Ok(())
}
Waoh... What a long piece of code! Let's go through it.
First, we defined a UserForm struct which derives the actix-multipart MultipartForm. This is mandatory. Since thumbnail is the file field, we used TempFile and limited its size to 1 MiB (1024 KiB) — this was also the default size allowed in the put_object_from_file method. Other fields were either type String or chrono::NaiveDate. Notice that all the fields were made optional. This is to ensure that only provided fields will be updated — hence the use of the PATCH HTTP verb.
Moving on, we used s3_client: actix_web::web::Data<crate::uploads::Client> as one of the parameters of the update_users_details handler definition. This makes this handler access the methods embedded in it. We also have form: actix_multipart::form::MultipartForm<UserForm as against the actix_web::web::Json<...> we were used to. This is because we are expecting multipart form data and not JSON data.
Inside the handler, we first checked whether or not the requesting user is logged in. From such a user's session, we retrieve user_id, incepted a transaction block and initialized the data that will be saved in the database. Next, we checked whether a profile image was included in the request. If it was, we retrieve the current user's image, delete it and then upload a new one such that the uploaded image's path will be media/rust-auth/{user_id}/.
Then, we checked whether other fields were included and each data was updated accordingly. Next was the actual update in the DB. This update used a separate abstraction, update_user_in_db. We have two relatively optimised queries — one updates the users table while the other user_profile. In the SQL queries, we only opted to update either table only if a distinct value was supplied. After the data were updated, we then retrieved the updated user data using the get_active_user_from_db function located in backend/src/utils/users.rs:
// backend/src/utils/users.rs
use sqlx::Row;
#[tracing::instrument(name = "Getting an active user from the DB.", skip(pool))]
pub async fn get_active_user_from_db(
pool: Option<&sqlx::postgres::PgPool>,
transaction: Option<&mut sqlx::Transaction<'_, sqlx::Postgres>>,
id: Option<uuid::Uuid>,
email: Option<&String>,
) -> Result<crate::types::User, sqlx::Error> {
let mut query_builder =
sqlx::query_builder::QueryBuilder::new(crate::queries::USER_AND_USER_PROFILE_QUERY);
if let Some(id) = id {
query_builder.push(" u.id=");
query_builder.push_bind(id);
}
if let Some(e) = email {
query_builder.push(" u.email=");
query_builder.push_bind(e);
}
let sqlx_query = query_builder
.build()
.map(|row: sqlx::postgres::PgRow| crate::types::User {
id: row.get("u_id"),
email: row.get("u_email"),
first_name: row.get("u_first_name"),
password: row.get("u_password"),
last_name: row.get("u_last_name"),
is_active: row.get("u_is_active"),
is_staff: row.get("u_is_staff"),
is_superuser: row.get("u_is_superuser"),
thumbnail: row.get("u_thumbnail"),
date_joined: row.get("u_date_joined"),
profile: crate::types::UserProfile {
id: row.get("p_id"),
user_id: row.get("p_user_id"),
phone_number: row.get("p_phone_number"),
birth_date: row.get("p_birth_date"),
github_link: row.get("p_github_link"),
},
});
let fetched_query = {
if pool.is_some() {
let p = pool.unwrap();
sqlx_query.fetch_one(p).await
} else {
let t = transaction.unwrap();
sqlx_query.fetch_one(&mut *t).await
}
};
match fetched_query {
Ok(user) => Ok(user),
Err(e) => {
tracing::event!(target: "sqlx",tracing::Level::ERROR, "User not found in DB: {:#?}", e);
Err(e)
}
}
}
In this function, we used yet another concept in SQLx called QueryBuilder. It is very useful to dynamically construct queries at runtime. In this case, we used a prepared SQL query in backend/src/queries/users.rs:
-- backend/src/queries/users.rs
SELECT
u.id AS u_id,
u.email AS u_email,
u.password AS u_password,
u.first_name AS u_first_name,
u.last_name AS u_last_name,
u.is_active AS u_is_active,
u.is_staff AS u_is_staff,
u.is_superuser AS u_is_superuser,
u.thumbnail AS u_thumbnail,
u.date_joined AS u_date_joined,
p.id AS p_id,
p.user_id AS p_user_id,
p.phone_number AS p_phone_number,
p.birth_date AS p_birth_date,
p.github_link AS p_github_link
FROM
users u
LEFT JOIN user_profile p ON p.user_id = u.id
WHERE
u.is_active = true AND
to get user's entire data by LEFT JOINING the user_profile table to the users table. This ensures that all the data are retrieved once! Then, depending on whether an ID or e-mail address was supplied, we made some filtering and then built the builder. Since either sqlx::postgres::Pool or sqlx::Transaction<'_, sqlx::Postgres> instance is expected, we dynamically set that also and finally execute the query which returns the user data. Back to the main handler, if everything goes well, we commit all the changes and returned the user's data as a response. That's it! Long but easy to reason with.
After that, we added the handler to our auth_routes_config in backend/src/routes/users/mod.rs and voila! our endpoint is up!
NOTE: We created a new user type,
UserProfile, and expanded the previous ones to accommodate that. You can check outbackend/src/types/users.rsfor the updated types.
Step 3: Current user handler logic
Apart from updating the users, we may need to also get the current user using their session token and that was done in backend/src/routes/users/current_user.rs:
// backend/src/routes/users/current_user.rs
#[tracing::instrument(
name = "Accessing retrieving current user endpoint.",
skip(pool, session)
)]
#[actix_web::get("/current-user/")]
pub async fn get_current_user(
pool: actix_web::web::Data<sqlx::postgres::PgPool>,
session: actix_session::Session,
) -> actix_web::HttpResponse {
match crate::routes::users::logout::session_user_id(&session).await {
Ok(id) => {
match crate::utils::get_active_user_from_db(Some(&pool), None, Some(id), None).await {
Ok(user) => {
tracing::event!(target: "backend", tracing::Level::INFO, "User retrieved from the DB.");
actix_web::HttpResponse::Ok().json(crate::types::UserVisible {
id: user.id,
email: user.email,
first_name: user.first_name,
last_name: user.last_name,
is_active: user.is_active,
is_staff: user.is_staff,
is_superuser: user.is_superuser,
date_joined: user.date_joined,
thumbnail: user.thumbnail,
profile: crate::types::UserProfile {
id: user.profile.id,
user_id: user.profile.user_id,
phone_number: user.profile.phone_number,
birth_date: user.profile.birth_date,
github_link: user.profile.github_link,
},
})
}
Err(e) => {
tracing::event!(target: "backend", tracing::Level::ERROR, "User cannot be retrieved from the DB: {:#?}", e);
let error_message = crate::types::ErrorResponse {
error: "User was not found".to_string(),
};
actix_web::HttpResponse::NotFound().json(error_message)
}
}
}
Err(e) => {
tracing::event!(target: "session",tracing::Level::ERROR, "Failed to get user from session. User unauthorized: {}", e);
actix_web::HttpResponse::Unauthorized().json(crate::types::ErrorResponse {
error: "You are not logged in. Kindly ensure you are logged in and try again"
.to_string(),
})
}
}
}
As usual, we retrieved the user's ID from the session and used it to retrieve the user's data using the get_active_user_from_db function discussed previously. Pretty basic! We also added it to auth_routes_config to make it accessible.
Step 4: Deploy changes to fly.io
Having made these changes, it's time to deploy them. We had already laid a solid foundation for that. What we only need to do is set our secret credentials for AWS S3 to fly.io:
~/rust-auth/backend$ flyctl secrets set AWS_REGION=<your_bucket_region>
~/rust-auth/backend$ flyctl secrets set AWS_S3_BUCKET_NAME=<your_bucket_name>
~/rust-auth/backend$ flyctl secrets set AWS_SECRET_ACCESS_KEY=<your_access_key>
~/rust-auth/backend$ flyctl secrets set AWS_ACCESS_KEY_ID=<your_access_id>
After that, you can then deploy:
~/rust-auth/backend$ fly deploy
That's it! Your changes are live and ready to be consumed! In the next article, we will build our front-end to do just that! See ya...
Outro
Enjoyed this article? Consider contacting me for a job, something worthwhile or buying a coffee ☕. You can also connect with/follow me on LinkedIn and Twitter. It isn't bad if you help share this article for wider coverage. I will appreciate it...
Top comments (0)