Introduction
Firebase is a suite of cloud-based tools and services on Google Cloud Platform for application development. It offers hosting, authentication, storage, and databases that can be easily integrated into your app. One of its most popular tools is called Firestore, a key-value non-relational database. That means that uses structures similar to JSON to store information.
During my professional experience with Firebase and Google Cloud, I encountered challenges with optimizing and securing applications. To help others overcome these difficulties, I have compiled some key insights that I wish I had known from the start. Whether you're a beginner or an experienced developer, the concepts outlined in this article are essential for a successful learning journey with these technologies.
Security Rules
Firebase documentation does not emphasize enough the importance of Security Rules, especially for those just starting to integrate Firebase into their projects. If you do not properly implement security rules in your production application it can have severe consequences. Without proper security, anyone can potentially modify your database through API access, such as through Bash or Powershell.
It is crucial to design your Firebase Security Rules before writing any code. If Security Rules are added at the end of the development process, you may be faced with the issue of accumulated technical debt and security vulnerabilities, which can be both time-consuming and costly to resolve.
Security Rules should be considered a critical component of database design and should be discussed throughout the entire project, especially in the early stages. It's more efficient to revise and make changes to a draft on paper or a Jamboard than to retroactively fix a production implementation affecting numerous users.
There are several patterns for controlling access to your application, including:
- Authenticated-only access
- Content-owner-only access
- Granular access control
- Functional access control
Authenticated-only access
You can restrict reading access to a collection of only signed-in users:
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match posts/{document=**} {
allow read: if request.auth != null
allow write: if false
}
}
}
Content-owner-only access
Also, you can implement a content-owner-only access pattern. In this way, users can access only their own documents, that is, the documents that have the same uid as the user.
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /user/{userId}/{documents=**} {
allow read, write: if request.auth != null && request.auth.uid == userId
}
}
}
Granular control access
Another option is to add more granular rules to restrict access depending on the specific operation.
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /posts/{documents=**} {
allow read: if request.auth != null && request.auth.uid && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == "Reader"
allow create, update: if request.auth != null if get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == "Writer"
allow delete: if request.auth != null if get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == "Admin"
}
}
}
In this example, the user only can read a post if he has reading access, and only can create and update a post if he is a writer. Also, only an Admin can delete content inside the posts collection.
Functional control access
If your project has a lot of duplicated code in your security rules, you can refactor it into functions and call those functions instead of copy and paste your rules.
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
function isAuthenticated(){
return request.auth != null
}
function isAdmin(){
return isAuthenticated() && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == "Admin"
}
match /products/{documents=**} {
allow read, write: if isAdmin()
}
}
}
Note: You have to keep in mind that {documents=**} means that it includes all the subcollections inside the products collection.
The Local Emulator Suite
It would be helpful if Firebase's documentation emphasized how important it is to use this tool, especially for those who use Firebase Functions. However, it doesn't seem like the documentation talks about it very much. There are entries on their website related to the Emulator Suite, but it feels a little messy sometimes. It took me months to understand that I could take advantage of that Emulator Suite and how to use it properly.
This tool lets you test your projects on your own computer without affecting the real services. The local emulators support Firestore, Realtime Database, Functions, Storage, Authentication, and PubSubs.
Testing your security rules and storage rules with the local emulators before you publish them to Google Cloud can help you find problems early. Plus, it can help you save money because fewer deploys mean fewer charges. Keeping all your rules in one place also makes it easier to keep track of them.
You can see the official documentation to learn more about the Local Emulator Suite
Optimizations for performance and scalability
It might seem like a waste of time to focus on scalability and performance at the start of a small project. This is correct, in a way. Engineers have to avoid over-engineering. However, it is important to have a solid understanding of algorithms and logic so you are prepared when the project grows and performance becomes an issue. Otherwise, you may end up having to spend a lot of time and money fixing slow execution time or high memory consumption. To be efficient, you should learn techniques for writing more efficient Firebase Functions, like how to delete big collections or update many documents with complicated conditions.
How to delete large collections
Some collections can grow to millions of documents, and deleting documents inside of them can be a computation-intensive task. The examples below come from the official Firebase documentation. These show the recommended way of handling this type of task.
async function deleteCollection(db, collectionPath, batchSize) {
const collectionRef = db.collection(collectionPath);
const query = collectionRef.orderBy('__name__').limit(batchSize);
return new Promise((resolve, reject) => {
deleteQueryBatch(db, query, resolve).catch(reject);
});
}
async function deleteQueryBatch(db, query, resolve) {
const snapshot = await query.get();
const batchSize = snapshot.size;
if (batchSize === 0) {
// When there are no documents left, we are done
resolve();
return;
}
// Delete documents in a batch
const batch = db.batch();
snapshot.docs.forEach((doc) => {
batch.delete(doc.ref);
});
await batch.commit();
// Recurse on the next process tick, to avoid
// exploding the stack.
process.nextTick(() => {
deleteQueryBatch(db, query, resolve);
});
}
If you're working with big files and running into processing issues, one solution could be to give your function more resources, like extra RAM or a longer timeout.
exports.convertLargeFile = functions
.runWith({
// Ensure the function has enough memory and time
// to process large files
timeoutSeconds: 300,
memory: "1GB",
})
.storage.object()
.onFinalize((object) => {
// Do some complicated things that take a lot of memory and time
})
Enqueued Cloud Tasks
For time-consuming, resource-intensive, or bandwidth-limited tasks that need to run asynchronously, Google Cloud offers the possibility to enqueue functions with Cloud Tasks. You can control the number of attempts, backoff second, and the max of concurrent dispatches. The example below comes from the official documentation:
exports.backupApod = functions
.runWith( {secrets: ["SOME_KEY"]})
.tasks.taskQueue({
retryConfig: {
maxAttempts: 5,
minBackoffSeconds: 60,
},
rateLimits: {
maxConcurrentDispatches: 6,
},
}).onDispatch(async (data) => {
//your complex task
})
exports.enqueueBackupTasks = functions.https.onRequest(
async (_request, response) => {
const queue = getFunctions().taskQueue("backupApod");
const enqueues = [];
for (let i = 0; i <= 10; i += 1) {
// Enqueue each task with i*60 seconds day. Our task queue function
// should process ~1 task/min.
const scheduleDelaySeconds = i * 60
enqueues.push(
queue.enqueue(
{ id: `task-${i}` },
{
scheduleDelaySeconds,
dispatchDeadlineSeconds: 60 * 5 // 5 minutes
},
),
);
}
await Promise.all(enqueues);
response.sendStatus(200);
});
Backups in Firebase
As far as I know, right now Google Cloud doesn't have a way to automatically backup Firestore. I use BackupFire to do it for me, but you can also write a solution using Firebase Functions. Just keep in mind that when you restore a backup, Firebase tries to merge it with the original database, not overwrite it. This can have important consequences. Consider this example:
There are two customers:
- User A, uid:
z1V5scLfVPX7YD8fRCFRhhRAwVN2 - User B, uid:
Zk5olEuvZ1WokRrCqIN5gGaIu353
User A gained a gift card or discount code and it was saved in a subcollection inside the user document.
users/z1V5scLfVPX7YD8fRCFRhhRAwVN2/discountCodes/dGn5OxSEPohqpZbIyLmCjqVEiwz2
{
code: 'dGn5OxSEPohqpZbIyLmCjqVEiwz2',
expirationDate: '2023-01-18T18:34:33.565Z',
isAGift: false
}
The app allows users to transfer codes to others. User A transferred the code to User B
users/z1V5scLfVPX7YD8fRCFRhhRAwVN2/discountCodes/dGn5OxSEPohqpZbIyLmCjqVEiwz2
{
code: 'dGn5OxSEPohqpZbIyLmCjqVEiwz2',
expirationDate: '2023-01-18T18:34:33.565Z',
isAGift: true
}
However, a backup from the previous day had to be restored due to an emergency. Upon checking, there were inconsistencies in the database. Both User A and User B now have the same discount code.
users/z1V5scLfVPX7YD8fRCFRhhRAwVN2/discoutCodes/dGn5OxSEPohqpZbIyLmCjqVEiwz2
{
code: 'dGn5OxSEPohqpZbIyLmCjqVEiwz2',
expirationDate: '2023-01-18T18:34:33.565Z',
isAGift: false
}
users/z1V5scLfVPX7YD8fRCFRhhRAwVN2/discoutCodes/dGn5OxSEPohqpZbIyLmCjqVEiwz2
{
code: 'dGn5OxSEPohqpZbIyLmCjqVEiwz2',
expirationDate: '2023-01-18T18:34:33.565Z',
isAGift: true
}
To avoid these issues in Firestore, you can create a script that verifies the correct documents and removes the duplicates. Another solution is to design your database in a way that avoids documents that can be transferred, for example, with a collection in the root path of your database.
discoutCodes/dGn5OxSEPohqpZbIyLmCjqVEiwz2
{
code: 'dGn5OxSEPohqpZbIyLmCjqVEiwz2',
expirationDate: '2023-01-18T18:34:33.565Z',
isAGift: true
createdBy: z1V5scLfVPX7YD8fRCFRhhRAwVN2
owner: Zk5olEuvZ1WokRrCqIN5gGaIu353
}
In this way, there is only one code available and transfer it to another user means just editing the owner field inside this document. Keep in mind that this is a specific problem related to the key-value internal structure of Firestore and may not occur in other databases.
Conclusions
In conclusion, it is important to consider scalability and performance efficiency in the early stages when designing a project, especially with Firebase; but without over-engineering. This can save time and resources in the long run. Additionally, it is essential to have a backup plan for Firestore, as restoring a backup can sometimes result in inconsistencies. To mitigate these issues, one can either use a third-party service or create their own solution based on Firebase Functions. It is also crucial to design the database in such a way as to avoid transferable subcollections that could lead to document duplications.
Bibliography
Backup Fire | 2022 | BackupFire Official Site
Firebase Documentation | 2022 | Emulator Suite
Firebase Documentation | 2022 | Security Rules Basics
Firebase Documentation | 2022 | Security Rules Getting Started
Google Developers | 2022 | Task Functions
Google Developers | 2022 | Manage Functions
Google Cloud | 2002 | Delete a Firestore collection
Top comments (0)