4.1 Getting Professional with Firebase V9 - 'System Hygiene' - Error-handling and Transactions

Last reviewed : June 2022

Introduction

Because these posts are intended primarily for readers who are still struggling to get a foothold in the IT world, the code samples I've provided so far assume that things will generally work as intended. To do otherwise would just have added confusion!

Even now, I'm going to hang back from overloading you with detail. I just intend to paint in the broad outline of the issues I want to cover and will supply references for you to follow up at your leisure. But there are things that I think it's really important that you are aware of.

In the real world, things don't always work as intended. Your code will almost certainly contain syntax or logic errors and will be dumped brutally by your browser the first time you run it. But even when you've straightened it out you'll see the same result when your users get at it and enter "stupid" data - ie data that causes your code to fail again because you didn't foresee that this might happen. Your fault again, I'm afraid and you need to put the appropriate validation checks in

These are predictable issues that can be reliably avoided by careful coding and testing

But another class of error - things that we refer to as contingencies - cannot be avoided. Examples would be the failure of a network connection or problems with the remote database host. All you can do in this situation is write code that recognises that a problem has occurred and then takes appropriate action. Sometimes the best that you can do is simply display a message along the lines of "Sorry - system currently unavailable". But this will always be better than leaving your users looking at a blank, frozen screen! This is where you can score points by demonstrating your professional concern for your users. As you'll see in a moment, this comes sharply into focus when you realise that in database applications, contingency errors that occur at inconvenient moments may lead to data loss.

So this post is all about how to respond to these various challenges: how to write sound code in the first place, how to keep your users informed when the unpredictable happens, and how to keep your database in a healthy, consistent state.

Fixing the foreseeable - Good Coding/Testing practices

Writing reliable, maintainable computer code is part art and part engineering discipline. There are many different views on what constitutes "good code". For Javascript, I refer you again to Eloquent Javascript. Elements of good coding practice will include layout, naming conventions and program structure. Beyond this, you'll really only learn what works and what doesn't from a mixture of practical experience and by looking at the work of others.

"Testing" is of course the procedure you follow to confirm your code's reliability. Your IDE and the browser (through its system debugging tool) can be relied upon to tell you firmly when your syntax is faulty or when you have put your program into a state where a statement cannot be run. A particularly helpful feature of the VSCode IDE is that it will alert you to errors while you are still writing code (ie before you try to run it). It will in fact make suggestions and help you create correct code in the first place - a huge time-saver. Beyond this, however, you need to create "scenarios" where, starting from known initial conditions, you follow planned routes through your application and check that the results match expectations. Of course, you realise that you will have to repeat this whenever you make changes to your system! You might like to look at the "test runner" systems used by mainstream professional developers to systematise the procedure. "Jest" is an example you might find interesting. As previously stated - serious, professional IT system development is hard work!

Fixing the unforeseeable - the Javascript "catch" facility

If you have concerns about the vulnerability of a block of code, the Javascript system enables you to wrap this in a try{.. vulnerable code block...} catch{.. do something about it ...} structure. This means that if anything in the code block "throws" an error, control is redirected to the code in the catch { } block.

What does "throw an error" mean? It means that a piece of code has recognised that something is going wrong and, in the simplest case, has executed a throw 'Explanation'; statement. Here, 'Explanation' is a string that explains the problem. The throw statement makes 'Explanation' available to catch(error) as error.message .

Those messages you'll have seen in the browser console when you've created faulty code had appeared because the browser has "thrown" them. If you put your code into try blocks (not that I'm suggesting this would always be a good idea), you could catch these errors and "handle" them.

So, for example, whereas webapp code like the following:

let x = 1 / a;

where a is a variable you've not defined, will be halted by your browser when you run it. While this will leave you looking at a blank screen, you'll know that you can find what's gone wrong by looking at the console in the browser system tools. Here you'll find a ReferenceError: a is not defined message. But your users won't know about this of course - all they will see will be a dead webapp.

On the other hand:

try {
    let x = 1 / a;
} catch (error) {
    alert("Oops Code has thrown the following error: " + error)
}

will produce an alert message that is clearly visible to the webapp user.

Given that the "thrown" error may be buried deep within a complex nested hierarchy of application code and SDK functions, you may also wonder how Javascript manages to deliver this arrangement. I refer you again to Eloquent Javascript (chapter 8).

For a Firebase webapp, you're most likely to want to "catch" errors thrown by Firestore or Cloud Storage functions. You have two options: while a whole stack of code can be wrapped inside the try/catch arrangement I've just described, if for some reason you want to monitor individual functions, Javascript offers you a .catch() method that you can attach to a Firestore function call. Here's an example from a Google code lab:

SpaceRace.prototype.deleteShip = function(id) {
    const collection = firebase.firestore().collection('ships');
    return collection.doc(id).delete().catch((error) => {
            console.error('Error removing document: ', error);
        });
};

I prefer this arrangement to try/catch blocks because I think it makes my code a bit more readable.

If you're wondering how .catch works, the answer is that Javascript provides this "method" automatically for any function that returns a Promise - and most Firestore functions return Promises. For background on promises and the await keyword, have a look at my earlier post: The "await" keyword

Transactions

As indicated above, unpredictable hardware problems can result in the corruption of a production database unless webapp software is sufficiently alert to the possibility and is equipped to handle it.

Here's an example. You'll remember that the "Shopping List" application introduced in "coding a simple webapp" allowed users to create lists of "purchase items". Imagine that "management" had decided that it would be a good idea to keep a running count of the number of times a purchase item appeared on users' Shopping Lists. Accordingly, a "purchaseMI" collection containing "running total" documents has been added to the database. Now every time a purchaseItem is added or removed from a shopping list, the webapp must adjust the corresponding entry in purchaseMI.

The problem with this is that an inconvenient failure halfway through such a procedure will leave the database in a corrupt state. With care, it would be possible to "catch" such a failure and attempt to deal with it, but in a more complex situation, this would not be a straightforward task.

Things look even bleaker when you consider what might happen when your database is handling "simultaneous" requests from multiple users.

Suppose that two users add a userPurchase for, say, "rolls" to their lists at the same time. Each of them thus accesses the purchaseMI collection for the running total for "rolls" - and each thus finds themselves holding identical values for the current total for that item - let's say it stands at "10". And yes - I'm sure you've seen the problem that now arises. After they've each applied their update to the running total, whereas this should read "12", it actually reads just "11". The database is now corrupt - the current value of the running total field for a "rolls" in purchaseMI doesn't square with the value you'd get if you searched for "rolls" in userSHoppingLists.

We need some help from Google here as these "concurrency" concerns are too complex for the webapp to address. What we need is some way of defining a "transaction" - a sequence of database commands which either all succeed, or are all discarded. With a transaction thus declared, the webapp just has to deal with the overall outcome - it doesn't have to concern itself with the internal minutiae of the process.

Google's response is to provide a transaction object with methods that can be used to launch CRUD commands in a way that enables them to communicate with each other. This transaction object is created by a runTransaction function that, in turn, launches a function with the transaction object as its argument. This wraps the sequence of CRUD commands and thus defines the transaction. Firestore is then able to take steps to ensure, without further effort on our part that, while the transaction may fail, if the database was consistent before a transaction starts, it remains consistent after it finishes.

To give you a feel for what this looks like, here's sample code for an updated version of the "Shopping Lists" webapp's Delete function.

 async function deleteShoppingListDocument(id, userPurchase) {

    // id =>  a userShoppingLists document
    // userPurchase =>  the userPurchase field for this document

    await runTransaction(db, async (transaction) => {

        const purchaseMIDocRef = doc(db, 'purchaseMI', userPurchase);
        const purchaseMIDoc = await transaction.get(purchaseMIDocRef);

        const shoppingListsDocRef = doc(db, 'userShoppingLists', id);
        transaction.delete(shoppingListsDocRef);

        const newUserPurchaseTotal = purchaseMIDoc.data().userPurchaseTotal - 1;
        transaction.update(purchaseMIDocRef, { userPurchaseTotal: newUserPurchaseTotal });

    }).catch((error) => {alert("Oops - Transaction failed : " + error)});
 }

By way of explanation:

1. I've had to add runTransaction to the import for firebase/firestore/lite. Additional preparations have been the creation of a purchaseMI collection with documents keyed on userPurchase and containing a userPurchaseTotal field. I've also added a rule permitting free read/write access to purchaseMI.

2. The deleteDoc function I previously used to delete a shoppingLists document is now replaced by a transaction.delete function. All the CRUD functions I might need to use are similarly subtly changed - see firebase.firestore.Transaction for Google's documentation on the Transaction object. Note that getDocs, the query form of getDoc isn't supported by the transaction object.

   • transaction.get replaces getDoc
   • transaction.set replaces setDoc
   • transaction.update replaces updateDoc
   • transaction.delete replaces deleteDoc

3. The order in which database commands are executed in the example may seem unnatural. This is because, in a Firestore transaction, all "read" operations must be completed before any updates are launched.

4. Whereas transaction.get still returns a promise and therefore needs to be called with a preceding "await" keyword, none of the other transaction methods do.

5. If Firestore detects that another user has modified the data that it has just read, it backs out anything that it may have done and reruns the transaction. A transaction may thus run more than once and so you need to take care over any statements that create "side-effects". For example, a counter field update statement could cause havoc.

6. Transactions can write to a maximum of 500 documents and there is a limit of approx 20MB on the volume of storage that may be affected by a transaction.

7. The Transaction concept used here - defined as "a set of read and write operations on one or more documents" - is paralleled by a Batched writes facility - "a set of write operations on one or more documents". Batched Writes are a lot simpler than Transactions and are preferred, where appropriate.

8. Cloud functions can also use transactions and, in this case, some of the restrictions described above are eased - for example, the Cloud Function transaction SDK supports the query form of get

As you can see, there's a lot to say about this. But now that I've introduced the topic and supplied an example, I think it would probably be best if I just stopped and left you to read Google's Transactions and batched writes documentation. You might want to run some test code too! There's an excellent video tucked into the Google docs referenced above that I also strongly recommend that you watch.

In conclusion, transactions aren't for the faint-hearted, but they are what will make your webapp a truly professional product. Good luck!

Other posts in this series

If you've found this post interesting and would like to find out more about Firebase you might find it worthwhile having a look at the Index to this series.