Data Validation In Low-Code

From Regular Expressions To OnValidate Events: Performing Data Validation In Five

Forms: the necessary evil of application design. Whether you are a developer building a form, or an end-user filling in a form: I’m sure everyone has had their fair share of form frustrations.

With easy-to-use tools such as Google Forms, everyone can design a form for surveys or questionnaires. But not everyone can design a good form, let alone make it part of a web application. I would argue that poorly designed forms are the reason that in any given data science project, one-quarter of the time is spent on data cleansing: incorrect date formats, invalid email addresses, or unclear descriptions. They all are the nightmare of data scientists. And they all contribute to poorly captured data.

So what can we do to make this process more efficient? How can we design & launch good forms?

Data Storage

Let’s start by looking at the most fundamental layer of data capture first: data storage.

Basic form builders, such as Google Forms store data inside a web-hosted spreadsheet. This is the ideal solution for small research projects, small-scale surveys, or questionnaires in your local football club (Should Johnny or Dave be the new captain? Do we prefer pink or blue jerseys?).

But spreadsheets are not a solution that can support businesses, let alone large-scale data capture.

The best place to store data is open-source, relational databases, with the two most popular ones being MySQL and PostgreSQL, according to StackOverflow’s Annual Developer Survey. Unlike spreadsheets, relational databases store data in a consistent, highly-efficient, safe, and secure manner. They are also scalable and portable, meaning they can support millions of read/write operations, and your data can be moved, copied, or transferred easily from one database, storage, or IT environment to another.

Just like a spreadsheet, a database needs to be hosted (or stored) somewhere. My recommendation is to go with one of the three big cloud providers, AWS, Azure, or GCP.

This setup (an open-source, relational database hosted by one of the big three cloud providers) gives you a tried-and-tested, scalable setup, with a large support community. In case you’re wondering, just how “tried-and-tested and scalable” this setup is, here’s a list of companies using MySQL on StackShare. The list includes Uber, Netflix, and Twitter.

And if now you are thinking: “We’re just a small business and nowhere near the scale of Netflix”, don’t underestimate how quickly data is growing over time, especially if you are in a transactional business. Or sometimes simply due to unforeseen circumstances, such as the Covid-pandemic, when Excel reached breaking point at health care providers.

How Five Stores Data

Every application built with Five comes with a MySQL database hosted. Developers can create and manage their database right inside Five. When applications are deployed to production, the application and its database are moved from local storage on your machine onto AWS.

Data Validation

The best way to ensure clean data is to create easy-to-use forms that validate data right at the time of end-user input. And there are multiple different ways to achieve this.

First, data & display types: data and display types define how data is stored and displayed. Data types aren’t visible to the end-user. They simply tell the database how to treat data (is something a string, an integer, a password?). Display types, on the other hand, are visible to the end-user, and can range from something as simple as a form field that accepts text to a date picker or a five-star rating input. The display type defines what the end-user can submit to the database.

A second way to validate data is regular expressions (RegExs). Regular expressions let developers define an accepted input. A good example of this is email addresses. To ensure a user enters a valid email address, a regular expression can be used. The regular expression that checks for a valid email address looks like this:

^(([^<>()[\]\\.,;:\s@”]+(\.[^<>()[\]\\.,;:\s@”]+)*)|(“.+”))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$

Looks intimidating? I agree. But most standard, regular expressions can be found or created online with helpful tools, such as this RegEx builder.

Last, functions and OnValidate events can be used to validate data. These go beyond the simple validations described above. For example, a function could check if there’s enough stock to fulfill an order: if yes, the order is processed. If no, an error occurs. Here’s a simple example: I want to order ten Baked Funfetti Doughnuts. But the store only has three in stock. A function that validates my order on submission will inform me that my order needs to be adjusted to three doughnuts (sadly).

Data Types, Display Types, and RegEx Inside Five

Here are a couple of built-in display types that Five has available out of the box:

In addition, developers can create custom display types. For example, if your end-user is asked to submit satisfaction with a service on a scale from 0 to 100, this can be created with a display type that has a minimum and maximum value of 0 and 100 respectively. If an end-user inserts any different input, a pop-up with an error message will appear. The same can be achieved by defining the accepted length of the input. Say you’re asking for a postcode and only accept four-digit postcodes. Then a display type with a required length of four characters can be created and applied to a field.

Developers can also create a custom display type using regular expressions.

Permissions

One last advantage of relational databases is user access and permissions. An acronym that is often used in this context is CRUD: create, read, update and delete. These are the four basic database operations. CRUD permissions can be granted on a table level, meaning one user might only be able to insert new records into a table that holds clients’ information, whereas another can only update a table storing clients’ orders.

Assigning Permissions Inside Five

Five lets developers assign CRUD permission on the table level in a point-and-click editor. Each user will have their own login credentials to the application, and based on the role that they have been assigned, they will have unique CRUD permissions. Beyond assigning CRUD permissions, each user can be assigned unique data record permissions. Let’s say you’re running a real estate business with four agents. Each agent is in charge of one area of town: North, South, East, or West. Even though all data is stored in one table, each user can only be given access to their territory’s data.

UI/UX Design

A good form is appealing to end-users. Clear descriptions, labels, and Call-to-Action (CTAs) all help end-users navigate through the form. Making smart use of conditional logic inside your form, in-app help buttons, or section labels and sub-headings can aid your end-user to navigate through the form. Especially for longer form, it is helpful to break it down into pieces so as to make the end-user isn’t overwhelmed from the get-go.

Designing Forms Inside Five

Show-If, Required-If, Read-Only: all of these are part of Five. Through their use, forms that guide users through each input step-by-step, or that show different questions based on different responses, can be created very quickly. Show-If conditions, for example, are written in JavaScript syntax. Here’ s a simple example of a form field that will only be shown to the end-user if two conditions are met:

(form.field !== ” && form.field !== null)

Developers can also add help buttons to form fields to provide additional in-app help to end-users.

To wrap it all up, check out our blog post to see a form that uses all of these elements: https://five.co/data-validation-in-low-code/

Learn More

Ready to learn more? Sign up for a free download through our website https://five.co and start building your first web application to capture and store data. For more help and useful tips on how to build applications with Five, visit our user community.