Save Data to Remote Storage For Later Retrieval

The entire reason I'm writing these articles is because I'm attempting to get to a place where I can :

1. Store a user's data
2. Retrieve the user's data
3. Make it possible for the user to save/retrieve data no matter where they are running my application.

Non-functional Requirements

Those three requirements lead us to one very important non-functional requirement:

Data security: Insure that only the owner of the data can access and manage ( CRUD operations ) the data.

Have you ever heard of non-functional requirements ? They are the parts of the system which indirectly effect the system.

Functional requirements are directly related to how your system provides services a user needs/wants from your App. They are things like :

• save data
• retrieve data
• view report
• delete data

Non-functional are the things that are necessary to insure the overall system works as expected (accessibility, availability, security, adaptability, etc.).
They are a part of your over all architecture which build a foundation for the functionality of your app.

Half-baked System

They are the (almost) intangible things that must be baked into the system. If they are not included then the system really isn't complete. It's only half-baked.

Examples of Non-functional Failure

1. Unavailable Web Site

For example, if you create a web site that is only available 50% of the time then no one can depend upon your web site and it won't be used. Availability is a non-functional requirement.

2. Exposed Data

In our case, if we have a system where the developer can store the user's data remotely but the data can be read by anyone, then the system is half-baked. If a user's data can be accessed by anyone then no user is going to use the system. Security is a a non-functional requirement.

During Design & Prototyping: Ignore Non-functional

However, to get to a solution without being overwhelmed by creating a perfect system, we often have to put non-functional concerns on the back burner. We often have to first just see if our idea can work. That's what we are doing here.

First Iteration of Web API

To keep this first iteration of the Storage Web API simply functional we will provide a way to :

1. Store data remotely
2. Retrieve remote data

Encrypting the User's Data

For this first revision we will not concentrate on securing the user data. We will store the user's data in clear text.

AES256 Encryption

However, in the next article in this series I will show you how to encrypt the data using AES256 via JavaScript. Once the data is encrypted on the client side we will post the data to our Web API.

Keeping the Article Shorter (Too Late)

In an attempt to keep the article shorter I will just walk you through how you can try out the Web API (instead of explaining every detail of how it has been created). Note: If you want to know more about how it was created, leave a comment on this article and I'll get back to you.

How To Use The Web API

Here's how easy it is to use the web api.

Where Is It Hosted?

https://newlibre.com/LibreStore

That's my website and it is HTTPS enabled so anything you post is protected. It is just a basic template right now but there is a Web API behind it that you can use to store data.

Since the web site is protected by HTTPS, your MainToken.Key won't be compromised when it is passed along in the URL.

Two Endpoints That Help You Store App Data

There are only two main endpoints that you can use at this point.

SaveData

To save your data, you just :

1. Create a (String) Key that is at least 10 bytes long & <= 128 bytes long. (Store it in a place where you won't lose it, because without it you're not getting your data back.)
2. Send your data (right now I just have it set up to accept HTTP Get commands) using the following URL: https://newlibre.com/LibreStore/Data/savedata?key=<your-key-here>&data=<your-data-here>; The easiest way to try it is to use the JavaScript fetch API ( Using Fetch - Web APIs | MDN[^] ).

Try It In Your Browser Dev Console

Here's how you can send data using your Browser's developer console.

1. Open your browser's dev console (F12 in most web browsers).
2. Paste the following code and alter to include your Key & Data.

fetch("https://newlibre.com/LibreStore/Data/SaveData?key=FirstOneForTest&data=First post to data for test.")
  .then(response => response.json())
  .then(data => console.log(data));

I already posted using the Key which means the MainToken record has already been created.

If you post using a previously used Key it just means the data will be bound to that Key. However, if you post data with a new (unique) Key then a new MainToken record will be created and then your data will be tied to that Key. To retrieve the data you'll need your original (unique) Key again.

What Happens When Calling SaveData?

When you send the data using the fetch above and your own key then here is what will happen.

1. A new unique entry (row) will be created in the MainToken table.
2. The data will be inserted in the Bucket table's Data field and will be tied to the MainToken.ID that was generated when the Key was created.
3. At this point your Bucket Data is stored.

What Is Returned?

When it completes the API returns:

JSON including two fields:

1. success : (true or false)
2. bucketId: id of the row just inserted into the bucket table It looks something like the following:

If you save that into a an object then you will have a object with those two properties (success & bucketId) and you'll be able to re-use the values.

GetData

Now, when you want to retrieve your data, you send to the following URL: https://newlibre.com/LibreStore/Data/GetData?key=<your-key>&bucketId=<your-bucket-id>

Again, you can use the Fetch API to get the data back.

Use the bucketId that was returned to you when you saved the data.

fetch("https://newlibre.com/LibreStore/Data/GetData?key=FirstOneForTest&bucketid=2")
  .then(response => response.json())
  .then(data => console.log(data));

It'll look something like the following in the browser console:

There are Two Major Rules For Using LibreStore

1. Never let your MainToken.Key out into the wild - I already broke this one with the example above. It means that anyone can attempt to use my Key to retrieve data now. Of course, when I save my real data I will create a long random Key to store my data.
2. Never Store Unencrypted Data - I also broke this rule so I could show you how it works, but in the future I will encrypt all my data using AES256. If you encrypt your data properly then you don't really have to worry about Rule #1 because no attacker should be able to decrypt your data.

Can You Use LibreStore Right Now?

Yes, you can post to it and retrieve data, but I haven't showed you how to easily encrypt your data yet so for now you (obviously) shouldn't post anything that really matters.

Next Article : Encrypting Data Via AES256

Since I'm just trying to get the Web API and this article out there and in an effort to keep this article shorter I will write up how to encrypt your data using the AES256 encryption algorithm via JavaScript.

Sneak Peak: AES256 Encryption

However, since I've already done the research and figured out how to encrypt & decrypt data using AES256 via JavaScript you can take a look at the code running at my Codepen.io and try it out if you like: https://codepen.io/raddevus/pen/VwMXawY

To try it:

1. type in your password
2. type in your data
3. click the [encrypt] button You will see a string of Base64 encoded bytes appear. That is the encrypted bytes converted into Base64 data. It is not the cleartext bytes. It is the cipher bytes converted to Base64.

If you click the decrypt button, the data will be:

1. Base64 decoded
2. Decrypted using the password
3. Clear text will be added to a div at the bottom so you can see it -- it will match the original text. If you change the password before decrypting then it will not be able to decrypt the bytes and you will see nothing.

See You Next Time

Next time we will update our ImageCat app to store it's data using the LibreStore Web API so it can be retrieved whenever the app is run.

I can think of a few challenges we are going to encounter but those will just lead to use learning more. 👍🏽