Were am I? A Streaming Service Experience

As mentioned in the description this is yet another component in building a simple streaming service and its components, not having to buy everything on the market.

I have since before written about creating a simple recommendation engine

One of the basics that you would expect of all streaming services out there, though it isn't implemented everywhere just yet, would be a continue watching service to be able to continue at the position where you left watching the last time. Here we'll implement the simplest of such solution for anyone to implement.

What do we want to to?

A Continue Watching service can be very complex in showing reasons (new episode, left in the middle of an episode, new season etc) but we will start with an implementation built for the simplest needs - to keep watching where you left intra episode or movie.

For that to exist we would want the following implementations

• Being able to post a position for a user on a specific asset.
• Get a position for a user on a specific asset.
• Get a list of the user's current saved positions, to show a Continue Watching carousel. Preferable in order.

Simplicity

Though you can of course implement this in the most complex ways of having data structure and metadata objects in infinity, stored in complex databases or graphs.

My ambition is rather to get both the architecture, as well as the application speed, as thin and light as possible.

I therefore choose to implement the storage in a simple key value store, in this case Redis.

Simplest implementation

When thinking about a continue watching service, it's essentially a bookmark in an asset for the specific user visiting the site, trying to keep on where he or she left. So which data do we need?

• A user watches an asset, until a specific position.

So for this to be stored in a key value storage, what are the unique part of this? What should be the key? In my optinion the best solution is to create a "unique" identifier of the userId of the user consuming the video and the assetId for the asset watching, i.e. userId:assetId, storing the position as value.

So first iteration is to use the SET and GET methods in redis for simply setting the value and later fetching the value for that specific asset only.

const util = require('util');
const redisClient = redis.createClient(6379, "127.0.0.1");
// Using util.promisify from the native node library to avoid callback hell
const set = util.promisify(redisClient.set);
const get = util.promisify(redisClient.get);

// This is the unique identifier which we create out of user and asset
const KEY_ASSET = (userId, assetId) => `${userId}:${assetId}`;

const store = async (userId, assetId, position) => {
  if (!userId || !assetId || !position) return false;

  const success = await set(
    KEY_ASSET(userId, assetId),
    position
  );

  return success;
};

const fetch = async (userId, assetId) => {
  if (!userId || !assetId) return false;

  const position = await get(KEY_ASSET(userId, assetId));

  return position;
};

Next up - User experience

Though in most services you would want a list of which assets you have currently ongoing - a "continue watching carousel" - to not have to search and find the specific asset before knowing that you have a progress in it.

Propably the most scalable and performance efficient way of doing this would be to generate Redis lists or sets on input, rather than on data request. Though for this simple solution we choose to find all listings for the user and fetching them all.

For this we will use the Redis KEY command, where you can get all matching keys given a pattern which you send in - in this case this would be userId:*, fetching all keys for one single user. This does only provide us with a list of keys though, so for each key we will have to do the basic GET command to get their values.

const util = require('util');
const redisClient = redis.createClient(6379, "127.0.0.1");
const set = util.promisify(redisClient.set);
const get = util.promisify(redisClient.get);
// adding the keys function which we will use
const keys = util.promisify(redisClient.keys);

const KEY_ASSET = (userId, assetId) => `${userId}:${assetId}`;
// This is our wildcard pattern
const KEY_USER = userId => `*${userId}:*`;

const list = async userId => {
  if (!userId) return false;
  const userKeys = await keys(KEY_USER(userId));
  // If no matches we return an empty array
  if (!userKeys) return [];

  // when we fetched as many values as we have keys, we'll be happy
  const expectedLength = userKeys.length;
  let keepGoing = true;
  const continueWatchingList = [];
  while (keepGoing) {
    const key = userKeys.shift();
    const val = await get(key);
    // we generate an object for each, to be able to list out for the end user
    const item = { assetId: key.split(":")[1], position: val };
    continueWatchingList.push(item);
    if (continueWatchingList.length >= expectedLength) {
      keepGoing = false;
      return continueWatchingList;
    }
  }
};

But what about order?

Now we have progress saved, ability to fetch it by asset and even a list of such objects to show in a carousel. Though this list is rather unordered, not showing the most relevant one up front. What if we should order it by the newest saved progress at first, as well as expiration when you haven't continued in a set amount of time?

Let's try to implement it using Redis EXPIRE command to set a expiration date, and then the TTL command to get the remaining time until expiration. If we update this on each progress report, the one with the highest time remaingin should be the newest input? Right?

// we add the expire command to be able to set expiration
const expire = util.promisify(redisClient.expire);

// we set it to one year, so each stored value will be deleted after one year if no progress is being made
const ONE_YEAR = 1 * 60 * 60 * 24 * 365;

const store = async (userId, assetId, position) => {
  if (!userId || !assetId || !position) return false;

  const setSuccess = await set(
    KEY_ASSET(userId, assetId),
    position
  );

  // when stored, we set expiration
  const expirationSuccess = await expire(
    KEY_ASSET(userId, assetId),
    ONE_YEAR
  );

  return success && expirationSuccess;
};

// we add the ttl method to read out remaingin expiration
const ttl = util.promisify(redisClient.ttl);

const list = async userId => {
  if (!userId) return false;
  const userKeys = await keys(KEY_USER(userId));
  if (!userKeys) return [];

  const expectedLength = userKeys.length;
  let keepGoing = true;
  const continueWatchingList = [];
  while (keepGoing) {
    const key = userKeys.shift();
    const val = await get(key);
    // we'll fetch the expiration for each key
    const expiration = await ttl(key);
    // storing it in each object
    const item = { assetId: key.split(":")[1], position: val, expiration };
    continueWatchingList.push(item);
    if (continueWatchingList.length >= expectedLength) {
      keepGoing = false;
      // before returning the list, we sort it by remaingin time
      continueWatchingList.sort((a, b) => b.expiration - a.expiration);
      return continueWatchingList;
    }
  }
};

Conclusion

So now we have the ability to set, get and list assets in order for an end user to be exposed by his or hers progress in a streaming service.

For the entire project implemented as an API to clone, fork or be inspired by - please have a look over at our Github .
We love contributions.
https://github.com/Eyevinn/continue-watching-api