DEV Community

Pouja
Pouja

Posted on

Making lodash function get type safe

#typescript #generics #safety #lodash

With the introduction of Typescript 4.1 we can use template literal types for writing our generics. This opens up a whole lot of new applications and better typing, also means they are more complex to read.

In this post I will be going over why and how I implemented the following result:

expectType<RegExp>(get(obj,'deeplvl1[1].deeplvl2.deeplvl3[88].deeplvl4.value'));
Enter fullscreen mode Exit fullscreen mode

Table of Content

  1. Why I needed
  2. The End Result
  3. How It Works
  4. Example
  5. Future Work

Why

Context

In my current project we are using react-final-form to create our forms. In one of our pages we have a huge object (or as we can call it a god object) that is allowed to be edited in several visible sections. This is bad design by nature, but unfortunately this technical debt is too large to currently solve. We wrote several hooks to make our live easier. One of them is:

export function useFormValue<T>(field: string): T {
  const fieldInput = useField(field, {
    subscription: {
      value: true
    }
  });
  return fieldInput.input.value;
}
Enter fullscreen mode Exit fullscreen mode

Which can be used as

export default function DeepNestComponent() {
  const price = useFormValue<number>('customers[3].contract.price');
  return (<div>{price}</div>);
}
Enter fullscreen mode Exit fullscreen mode

This allows us to create deep nested components that can subscribe to form changes of a specific form input and display the value.

The problem

What if the structure of the large object that is being edited changes? We then have to go over all the components and adjust the path. If you have full unit test coverage, a (snapshot) test will probably fail. But I rather have my compiler give me an error than hoping that my unit coverage is sufficient.

The End Result

I ended up writing the following generic

export type ResolveType<ObjectType, Path extends string, OrElse> =
    Path extends keyof ObjectType ? ObjectType[Path] :
    Path extends `${infer LeftSide}.${infer RightSide}` ? LeftSide extends keyof ObjectType ? ResolveType<ObjectType[LeftSide], RightSide, OrElse> : 
    Path extends `${infer LeftSide}[${number}].${infer RightSide}` ? LeftSide extends keyof ObjectType ? ObjectType[LeftSide] extends Array<infer U>? ResolveType<U,RightSide, OrElse> : OrElse : OrElse : OrElse :
    Path extends `${infer LeftSide}[${number}]` ? LeftSide extends keyof ObjectType ? ObjectType[LeftSide] extends Array<infer U> ? U : OrElse : OrElse : OrElse;
Enter fullscreen mode Exit fullscreen mode

Which is uploaded to

GitHub logo Pouja / typescript-deep-path-safe

Collections of generics to make deep nested string paths type safe

And it can be used as:


function get<ObjecType extends object, Path extends string, OrElse extends unknown>(obj:ObjecType, path: Path, orElse?: OrElse): ResolveType<ObjecType, Path, OrElse> {
  return _get(obj, path, orElse);
}

const nested = get(obj, 'my.deep[3].nested', 'orElse');
Enter fullscreen mode Exit fullscreen mode

How it works

My starting point was the implementation of template literal types:

Template literal types and mapped type 'as' clauses #40336

ahejlsberg avatar
ahejlsberg posted on

This PR implements two new features:

  • Template literal types, which are a form of string literals with embedded generic placeholders that can be substituted with actual string literals through type instantiation, and
  • Mapped type as clauses, which provide the ability to transform property names in mapped types.

Template literal types

Template literal types are the type space equivalent of template literal expressions. Similar to template literal expressions, template literal types are enclosed in backtick delimiters and can contain placeholders of the form ${T}, where T is a type that is assignable to string, number, boolean, or bigint. Template literal types provide the ability to concatenate literal strings, convert literals of non-string primitive types to their string representation, and change the capitalization or casing of string literals. Furthermore, through type inference, template literal types provide a simple form of string pattern matching and decomposition.

Template literal types are resolved as follows:

  • Union types in placeholders are distributed over the template literal type. For example `[${A|B|C}]` resolves to `[${A}]` | `[${B}]` | `[${C}]`. Union types in multiple placeholders resolve to the cross product. For example `[${A|B},${C|D}]` resolves to `[${A},${C}]` | `[${A},${D}]` | `[${B},${C}]` | `[${B},${D}]`.
  • String, number, boolean, and bigint literal types in placeholders cause the placeholder to be replaced with the string representation of the literal type. For example `[${'abc'}]` resolves to `[abc]` and `[${42}]` resolves to `[42]`.
  • Any one of the types any, string, number, boolean, or bigint in a placeholder causes the template literal to resolve to type string.
  • The type never type in a placeholder causes the template literal to resolve to never.

Some examples:

type EventName<T extends string> = `${T}Changed`;
type Concat<S1 extends string, S2 extends string> = `${S1}${S2}`;
type ToString<T extends string | number | boolean | bigint> = `${T}`;
type T0 = EventName<'foo'>;  // 'fooChanged'
type T1 = EventName<'foo' | 'bar' | 'baz'>;  // 'fooChanged' | 'barChanged' | 'bazChanged'
type T2 = Concat<'Hello', 'World'>;  // 'HelloWorld'
type T3 = `${'top' | 'bottom'}-${'left' | 'right'}`;  // 'top-left' | 'top-right' | 'bottom-left' | 'bottom-right'
type T4 = ToString<'abc' | 42 | true | -1234n>;  // 'abc' | '42' | 'true' | '-1234'
Enter fullscreen mode Exit fullscreen mode

Beware that the cross product distribution of union types can quickly escalate into very large and costly types. Also note that union types are limited to less than 100,000 constituents, and the following will cause an error:

type Digit = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9;
type Zip = `${Digit}${Digit}${Digit}${Digit}${Digit}`;  // Error
Enter fullscreen mode Exit fullscreen mode

A template literal placeholder may optionally specify an uppercase, lowercase, capitalize, or uncapitalize modifier before the type. This modifier changes the casing of the entire replacement string or the first character of the replacement string. For example:

EDIT: Based on feedback, the casing modifiers have been replaced by intrinsic string types in #40580.

type GetterName<T extends string> = `get${Capitalize<T>}`;
type Cases<T extends string> = `${Uppercase<T>} ${Lowercase<T>} ${Capitalize<T>} ${Uncapitalize<T>}`;
type T10 = GetterName<'foo'>;  // 'getFoo'
type T11 = Cases<'bar'>;  // 'BAR bar Bar bar'
type T12 = Cases<'BAR'>;  // 'BAR bar BAR bAR'
Enter fullscreen mode Exit fullscreen mode

Template literal types are all assignable to and subtypes of string. Furthermore, a template literal type `${T}` is assignable to and a subtype of a template literal type `${C}`, where C is a string literal type constraint of T. For example:

function test<T extends 'foo' | 'bar'>(name: `get${Capitalize<T>}`) {
    let s1: string = name;
    let s2: 'getFoo' | 'getBar' = name;
}
Enter fullscreen mode Exit fullscreen mode

Type inference supports inferring from a string literal type to a template literal type. For inference to succeed the starting and ending literal character spans (if any) of the target must exactly match the starting and ending spans of the source. Inference proceeds by matching each placeholder to a substring in the source from left to right: A placeholder followed by a literal character span is matched by inferring zero or more characters from the source until the first occurrence of that literal character span in the source. A placeholder immediately followed by another placeholder is matched by inferring a single character from the source.

Some examples:

type MatchPair<S extends string> = S extends `[${infer A},${infer B}]` ? [A, B] : unknown;

type T20 = MatchPair<'[1,2]'>;  // ['1', '2']
type T21 = MatchPair<'[foo,bar]'>;  // ['foo', 'bar']
type T22 = MatchPair<' [1,2]'>;  // unknown
type T23 = MatchPair<'[123]'>;  // unknown
type T24 = MatchPair<'[1,2,3,4]'>;  // ['1', '2,3,4']

type FirstTwoAndRest<S extends string> = S extends `${infer A}${infer B}${infer R}` ? [`${A}${B}`, R] : unknown;

type T25 = FirstTwoAndRest<'abcde'>;  // ['ab', 'cde']
type T26 = FirstTwoAndRest<'ab'>;  // ['ab', '']
type T27 = FirstTwoAndRest<'a'>;  // unknown
Enter fullscreen mode Exit fullscreen mode

Template literal types can be combined with recursive conditional types to write Join and Split types that iterate over repeated patterns.

type Join<T extends unknown[], D extends string> =
    T extends [] ? '' :
    T extends [string | number | boolean | bigint] ? `${T[0]}` :
    T extends [string | number | boolean | bigint, ...infer U] ? `${T[0]}${D}${Join<U, D>}` :
    string;
type T30 = Join<[1, 2, 3, 4], '.'>;  // '1.2.3.4'
type T31 = Join<['foo', 'bar', 'baz'], '-'>;  // 'foo-bar-baz'
type T32 = Join<[], '.'>;  // ''
Enter fullscreen mode Exit fullscreen mode 
type Split<S extends string, D extends string> =
    string extends S ? string[] :
    S extends '' ? [] :
    S extends `${infer T}${D}${infer U}` ? [T, ...Split<U, D>] :
    [S];

type T40 = Split<'foo', '.'>;  // ['foo']
type T41 = Split<'foo.bar.baz', '.'>;  // ['foo', 'bar', 'baz']
type T42 = Split<'foo.bar', ''>;  // ['f', 'o', 'o', '.', 'b', 'a', 'r']
type T43 = Split<any, '.'>;  // string[]
Enter fullscreen mode Exit fullscreen mode

The recursive inference capabilities can for example be used to strongly type functions that access properties using "dotted paths", and pattern that is sometimes used in JavaScript frameworks.

type PropType<T, Path extends string> =
    string extends Path ? unknown :
    Path extends keyof T ? T[Path] :
    Path extends `${infer K}.${infer R}` ? K extends keyof T ? PropType<T[K], R> : unknown :
    unknown;

declare function getPropValue<T, P extends string>(obj: T, path: P): PropType<T, P>;
declare const s: string;

const obj = { a: { b: {c: 42, d: 'hello' }}};
getPropValue(obj, 'a');  // { b: {c: number, d: string } }
getPropValue(obj, 'a.b');  // {c: number, d: string }
getPropValue(obj, 'a.b.d');  // string
getPropValue(obj, 'a.b.x');  // unknown
getPropValue(obj, s);  // unknown
Enter fullscreen mode Exit fullscreen mode

Mapped type as clauses

With this PR, mapped types support an optional as clause through which a transformation of the generated property names can be specified:

{ [P in K as N]: X }
Enter fullscreen mode Exit fullscreen mode

where N must be a type that is assignable to string | number | symbol. Typically, N is a type that transforms P, such as a template literal type that uses P in a placeholder. For example:

type Getters<T> = { [P in keyof T & string as `get${Capitalize<P>}`]: () => T[P] };
type T50 = Getters<{ foo: string, bar: number }>;  // { getFoo: () => string, getBar: () => number }
Enter fullscreen mode Exit fullscreen mode

Above, a keyof T & string intersection is required because keyof T could contain symbol types that cannot be transformed using template literal types.

When the type specified in an as clause resolves to never, no property is generated for that key. Thus, an as clause can be used as a filter:

type Methods<T> = { [P in keyof T as T[P] extends Function ? P : never]: T[P] };
type T60 = Methods<{ foo(): number, bar: boolean }>;  // { foo(): number }
Enter fullscreen mode Exit fullscreen mode

When the type specified in an as clause resolves to a union of literal types, multiple properties with the same type are generated:

type DoubleProp<T> = { [P in keyof T & string as `${P}1` | `${P}2`]: T[P] }
type T70 = DoubleProp<{ a: string, b: number }>;  // { a1: string, a2: string, b1: number, b2: number }
Enter fullscreen mode Exit fullscreen mode

Fixes #12754.

Playground: https://www.typescriptlang.org/play?ts=4.1.0-pr-40336-88

View on GitHub

Where already a start was made which works with only objects without arrays.

But I want to go over step by step in how it works and how I extend it.

Single Nested Object

To make it easier to reason with generics, I will view them as functions. Example type ResolveType<ObjectType extends object, Path extends string, OrElse> is function called ResolveType which accepts 3 arguments. First argument should be of (sub)type object, second argument must be a string and the third argument is of type any.

Our function must return type of the single level nested path or if it does not exists then return the third argument. In pseudo code it will look like:

function ResolveType(obj:Object, path:String, orElse:any) {
 if(path exists in obj) return type of obj[path]
 else return orElse;
}
Enter fullscreen mode Exit fullscreen mode

To retrieve all the keys of a object we can use keyof operator. To check if type a is of type b we can use the extends operator. This covers our if condition. The if-else equivalent in generics are ternary statements. We then get the following result:

export type ResolveType1<ObjectType, Path extends string, OrElse> =
    Path extends keyof ObjectType ? ObjectType[Path] : OrElse;
Enter fullscreen mode Exit fullscreen mode

Unlimited Nested Objects

Now we must use the new template literal types of Typescript, introduced in 4.1. This allows us, as I call it, to do pattern matching. Furthermore because nested objects can be nested more than 10 levels deep, we have to either loop or recursively call our generic type resolver. In 4.1 they also introduced Recursive Conditional Types which allows us to recursively walk through all the nested levels of our object.

Our pseudo code will look like:

function ResolveType(obj:Object, path:String, orElse:any) {
 if(path exists in obj) return type of obj[path]
 else if(path can be splitted between '.') {
   if(leftside of '.' is a key of obj) {
     return ResolveType(obj[leftSide], rightSide, orElse);
   }
 }
 return orElse;
}
Enter fullscreen mode Exit fullscreen mode

First condition can be written as

Path extends `${string}.${string}`

But we don't want to just check if the path matches the pattern, but we also want to use the matched string. This can be achieved with the infer keyword, which is not documented in detail in the handbook of Typescript. Our condition becomes

Path extends `${infer LeftSide}.${infer RightSide}`

The last condition we have to check is the same we already covered for single level nested objects.

Typescript doesn't support a AND operator in generics, so we have to used nested ternary.

Our function becomes:

export type ResolveType<ObjectType, Path extends string, OrElse> =
    Path extends keyof ObjectType ? ObjectType[Path] :
    Path extends `${infer LeftSide}.${infer RightSide}` ? LeftSide extends keyof ObjectType ? ResolveType<ObjectType[LeftSide], RightSide, OrElse> : OrElse : OrElse;
Enter fullscreen mode Exit fullscreen mode

Arrays

The pattern of arrays are as followed: nested[3] which is the simple variant. Our pseudo code becomes:

function ResolveType(obj:Object, path:String, orElse:any) {
 if(path exists in obj) return type of obj[path]
 else if(path can be splitted between '.') {
   if(leftside of '.' is a key of obj) {
     return ResolveType(obj[leftSide], rightSide, orElse);
   }
 } else if(path matches with pattern 'string.[number]') {
   if(leftside of [number] is a key of obj) {
     if(obj[leftside] is type of array) {
       return type of array
     }
   }
 }
 return orElse;
}
Enter fullscreen mode Exit fullscreen mode

Array notation can be pattern matched with ${infer LeftSide}[${number}]. We are just matching if a number is being used, since we can not infer the size of an array are compile time. If you use an index outside of the range you will get a Index Out of Bounds Exception, which is the only downside of the constructed generic.

We can also use the infer key word to infer the type of array we are using: Array<infer U>. Typescript will then infer the type of the array.

We then get the following function:

export type ResolveType<ObjectType, Path extends string, OrElse> =
    Path extends keyof ObjectType ? ObjectType[Path] :
    Path extends `${infer LeftSide}.${infer RightSide}` ? LeftSide extends keyof ObjectType ? ResolveType<ObjectType[LeftSide], RightSide, OrElse> : OrElse :
    Path extends `${infer LeftSide}[${number}]` ? LeftSide extends keyof ObjectType ? ObjectType[LeftSide] extends Array<infer U> ? U : OrElse : OrElse : OrElse;
Enter fullscreen mode Exit fullscreen mode

Nested arrays

We can now match the following constructions:

expectType<number>(get(obj, 'nested.a'));
expectType<string>(get(obj, 'normal'));
expectType<number>(get(obj,'nested.a'));
expectType<boolean>(get(obj,'nested.b.c'));
expectType<number[]>(get(obj,'arr'));
expectType<number>(get(obj,'arr[13]'));
expectType<string[]>(get(obj,'anothernested.deep'));
Enter fullscreen mode Exit fullscreen mode

I am using the following library for testing my types:

GitHub logo SamVerschueren / tsd

Check TypeScript type definitions

But we can not match for example arr[13].length because we only check for the pattern string[number]. We need to match for string[number].string and infer the types. But arr[13].length will match with our second condition in our pseudo code:

if(path can be splitted between '.')
Enter fullscreen mode Exit fullscreen mode

So in our else statement of the second condition we have to check if it matches the pattern string[number].string. And then recursively call our function with the right side of the dot. Our pseudo code becomes:

function ResolveType(obj:Object, path:String, orElse:any) {
 if(path exists in obj) return type of obj[path]
 else if(path can be splitted between '.') {
   if(leftside of '.' is a key of obj) {
     return ResolveType(obj[leftSide], rightSide, orElse);
   } else if(path contains pattern 'string[number].string') {
     if(leftSide of [number] is a key of obj) {
       if(obj[leftSide] is an array) {
         return ResolveType(type of array, rightSide, orElse)
       }
     }
   }
 } else if(path matches with pattern 'string.[number]') {
   if(leftSide of [number] is a key of obj) {
     if(obj[leftSide] is type of array) {
       return type of array
     }
   }
 }
 return orElse;
}
Enter fullscreen mode Exit fullscreen mode

As you can see the added if-else condition is similar to last one, except we recursively walk through the right side of the dot.

Combining everything until now we get the final result:

export type ResolveType<ObjectType, Path extends string, OrElse> =
    Path extends keyof ObjectType ? ObjectType[Path] :
    Path extends `${infer LeftSide}.${infer RightSide}` ? LeftSide extends keyof ObjectType ? ResolveType<ObjectType[LeftSide], RightSide, OrElse> : 
    Path extends `${infer LeftSide}[${number}].${infer RightSide}` ? LeftSide extends keyof ObjectType ? ObjectType[LeftSide] extends Array<infer U>? ResolveType<U,RightSide, OrElse> : OrElse : OrElse : OrElse :
    Path extends `${infer LeftSide}[${number}]` ? LeftSide extends keyof ObjectType ? ObjectType[LeftSide] extends Array<infer U> ? U : OrElse : OrElse : OrElse;
Enter fullscreen mode Exit fullscreen mode

Example

Lets take the lodash get example and step by step walk through how the generic will be called and how it will resolve:

const barLength = get(obj,'foo.bar[3].length', -1);
Enter fullscreen mode Exit fullscreen mode

We will assume that obj is of type FooBar and the path is valid. The generic will then be called with ResolveType<FooBar, 'foo.bar[3].length', number)

  1. foo.bar[333].length matches the pattern string.string, Typescript will infer the following values leftSide=foo and rightSide=bar[333].length
  2. foo is a key of FooBar, lets assume that FooBar['foo'] is of type Foo.
  3. ResolveType will be called with ResolveType<Foo, 'bar[3].length', number>
  4. bar[3].length matches the pattern string.string, Typescript will infer the following values leftSide=bar[3] and rightSide=length
  5. bar[3] is not a key of Foo
  6. bar[3].length matches the pattern string[number].string, Typescript will infer the following values leftSide=bar and rightSide=length
  7. bar is a key of Foo
  8. Foo['bar'] is an array of type Bar
  9. ResolveType will be called with ResolveType<Bar, 'length', number>
  10. length is a key of bar since bar is a string
  11. ResolveType returns the type number

Future Work

Publishing

Currently you can copy the type from this article or from my repository:

GitHub logo Pouja / typescript-deep-path-safe

Collections of generics to make deep nested string paths type safe

Type safe paths

Collections of generics to make deep nested string paths type safe.

ResolveType

It accepts 3 arguments:

  1. The object type
  2. The string path
  3. The type to return if it fails to resolve

Usage

For example you can augment the lodash get:

import { get as _get } from 'lodash';
function get<ObjecType extends object, Path extends string, OrElse extends unknown>(obj:ObjecType, path: Path, orElse?: OrElse): ResolveType<ObjecType, Path, OrElse> {
  return _get(obj, path, orElse);
}
get(obj,'deeplvl1[1].deeplvl2.deeplvl3[88].deeplvl4.value');
Enter fullscreen mode Exit fullscreen mode

Or for react-final-form:

// file: form-value-hook.tsx
export function useFormValue<Path extends string>(field: Path): ResolveType<YourFormType, Path, unknown> {
  const fieldInput = useField(field, {
    subscription: {
…
Enter fullscreen mode Exit fullscreen mode
View on GitHub

I hope in the near future to publish it to NPM so you can install it (when I figure out how).

Autocomplete

I want to look into making generic type where is accepts variable list of function arguments to determine the type. This allows you to use auto completion of typescript itself.
The end result should look like:

get(obj, 'orElse', 'lvl1path', 3, 'lvl2path', '[your cursor]
Enter fullscreen mode Exit fullscreen mode

Where in the last argument you will get the autocomplete of Typescript for the type obj.lvl1path[3].lvl2path.

Top comments (2)

Collapse
 
wombbu profile image
Lauri Nevanperä •

This is incredible. Thank you. Final-form user here.

Collapse
 
vskrivoshein profile image
Vadim Krivoshein •

Excellent work!

Read next

railsstudent profile image

Text translation using Google Cloud Translation API in a NestJS application

Connie Leung -

sadanandgadwal profile image

Mastering TypeScript: A Comprehensive Guide

Sadanand gadwal -

flameface profile image

Everyday AI Tools

Kunal Kande Patil -

almaju profile image

Someone finally fixed Javascript

Almaju -