Mike Talbot ⭐

Posted on Aug 16, 2021 • Updated on Sep 7, 2021

14 functions I made to dump lodash and reduce my bundle size...

#javascript #webdev #showdev #tutorial

Lodash and underscore changed the way I write Javascript forever, but today there might be better options for the most common functions.

I recently went through our main app looking to reduce the bundle size and quickly identified that we were still getting most of lodash imported despite our best efforts to do specific functional imports.

We moved to lodash-es and that helped a bit, but I was still looking at a couple of utility functions taking up around 30% of the bundle.

The problem is that, as a node module, many of the choices about polyfilling old functionality have already been made by the library, so depending on your target browser you might have a lot of code you don't need.

I identified 14 core functions we used from lodash and went about re-writing them in modern Javascript so the bundling process can decide what it needs to provide in terms of polyfills depending on the target. The reductions in import size were significant.

Lodash-es after tree shaking, before my functions:

My code: 4.1kb (uncompressed/unminified, though it will need polyfills on older browsers)

The core functions

Here's what I did about that list of functions:

Matched functionality

filter
forEach (arrays and objects)
groupBy
keyBy
map (arrays and objects)
merge
omit
sortBy
uniq
uniqBy

Implemented "enough"

pick
get (doesn't support array syntax)
set (doesn't support array syntax)
debounce (with maxWait, flush, cancel)

The functions

So here are those functions, what they do and how I implemented them:

pick(function(item)=>value | propertyName)

We will start with pick because it's pretty useful for everything else. pick will return a function to extract a property from an object - my implementation will convert a string to this, but leave other values alone.

You can use pick yourself like this:

const array = [{ name: "mike", a: 1 }, { name: "bob", a: 2 }]

console.log(array.map(pick('name')) //=> ["mike", "bob"]

Implementation

import {get} from './get'
export function pick(fn) {
  return typeof fn === "string" ? (v) => get(v,fn) : fn
}

filter(array, function(item)=>boolean | string)

We used filter with a name property quite a lot, so filter is basically just pick and the existing filter function:

const array = [{ name: "mike", a: 1 }, { name: "bob", a: 2 }, { a: 4 }]

console.log(filter(array, 'name')) //=> [{ name: "mike", a: 1 }, { name: "bob", a: 2 }]

Implementation

import { pick } from "./pick"

export function filter(target, fn) {
  return target.filter(pick(fn))
}

forEach(array|object, function(value, key))

In lodash we can use either an object or an array for a forEach and so we needed an implementation that can do that. The callback gets the parameters value and key. It works like this:

const data = { a: 1, b: 2, d: "hello" }
forEach(data, (value, key)=>console.log(`${key}=${value}`) 
      //=> a=1
      //=> b=2
      //=> d=hello

Implementation

import { pick } from "./pick"

export function applyArrayFn(target, fnName, fn) {
  fn = pick(fn)
  if (Array.isArray(target)) return target[fnName](fn)
  if (target && typeof target === "object")
    return Object.entries(target)[fnName](([key, value], index) =>
      fn(value, key, target, index)
    )
  throw new Error(`Cannot iterate ${typeof target}`)
}

export function forEach(target, fn) {
  return applyArrayFn(target, "forEach", fn)
}

get(object, propertyPath, defaultValue)

get allows you to read properties from an object and if any intermediaries or the final value are not found it will return the default value

const data = { a: { b: {d: 1 } } }
get(data, "a.b.d") //=> 1
get(data, "a.c.d", "hmmm") //=> hmmm

Implementation

export function get(object, path, defaultValue) {
  const parts = path.split(".")
  for (let part of parts) {
    if(!object) return defaultValue
    object = object[part]
  }
  return object ?? defaultValue
}

groupBy(array, function(item)=>key | propertyName)

Create an object keyed by the result of a function (or picked property name) where every value is an array of the items which had the same key.

const array = [{ name: "mike", type: "user" }, { name: "bob", type: "user" }, { name: "beth", type: "admin"} ]

console.log(groupBy(array, 'type'))
    /*=>
       {
          admin: [{name: "beth", type: "admin" }],
          user: [{name: "mike", type: "user" }, {name: "bob", type: "user"}]
       }
    */

Implementation

import { pick } from "./pick"

export function groupBy(target, fn) {
  fn = pick(fn)
  return target
    .map((value) => ({ value, key: fn(value) }))
    .reduce((c, a) => {
      c[a.key] = c[a.key] || []
      c[a.key].push(a.value)
      return c
    }, {})
}

keyBy(array, function(item)=>key | propertyName)

Similar to groupBy but the result is the last item which matched a key - usually this is given something where the key will be unique (like an id) to create a lookup

const array = [{ id: "a7", name: "mike", type: "user" }, { id: "z1", name: "bob", type: "user" }, { id: "a3", name: "beth", type: "admin"} ]

console.log(keyBy(array, 'id'))
    /*=>
       {
          "a3": {name: "beth", type: "admin", id: "a3" },
          "a7": {name: "mike", type: "user", id: "a7" },
          "z1": {name: "bob", type: "user", id: "z1"}
       }
    */

Implementation

import { pick } from "./pick"

export function keyBy(target, fn) {
  fn = pick(fn)
  return target
    .map((value) => ({ value, key: fn(value) }))
    .reduce((c, a) => {
      c[a.key] = a.value
      return c
    }, {})
}

map(array|object, function(value, key)=>value | propertyName)

Maps both objects and arrays (like forEach)

const records = {
          "a3": {name: "beth", type: "admin" },
          "a7": {name: "mike", type: "user" },
          "z1": {name: "bob", type: "user"}
       }
console.log(map(records, 'name')) /=> ["beth", "mike", "bob"]

Implementation

import { pick } from "./pick"

export function applyArrayFn(target, fnName, fn) {
  fn = pick(fn)
  if (Array.isArray(target)) return target[fnName](fn)
  if (target && typeof target === "object")
    return Object.entries(target)[fnName](([key, value], index) =>
      fn(value, key, target, index)
    )
  throw new Error(`Cannot iterate ${typeof target}`)
}

export function forEach(target, fn) {
  return applyArrayFn(target, "map", fn)
}

merge(target, ...sources)

Works like Object.assign but recurses deep into the underlying structure to update the deeper objects rather than replacing them.

const record = { id: "2", name: "Beth", value: 3, ar: ["test", { a: 3, d: { e: 4 } }] }
console.log(merge(record, { ar: [{ b: 1 }, { c: 3, d: { f: 5 } }]))

   /*=>
    {
      id: "2",
      name: "Beth",
      value: 3,
      ar: [{ b: 1 }, { c: 3, d: { f: 5, e: 4 } }]
    }
   */

Implementation

export function merge(target, ...sources) {
  for (let source of sources) {
    mergeValue(target, source)
  }

  return target

  function innerMerge(target, source) {
    for (let [key, value] of Object.entries(source)) {
      target[key] = mergeValue(target[key], value)
    }
  }

  function mergeValue(targetValue, value) {
    if (Array.isArray(value)) {
      if (!Array.isArray(targetValue)) {
        return [...value]
      } else {
        for (let i = 0, l = value.length; i < l; i++) {
          targetValue[i] = mergeValue(targetValue[i], value[i])
        }
        return targetValue
      }
    } else if (typeof value === "object") {
      if (targetValue && typeof targetValue === "object") {
        innerMerge(targetValue, value)
        return targetValue
      } else {
        return value ? { ...value } : value
      }
    } else {
      return value ?? targetValue ?? undefined
    }
  }
}

omit(object, arrayOfProps)

Returns an object with the props listed removed

const record = { a: 1, b: 2, c: 3}
console.log(omit(record, ['b', 'c'])) //=> {a: 1}

Implementation

export function omit(target, props) {
  return Object.fromEntries(
    Object.entries(target).filter(([key]) => !props.includes(key))
  )
}

set(object, propertyPath, value)

Sets a value on an object, creating empty objects {} along the way if necessary.

const record = { a: 1, d: { e: 1 } }
set(record, "a.d.e", 2) //=> { a: 1, d: { e: 2 } }
set(record, "a.b.c", 4) //=> { a: 1, b: { c: 4 }, d: { e: 2 } }

Implementation

export function set(object, path, value) {
  const parts = path.split(".")
  for (let i = 0, l = parts.length - 1; i < l; i++) {
    const part = parts[i]
    object = object[part] = object[part] || {}
  }
  object[parts[parts.length - 1]] = value
}

sortBy(array, function(item)=>value | propertyName)

Sort an array by a sub element.

const array = [{ id: "a7", name: "mike", type: "user" }, { id: "z1", name: "bob", type: "user" }, { id: "a3", name: "beth", type: "admin"} ]
console.log(sortBy(array, 'name'))
     /*=>
      [
        { id: "a3", name: "beth", type: "admin"} 
        { id: "z1", name: "bob", type: "user" }, 
        { id: "a7", name: "mike", type: "user" }, 
      ]
     */

Implementation

import { pick } from "./pick"

export function sortBy(array, fn) {
  fn = pick(fn)
  return array.sort((a, b) => {
    const va = fn(a)
    const vb = fn(b)
    if (va < vb) return -1
    if (va > vb) return 1
    return 0
  })
}

uniq(array)

Make a unique array from an existing array

const array = ['a', 'b', 'c', 'b', 'b', 'a']
console.log(uniq(array)) //=> ['a', 'b', 'c']

Implementation

export function uniq(target) {
  return Array.from(new Set(target))
}

uniqBy(array, function(item)=>value | propertyName)

Make a uniq array using a property of objects in the array.

const array = [{a: 1, b: 2}, {a: 4, b: 2}, {a: 5, b: 3}]
console.log(uniqBy(array, 'b')) //=> [{a: 1, b: 2}, {a: 5, b: 3}]

Implementation

import { pick } from "./pick"

export function uniqBy(target, fn) {
  fn = pick(fn)
  const dedupe = new Set()
  return target.filter((v) => {
    const k = fn(v)
    if (dedupe.has(k)) return false
    dedupe.add(k)
    return true
  })
}

Partially Implemented `debounce`

lodash debounce is very powerful - too powerful for me and too big. I just need a function I can debounce, a maximum time to wait and the ability to flush any pending calls or cancel them. (So what is missing is trailing and leading edges etc, + other options I don't use).

const debounced = debounce(()=>save(), 1000, {maxWait: 10000})
...
debounced() // Call the debounced function after 1s (max 10s)
debounced.flush() // call any pending 
debounced.cancel() // cancel any pending calls

Implementation

export function debounce(fn, wait = 0, { maxWait = Infinity } = {}) {
  let timer = 0
  let startTime = 0
  let running = false
  let pendingParams
  let result = function (...params) {
    pendingParams = params
    if (running && Date.now() - startTime > maxWait) {
      execute()
    } else {
      if (!running) {
        startTime = Date.now()
      }
      running = true
    }

    clearTimeout(timer)
    timer = setTimeout(execute, Math.min(maxWait - startTime, wait))

    function execute() {
      running = false
      fn(...params)
    }
  }
  result.flush = function () {
    if (running) {
      running = false
      clearTimeout(timer)
      fn(...pendingParams)
    }
  }
  result.cancel = function () {
    running = false
    clearTimeout(timer)
  }
  return result
}

Conclusion

It's possible to drop the need for lodash if you only use these functions. In our app we do use other lodash functions, but they are all behind lazy imports (so template for instance) - our app is way faster to load as a result.

Feel free to use any of the code in your own projects.

Top comments (19)

artydev • Mar 24

Great as always :-)

Muhammad Hasnain • Aug 16 '21

I have personally never used lodash because there has never been a need for me to do so. I just want to point out that lodash is tree shakable and will not contribute to bundle size a lot expect for the functions you've imported of course.

Mike Talbot ⭐ • Aug 16 '21 • Edited

See my point about the decisions it makes around polyfilling. For these 14 functions with lodash-es - which is better designed for tree shaking - I still saved a significant amount of bundle size.

Jeremy Mill • Aug 17 '21

your implementation of merge is vulnerable to prototype pollution. You should go and read lodash's implementation before re-implementing it.

portswigger.net/daily-swig/prototy...

Mike Talbot ⭐ • Aug 17 '21

Thanks :) Wilco

Roberto Maurizzi • Aug 16 '21

I recently added lodash to a project I'm working on.Docs are inconclusive, some recommendations I found about importing lodash functions from submodules do not work (either they were using the now deprecated standalone submodules, or something changed somewhere).
In the end I added a babel module designed to strip away the unused pieces of lodash and that gave me a very decent subset of it in my bundle.
Again, my impression? The Javascript ecosystem is a royal mess that can continue only because it has no competition.

Mike Talbot ⭐ • Aug 16 '21

Here here to that... :)

Quozzo • Aug 16 '21

I'm not sure if this is a smaller implementation of your merge function, except mine returns a new value instead of mutating the first value passed in.

gist.github.com/Quozzo/3715ad741cf...

If you use Object.assign with an empty object/array as the first argument then it should be a non-issue either way.

Daniel • Aug 16 '21

So... What is exactly the difference between using functions from Lodash and using your own function? Is your code lighter than Lodash just because you wrote it?

Mike Talbot ⭐ • Aug 16 '21

Well see above, partly down to the fact that I wrote tighter implementations with only the functionality I needed but a lot down to the face I used the same bundler to add polyfills I already need for the ones there. The different was around 22kb reduction from the tree shaken version. TBH debounce is probably the killer in there. But there are lots of "BaseClone" etc. Code that would work anywhere but I certainly don't use.

As you can see, this isn't all of lodash-es... so tree shaking worked as far as possible:

Daniel • Aug 16 '21 • Edited

Then, to be fair, you didn't write functions that would replace Lodash. You wrote functions that offer a subset of the functionality offered by Lodash, and that's why they are lighter.

But something that it's true is that it would be really nice if they released a version 5 of Lodash that would be compatible only with evergreen browsers. I am sure that they could get rid of a lot of code.

Mike Talbot ⭐ • Aug 16 '21

Yeah I did document above which functions didn't work the same. Debounce, get and set basically. I don't have the need for the advanced versions of those.

Abhishek Raj • Aug 16 '21

Better to tree shake it to decrease its size

Mike Talbot ⭐ • Aug 16 '21 • Edited

Didn 't work see above. The libraries were still much bigger than my implementations.

Amin • Aug 16 '21

Your get function won't return the value if it is 0 (or any other falsy value). You'll get the defaultValue every time (which may be something else like "none").