Marina Mosti

Posted on Feb 4, 2019 • Edited on Sep 23, 2019

Removing duplicates in an Array of Objects in JS with Sets

#javascript #tip #array

The other day at work I was faced with what I think is a rather common problem when dealing with data coming from an API.

I was getting from my async http call an array of objects (addresses in this case), but for unimportant reasons this array was returning objects that could be duplicate (so two identical ids).

My first instinct was to make a new array, and loop through the addresses' array and do a .findIndex on each for my "copy". If the "copy" didn't have the address, then I would push it. In the end the code was readable, but I wanted a simpler more straightforward way.

There are many ways to solve this particular problem!

Now that we got that out of the way 😋, I wanted to show this solution that uses the power of JavaScript Set.

const addresses = [...]; // Some array I got from async call

const uniqueAddresses = Array.from(new Set(addresses.map(a => a.id)))
 .map(id => {
   return addresses.find(a => a.id === id)
 })

Let's dissect this mess:

Array.from(new Set()) I'm going to make a new set, and I want to turn it back into an array with Array.from so that I can later re-map it.
new Set(addresses.map(a => a.id)) Set will only allow unique values in it, so i'm going to pass it the ids of each object. If the loop tries to add the same value again, it'll get ignored for free.
.map(id => [...]) With the array of ids I got on step 1, I run a map function on it and return the actual address from the original address array

That's it! Simple, clear, and I had fun learning about/using Set 😎

Huge shout out to my bae Natalia Tepluhina who endured and helped me come up with crazy ideas for this, and her awesome solution to do it with Array.reduce showcased below ❤️👩‍💻

const arr = [
  { id: 1, name: "test1" },
  { id: 2, name: "test2" },
  { id: 2, name: "test3" },
  { id: 3, name: "test4" },
  { id: 4, name: "test5" },
  { id: 5, name: "test6" },
  { id: 5, name: "test7" },
  { id: 6, name: "test8" }
];

const filteredArr = arr.reduce((acc, current) => {
  const x = acc.find(item => item.id === current.id);
  if (!x) {
    return acc.concat([current]);
  } else {
    return acc;
  }
}, []);

Latest comments (41)

Jacob Sides • Jun 24 '25 • Edited

I feel like this could be even better. This is the solution I came up with.

const uniqueItemsInObj = allItems
    .reduce((accu, crnt) => ({...accu, [crnt.id]: crnt}), {});
return Object.values(uniqueItemsInObj)

I suppose it might be a problem if order mattered, but usually that's done using a different property/procedure.

feldmanovitch • Jan 21 '23

Man, I only created an account here just for saying thanks. So... thanks! :D

I came here because Chat GPT suggested to me a filter-based solution that did not work as filter does consider two objects unequal due to different spots in memory, eventhough they contain the same values.

You saved me a lot of headaches.

JuanDouek • Oct 28 '22

Other solutions...

Leave last appearance:

const arr = [
    { id: 1, name: "test1" },
    { id: 2, name: "test2" },
    { id: 2, name: "test3" },
    { id: 2, name: "test4" },
    { id: 3, name: "test5" },
    { id: 4, name: "test5" }
];

const by_id = {};

for (item of arr) by_id[item.id] = item;

const uniques = Object.values(by_id);

Leave first appearance:

const arr = [
    { id: 1, name: "test1" },
    { id: 2, name: "test2" },
    { id: 2, name: "test3" },
    { id: 2, name: "test4" },
    { id: 3, name: "test5" },
    { id: 4, name: "test5" }
];

const by_id = {};

for (item of arr)
    if(!by_id[item.id]) by_id[item.id] = item;

const uniques = Object.values(by_id);

Remove duplicated names also:

const arr = [
    { id: 1, name: "test1" },
    { id: 2, name: "test2" },
    { id: 2, name: "test3" },
    { id: 2, name: "test4" },
    { id: 3, name: "test5" },
    { id: 4, name: "test5" }
];

const by_id = {};
const by_name = {};

for (item of arr)
    if(!by_id[item.id] && !by_name[item.name])
    {
        by_id[item.id] = item;
        by_name[item.name] = 1;
    }

const uniques = Object.values(by_id);

Aissa0347 • Jul 30 '22 • Edited

Thanks for chairing, I found this also helpful

const arr = [
  { id: 1, name: "test1" },
  { id: 2, name: "test2" },
  { id: 2, name: "test3" },
  { id: 3, name: "test4" },
  { id: 4, name: "test5" },
  { id: 5, name: "test6" },
  { id: 5, name: "test7" },
  { id: 6, name: "test8" }
];
filteredArr= arr.filter((currentUser, index) => {
      return (
        arr.findIndex((user) => user.id === currentUser.id) === index
      );
    });

Fahim Uddin • Jun 9 '22

Thanks for the solution.

Laura Pereyra • Aug 17 '21

I have two list of objects and I need to remove all duplicates, something like this:

INPUTS
var array1 = [(1, 'banana', 'yellow'), (1, 'apple', 'red'), (1, 'orange', 'orange')];
var array2 = [(1, 'banana', 'yellow'), (1, 'apple', 'red'), (2, 'grapes', 'purple')];

OUTPUT
array1 = [(1, 'orange', 'orange')]

I tried using filter but it doesn't work for objects :(