DEV Community

Cover image for 3 ways to remove duplicates in an Array in Javascript
Lenin Felix
Lenin Felix

Posted on

3 ways to remove duplicates in an Array in Javascript

promo

Let's check, many times (or few) arises the need to remove duplicate elements given in an array, I don't know... it can be because you have to print a list from the super, remove a student that duplicated his record in a form, infinity of things, so let's see some ways to do this:

1) Use Set

Using Set(), an instance of unique values will be created, implicitly using this instance will delete the duplicates.

So we can make use of this instance and from there we will have to convert that instance into a new array, and that would be it:

let chars = ['A', 'B', 'A', 'C', 'B'];
let uniqueChars = [...new Set(chars)];

console.log(uniqueChars);
Enter fullscreen mode Exit fullscreen mode

Output:

['A', 'B', 'C']
Enter fullscreen mode Exit fullscreen mode

2) Using the indexOf() and filter() methods

The indexOf() method returns the index of the first occurrence of the element in the array:

let chars = ['A', 'B', 'A', 'C', 'B'];
chars.indexOf('B');
Enter fullscreen mode Exit fullscreen mode

Output:

1
Enter fullscreen mode Exit fullscreen mode

The duplicate element is the element whose index is different from its indexOf() value:

let chars = ['A', 'B', 'A', 'C', 'B'];

chars.forEach((element, index) => {
    console.log(`${element} - ${index} - ${chars.indexOf(element)}`);
});
Enter fullscreen mode Exit fullscreen mode

Output:

A - 0 - 0
B - 1 - 1
A - 2 - 0
C - 3 - 3
B - 4 - 1
Enter fullscreen mode Exit fullscreen mode

To eliminate duplicates, the filter() method is used to include only the elements whose indexes match their indexOf values, since we know that the filer method returns a new array based on the operations performed on it:

let chars = ['A', 'B', 'A', 'C', 'B'];

let uniqueChars = chars.filter((element, index) => {
    return chars.indexOf(element) === index;
});

console.log(uniqueChars);
Enter fullscreen mode Exit fullscreen mode

Output:

['A', 'B', 'C']
Enter fullscreen mode Exit fullscreen mode

And if by chance we need the duplicates, we can modify our function a little, just by changing our rule:

let chars = ['A', 'B', 'A', 'C', 'B'];

let dupChars = chars.filter((element, index) => {
    return chars.indexOf(element) !== index;
});

console.log(dupChars);
Enter fullscreen mode Exit fullscreen mode

Output:

['A', 'B']
Enter fullscreen mode Exit fullscreen mode

3) Using the includes() and forEach() methods

The include() function returns true if an element is in an array or false if it is not.

The following example iterates over the elements of an array and adds to a new array only the elements that are not already there:

let chars = ['A', 'B', 'A', 'C', 'B'];

let uniqueChars = [];
chars.forEach((element) => {
    if (!uniqueChars.includes(element)) {
        uniqueChars.push(element);
    }
});

console.log(uniqueChars);
Enter fullscreen mode Exit fullscreen mode

Output:

['A', 'B', 'C']  
Enter fullscreen mode Exit fullscreen mode

Basically, we have options to solve this type of problem, so don't get stuck anymore and you can use whichever one appeals to you.


If you liked the content you can follow me on my social networks as @soyleninjs

ko-fi

promo

Discussion (12)

Collapse
eaallen profile image
Elijah Allen

The last two are problematic because you are essentially calling a for loop in a for loop which heavily increases how long the algorithms are going to take.

Using a set to remove duplicates is a great to solve this problem.

Collapse
sash20m profile image
Alex Matei

The first one is sexier

Collapse
jasimalrawie profile image
JasimAlrawie

Or
let chars = ['A', 'B', 'A', 'C', 'B'];

let uniqueChars = [];
chars.forEach((e) => {
if (!(e in chars)) {
uniqueChars.push(e);
}
});

console.log(uniqueChars);

Collapse
vedranbasic profile image
Vedran-Basic

Imagine having array of objects with n props having only to remove duplicate props with m same properties where m < n and use first of the two or more duplicates of the same unique constraint rule.

That's where science begins. Would be much more interested in hearing different solutions to this topic.

Collapse
phibya profile image
Phi Byă • Edited

Simply use reduce and Map:

[{a: 1, b: 2, c: 3}, {a: 2, b: 3, c: 4}, {a: 1, b: 2, c: 5}].reduce((p, c) => p.set([c.a, c.b].join('|'), c), new Map()).values()

Edit: For selecting the first value, use Map.has before setting the value.

Collapse
monello profile image
Morné Louw

Then write your own article an stop trying to sound smart on someone else's post

Collapse
tutrinh profile image
Tu Trinh

Thanks for the article. This is useful.

Collapse
phibya profile image
Phi Byă

For big array of objects, use reduce and Map:

[{a: 1, b: 2}, {a: 2, b: 3}, {a: 1, b: 2}].reduce((p, c) => p.set(c.a, c), new Map()).values()

Collapse
antonvryckij profile image
Anton Vryckij

Thank you for the article!

Collapse
devman profile image
DevMan • Edited

Complexities are:

  1. O(N)

2 and 3 are O(N^2)

So 1 should always be used imho.

Collapse
lucianodouglasm2 profile image
Luciano Douglas Machado Chagas

I usually do it using Define, it seems to be more performative. Could you show the performance of each of these modalities?

Collapse
jimjam88 profile image
James Morgan

Don't forget reduce!

chars.reduce((acc, char) => acc.includes(char) ? acc : [...acc, char], []);