DEV Community

Cover image for How to remove duplicates from JavaScript array
Kristijan Pajtasev
Kristijan Pajtasev

Posted on

How to remove duplicates from JavaScript array

Recently, I posted a small snippet on how to remove duplicates from an array using a filter function. That started a thread on all the different ways to achieve this. Therefore, in this post, I am covering different ways to solve this problem.
JavaScript logo

Alt Text

1. Creating new array and for loop

For loop and the new array would be the oldest possible solution. A solution you would use before array functions. Today you would do it this way.

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const newNumbers = [];

for(let i = 0; i < numbers.length; i++) {
  if(newNumbers.indexOf(numbers[i]) < 0)
    newNumbers.push(numbers[i])
}

console.log(newNumbers);
Enter fullscreen mode Exit fullscreen mode

2. Filter function

JavaScript arrays contain many different built-in functions, and a filter is one of them. This function takes one parameter, the filter function, which then has three parameters. The first one is the element you are testing, the second is its index, and the last one is the original array.

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const newNumbers =
  numbers.filter(
    (element, index, array) =>
      array.indexOf(element) === index
    )

console.log(newNumbers);
Enter fullscreen mode Exit fullscreen mode

3. Using the Set object

Using a set is one of the more modern solutions. Sets are part of the ES6 version of JavaScript, and they are objects that do not contain duplicates. You can create a set object using the original array and then convert it back to the array.

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const newNumbers = Array.from(new Set(numbers));

console.log(newNumbers);
Enter fullscreen mode Exit fullscreen mode

These are the three most common ways to remove duplicates from an array. But programming is creative, and I am sure there are many more. If you know of any, please write it in the comments.


For more, you can follow me on Twitter, LinkedIn, GitHub, or Instagram.

Top comments (11)

Collapse
 
chrismilson profile image
Chris Milson

You could sort the array first and then all duplicates will be next to each other:

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const unique = [...numbers].sort().filter((v, i, arr) => i === 0 || arr[i - 1] !== v)

console.log(unique) // [1, 2, 3]
Enter fullscreen mode Exit fullscreen mode
Collapse
 
ibnsamy96 profile image
Mahmoud Ibn Samy

why did you use an instance of the numbers array, not the number array itself? " [...numbers].sort() not numbers.sort() "

Collapse
 
hi_iam_chris profile image
Kristijan Pajtasev

That is a good question, sort is a destructive function. This means running sort on numbers array will change original numbers array.

Thread Thread
 
ibnsamy96 profile image
Mahmoud Ibn Samy

wow I forgot that

Collapse
 
ibnsamy96 profile image
Mahmoud Ibn Samy

Great ways but is it efficient to use the sets method? Especially in the big arrays.

Collapse
 
andrewbridge profile image
Andrew Bridge

I'd imagine if you were using a Set from the start, and adding values via the .add method on the Set object, you wouldn't need to store or iterate through the all the duplicates. That won't help if you needed to retain all the dupe data for some other uses though.

Collapse
 
ibnsamy96 profile image
Mahmoud Ibn Samy

I agree that in some use cases you can't use Set. Actually, in some cases, you can't even use it from the beginning, As an example, if you want to have a collection of unique objects, Set won't help.

Collapse
 
hi_iam_chris profile image
Kristijan Pajtasev

There is a topic on the StackOverflow showing the set is faster. I do personally like filter more, but in most cases, you won't see the difference, and you can go for what looks best in your code.

Collapse
 
ibnsamy96 profile image
Mahmoud Ibn Samy

That's good. Thanks.

Collapse
 
dansvel profile image
dan

and,, you can create a funtion for it,,

const unique = arr => [...new Set(arr)];

console.log(unique(numbers))
Enter fullscreen mode Exit fullscreen mode
Collapse
 
hi_iam_chris profile image
Kristijan Pajtasev

Ah yes, spread operator, love that one :smilery: