Removing duplicates from arrays and arrays of objects

Published June 22, 2021

In this post, we'll look at quick and easy ways to remove duplicates in JavaScript from:

  • arrays
  • arrays of objects

Removing duplicates from arrays

Using Set

const brands = ['Wilson', 'Babolat', 'Wilson'];

console.log([...new Set(brands)]);
// ["Wilson", "Babolat"] 

Using Filter

const brands = ['Wilson', 'Babolat', 'Wilson'];

const filtered = brands.filter((item, index) => brands.indexOf(item) === index);

console.log(filtered);
// ["Wilson", "Babolat"] 

The filter method will create a new array of elements that meet the boolean condition that we provide to it. If the condition for an element returns true, it will be included in the filtered array. If the condition for an element returns false, it will not be in the filtered array.

The indexOf method finds the first index position of the item. If the array contains duplicates for an item, the first index position returned by indexOf will not match the current (duplicate) item's index position.

We can also get back the duplicates from the array if we need them by switching the condition on indexOf to an inequality one rather than an equality one.

const brands = ['Wilson', 'Babolat', 'Wilson'];

const filtered = brands.filter((item, index) => brands.indexOf(item) !== index);

console.log(filtered);
// ["Wilson"] 

Using Reduce

The reduce function might be one of the hardest to understand in JavaScript.

reduce applies a reducer function to the elments of the array. The reduce function's first two parameters are a function and an initial value. The reducer function is called on each element of the array with the accumulator as the first parameter. The accumulator starts with the initial value. The return value of each reducer function call becomes the new accumulator.

const brands = ['Wilson', 'Babolat', 'Wilson'];

const result = brands.reduce((accumulator, current) => {
  if (accumulator.includes(current)) {
    return accumulator;
  } 
  return [...accumulator, current];
}, []);

console.log(result);
// ["Wilson", "Babolat"] 

The initial value for reduce() is an empty array []. This is the initial value of the accumulator. If the accumulator does not contain the current item, then the current item gets returned along with any items in the accumulator as a new array. If the accumulator already contains the current item, then only the accumulator is returned for the next iteration of the reduce function.

Here is a look at the values of the accumulator during the reduce() operation on the brands array.

[LOG]: [] 
[LOG]: ["Wilson"] 
[LOG]: ["Wilson", "Babolat"] 

Removing duplicates from arrays of objects

Removing duplicate objects from an array of objects is a bit more complex.

Set and filter

We can use a Set and the array filter method to remove duplicate objects from an array of objects.

const brands = [
  { name: 'Wilson' },
  { name: 'Babolat' },  
  { name: 'Wilson' },
];

// a Set to track seen brands
const seen = new Set();

const filtered = brands.filter(brand => {
  // check if the current brand is a duplicate
  const isDuplicate: boolean = seen.has(brand.name);
  // add the current brand to the Set
  seen.add(brand.name);
  // filter() returns the brand when isDuplicate is false
  return !isDuplicate;
});

console.log(filtered);
// [{ "name": "Wilson" }, { "name": "Babolat" }] 

Map

We can also use the JavaScript Map object to remove duplicates from an array of objects. Map holds key-value pairs and remembers the original order that keys were inserted. Map is similar to an Object, but the difference is that Map allows keys of any type, not just strings.

const myMap = new Map();

myMap.set(1, 'foo');
myMap.set(1, 'bar');

console.log(map1.get(1)); 
// bar

As we can see from the example above, keys must be unique. The bar value replaces foo because Map can only hold one pair with the key 1.

Now, let's use Map to help us remove duplicates from an array of objects.

const brands = [
  { name: 'Wilson' },
  { name: 'Babolat' },  
  { name: 'Wilson' },
];

// brand name is the key for the Map
const map = new Map(brands.map(brand => [brand.name, brand]));
// put unique brand objects in an array
const uniques = [....map.values()];

console.log(uniques);
// [{ "name": "Wilson" }, { "name": "Babolat" }]