How to Remove Duplicate Objects from an Array in JavaScript: Best Methods Explained

Table of Contents#

1. Understanding the Problem: Why Objects Are Tricky
2. Method 1: Using Set with a Unique Key
3. Method 2: Using Array.reduce()
4. Method 3: Using Array.filter() with findIndex
5. Method 4: Using Lodash’s uniqBy
6. Method 5: Using Map for Key-Value Tracking
7. Method 6: Deep Comparison (No Unique Key)
8. Comparison of Methods: When to Use Which?
9. Conclusion
10. References

Understanding the Problem: Why Objects Are Tricky#

In JavaScript, primitives (e.g., 'a', 42) are compared by value, so duplicates in an array of primitives can be easily removed with a Set:

const primitiveArray = [1, 2, 2, 3, 3, 3];
const uniquePrimitives = [...new Set(primitiveArray)]; // [1, 2, 3]

Objects, however, are reference types. Two objects with identical properties are stored at different memory addresses, so === returns false:

const obj1 = { id: 1 };
const obj2 = { id: 1 };
console.log(obj1 === obj2); // false (different references)

Thus, removing duplicates requires comparing object content, not references. The solution depends on whether your objects have a unique identifier (e.g., id) or require deep comparison (checking all properties).

Method 1: Using Set with a Unique Key#

How It Works#

If your objects have a unique identifier (e.g., id, email), you can track duplicates by storing these keys in a Set. A Set automatically handles uniqueness, so you can filter the array to include only objects whose keys haven’t been seen before.

Code Example#

Suppose we have an array of user objects with a unique id:

const users = [
  { id: 1, name: "Alice", email: "[email protected]" },
  { id: 2, name: "Bob", email: "[email protected]" },
  { id: 1, name: "Alice", email: "[email protected]" }, // Duplicate (same id)
  { id: 3, name: "Charlie", email: "[email protected]" },
  { id: 2, name: "Bob", email: "[email protected]" }, // Duplicate (same id)
];
 
// Step 1: Track seen unique keys (e.g., `id`) in a Set
const seenIds = new Set();
 
// Step 2: Filter the array to keep only objects with unseen keys
const uniqueUsers = users.filter(user => {
  if (!seenIds.has(user.id)) {
    seenIds.add(user.id);
    return true; // Keep the object
  }
  return false; // Skip duplicates
});
 
console.log(uniqueUsers); 
// Output: [{ id: 1, ... }, { id: 2, ... }, { id: 3, ... }] (no duplicates)

Pros#

• Efficient: Set operations (has, add) are O(1), so the overall time complexity is O(n).
• Simple: Easy to read and implement.

Cons#

• Requires a Unique Key: Only works if objects have a distinct identifier (e.g., id).
• Key Type Sensitivity: If keys are of mixed types (e.g., 1 vs "1"), they’ll be treated as unique.

Method 2: Using Array.reduce()#

How It Works#

The reduce method iterates over the array and builds a new array of unique objects. We use an accumulator (an object or Set) to track seen keys, adding objects to the result only if their key is new.

Code Example#

Using the same users array with id as the unique key:

const uniqueUsers = users.reduce((acc, user) => {
  // Check if the user's id is already in the accumulator's `seenIds` Set
  const seenIds = acc.seenIds;
  if (!seenIds.has(user.id)) {
    seenIds.add(user.id);
    acc.result.push(user); // Add to result if id is new
  }
  return acc;
}, { seenIds: new Set(), result: [] }).result; // Initial accumulator
 
console.log(uniqueUsers); 
// Output: [{ id: 1, ... }, { id: 2, ... }, { id: 3, ... }]

Pros#

• Flexible: Can track multiple keys (e.g., id + email) by combining them into a composite key.
• No External Dependencies: Uses native JavaScript methods.

Cons#

• More Verbose: Slightly longer than the Set + filter approach.

Method 3: Using Array.filter() with findIndex#

How It Works#

Array.filter() creates a new array with objects that pass a test. For each object, we check if its first occurrence in the array (via findIndex) is the same as its current index. If findIndex returns the current index, the object is unique; otherwise, it’s a duplicate.

Code Example#

Using id as the unique key:

const uniqueUsers = users.filter((user, index) => {
  // Find the first index where the `id` matches the current user's id
  const firstOccurrenceIndex = users.findIndex(u => u.id === user.id);
  // Keep the object only if it's the first occurrence
  return firstOccurrenceIndex === index;
});
 
console.log(uniqueUsers); 
// Output: [{ id: 1, ... }, { id: 2, ... }, { id: 3, ... }]

Pros#

• Intuitive: Directly checks for the first occurrence of an object.

Cons#

• Inefficient for Large Arrays: findIndex iterates the array for every object, leading to O(n²) time complexity. Avoid for arrays with 1000+ items.

Method 4: Using Lodash’s uniqBy#

How It Works#

Lodash, a popular utility library, provides _.uniqBy, which removes duplicates by a specified key. It’s concise and handles edge cases (e.g., null/undefined keys) gracefully.

Code Example#

First, install Lodash (npm install lodash), then import uniqBy:

import { uniqBy } from "lodash";
 
const uniqueUsers = uniqBy(users, "id"); // Deduplicate by `id`
 
console.log(uniqueUsers); 
// Output: [{ id: 1, ... }, { id: 2, ... }, { id: 3, ... }]

Pros#

• Concise: One line of code.
• Battle-Tested: Handles edge cases (e.g., nested keys with _.uniqBy(users, u => u.profile.id)).

Cons#

• Adds a Dependency: Increases bundle size (Lodash core is ~70KB minified). Use lodash-es with tree-shaking to reduce size.

Method 5: Using Map for Key-Value Tracking#

How It Works#

A Map object stores key-value pairs and remembers the insertion order. Like Set, it can track unique keys, but it also allows you to store the entire object for quick access.

Code Example#

const userMap = new Map();
 
// Use `id` as the key; if the key doesn't exist, add the object to the Map
users.forEach(user => {
  if (!userMap.has(user.id)) {
    userMap.set(user.id, user);
  }
});
 
// Convert Map values back to an array
const uniqueUsers = Array.from(userMap.values());
 
console.log(uniqueUsers); 
// Output: [{ id: 1, ... }, { id: 2, ... }, { id: 3, ... }]

Pros#

• Readable: Explicitly maps keys to objects, making intent clear.
• Order-Preserving: Maintains the order of first occurrences (unlike older Set implementations in some browsers).

Cons#

• Slightly Overkill: For simple cases, Set + filter is more concise.

Method 6: Deep Comparison (No Unique Key)#

When to Use This#

If your objects lack a unique key (e.g., { name: "Alice" } and { name: "Alice" }), you need to compare all properties. This is called "deep comparison."

Approach 1: JSON.stringify + Set#

Convert objects to JSON strings and use a Set to track duplicates. This works for simple objects with serializable values (strings, numbers, arrays).

Code Example#

const objects = [
  { name: "Alice", age: 30 },
  { age: 30, name: "Alice" }, // Same properties, different order
  { name: "Bob", age: 25 },
  { name: "Alice", age: 30 }, // Duplicate
];
 
// Sort object keys to handle inconsistent property order
const stringifyWithSortedKeys = (obj) => 
  JSON.stringify(Object.entries(obj).sort());
 
const seenStrings = new Set();
const uniqueObjects = objects.filter(obj => {
  const str = stringifyWithSortedKeys(obj);
  if (!seenStrings.has(str)) {
    seenStrings.add(str);
    return true;
  }
  return false;
});
 
console.log(uniqueObjects); 
// Output: [{ name: "Alice", age: 30 }, { name: "Bob", age: 25 }]

Approach 2: Custom Deep Compare Function#

For complex objects (e.g., with nested objects, Date, or RegExp), JSON.stringify may fail (e.g., Date objects serialize to strings, but new Date(2023) and new Date("2023-01-01") are different strings). A custom deep compare function is more reliable but complex.

Code Example (Simplified Deep Compare)#

const isDeepEqual = (obj1, obj2) => {
  // Handle primitive values and references
  if (obj1 === obj2) return true;
  // Check if both are objects and not null
  if (typeof obj1 !== "object" || obj1 === null || typeof obj2 !== "object" || obj2 === null) {
    return false;
  }
  // Compare keys
  const keys1 = Object.keys(obj1);
  const keys2 = Object.keys(obj2);
  if (keys1.length !== keys2.length) return false;
  // Recursively compare values
  for (const key of keys1) {
    if (!isDeepEqual(obj1[key], obj2[key])) return false;
  }
  return true;
};
 
// Filter duplicates using deep comparison
const uniqueObjects = objects.filter((obj, index) => {
  return objects.findIndex(otherObj => isDeepEqual(obj, otherObj)) === index;
});

Pros#

• No Unique Key Needed: Compares object content directly.

Cons#

• Performance: JSON.stringify and custom deep compare are O(n²) for large arrays.
• Limitations: JSON.stringify fails for non-serializable values (e.g., Symbol, Function, undefined).

Comparison of Methods: When to Use Which?#

Conclusion#

Removing duplicate objects in JavaScript depends on your data structure and needs:

• Use Set + filter or reduce if you have a unique key (fast and native).
• Use Lodash’s uniqBy for concise code in Lodash projects.
• Use deep comparison only when no unique key exists (e.g., ad-hoc objects).

Always prioritize methods with O(n) time complexity for large datasets, and test edge cases like missing keys or non-serializable values.

References#

• MDN: Set
• MDN: Array.reduce()
• Lodash uniqBy
• MDN: JSON.stringify
• MDN: Map