Filter unique & duplicate objects in array using underscore

Question

I have an array of objects:

data = [{"origin":"SJU","dest":"JFK","rank":48},{"origin":"JFK","dest":"SJU","rank":21},{"origin":"IAD","dest":"LAX","rank":31},{"origin":"LAS","dest":"SJU","rank":21}]

I am trying to extract all duplicates & unique objects BY "origin" & "dest". So that these 2 are the same, ignoring the rank key

{origin:'JFK',dest:'SJU',rank:21}
{"origin":"SJU","dest":"JFK","rank":48}

Basically I want 2 seperate arrays:

duplicates=[{"origin":"SJU","dest":"JFK","rank":48},{"origin":"JFK","dest":"SJU","rank":21}]

unique = [{"origin":"IAD","dest":"LAX","rank":31},{"origin":"LAS","dest":"SJU","rank":21}]

using underscore, I was able to throw something like this together. But It seems inefficient and only returns an array of duplicates:

duplicates = _.chain(data).map(function (d) {
    var ar = [d.origin, d.dest];
    return ar.sort();
}).sortBy(function (d) {
    return d
}).groupBy(function (d) {return d}).map(function (d) {
    if (d.length > 1) {
        return d[0]
    }
}).compact().value()
single = _.chain(data).map(function (d) {
    var ar = [d.origin, d.dest];
    return ar.sort();
}).sortBy(function (d) {
    return d
}).groupBy(function (d) {
    return d
}).map(function (d) {
    if (d.length == 1) {
        return d[0]
    }
}).compact().value()

I can't help but feel there is a much easier way to get this.

Ja͢ck · Accepted Answer · 2014-06-20 14:15:05Z

2

It might be easier to introduce a temporary variable to hold the groups:

var data = [{"origin":"SJU","dest":"JFK","rank":48},{"origin":"JFK","dest":"SJU","rank":21},{"origin":"IAD","dest":"LAX","rank":31},{"origin":"LAS","dest":"SJU","rank":21}]

var groups = _.groupBy(data, function(item) {
  return [item.origin, item.dest].sort();
});

Then:

var duplicates = [],
singles = [];

_.each(groups, function(group) {
  if (group.length > 1) {
    duplicates.push.apply(duplicates, group);
  } else {
    singles.push(group[0]);
  }
});

Demo

edited Jun 20, 2014 at 14:15

answered Jun 20, 2014 at 13:45

Ja͢ck

174k39 gold badges269 silver badges317 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Kyle Kalwarski Over a year ago

Ahhhh - I am not sure about the [0] at the end of your code. But from this, I was able to get what I needed. Much simpler. Thanks!: singles = _.chain(groups).filter(function(group) { return group.length == 1; }).flatten().value() and duplicates = _.chain(groups).filter(function(group) { return group.length > 1; }).flatten().value()

Ja͢ck Over a year ago

@user3760167 Yeah, I was struggling with Underscore there for a bit :) it should return the correct values now.

plalx Over a year ago

@user3760167 It would be more efficient to avoid filtering twice and there's no need for it. Just build dupes and uniques in a single run.

Kyle Kalwarski Over a year ago

thanks! I didn't think about doing it without chaining... I think the final thing missing is returning the single array of duplicates:

duplicates = _.flatten(_.filter(groups, function(group) {   return group.length > 1; }).map(function(item) {    return item; }));

plalx Over a year ago

@Jack That's cool, since when can we notify more than a single user like this /cc:@user3760167 ? Anyway, the algo is better that way ;)

|

user1835565 · Accepted Answer · 2014-06-20 13:42:27Z

0

I don't understand your code. However I got the problem.

The way I see it is 2 steps:

identify duplicates and push them in an array duplicates
create the uniquearray by copying the inital array and remove all the elements that match duplicates array.

Maybe it has lower performance but it could be clearer.

answered Jun 20, 2014 at 13:42

user1835565

1411 silver badge8 bronze badges

Comments

Ingmars · Accepted Answer · 2014-06-20 13:46:12Z

Here is the algorithm using plain javascript. It is probably not the most efficient one if you will have thousands of records, but it get's the job done.

var data = [
    {"origin": "SJU", "dest": "JFK", "rank":48},
    {"origin": "JFK", "dest": "SJU", "rank":21},
    {"origin": "IAD", "dest": "LAX", "rank":31},
    {"origin": "LAS", "dest": "SJU", "rank":21}
];

var uniques = [];
var doubles = [];

var i, j, l = data.length, origin, dest, foundDouble;

for ( i = 0; i < l; i += 1 ) {
    origin = data[i].origin;
    dest = data[i].dest;
    foundDouble = false;

    for ( j = 0; j < l; j += 1 ) {
        //skip the same row
        if ( i == j ) {
            continue;
        }

        if ( (data[j].origin == origin || data[j].origin == dest) && (data[j].dest == origin || data[j].dest == dest) ) {
            doubles.push( data[i] );
            foundDouble = true;
        }
    }

    if ( !foundDouble ) {
        uniques.push( data[i] );
    }
}

console.log( 'Uniques', uniques );
console.log( 'Doubles', doubles );

plalx · Accepted Answer · 2014-06-20 14:07:35Z

There are many ways to do this, but here's one. I never used underscore, but the algorithm is pretty straight forward and you should be able to convert it easily.

var voyages = [{"origin":"SJU","dest":"JFK","rank":48},{"origin":"JFK","dest":"SJU","rank":21},{"origin":"IAD","dest":"LAX","rank":31},{"origin":"LAS","dest":"SJU","rank":21}],
    dupes = [],
    uniques = [],
    countMap = new Map(), //Map is from Harmony, but you can use a plain object
    voyageKeyOf = function (voyage) { return [voyage.origin, voyage.dest].sort().join(''); },
    uniques;

//Create a map that stores how many times every voyage keys were seen
voyages.forEach(function (voyage) {
    var key = voyageKeyOf(voyage),
        hasCount = countMap.get(key);

    if (!hasCount) countMap.set(key, 1);
    else {
        let count = countMap.get(key);
        countMap.set(key, ++count);
    }
});

voyages.forEach(function (voyage) {
    var key = voyageKeyOf(voyage),
        isUnique = countMap.get(key) == 1;

    if (isUnique) uniques.push(voyage)
    else dupes.push(voyage);
});


console.log('uniques', uniques);
console.log('dupes', dupes);

@Jack developer.mozilla.org/fr/docs/Web/JavaScript/Reference/… It's from Harmony. I added a note ;)
The reason people use Underscore is so that they don't have to worry about different browser versions ...
@Jack Yes and underscore is a great abstraction. I am not suggesting to use this code, I said it could easily be ported to underscore, but I never used it so it would take more time for me to port it compared to someone who knows it already.

Collectives™ on Stack Overflow

Filter unique & duplicate objects in array using underscore

4 Answers 4

6 Comments

Comments

Comments

6 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

6 Comments

Comments

Comments

6 Comments

Linked

Related