9

I know this is probably an easy answer but I can't figure it out. What is the best way in Python to keep the duplicates in a list:

x = [1,2,2,2,3,4,5,6,6,7]

The output should be:

[2,6]

I found this link: Find (and keep) duplicates of sublist in python, but I'm still relatively new to Python and I can't get it to work for a simple list.

6
  • Have you tried one of the one liners in that post? Commented Apr 4, 2013 at 13:25
  • Do you need to preserve order?
    – DSM
    Commented Apr 4, 2013 at 13:28
  • @DSM -- Looks like we were thinking the same thing ...
    – mgilson
    Commented Apr 4, 2013 at 13:28
  • Post the code you are having trouble with, otherwise this is a complete retread of that other question. Commented Apr 4, 2013 at 13:28
  • 2
    @StevenRumbalski: Not precisely, the other question is also flattening nested lists at the same time.
    – MattH
    Commented Apr 4, 2013 at 13:31

5 Answers 5

16

I'd use a collections.Counter:

from collections import Counter
x = [1, 2, 2, 2, 3, 4, 5, 6, 6, 7]
counts = Counter(x)
output = [value for value, count in counts.items() if count > 1]

Here's another version which keeps the order of when the item was first duplicated that only assumes that the sequence passed in contains hashable items and it will work back to when set or yeild was introduced to the language (whenever that was).

def keep_dupes(iterable):
    seen = set()
    dupes = set()
    for x in iterable:
        if x in seen and x not in dupes:
            yield x
            dupes.add(x)
        else:
            seen.add(x)

print list(keep_dupes([1,2,2,2,3,4,5,6,6,7]))
11
  • However you lose the order of the elements in the output. Commented Apr 4, 2013 at 13:30
  • Yep. There are a lot of situations where this isn't the best way to go. It also requires the input be hashable... But, it's O(n) even for un-sorted lists which is nice.
    – mgilson
    Commented Apr 4, 2013 at 13:32
  • The shortest ordered variant I can think of offhand is [k for k in OrderedDict.fromkeys(x) if counts[k] > 1].
    – DSM
    Commented Apr 4, 2013 at 13:33
  • 1
    @DSM -- Why do you need an OrderedDict there? why not just [k for k in x if counts[k] > 1]? Actually, that's better than what I have. I'll update...
    – mgilson
    Commented Apr 4, 2013 at 13:40
  • @mgilson: try it and see.. :^)
    – DSM
    Commented Apr 4, 2013 at 13:40
10

This is a short way to do it if the list is sorted already:

x = [1,2,2,2,3,4,5,6,6,7]

from itertools import groupby
print [key for key,group in groupby(x) if len(list(group)) > 1]
2
  • 1
    This will also work with python2.6 which is a problem with mine.
    – mgilson
    Commented Apr 4, 2013 at 13:29
  • @luchosrock: No, groupby groups consecutive elements Commented Apr 4, 2013 at 14:30
3

List Comprehension in combination with set() will do exactly what you want.

>>> list(set([i for i in x if x.count(i) > 1]))

[2, 6]
0

keepin' it simple:

array2 = []
aux = 0
aux2=0
for i in x:
    aux2 = i
    if(aux2==aux):
        array2.append(i)
    aux= i
list(set(array2))

That should work

2
  • Won't that give [2,2,6]?
    – DSM
    Commented Apr 4, 2013 at 13:35
  • @DSM ahaha you're totally right, I edited my answer, Thanks :)
    – luchosrock
    Commented Apr 4, 2013 at 13:43
0

Not efficient but just to get the output, you could try:

import numpy as np

def check_for_repeat(check_list):
    repeated_list = []

    for idx in range(len(check_list)):
        elem = check_list[idx]
        check_list[idx] = None

        if elem in temp_list:
            repeated_list.append(elem)

    repeated_list = np.array(repeated_list)

    return list(np.unique(repeated_list))

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.