Remove duplicates from list python

Question

I'm trying to write a program that removes duplicates from a list, but my program keeps throwing the error "list index out of range" on line 5, if n/(sequence[k]) == 1:. I can't figure this out. Am I right in thinking that the possible values of "k" are 0, 1, and 2? How is "sequence" with any of those as the index outside of the possible index range?

def remove_duplicates(sequence):
    new_list = sequence
    for n in sequence:
        for k in range(len(sequence)):
            if n/(sequence[k]) == 1:
                new_list.remove(sequence[k])
    print new_list

remove_duplicates([1,2,3])

Answer 1

I strongly suggest Akavall's answer:

list(set(your_list))

As to why you get out of range errors: Python passes by reference, that is sequence and new_list still point at the same memory location. Changing new_list also changes sequence.

And finally, you are comparing items with themselves, and then remove them. So basically even if you used a copy of sequence, like:

new_list = list(sequence)

or

new_list = sequence[:]

It would return an empty list.

Answer 2

Your error is concurrent modification of the list:

for k in range(len(sequence)):
    if n/(sequence[k]) == 1:
        new_list.remove(sequence[k])

It may seem removing from new_list shouldn't effect sequence, but you did new_list = sequence at the beginning of the function. This means new_list actually literally is sequence, perhaps what you meant is new_list=list(sequence), to copy the list?

If you accept that they are the same list, the error is obvious. When you remove items, the length, and the indexes change.

P.S. As mentioned in a comment by @Akavall, all you need is:

sequence=list(set(sequence))

To make sequence contain no dupes. Another option, if you need to preserve ordering, is:

from collections import OrderedDict
sequence=list(OrderedDict.fromkeys(sequence))

Answer 3

If you don't like list(set(your_list)) because it's not guaranteed to preserved order, you could grab the OrderedSet recipe and then do:

from ordered_set import OrderedSet

foo = list("face a dead cabbage")
print foo
print list(set(foo)) # Order might change
print list(OrderedSet(foo)) # Order preserved

Answer 4

# like @Akavall suggested
def remove_duplicates(sequence):
    # returns unsorted unique list
    return list(set(sequence))

# create a list, if ele from input not in that list, append.
def remove_duplicates(sequence):
    lst = []
    for i in sequence:
        if i not in lst:
            lst.append(i)
    # returns unsorted unique list
    return lst