Why does a==b call b.__eq__ when derived from list and tuple with missing override?

Question

Background

I am writing math utility classes ListVector and TupleVector, inheriting from list and tuple respectively:

class ListVector(list):
  ...

class TupleVector(tuple):
  ...

(Aside: I'm not necessarily claiming this is really a good idea; in fact, I'm well aware that, arguably, I should not do this, since my intended relationships are logically "has-a" rather than "is-a", and inappropriately making it "is-a" is dangerous since, if I'm not careful, undesired behavior will leak from the base classes into my classes, e.g. the behavior of operators +, +=, *, *=, ==, and other gotchas described here.); one possible advantage of using "is-a" anyway is that I expect there will be little-to-no overhead in terms of memory and for getting/setting the i'th element, compared to using list and tuple directly.)

I want to support comparison of ListVector against TupleVector using == and !=; e.g. I want this to succeed:

  assert ListVector((1,2,3)) == TupleVector((1,2,3))

Note that that's different from the base class behavior:

  assert list((1,2,3)) != tuple((1,2,3))

i.e.

  assert [1,2,3] != (1,2,3)

Therefore I'll need to override the __eq__ and __ne__ methods in both of my vector classes.

The problem

I implemented the overrides for __eq__ and __ne__ in my TupleVector class, but I initially forgot to implement them in my ListVector class.

No problem so far: I'm doing TDD, so my unit test should catch that mistake and force me to fix it.

But the unit test assertion that's supposed to catch the mistake unexpectedly succeeds:

  assert ListVector((1,2,3)) == TupleVector((1,2,3))  # unexpectedly succeeds!

Expected behavior: since I forgot to override __eq__ and __ne__ in ListVector, I expect the == call to fall through to list.__eq__, which should return False, and so the assertion should fail.

Actual behavior: it calls reflected TupleVector.__eq__ instead, which returns True, and so the assertion succeeds!

The question

So my question is: why is it calling reflected TupleVector.__eq__ instead of (non-reflected) list.__eq__?

According to the rules described here (which is taken from this faq), I think it should call list.__eq__. Specifically, it looks to me like the 2nd clause applies:

If type(a) has overridden __eq__ (that is, type(a).__eq__ isn’t object.__eq__), then the result is a.__eq__(b)." where my type(a) and type(b) are ListVector and TupleVector respectively.

My reading of the documentation also seems to lead to the same conclusion as the faq (that is, the left operand's method, i.e. list.__eq__, should be called):

If the operands are of different types, and the right operand’s type is a direct or indirect subclass of the left operand’s type, the reflected method of the right operand has priority, otherwise the left operand’s method has priority.

Here is the code:

#!/usr/bin/python3

class ListVector(list):
  # OOPS! Forgot to implement __eq__ and __ne__ for ListVector
  # ...
  pass

class TupleVector(tuple):
  def __eq__(self, other):
    print("TupleVector.__eq__ called")
    # strict=True so comparing Vectors of unequal length will throw
    return all(x==y for x,y in zip(self,other, strict=True))
  def __ne__(self, other):
    return not self.__eq__(other)
  # ...

# Unit test 
assert repr(ListVector((1,2,3))) == "[1, 2, 3]"  # succeeds as expected
assert repr(TupleVector((1,2,3))) == "(1, 2, 3)"  # succeeds as expected
assert TupleVector((1,2,3)) == ListVector((1,2,3))  # emits "TupleVector.__eq__ called" and succeeds, as expected
assert ListVector((1,2,3)) == TupleVector((1,2,3))  # WTF: unexpectedly emits "TupleVector.__eq__ called" and succeeds!

# Confirm that the condition "type(a).__eq__ isn’t object.__eq__", mentioned
# in the decision procedure in the FAQ, holds:
assert ListVector.__eq__ is list.__eq__  # because I forgot to override that
assert ListVector.__eq__ is not object.__eq__  # because list.__eq__ is not object.__eq__
assert TupleVector.__eq__ is not tuple.__eq__  # because I remembered that override
assert TupleVector.__eq__ is not object.__eq__  # definitely not

The (surprising) output is:

TupleVector.__eq__ called
TupleVector.__eq__ called

I expected that, instead, "TupleVector.__eq__ called" should be emitted only once instead of twice, and the "assert ListVector((1,2,3)) == TupleVector((1,2,3))" should fail.

Answer 1

list.__eq__ does get called. But it doesn't return False. It doesn't know what to do with a non-list and thus returns NotImplemented. And then the == operator tries the other way around, as if the tuple had been on the left and the list on the right. Which then succeeds.

You can see it with this:

class ListVector(list):
  def __eq__(self, other):
    print("ListVector.__eq__ called")
    return NotImplemented

Output for the single comparison then:

ListVector.__eq__ called
TupleVector.__eq__ called

Answer 2

If we have the setup:

class TupleVector(tuple):
    def __eq__(self, other: object) -> bool:
        print("TupleVector.__eq__")
        return super().__eq__(other)
        
class ListVector(list):
    ...

Then:

values = (1,)
print(f"== comparison: {ListVector(values) == TupleVector(values)}")

Outputs:

TupleVector.__eq__
== comparison: False

Showing that the __eq__ call has been reflected to call the right-hand side TupleVector.__eq__().

The reason can be seen if we directly call ListVector.__eq__():

print(f"__eq__ comparison: {ListVector(values).__eq__(TupleVector(values))}")

Which outputs:

__eq__ comparison: NotImplemented

The Python documentation states:

NotImplemented

A special value which should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. __imul__(), __iand__(), etc.) for the same purpose. It should not be evaluated in a boolean context. NotImplemented is the sole instance of the types.NotImplementedType type.

Note: When a binary (or in-place) method returns NotImplemented the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator). If all attempts return NotImplemented, the interpreter will raise an appropriate exception. Incorrectly returning NotImplemented will result in a misleading error message or the NotImplemented value being returned to Python code. See Implementing the arithmetic operations for examples.

Since listVector.__eq__(other) is not implemented then it goes up the class hierarchy and calls list.__eq__() which returns the NotImplemented object and this causes Python to try to evaluate the reflected operation other.__eq__(listVector), which is implemented and returns a boolean.