1

I am trying to create types with certain constraints.

Because I want these constraints to be arbitrarily complex, I've decided that I do not need them to be type-checked, but I do want them to travel with the type definitions.

As an example, say I want to create a homogeneous container (i.e. typing.List) constrained to a size. For this particular example, I know I should use Tuple[int, int] (as per this question) but that's not flexible enough for other use cases.

My desired functionality:

from typing import List

class MyList(List):
    def __init__(self, *, num_elements: int) -> None:
        self.num_elements = num_elements

    def validate(self, input: List) -> None:
        if len(to_validate) > self.num_elements:
            raise ValueError


class MyClass:
    myvar: MyList(num_elements=2)[int]  # should look like List[int] to a type checker

    def __init__(self, *, myvar: MyList(num_elements=2)[int]):  # I guess I need to define twice?
        self.myvar = myvar
        self.validate_all()
    
    def validate_all(self):
        for var in self.__annotations__:
            if hasattr(self.__annotations__[var], "validate"):
                self.__annotations__[var].validate(getattr(self, var))


MyClass(myvar=(1, 2))  # pass

MyClass(myvar=(1, 2, 3))  # fail

As noted above, the annotation for myvar would have to look like List[int] to a type checker like mypy so that at least that part can be handled by existing frameworks.

I know I probably need to do something with typing.Generic or typing.TypeVar but I tried and honestly don't understand how they would be applicable to this situation.

2
  • When you say it could be arbitrarily complex, what do you have in mind? And why do you want to avoid type checking it? Commented Jul 5, 2020 at 15:41
  • It could be something like checking a regex pattern on a string. For example, I define a new type called "Barcode" that to a type checker appears looks like str (so at least the fact that it is a string is taken care of) but my validate method checks that the string matches the regex pattern. Commented Jul 5, 2020 at 16:40

3 Answers 3

1

To my knowledge typing module is quite primitive because we needed a basic type checking with some extra flavors and that's all it does - allows to valiadate the type of an input. What you want is a logical validation that is't really defined by a type. List with 3 or 2 elements is still a list.

With pydantic you can do

from typing import List

from pydantic import validator, BaseModel

num_elements = 2


class MyClass(BaseModel):
    myvar: List[int]

    @validator('myvar')
    def check_myvar_length(cls, v):
        if len(v) > num_elements:
            raise ValueError("Myvar too long!")
        return v

MyClass(myvar=(1, 2))  # pass
MyClass(myvar=(1, 2, 3))  # fail

or with dataclasses

from dataclasses import dataclass
from typing import List

num_elements = 2


@dataclass
class MyClass:
    myvar: List[int]

    def __post_init__(self):
        if len(self.myvar) > num_elements:
            raise ValueError("Myvar too long!")


MyClass(myvar=(1, 2))  # pass
MyClass(myvar=(1, 2, 3))  # fail

I know what you want to accomplish but I don't think it's possible. You can always create a regular class with validate method and run it in __init__ but I don't think that's what you want nor it is readable.

Sign up to request clarification or add additional context in comments.

1 Comment

I'm open to using pydantic! Do you know if there's any way to make a pydantic class type check against a list? For example: class MyList(BaseModel): ... And then: myvar: MyList(args) = [1, 2, 3] # type checker would be okay with this Thanks!
0

This doesn't use typing, but you could solve this using the descriptor pattern. This would be something like this:

from collections import Iterable


class MyList:
    def __init__(self, name):
        self.name = name

    def __get__(self, instance, cls):
        if instance is None:
            return self
        else:
            return instance.__dict__[self.name]

    def __set__(self, instance, value):
        if not isinstance(value, Iterable):
            raise TypeError("This isn't iterable!")
        elif len(value) != 2 or not all(isinstance(val, int) for val in value):
            raise TypeError("List must be length 2 and all integers")

        instance.__dict__[self.name] = value

    def __delete__(self, instance):
        del instance.__dict__[self.name]


class MyClass:
    variable = MyList("variable")
    def __init__(self, var):
        self.variable = var


MyClass(myvar=(1, 2))  # pass
MyClass(myvar=[1, 2])  # pass
MyClass(myvar=(1, 2, 3))  # fail
MyClass(myvar="1,2,3")  # fail

Comments

0

I think I found a solution using pydantic (credit to Tomasz Wojcik for suggesting it in their answer).

This is a trivial example for a fixed size list. I'm using this example because it is what was in my original question, however I will note that pydantic has this constrained type built in (see their docs). Still, this general approach does allow arbitrary constrained types that pydantic will validate.

I do not know how this will interact with other type-checking systems (thinking of mypy), but since pydantic is compatible with mypy I think this should too.

Most of this code/solution was borrowed from the implementation of pydantic's existing constrained types (here). I would suggest that anyone trying to apply this method take a look there for a starting point.

from typing import Optional, List, TypeVar, Type, Dict, Any, Generator, Callable
from types import new_class

from pydantic import BaseModel
from pydantic.error_wrappers import ValidationError
from pydantic.fields import ModelField
from pydantic.utils import update_not_none
from pydantic.validators import list_validator

T = TypeVar('T')

# This types superclass should be List[T], but cython chokes on that...
class ConstrainedListClass(list):  # type: ignore
    # Needed for pydantic to detect that this is a list
    __origin__ = list
    __args__: List[Type[T]]  # type: ignore
    item_type: Type[T]  # type: ignore

    length: Optional[int] = None
    
    @classmethod
    def __get_validators__(cls) -> Generator[Callable, None, None]:
        yield cls.validator

    @classmethod
    def __modify_schema__(cls, field_schema: Dict[str, Any]) -> None:
        update_not_none(field_schema, length=cls.length)

    @classmethod
    def validator(cls, v: 'Optional[List[T]]', field: 'ModelField') -> 'Optional[List[T]]':
        if v is None and not field.required:
            return None

        v = list_validator(v)

        if cls.length != len(v):
            raise ValueError

        return v

def ConstrainedList(item_type: Type[T], *, length: int = None) -> Type[List[T]]:
    """Factory function for ConstrainedListClass

    Returns
    -------
    Type[List[T]]
        [description]
    """
    cls = ConstrainedListClass
    # __args__ is needed to conform to typing generics api
    namespace = {'length': length, 'item_type': item_type, '__args__': [item_type]}
    # We use new_class to be able to deal with Generic types
    return new_class(name=cls.__name__ + 'Value', bases=(cls, ), kwds={}, exec_body=lambda ns: ns.update(namespace))

ConstrainedList(int, length=2)

class UserClass(BaseModel):
    myvar: ConstrainedList(int, length=2)


UserClass(myvar=[1, 2])   # pass

for myvar in [
    [1],  # wrong number of items
    ["str1", "str2"],   # wrong type
]:
    try:
        UserClass(myvar=myvar)  # fail
    except ValidationError:
        continue
    raise RuntimeError

I will also note that it would be nice if the factory function could be put into the class' __new__ so that the syntax could stay the same but avoid having the extra function. I couldn't get this or anything similar to work.

2 Comments

Wow, that looks quite complicated. I'm glad you've found what you were looking for. Pydantic has a mypy plugin but I never used it. Another thing, I'm not sure if you inherit from list on purpose but if so, you should rather inherit from UserList. docs.python.org/3/library/… Good luck!
Ah, I see that's how pydantic inherit from the list. Well, maybe :shrug: I'd rather go with UserList as you're not supposed to inherit from builtins.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.