New-Style Classes

1. New-Style Classes Thomas Wouters XS4ALL thomas@xs4all.net Yhg1s @ #python

2. Overview • Old situation • types, classes • shape of Python • Type unification • subtyping • descriptors and properties • class- and static-methods • Metaclasses

3. Old situation: types • Python types implemented in C • C structs represent types • C function pointers define functionality • No real class data • C structs represent objects • All instance data in object structs • 'Manual' attribute retrieval • Explicit methods

4. Old situation: Python classes • Implemented in terms of Python types • "bolted on top" • 'classes' and 'instances' as distinct types • C function pointers delegated to 'magic' methods to define behaviour • tp_getattro function searches base classes • Implicit methods from Python functions • Class and instance data in '__dict__' attribute

5. Old situation: limitations • Python and C as two distinct worlds • No real accessors • No class/static methods • No immutable classes • Simplistic inheritance order • Less control over behaviour

6. Old situation: consequences • Interface-based functionality • informal interfaces rather than inheritance • Containment rather than inheritance • inheritance is not always the answer • Simple inheritance trees • Easy to use

7. Type unification • Allow (better) mingling of Python and C types (or classes) • bring C and Python closer • __dict__ and inheritance for C types • descriptors, properties, class/staticmethods for Python classes • subclassing C types in C • subclassing C types in Python • Correct 'warts' in classic classes • Generalize special cases

8. Type unification (2) • Metaclasses: a new type of type • Explicit base class: object • container of generic functionality • Old-style classes for compatibility • hidden metaclass • Translation from C function-pointers to Python __methods__ (slots) • __dict__ for C types and __slots__ for Python classes

9. Subclassing • Subclass C types from Python in the expected manner: class mylist(list): def __getitem__(self, i): try: return list.__getitem__(self, i) except IndexError: return None • Resricted multiple inheritance • Not always a good idea!

10. Subclassing (2) • __new__, Python's constructor • called to construct (allocate) the object • static method, called with class as first argument • may return an existing value • __slots__: store data almost like C would • no __dict__, less memory consumption

11. Immutable Python types class tristate(int): __slots__ = [] nstates = 3 def __new__(cls, state=0): state %= cls.nstates return int.__new__(cls, state) def __add__(self, o): return tristate(int.__add__(self, o)) def __sub__(self, other): return self.__add__(-other)

12. Subclassing C types in C • Make sure base type supports subclassing • Py<type>_Check(), Py<type>_CheckExact() • PyMethodDef, PyMemberDef, PyGetSetDef • PyObject_GenericGetAttr as tp_getattro • no type-object hardcoding • Subclass's PyType object • leave unchanged behaviour up to base type • set tp_base to base class • call base class's tp_init • Provide compatible object struct

13. Type checking • Type-checking is a necessary evil (in C) • PyObject_TypeCheck() for inheritance-aware PythonC type check • Define Py<type>_Check() in terms of PyObject_TypeCheck() • Define Py<type>_CheckExact() as type-pointer comparison • Use Py<type>_CheckExact() for internal optimizations

14. PyList_Check*() #define PyList_Check(op) \ PyObject_TypeCheck(op, \ &PyList_Type) #define PyList_CheckExact(op)\ ((op)->ob_type == \ &PyList_Type)

15. Py*Def • Allow subclasses to extend/override parts • PyMemberDef (tp_members) for instance data • maps C structs to Python attributes • tp_members in type struct • PyMethodDef (tp_methods) for all methods • wraps C functions in Python objects • specifies argument style and type of method • PyGetSetDef (tp_getset) for accessors • maps functions to attributes and vice versa

16. Subclass struct • Include base class struct in subclass struct typedef struct { PyListObject list; PyObject * default; } defaultlistobject; • No changes to original memory layout • Multiple inheritance is only possible with 'compatible memory layouts' • C subclasses subclassable in Python

17. Method Resolution Order • Old MRO not suited to complex inheritance trees • base classes get queried before some of their derived classes • base classes get queried multiple times • No convenient way to access base classes • hardcode base class names • guess about attributes / methods

18. New MRO • Published algorithm: C3 • http://www.webcom.com/haahr/dylan/linearization-oopsla96.html • Relatively easy to explain • same order as before • eliminates all but the last occurance of the same class • Same order for simple inheritance

19. A C(A) B(A) Old-style MRO: D, B, A, C, A New-style MRO: D, B, C, A (see __mro__) D(B, C)

20. A C(A) B(A) D(B, C) E(C, B) F(D, E) Old-style MRO: F, D, B, A, C, A, E, C, A, B, A New-style MRO (2.2): F, D, E, B, C, A

21. super() • Proxy'object for accessing 'base' classes • Continues MRO where it left off • requires current class and (derived) instance • Somewhat inconvenient to use • Very important for consistency • Use it anyway

22. super() use class BStore(Storage): def __init__(self, state): Storage.__init__(self, state) class BStore(Storage): def __init__(self, state): super(BStore, self).__init__(state)

23. Descriptors • Generalization of class-getattr magic and C tp_getattr tricks • Trigger functioncalls when retrieved or stored from an object (getattr/setattr) • __get__() • __set__() • __delete__()

24. Properties • Accessors for Python • An application of descriptors • class R(object): def _get_random(self): return random.random() random = property(_get_random) • Also hold docstrings for attributes • 'set' and 'delete' functions don't work with old-style classes

25. Caching Property class cachingprop(object): __slots__ = ["_name", "_fget"] def __init__(self, name, fget): self._name = name self._fget = fget def __get__(self, inst, type=None): if inst is None: return self v = self._fget(inst) inst.__dict__[self._name] = v return v

26. Special method types • classmethods • Passes class as implicit first argument • can be called through class or through instance • allow for factory functions (or 'alternate initializers') that create subclasses • dict.fromkeys • tarfile.TarFile.open

27. Special Method Types (2) • staticmethods • Passes no special arguments • Necessary for object.__new__ (or is it?) • Allows for regular (non-method) Python functions as attributes

28. Special Method Types (3) class Buffer(object): def __init__(self, data): self.data = data[:] def fromstring(cls, s): return cls(s.splitlines()) fromstring = classmethod(fromstring) def send(self): self._extern_send(self.data) _extern_send = staticmethod(sendmodule.send)

29. Metaclasses • The class of class • Usually derives from type • Relate to classes like classes relate to instances • Define class behaviour • Allow for convenient post-processing of classes

30. Class/instance relation • Creating the instance passes the contents (arguments) to the class __init__: class Send(object): def __init__(self, what, who): ... Send("my data", him)

31. Metaclass/class relation class Meta(type): def __init__(self, name, bases, attrs): type.__init__(self, name, bases, attrs) class Impl(base1, base2): __metaclass__ =Meta X = 1 def method(self, it): return not it stat = staticmethod(...)

32. Metaclasses • Behave like classes: • __new__ called for class creation • __init__ called for class initialization • inheritance • Mixing metaclasses requires compatibility • derived classes must have same or derived metaclasses • metametaclasses can automatically derive metaclasses

33. Questions ? Slides will be on http://www.xs4all.nl/~thomas/python/