1 / 18

Iterators and Generators

This overview covers the concepts of iteration, iterators, generators, and tasklets in Python. Learn how to create, utilize, and manage iterators and generators to enhance your coding experience. Discover examples and best practices for implementing these features effectively.

meriw
Download Presentation

Iterators and Generators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Iterators and Generators Thomas Wouters XS4ALL thomas@xs4all.net

  2. Overview • Iteration • Iterators • Generators • Tasklets • Questions

  3. Iteration • The act of going over a collection • Explicit in the form of ‘for’ • Implicit in many forms: • list(), tuple(), dict(), … • map(), reduce(), zip(), … • ‘in’ in absence of __contains__() • ‘extended call’ syntax: func(*…) • but not apply() • Uses __getitem__ in Python before 2.2

  4. Iterators • Added in Python 2.2 • Protocol of 2 methods (no special class): • __iter__(): get iterator • next(): get next value • raises StopIteration when done • Explicit iterator creation with iter() • Turn iteration(-state) into objects • Interchangeable with iterable for iteration

  5. iter() • Creates a new iterator for an object • Calls __iter__() for creation • Falls back to __getitem__() • Called implicitly for iteration • Wraps a function in an iterator • iter(f, o) calls f until o is returned: for line in iter(file.readline, ""): handle(line)

  6. Examples >>> l = [1, 2, 3, 4, 5, 6] >>> it = iter(l) >>> for num in it: ... if num > 1: break >>> for num in it: ... print num; break 3 >>> print list(it) [4, 5, 6]

  7. Writing Iterators class IRange: def __init__(self, end): self.end = end self.cur = 0 def next(self): cur = self.cur if cur >= self.end: raise StopIteration self.cur += 1 return cur def __iter__(self): return self

  8. Generators • Optional in Python 2.2 • from __future__ import generators • Use new keyword 'yield' • any function with 'yield' is special • Turn function-state into objects • Use the iterator protocol • Not unavoidable • just very very convenient

  9. IRange generator >>> def irange(end): ... cur = 0 ... while cur < end: ... yield cur ... cur += 1 >>> print irange(10) <generator object at 0x4701c0> >>> print list(irange(10)) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] • Many useful generators in the 'itertools' module (Python 2.3)

  10. Example def map(func, iterable): result = [] for item in iterable: result.append(func(item)) return result def imap(func, iterable): for item in iterable: yield func(item)

  11. 'yield' and 'try'/'finally' • Python does not allow 'yield' inside a 'try' block with a 'finally' clause: try: yield x yield x finally: print x • 'yield' inside 'finally' or in 'try'/'except' is allowed

  12. Generators as Tasklets • Write function as generator • at all pause points before the final result, yield a sentinel value (such as None) • avoid blocking functions, or write them as generators and call in a loop • Convince callers to use iteration • Remember: you may never be called again

  13. TryAgainLater = object() class Bucket: def __init__(self): self.data = [] self.done = False def __iter__(self): return self def next(self): if self.data: return self.data.pop(0) if self.done: raise StopIteration return TryAgainLater def add(self, item): self.data.append(item) def end(self): self.done = True

  14. class Tasklet: def __init__(self, *args, **kwargs): self._work = self._workgen(*args, **kwargs) def run(self): try: return self._work.next() except StopIteration: return None def _workgen(self, bucket): for item in bucket: if item is TryAgainLater: yield TryAgainLater continue for returnval in self._process(item): yield returnval def _process(self, item): while not item.done: yield TryAgainLater yield item.value

  15. class Tasklet: def __init__(self, *args, **kwargs): self.run = self._workgen(*args, **kwargs).next def _workgen(self, bucket, process): for item in bucket: if item is TryAgainLater: yield TryAgainLater continue for endmarker in process(item): if endmarker is TryAgainLater: yield TryAgainLater break else: raise SuitableError, "process never returned anything" for skipitem in bucket: # eat bucket items until endmarker if skipitem is not endmarker: yield TryAgainLater break else: warnings.warn(SuitableWarning, "marker not found") yield DoneProcessing raise InternalError, "Tasklet finished"

  16. class Tasklet: def __init__(self, *args, **kwargs): self.run = self._workgen(*args).next def _workgen(self, bucket, process): for item in bucket: if item is TryAgainLater: yield TryAgainLater continue for endmarker in process(item): if endmarker is TryAgainLater: yield TryAgainLater break else: raise SuitableError

  17. # for item in bucket: # ... for skipitem in bucket: # eat bucket items until # endmarker if skipitem is not endmarker: yield TryAgainLater break else: warnings.warn(SuitableWarning, "marker not found") yield DoneProcessing raise TaskletError, \ "Tasklet finished"

  18. Questions ?

More Related