1 / 34

October 4, 2011 Joe Cross

Useful Python Techniques: A brief introduction to List Comprehensions, Functional Programming , and Generators. October 4, 2011 Joe Cross. Contents. Looping and List Comprehensions Filter, Map, Reduce (and lambda!) Functional Programming Basics + Decorators Generators.

dseal
Download Presentation

October 4, 2011 Joe Cross

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Useful Python Techniques:A brief introduction to List Comprehensions, Functional Programming, and Generators October 4, 2011 Joe Cross

  2. Contents • Looping and List Comprehensions • Filter, Map, Reduce (and lambda!) • Functional Programming Basics + Decorators • Generators

  3. 1. Looping and List Comprehensions for(inti = 0; i < 5; i++){ cout << witty_banter(i) << endl; }

  4. Looping compared to Java/C++ for i inrange(0, 10, 1):    a /= 2.0for i inxrange(10):    a /= 2.0 Python Java/C++ for(int i =0; i <10; i++){    a /=2.0;} Looping

  5. Looping Range + Iter aList = [0, 1, 2, 'hello', 2**-4]for i inrange(len(aList)):print aList[i]for item in aList:print item Looping

  6. Looping still not as fun as… To double the values in a list and assign to a new variable: winning_lottery_numbers = [0, 4, 3, 2, 3, 1]fake_lottery_numbers = []for i inrange(len(winning_lottery_numbers)):    fake_lottery_numbers.append(2* winning_lottery_numbers[i])fake_lottery_numbers = []for number in winning_lottery_numbers:    fake_lottery_numbers.append(2* number) Even though it’s an improvement over the tedium of c++ et. al, we can still do better. Looping

  7. List Comprehensions Woohooo! Syntax: [<expression>for<value>in<collection>if<condition>] winning_lottery_numbers = [0, 4, 3, 2, 3, 1]fake_lottery_numbers = [2*n for n in winning_lottery_numbers] • List Comprehensions allow us to do all sorts of things: • Single-function single-line code • Apply a function to each item of an iterable • Filter using conditionals • Cleanly nest loops List Comprehensions

  8. List Comprehensions Don’t nest too many! Multi-variable functions in a single line using zip: vec1 = [3, 10, 2]vec2 = [-20, 5, 1]dot_mul = [u*v for u, v inzip(vec1, vec2)]dot_prod = sum(dot_mul) Filtering: readings = [-1.2, 0.5, 12, 1.8, -9.0, 5.3]good_readings = [r for r in readings if r >0] Bad: orig = [15, 30, 78, 91, 25]finals = [min(s, 100)for s in[f+5for f in orig]] List Comprehensions

  9. 2. Filter, Map, Reduce Life = map(evolution, irreducible complexity) assert(sum(Life) == 42)

  10. Filter Syntax: result = filter(aFunction, aSequence) def isPos(number, lim = 1E-16):return number > lim>>> a = [-1,2,-3,4,-5,6,-7,8,-9,10]>>>filter(isPos, a)[2, 4, 6, 8, 10] >>>filter(not isPos, a)Traceback (most recent call last):  File "<pyshell#7>", line 1filter(not isZero, a)TypeError: 'bool'objectisnotcallable Filter, Map, Reduce

  11. Filter + Lambda Syntax: [fnName] = lambda [args]: expression def isPos(number, lim = 1E-16):return number > lim >>>filter(lambda n: not isPos(n), a)[-1, -3, -5, -7, -9] Filter, Map, Reduce

  12. Lambda vs. def def add(x, y):return x + yLadd = lambda x, y: x+ydef printWords():print"Words"LprintWords = lambda: print"Words" Filter, Map, Reduce

  13. So… why use lambda? When using verbose function declaration it is often the case that the function’s verbose declaration can be verbose, even for functions that don’t require such verbosity. Verbose b = []for a in aList:if a >0:        b.append(a) def ispos(n):return n >0b = filter(ispos, aList) Vs. b = filter(lambda n: n >0, aList) Not Verbose Also, there are some valid concerns about namespace clutter and the like. Verbose verbose verbose. Filter, Map, Reduce

  14. Map Syntax: result = map(aFunction, aSequence) Compare to list comprehension: winning_lottery_numbers = [0, 4, 3, 2, 3, 1] fake_lottery_numbers = [2*n for n in winning_lottery_numbers] 1. 2. fake_lottery_numbers = map(lambda n: 2*n, winning_lottery_numbers) Filter, Map, Reduce

  15. Reduce Syntax: result = reduce(aFunction, aSequence, [initial]) NOTE: results get accumulated on the left, and new values applied to the right. so reduce(add, [1,2,3,4]) is processed as (((1+2)+3)4) lambda factorial n: reduce(operator.mul, xrange(1, n)) Filter, Map, Reduce

  16. 3. Functional Programming + Decorators This isn’t your dad’s Procedural (imperative) programming

  17. A Simple Example def add(x, y):return x + ydef sub(x, y):return x - ydef mul(x, y):return x * ydef div(x, y):return x / ydef mod(x, y):return x % y def op(fn, x, y):return fn(x, y) Functional Programming

  18. Nested Functions Speed vs. Obfuscation defrandNoGap(min_, max_):#random.random() -> [0,1)    v = random.random()return(max_ - min_)* v - min_defrandWithGap(min_, max_):    s = random.random()    v = randNoGap(min_, max_)if s <0.5:return velse:return -v#Same conditional using Python’s #Ternary operator#return v if s < 0.5 else -v def rand(min_, max_, hasGap = False):ifhasGap:returnrandWithGap(min_, max_)else:returnrandNoGap(min_, max_) Functional Programming

  19. Nested Functions Speed vs. Obfuscation (Continued) def randomExplosion(minv, maxv, n):    particles = []for _ inxrange(n):        vx = rand(minv, maxv, True)        vy = rand(minv, maxv, True)        vz = rand(minv, maxv, True)        vx2 = rand(minv, maxv, True)        vy2 = rand(minv, maxv, True)        vz2 = rand(minv, maxv, True)        r = rand(0,255,False)        g = rand(0,255,False)        b = rand(0,255,False)        mainParticle = [vx,vy,vz,r,g,b]        secondParticle = [vx2,vy2,vz2,r,g,b]        particleGroup = (mainParticle, secondParticle)        particles.append(particleGroup)return particles NO Functional Programming

  20. Nested Functions Speed vs. Obfuscation (Continued) What we’d like to do: velocities = [rndV() for _ inxrange(6)] What it actually looks like: velocities = [rand(minv,maxv,True)for i inxrange(6)] With a functional wrapper, we re-map: rndV -> make_rand_fnc(minv, maxv, True) rndV() -> make_rand_fnc(minv, maxv, True)() Functional Programming

  21. Nested Functions Speed vs. Obfuscation (Continued) rand(minv, maxv, True)rand(minv, maxv, True)rand(minv, maxv, True) def rand(min_, max_, hasGap = False): def mkRand(min_, max_, hasGap = False):def wrapper():return rand(min_, max_, hasGap)return wrapper def randomExplosion(minv, maxv, n):    rVel = mkRand(minv, maxv, True)    rCol = mkRnd(0,255,False)for _ inxrange(n):        vx = rVel()        vy = rVel()        vz = rVel()        vx2 = rVel()        vy2 = rVel()        vz2 = rVel()        r = rCol()        g = rCol()        b = rCol() Functional Programming

  22. Nested Functions Speed vs. Obfuscation (Continued) def randomExplosion(minv, maxv, n):    particles = []    rndV = mkRand(minv, maxv, True)    rndC = mkRnd(0,255,False)for _ inxrange(n):        velocities = [rndV()for i inxrange(6)]        r,g,b = [rndC()for i inxrange(3)]        mainParticle = velocities[:3] + [r,g,b]        secondParticle = velocities[3:] + [r,g,b]        particleGroup = (mainParticle, secondParticle)        particles.append(particleGroup)return particles Functional Programming

  23. Decorators Quickly apply common tasks to methods • Common pre + post function call tasks, such as: • Caching • Timing • Counting function calls • Access rights @f1(arg)@f2def func(): passdef func(): passfunc = f1(arg)(f2(func)) @decoratordef myFunc(arg1):print “arg1: “, arg1myFunc = decorator(myFunc) Decorators

  24. Decorators Quickly apply common tasks to methods def decorator(f):print"This line is run once during func = decorator(func)"def wrapper(*args, **kwargs):print"This line is executed just before the function is called"#Call the function        ret = f(*args, **kwargs)print"This line is executed just after the function is called"#Return the function's returnreturn retreturn wrapper@decoratordef foo(bar):print bar On running, we get this output: >>> ================================ RESTART ================================>>>This line is run once during func = decorator(func)>>> foo(1)This line is executed just before the function is called1This line is executed just after the function is called Decorators

  25. Decorators Quickly apply common tasks to methods Decorators using classes class decorator(object):def__init__(self, f):print"This line is run once during func = decorator(func)"self.f = fdef__call__(self, *args, **kwargs):print"This line is executed just before the function is called" #Call the function        ret = self.f(*args) print"This line is executed just after the function is called" #Return the function's returnreturn ret Decorators

  26. Decorators Quickly apply common tasks to methods (Rough) Timing importtimeclass TIMED(object):def__init__(self, f):self.f = f def__call__(self, *args):        start = time.clock()        ret = self.f(*args)        stop = time.clock()print"{0}: {1} ms.".format(self.f.func_name, 1000*(stop-start)) return ret Decorators

  27. Decorators Quickly apply common tasks to methods @TIMEDdef euler(f, t0, y0, h):""" Euler's Method """   yn = y0 + h*f(t0,y0)return yn@TIMEDdef RK2(f, t0, y0, h):""" Heun's Method """   y_hat = y0 + h*f(t0,y0)   yn = y0 + h/2.0*(f(t0,y0)+f(t0+h, y_hat))return yn@TIMEDdef RK4(f, t0, y0, h):""" Standard RK4 """   k1 = f(t0, y0)   k2 = f(t0+h/2.0,y0 + h*k1/2.0)   k3 = f(t0+h/2.0,y0 + h*k2/2.0)   k4 = f(t0+h/2.0,y0 + h*k3)   yn = y0 + 1.0/6.0*h*(k1 + 2.0*k2 + 2.0*k3 + k4)return yn Decorators

  28. Decorators Quickly apply common tasks to methods fns = [euler, RK2, RK3, RK4, jRK4]t0 = scipy.linspace(-1,1)y0 = scipy.ones(50)h = 0.025args = (f, t0, y0, h)for fn in fns:print fn(*args)print >>>euler: 0.0181114469778 ms.[ ... ]RK2: 0.041656328049 ms.[  ... ]RK3: 0.0606733473757 ms.[  ... ]RK4: 0.0745587900587 ms.[ ... ]jRKN: 0.00150928724815 ms.jRK4: 1.57358288492 ms.[ ... ] Decorators

  29. 4. Generators Memory-conscious patterns are kind of a big deal in scientific computing

  30. Binary Tree from Array(simplified interface) class T(object):def__init__(self, values = None, index = 0):self.left = Noneself.right = Noneself.v = Noneif values isnotNone:self.loadValues(values, index)def loadValues(self, values, index):self.v = values[index]        n = len(values)if index *2 + 1< n:self.left = T(values, index *2 + 1)if index *2 + 2< n:self.right = T(values, index *2 + 2) Generators

  31. Guessing Game def makeT(val, delta, levels, level = 0):if level < levels:        t = T()        t.v = val        t.left = makeT(val-delta, delta/2, levels, level+1)        t.right = makeT(val+delta, delta/2, levels, level+1)return t Generators

  32. Clean Code def inorder(t):if t:for v in inorder(t.left):yield vyield t.vfor v in inorder(t.right):yield v Generators

  33. Using our Generator for v in inorder(a_tree):print va = []for v in inorder(a_tree):    a.append(v)b = [v for v in inorder(a_tree)] Generators

  34. Questions? • “Yes, the slide with the code. What did that one do?”

More Related