We've spent some time playing with lists -- they are ordered collections that are iterable, indexable -- and finite. But, what if we'd like to have an iterable collection -- that isn't necessarily finite. Imagine an ordered collection of the even numbers or the digits of pi? Although it isn't indexable, Python does give us a way of representing such an ordered sequence in an iterable way -- generators.
To do this, we write what looks a lot like a function that computes each item in the sequence, one at a time, stopping after each one. Then, each time we "iterate" upon the generator, it computes and yields the next item. The syntax of the generator looks a lot like the syntax of a function, but we "yield" a value instead of "return"ing the value. A "yield" is like a reusmable return -- the next time we poke the generator, it will pick up there.
In class we built up the example below. Notice that we used a loop to genrate the sequence; that if the loop isn't infinite, the sequence isn't either; that we can iterate using next(); and we can also iterate using a for-loop, as we can for any other iterable
def generateEvens(start): if ((start % 2) != 0): start += 1 even = start while True: yield even even +=2 evens = generateEvens(3) print next(evens) print next(evens) print next(evens) print next(evens) # Reset the generator: # Technically, create a new one and assign it to the same variable evens = generateEvens(3) # This would be infinite! # for even in evens: print even def generateEvensRangeInclusive(start, end): if ((start % 2) != 0): start += 1 even = start while (even <= end): yield even even +=2 evens = generateEvensRangeInclusive(3, 10) for even in evens: print even # Reset evens = generateEvensRangeInclusive(3, 10) print next(evens) print next(evens) print next(evens) print next(evens) print next(evens) # Notice the StopIterator exception
Generators "Comprehensions" Expressions
Just like we could use a closed form to initialize a list, we can do the same for generators. Instead of using -brackets to define the comprehension, we use ()-parenthesis to define the generator "expression".
The example below shows a couple of genertor expressions and their use:
sentence = "The quick brown fox jumps over the lazy old dog." # List comprehension wordList = [word for word in sentence.split()] print wordList # Prints the list as a list for word in wordList: print word # Works as expected print wordList # Legal # Similar generator expression wordGenerator = (word for word in sentence.split()) print wordGenerator # Prints nothing useful for word in wordList: print word # Works as expected # print wordGenerator # Syntax Error
One of my favorite features of generators is the ability to use them in a pipeline for filtering. The example below does several steps in a pipeline: parses words, removes periods from them, upper cases them, numbers them:
sentence = "The quick brown fox jumps over the lazy old dog." words = (sentence.split()) #for word in words: print word wordsNoPeriods = (word.replace(".","") for word in words) #for word in wordsNoPeriods: print word wordsUpper = (word.upper() for word in wordsNoPeriods) #for word in wordsUpper: print word def numbersGenerator(): number = 0 while True: yield number number = number+1 numbers = numbersGenerator() numberedWords = (str(numbers.next()) + ":" + word for word in wordsUpper) for word in numberedWords: print word