Iterators - Python

An iterator is an object i such that you can call i.next() with no arguments. i.next() returns the next item of iterator i  or, when iterator i  has no more items, raises a StopIteration exception. When you write a class (see "Classes and Instances" on page 82), you can allow instances of the class to be iterators by defining such a method next. Most iterators are built by implicit or explicit calls to built-in function iter, covered in iter on page 163. Calling a generator also returns an iterator, as we'll discuss in "Generators" on page 78.

The for statement implicitly calls iter to get an iterator. The following statement:

  for x in c:
      statement(s)

is exactly equivalent to:

  _temporary_iterator = iter(c)
  while True:
      try: x = _temporary_iterator.next()
      except StopIteration: break
      statement(s)

where _temporary_iterator  is some arbitrary name that is not used elsewhere in the current scope.

Thus, if iter(c) returns an iterator i  such that
i.next() never raises StopIteration (an unbounded iterator), the loop for x  in c  never terminates (unless the statements in the loop body include suitable break or return statements, or raise or propagate exceptions). iter(c), in turn, calls special method c.__iter__() to obtain and return an iterator on c . I'll talk more about the special method __iter__ in __iter__ on page 112.

Many of the best ways to build and manipulate iterators are found in standard library module itertools, covered in "The itertools Module" on page 183.

range and xrange

Looping over a sequence of integers is a common task, so Python provides built-in functions range and xrange to generate and return integer sequences. The simplest way to loop n  times in Python is:

  for i in xrange(n):
      statement(s)

range(x) returns a list whose items are consecutive integers from 0 (included) up to x  (excluded).
range(x, y) returns a list whose items are consecutive integers from x  (included) up to y (excluded). The result is the empty list if x  is greater than or equal to y . range(x, y, step) returns a list of integers from x  (included) up to y  (excluded), such that the difference between each two adjacent items in the list is step. If step  is less than 0, range counts down from x  to y . range returns the empty list when x  is greater than or equal to y  and step  is greater than 0, or when x  is less than or equal to y  and step is less than 0. When step  equals 0, range raises an exception.

While range returns a normal list object, usable for all purposes, xrange returns a special-purpose object, specifically intended for use in iterations like the for statement shown previously (unfortunately, to keep backward compatibility with old versions of Python, xrange does not return an iterator, as would be natural in today's Python; however, you can easily obtain such an iterator, if you need one, by calling iter(xrange(...))). The special-purpose object xrange returns consumes less memory (for wide ranges, much less memory) than the list object range returns, but the overhead of looping on the special-purpose object is slightly higher than that of looping on a list. Apart from performance and memory consumption issues, you can use range wherever you could use xrange, but not vice versa. For example:

  >>> print range(1, 5)
  [1, 2, 3, 4]
  >>> print xrange(1, 5)
  xrange(1, 5)

Here, range returns a perfectly ordinary list, which displays quite normally, but xrange returns a special-purpose object, which displays in its own special way.

Please check back next week for the continuation of this article.