⬅ Previous Topic
Python IteratorsNext Topic ⮕
Python Closures⬅ Previous Topic
Python IteratorsNext Topic ⮕
Python ClosuresWhen working with large datasets or continuous data streams, we often seek smarter ways to process information without gobbling up memory. That’s where Python Generators come in — they allow you to iterate lazily, computing values only as needed.
A generator is a special type of iterator in Python. It lets you generate values on the fly instead of storing them in memory like lists. You define a generator using a regular function but with a twist — the yield
keyword.
Let’s take a simple example to see generators in action.
def count_up_to(n):
count = 1
while count <= n:
yield count
count += 1
gen = count_up_to(3)
print(gen) # Just a generator object
for num in gen:
print(num)
<generator object count_up_to at 0x...>
1
2
3
Unlike regular functions, calling a generator function doesn't run it. Instead, it returns a generator object
. When used in a for
loop, it executes until the next yield
is hit — pausing there and resuming from the same spot in the next iteration.
Generators don’t compute values until you ask for them. This approach is known as lazy evaluation.
def infinite_numbers():
num = 0
while True:
yield num
num += 1
gen = infinite_numbers()
for _ in range(5):
print(next(gen))
0
1
2
3
4
This example shows how generators can theoretically run forever — only producing the next value when needed.
import sys
def gen_func():
for i in range(1000):
yield i
gen_obj = gen_func()
list_obj = [i for i in range(1000)]
print(sys.getsizeof(gen_obj)) # Lightweight
print(sys.getsizeof(list_obj)) # Much heavier
112
9024
Notice how the generator object is tiny in memory compared to the full list — even though they represent the same range of values.
next()
You can manually control a generator using the next()
function.
def greeting():
yield "Hi"
yield "Hello"
yield "Hey"
g = greeting()
print(next(g))
print(next(g))
print(next(g))
Hi
Hello
Hey
If you call next()
again after it's exhausted, it will raise a StopIteration
exception.
Much like list comprehensions, Python allows you to create generators in a compact form using generator expressions.
gen_exp = (x * x for x in range(5))
for val in gen_exp:
print(val)
0
1
4
9
16
They’re shorter to write and just as memory-efficient.
yield
doesn’t return data like return
. Use return
only when you want to terminate the generator early.from types import GeneratorType
def sample():
yield 1
print(isinstance(sample(), GeneratorType)) # True
True
Generators are a brilliant tool in the Python ecosystem. They let you write efficient, elegant, and scalable code. From data pipelines to game loops, they pop up in surprising places — once you embrace them, you'll wonder how you lived without them.
itertools
— many of its tools work seamlessly with generators.⬅ Previous Topic
Python IteratorsNext Topic ⮕
Python ClosuresYou can support this website with a contribution of your choice.
When making a contribution, mention your name, and programguru.org in the message. Your name shall be displayed in the sponsors list.