This content originally appeared on DEV Community and was authored by Anik Sikder
Chefs, Sous-chefs, and Lazy Pipelines
By now you’ve met:
- The buffet (iterable: the source of food 🍱)
- The waiter (iterator: walks forward 🚶♂️)
- The autopilot waiter (generator: powered by
yield 🤖)
But real restaurants need teams sous-chefs, assistants, conveyor belts, and clever tricks to keep food moving.
Today we explore advanced iteration: itertools, yield from, generator delegation, cloning with tee, and lazy pipelines.
And this time, we’ll tie it to real-world mini-projects so you see the practical magic.
🍴 1. itertools → Kitchen Gadgets
Think of itertools as Python’s drawer of restaurant gadgets simple tools that make iteration effortless.
Fun Example: Endless Breadsticks
import itertools
for bread in itertools.count(1): # waiter counts forever
if bread > 5:
break
print(bread)
🍞 Output:
1
2
3
4
5
Behind the curtain:
-
countis just a tiny iterator with a counter. - Each
next()increments, no giant list needed.
🍟 Real-World Mini-Project: Streaming Logs
import itertools
def read_logs(filename):
with open(filename) as f:
for line in f:
yield line.strip()
# Look only at the first 5 errors lazily
logs = read_logs("server.log")
errors = (l for l in logs if "ERROR" in l)
for e in itertools.islice(errors, 5):
print("🚨", e)
👉 You can process gigantic logs without loading them all into memory.
🥡 2. yield from → Waiter Delegation
When a waiter has too many trays, he calls an assistant:
“Hey intern, serve these dishes for me.”
Example:
def menu():
yield 1
yield 2
yield from [3, 4] # delegation
Output → [1, 2, 3, 4].
Behind the curtain:
-
yield from iterableexpands into aforloop. - It even forwards exceptions and return values.
🍕 Real-World Mini-Project: Nested Config Loader
def load_base():
yield "db=sqlite"
yield "timeout=30"
def load_dev():
yield from load_base()
yield "debug=true"
print(list(load_dev()))
# ['db=sqlite', 'timeout=30', 'debug=true']
👉 yield from = clean delegation of tasks (like dev configs building on base configs).
🍝 3. Generator Delegation (Sous-Chefs)
Restaurants don’t have one chef for all courses. They delegate: appetizers, mains, desserts.
def appetizer():
yield "salad 🥗"
yield "soup 🍜"
def main_course():
yield "steak 🥩"
yield "pasta 🍝"
def dessert():
yield "cake 🍰"
def full_meal():
yield from appetizer()
yield from main_course()
yield from dessert()
print(list(full_meal()))
Output → ['salad 🥗','soup 🍜','steak 🥩','pasta 🍝','cake 🍰'].
🍜 Real-World Mini-Project: File Processing Pipeline
def read_lines(path):
with open(path) as f:
for line in f:
yield line
def parse_csv(lines):
for line in lines:
yield line.strip().split(",")
def filter_users(rows):
for row in rows:
if int(row[2]) > 30: # age > 30
yield row
def users_over_30(path):
yield from filter_users(parse_csv(read_lines(path)))
for u in users_over_30("users.csv"):
print(u)
👉 This is a real pipeline: one generator hands off to the next like sous-chefs in a line.
🥂 4. itertools.tee → Cloning Waiters
Normally: one waiter, one trip. If Alice eats, Bob gets an empty plate.
tee = photocopy the waiter so both can eat.
import itertools
it = iter([1, 2, 3])
alice, bob = itertools.tee(it)
print(list(alice)) # [1, 2, 3]
print(list(bob)) # [1, 2, 3]
Behind the curtain:
-
teebuffers values if one consumer lags behind. - Watch out: memory grows if consumers are very out of sync.
🍤 Real-World Mini-Project: Dual Consumers (Analytics + Logging)
orders = (f"Order {i}" for i in range(1, 6))
# Clone the stream
chef, cashier = itertools.tee(orders)
print("Chef prepares:", list(chef))
print("Cashier logs:", list(cashier))
👉 Same data stream, used by two different subsystems.
🍧 5. Lazy Pipelines → Conveyor Belts of Food
Now the coolest trick: combine everything into a conveyor belt where dishes flow step by step.
import itertools
nums = range(1, 1000000)
pipeline = itertools.islice(
(n**2 for n in nums if n % 2), # odd squares
5
)
print(list(pipeline)) # [1, 9, 25, 49, 81]
Behind the curtain:
- Each stage is lazy → nothing happens until
next()is called. - You can chain infinite data sources safely.
🍜 Real-World Mini-Project: Live Order Filter
import itertools
def infinite_orders():
n = 1
while True:
yield f"Pizza #{n}"
n += 1
orders = itertools.islice(
(o for o in infinite_orders() if "3" not in o), # skip unlucky 3s
5
)
print(list(orders))
Output:
['Pizza #1', 'Pizza #2', 'Pizza #4', 'Pizza #5', 'Pizza #6']
👉 Streaming infinite orders, but filtered + capped safely.
🎨 ASCII Mental Model
[ Buffet ] → [ Waiter ] → [ Gadget ] → [ Filter ] → [ Map ] → [ Eater ]
Think of it as a restaurant conveyor belt each chef only touches one dish at a time, dishes move lazily, no giant tray dumped all at once.
🎬 Wrap-Up
Today we upgraded from one waiter to a whole restaurant team:
- itertools: kitchen gadgets (count, cycle, chain, islice).
- yield from: waiter delegation.
- generator delegation: sous-chefs working in harmony.
- tee: cloning waiters with buffering.
- lazy pipelines: conveyor belts of infinite food.
👉 Next up (Part 4): Coroutines waiters who don’t just serve, but can also take your orders mid-service (send, throw, async/await).
This content originally appeared on DEV Community and was authored by Anik Sikder
Anik Sikder | Sciencx (2025-09-25T15:00:00+00:00) 🍲 Part 3: Advanced Iteration Tricks. Retrieved from https://www.scien.cx/2025/09/25/%f0%9f%8d%b2-part-3-advanced-iteration-tricks/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.