Understanding Python Generators, Yield/Yield from, and Coroutines (asyncio)

Because of the Global Interpreter Lock (GIL), Python's multithreading performance can be worse than single‑threaded execution, which leads to the need for coroutines.

GIL (Global Interpreter Lock) synchronizes threads so that only one thread runs at a time, even on multi‑core CPUs.

Coroutines (also called micro‑threads) allow a function to be paused and resumed without the overhead of thread switching, making them ideal for I/O‑bound tasks.

Before Python 3.4, coroutine support required third‑party libraries such as gevent or Tornado; from 3.4 onward the built‑in asyncio library provides native coroutine functionality.

Python implements coroutines using generators, so understanding generators is a prerequisite.

Generators

We focus on the yield and yield from expressions, which are closely related to coroutine implementation.

Python 2.5 introduced yield (PEP 342).

Python 3.3 added yield from (PEP 380).

When a function contains a yield expression, Python treats it as a generator object rather than a normal function.

<code>def test():
    print("generator start")
    n = 1
    while True:
        yield_expression_value = yield n
        print("yield_expression_value = %d" % yield_expression_value)
        n += 1

# Create generator
generator = test()
print(type(generator))

# Start generator
next_result = generator.__next__()
print("next_result = %d" % next_result)

# Send value to yield expression
send_result = generator.send(666)
print("send_result = %d" % send_result)</code>

Output shows the generator type, the first yielded value, and the value sent back after resuming.

The __next__() method starts or resumes a generator (equivalent to send(None) ), while send(value) sends a value to the yield expression.

Producer‑Consumer Model with Coroutines

<code>def consumer():
    print("[CONSUMER] start")
    r = 'start'
    while True:
        n = yield r
        if not n:
            print("n is empty")
            continue
        print("[CONSUMER] Consumer is consuming %s" % n)
        r = "200 ok"

def producer(c):
    start_value = c.send(None)
    print(start_value)
    n = 0
    while n < 3:
        n += 1
        print("[PRODUCER] Producer is producing %d" % n)
        r = c.send(n)
        print('[PRODUCER] Consumer return: %s' % r)
    c.close()

c = consumer()
producer(c)</code>

The producer sends values to the consumer via yield , and the consumer returns results back, all within a single thread and without locks.

Asyncio Coroutines

Python 3.4 introduced @asyncio.coroutine and yield from to implement coroutines; Python 3.5 added the syntactic sugar async / await (PEP 492).

<code>@asyncio.coroutine
def compute(x, y):
    print("Compute %s + %s ..." % (x, y))
    yield from asyncio.sleep(1.0)
    return x + y

@asyncio.coroutine
def print_sum(x, y):
    result = yield from compute(x, y)
    print("%s + %s = %s" % (x, y, result))

loop = asyncio.get_event_loop()
print("start")
loop.run_until_complete(print_sum(1, 2))
print("end")
loop.close()</code>

Running this prints the start message, the computation, the result, and the end message. The @asyncio.coroutine decorator will be removed in Python 3.10.

With async / await , the same logic becomes more readable:

<code>async def compute(x, y):
    print("Compute %s + %s ..." % (x, y))
    await asyncio.sleep(1.0)
    return x + y

async def print_sum(x, y):
    result = await compute(x, y)
    print("%s + %s = %s" % (x, y, result))

loop = asyncio.get_event_loop()
print("start")
loop.run_until_complete(print_sum(1, 2))
print("end")
loop.close()</code>

Future Example

<code>future = asyncio.Future()

async def coro1():
    print("wait 1 second")
    await asyncio.sleep(1)
    print("set_result")
    future.set_result('data')

async def coro2():
    result = await future
    print(result)

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait([coro1(), coro2()]))
loop.close()</code>

The first coroutine sets a result after a delay; the second coroutine awaits the future and prints the result, demonstrating how await pauses execution until the future is resolved.

Overall, this note provides an introductory overview of generators, the yield / yield from mechanics, and how they form the basis for modern asynchronous programming in Python using asyncio , async / await , and futures.