Easy Python Event Async Revisited: Efficient Concurrent Programming Unveiled

Introduction

Sometimes, when your program has a task which takes a lot of time, like working with databases, web services, or complex calculations, you might want to let it run in the background. This way, your program or to be more precise the main thread, can keep running smoothly without waiting for the time-consuming task to finish. In Python, we can achieve this using threads. This article explores a straightforward example involving a virtual windowing system.

Implementation in Python

Before we start we need to import four packages:

import threading
import queue
from typing import Callable, Optional, Tuple
from dataclasses import dataclass

Line by line:

1. The threading module provides tools for creating and managing threads allowing your program to perform multiple operations simultaneously.
2. The queue module provides multi-producer multi-consumer queues. These are used for exchanging data between threads in a safe way. These queues handles the locking necessary for thread synchronization internally.
3. The typing module provides two type hinting constructs. The first is Callable which allows annotating function which can be called with a certain input and certain return types. The second is Optional which indicates that a value can be of a certain type or None.
4. The last one is the dataclass module which imports the dataclass decorator. Applying this decorator to a class, it examines the class variables, and creates a proper initialization method, and other supporting functionality.

The ResizeEvent just consists of a new width and height, with their respective accessors:

@dataclass(frozen=True)
class ResizeEvent:
    width: int
    height: int

The ResizeEvent is a clean immutable container (frozen=True) which stores a width and a height value.

Next we need a type to handle the events, this could also be used in more complex cases to filter events, or even transform. The ResizeEventHandler has a function handler which handles the event. In our case, the event is neither filtered nor transformed, but passed as it is to the handler:

class ResizeEventHandler:
    _handler: Optional[Callable[["ResizeEvent"], Tuple[int, int, Optional[str]]]]

    def __init__(self,initial_handler):
        self._handler = initial_handler

    def handle(self, event: ResizeEvent):
        if self._handler:
            return self._handler(event)
        return None, None, 'No handler defined'

Line by line:

1. The _handler instance variable which can be either:
   • A callable which takes a ResizeEvent parameter, and returns a type of (int, int, Optional[str])
   • Or None when no handler has been provided.
2. The forward reference “ResizeEvent” which is in quotes is used because the ResizeEvent class is already defined in this file, but Python’s type checker needs this syntax to reference a class which might not have been full processed at this point.
3. The constructor __init__ stores the provided handler for later use.
4. The handle() method has one parameter, which is of type ResizeEvent.
   • If a handler has been provided, the handler is called and the result is returned.
   • If no handler is available, an error tuple is returned.

Now implement a ResizeEventListener to make sure the events which are sent to the window are handled by the appropiate handler.

The code:

class ResizeEventListener:
    _events: queue.Queue
    _handler: ResizeEventHandler

    def __init__(self, init_handler):
        self._events = queue.Queue()
        self._handler = ResizeEventHandler(init_handler)
        self._thread: Optional[threading.Thread] = None

    def start(self,initial_window):
        self._thread = threading.Thread(target=self._run, args=(initial_window,), daemon=True)
        self._thread.start()

    def _run(self, initial_window):
        while True:
            event = self._events.get()
            try:
                if event is None:
                    break
                
                new_width, new_height, error = self._handler.handle(event)
                if error:
                    print(f"Error handling resize event: {error}")
                    break
                
                initial_window.width = new_width
                initial_window.height = new_height
            finally:
                self._events.task_done()

    def stop(self):
        self._events.put(None)
        if self._thread is not None:
            self._thread.join()

    def send(self, event: ResizeEvent):
        self._events.put(event)

Line by line:

1. At initialization the listener creates an empty event queue, and a handler wrapper in a ResizeEventHandler. The handler is a Callable and it processes the resize events on the queue. It returns the updated size, or in some cases an error. When initialized the listener does not start processing immediately, as the thread reference is set to None. Only in the start() method does processing begin.
2. When start() is called a daemon thread is creates a daemon thread which runs the _run() method. Why a daemon thread? Having a daemon thread ensure it will not prevent the application from exiting.
3. The _run() method which is heart of the event handling mechanism constantly pulls events from the queue in an infinite loop using the get() method. If no events are available the call to this method blocks, but since the _run() method runs in a separate thread, this does not affect the main program. The try block is used to make sure the task_done() method is called for every element in the queue, even if an error has occurred. The main reason for this is to keep the queue’s internal counter consistent, as a call to this method decrements this counter.
4. The stop() method sends a None value to the queue which will terminate the event loop. After which, it waits for the thread to terminate.
5. The send() method adds a ResizeEvent to the queue.

Next we need to define the Window class. The Window class has a title, width and height, but also a resize listener, which is initialized in the constructor.

class Window:
    _listener: ResizeEventListener
    _title: str
    _width: int
    _height: int

    def __init__(self, title: str, width: int, height: int, 
                 resize_handler: Optional[Callable[[ResizeEvent], Tuple[int, int, Optional[str]]]] = None):
        self._title = title
        self._width = width
        self._height = height
        self._listener = ResizeEventListener(resize_handler)
    
    def open(self) -> None:
        print(f"Opening window '{self._title}' with size {self._width}x{self._height}")
        self._listener.start(self)
    
    def close(self) -> None:
        self._listener.stop()
        print(f"Window '{self._title}' closed.")
    
    @property
    def width(self) -> int:
        return self._width
    
    @width.setter
    def width(self, value: int):
        self._width = value

    @property
    def height(self) -> int:
        return self._height
    
    @height.setter
    def height(self, value: int):
        self._height = value
    
    def resize(self, width: int, height: int) -> None:
        print(f"Resizing window '{self._title}' to {width}x{height}")
        self._listener.send(ResizeEvent(width, height))

Line by line:

1. A Window object in this set up has four properties: a width and a height which both are integers, a title which is of type string, and a listener which is of type ResizeEventListener
2. In the constructor these values are set, and the provided handler is wrapped inside ResizeEventListener object. Note the type of the resize_handler parameter. This is an Optional which means that a resize handler does not have to provided.
3. The open() method prints a message to the screen, and starts the resize event listener.
4. Conversely the close() method stops event processing immediately and prints a message to the screen.
5. Then there are several property definitions so the clients of this class, like the ResizeEventListener can access and change them.
6. Finally there the resize() method itself which receives a new width and a new height, and sends these wrapped in a ResizeEvent to the listener.

Time to test

We will start our test by setting up a handler function, which we pass to the window constructor.

Then we open the window and resize it several times, before closing it, and printing out the final width and height.

if __name__ == '__main__':
    def handler(event: ResizeEvent):
        print(f"Window resized to {event.width}x{event.height}")
        return event.width, event.height, None


    window = Window("My Window", 800, 600, handler)
    window.open()
    window.resize(1024, 768)
    window.resize(644, 484)
    window.resize(800, 600)
    window.close()
    print(f"Height is {window.height}")
    print(f"Width is {window.width}")

Conclusion

Using multithreading this way in Python was surprisingly simple, although it took me some experimentation to get things right. Possible enhancements could be to make the eventhandler more generic and more robust.

As you can see dispatching calculations to the background in Python can be both powerful and simple to implement. It is however fair to say that this is a deliberately simple example. In a next blog post I will discuss a somewhat more involved example.