Asynchronous programming. A tale of reading from a file.

As most probably know, asynchronous programming is the thing that enables your scripts and programs to spend less time being blocked by I/O operations and spend more time doing more work. Because of Python's GIL, this is more important than ever. A lot of time is spend needlessly waiting for the network or the disk.

In this article, I am going to explain some concepts about asynchronous programming and explain broader concepts, rather than listing some Python syntax related to asyncio, in a manner that I hope will be easier to digest by absolute beginners. My goal here is to offer an analogy on how language components of a script work at a lower level when dealing with I/O operations in a regular blocking workflow and in an asynchronous context as well.

The synchronous way

Usually, a script or application does a lot of things. In order to do something useful, it has to interact with other things from the outside. Even though it does mostly computational bound things, at some point it's very possible it will need to interact with the file system or the network. But how does this interaction looks like at a lower level?

When your script/app wants to use operating system resources, it asks the OS, and then the OS responds. It's a very low level cycle of request-response interactions between the application specific primitives (defined in some OS libraries) and the kernel.

For example, when your script wants to read from a file, the interaction looks something like this:

  • Script: hey OS, I want to open the file "example.txt" to read from it (f = open("example.txt", "r")).
  • OS: Ok, lemme check if you can do that. OS checks the file permissions and the owner of the process making the request. If the user has the read permission on that file, a file handle is returned. If not, a request denied response is returned and the script raises a PermissionDenied error. Let's assume that the opening succeeded. The OS keeps track of that file handle until the script no longer needs it, knowing that the specific file handle signifies the script's access to the opened file.
  • Script: Neat. Now I want to read from it. Here is the file handle I got before, I want to read 1000 bytes please (f.read(1000)). I'll wait here doing nothing until you give me those 1000 bytes of content.
  • OS: Ok. Gimme a second. finds the file on disk, loads the content in the memory, returns the data to the process. Here it is.
  • Script: Alright. Thanks. I don't need access to that file anymore. Here is the handle, make sure you don't keep it in your books anymore (f.close())

Bonus, a little interaction with a bug that sometimes occurs when file handling is not done properly:

  • Script, 10 seconds later: I want to read from that file again. Here is the handle to the file, give me 1000 bytes again (f.read(1000))
  • OS: Well, that handle is not valid anymore because you closed it. Here is a ValueError: I/O operation on closed file.

We see that when reading was done, the script just waited doing nothing until the content was available. These request-response cycle between the script and the operating system are pretty slow.

Next, we will see what happens when the asynchronous programming model comes in.

The asynchronous way

The asynchronous model is becoming more and more widespread because it enables our programs to do more with less resources. When we parallelize our workload through processes, we waste memory by loading the process again and again in memory for each worker process we spawn. And the communication between different processes is heavy and relies on external resources as well that are blocking as well.

More and more programming languages adopt the asynchronous programming model that enable our single threaded scripts to do more work instead of waiting for the operating system to fulfill read/write requests. This is being done using some kernel capabilities named epoll which enables picking up responses as they become available.

So the interaction between the script and the operating system turns into:

  • Script: Hey OS, I want to read data from 1.txt and 2.txt please (we'll use two files to show how asynchronous reading is done between multiple sources.
  • OS: Ok, permissions seems fine, here are two file handles, H1 and H2. One for each file.
  • Script: Thanks. I want to read from 1.txt 1000 bytes. But I won't wait for it to finish. I suspend the current function and continue with executing another function until you give me the content.
  • OS: Ok, be right back with the content.
  • Script: In another function I want to read 1000 bytes from 2.txt. But again, I wont wait. I continue with another async function until you give me the content.
  • Script's first function and second function: Is any content available?
  • OS: for the first file I have those 1000 bytes, here they are. For the 2nd, not yet.
  • Script's first function: Ok, thanks, I'll continue doing from where I stopped.
  • Script's second function: Ok, I'll keep waiting.

... a while later, after the script's second function keeps asking for the contenet

  • OS: Ok here they are, your 1000 bytes.
  • Script's second function: Nice, thanks, now I can continue doing what I need to do.

We can see, that at a point, two functions ended up waiting for blocking input from the operating system (from the disk in this case). So from time to time, the respective points where the functions were suspended, the OS was being asked about the status of all blocking operations in a repeating pattern (most of the times called event loop). On each iteration, if an operation is completed, the code continues to execute until it's suspended again. Because of this pattern, a single thread manages to run much more code than it would could using regular blocking operations.

Of course, what happens under the hood is much more complicated and the complexity of the interactions between the script and the OS are more numerous and more complex. But I still hope that this simplification helps understanding the basics of asynchronous programming and what are the key differences between it and synchronous I/O operations.

Show Comments