The relationship between concepts like concurrency, multi-threading, multi-processing, asynchronous programming and event loop can be confusing. Especially what techniques can achieve concurrency. This post is an attempt to explain them in a simple and visual way.
First of all, concurrency is a concept that one has the ability to handle multiple tasks out-of-order. Here out-of-order is the key, it doesn’t mean they are handled at the same time, and it doesn’t mean they are started or ended at the same time. This captures a wide variety of techniques that can achieve concurrency, including multi-threading, multi-processing and asynchronous programming.
Now let’s look at what are multi-threading and multi-processing. I group them because they are similar compare to asynchronous programming and event loop. It’s pretty clear from the words that they allow to use multiple threads or processes to execute tasks at the same time. They are different techniques to achieve concurrency and what’s the difference is out of the scope of this post. You can refer to reference likes to find out the detail.
Asynchronous programming is a technique to defer certain tasks to a later time and allows the main thread to handle other tasks. The most common example is asynchronous I/O used by node.js. In node.js, it hands off I/O operation to OS kernel and defers the callback of I/O operation to a later time. This allows node.js to process other tasks immediately after the I/O operation is handed off to OS and doesn’t need to wait until the OS finishes the I/O task. This is the core part of concurrency of node.js. As you may know, Javascript and node.js are single-threaded. This shows you that even a single-threaded system can have concurrency. Some people may argue that asynchronous programming doesn’t provide concurrency, this depends on what’s the definition of task. If people define callbacks as independent tasks, then they are correct. Because as you will see in the diagram later when we talk about event loop, callbacks are nothing more than tasks that being pushed to an event queue. In this definition, tasks are not executed out-of-order. Conventionally, most people treat callbacks as a part of the original task that is deferred to be finished later. So they consider this task is broken into several parts and executed out-of-order among other tasks, which made such system concurrent. I agree with the statement that asynchronous programming provides concurrency.
Finally let’s look at event loop. Event loop is a design pattern that waits for and dispatches events in a system. When it is used with techniques like Asynchronous programming and multi-processing, it also provides concurrency to the system. And event loop is typically constructed with a queue(for example FIFO queue), a loop and workers. Let’s look at a simple event loop that only has one worker shown in the picture. Events are pushed to the event queue, and event loop pulls event from the event queue and dispatches them to the worker. In this case, let’s assume the queue is FIFO queue, in such a system, events are processed one by one without concurrency. If we add Asynchronous programming to the picture, as you can see the line at the bottom, the worker adds the callback event back to the event queue. This provides concurrency even though there is only one worker or thread. This is what happens when you call setTimeout(myCallback, 100). Behind the scene, the worker pushes the myCallback to the event queue after 100ms. Now you understand why myCallback passed to setTimeout() won’t be called exactly after 100ms? It’s because there might be other events in the queue that need to be processed before myCallback. The 100ms is the minimum time myCallback needs to wait until it’s executed. Now you also understand why people use setTimeout(myCallback, 0). This doesn’t mean myCallback will be called after 0ms meaning immediately. This is a technique to free the main worker(thread) to handle other events in the queue and defer myCallback to a later time.
Event loop can also use multi-threading. One can simply add more workers and a thread poll in the system as shown in the picture below. The only difference is now multiple events can be pulled from the event queue and handled at the same time by different workers in different threads.
One can be very creative and extend the event loop as they see fit. Using node.js as an example, it divides the entire event loop into multiple phases, and each phase itself is like a smaller event loop. If you want to know more, refer to the reference at the end.
References:
Concurrency
Event Loop
Multi-threading
Multi-processing
Node.js Event Loop