How does Windows allocate CPU time?
CPU (Central Processing Unit) time is incredibly valuable. It determines how much and how fast code can be executed (in the context of length of time in regards to amount). Because of this, the operating system has to be conscious of how it split processing time between the different programs and services that are running. In order to make this a reality, various structures are needed to represents the executing code.
Applications – Processes – Threads
An application consists of one or more processes. A process is an executing program. One or more threads run in the context of the process. A thread is the basic unit to which the operating system allocates processor time. A thread can execute any part of the process code, including ?parts currently being executed by another thread. All threads in a process share the same memory and system resources. Each CPU core can execute one thread at a time, so multi core systems can execute as many threads in parallel at the same exact time as the amount of CPU cores available.
To recap, a threads make up a process which makes up an application. The simplest application will have one process consisting of one thread. Multi threaded applications can have multiple processes with multiple threads.
Thread Pools
Creating and terminating a thread is very time consuming. Memory allocation, thread management, and processing time are all involved in handling multiple threads. The more threads an application has, the more of these resources it will consume.
Some applications are written to take advantage of multiple threads and run processes (not windows process, but a literal one) in parallel. Some of these processes are very short lived and make the cost of creating and terminating the thread very expensive. In a way, the ROI is not high enough.
In order to make multi threading less expensive resource wise, thread pools were introduced. Thread pools are collections of worker threads that handle processes on behalf of the main thread.
A good analogy to this would be a restaurant. A new waiter (thread) would have to be hired every time a customer walked in the door. The waiter would take the order, bring the food, charge the fee, and do other tasks. When the waiter would be done serving that customer, they would be fired (thread terminated) and a new waiter hired for the next customer. This is incredibly inefficient if there are many customers and they spend very little time in the restaurant. A thread pool would be like having a staff of waiters ready to serve new customers as they walked in.
Process & Thread Priority
The operating system dictates how much processing time is given to each process based on the process priority. In windows, you can change the priority of a running process by using the Windows Task Manager and the processes tab. Similarly, processes can change the priority of the treads that make it up. By assigning a higher priority, that particular thread would get more processing time allocated to it.
A good use of this would be in an application that has heavy background processing. The thread responsible for the UI should have a higher priority then the thread which is running background processes. This would ensure that the application is responsive to the user input.
Memory Sharing
While multiple threads can execute the same exact code at the same time, they can not read and write to the same memory space. Safeguards are built in to ensure that memory access by multiple threads don’t overlap. Race conditions are one of the biggest problems in multi threaded applications. When one thread starts reading or writing to a piece of memory, other threads have to wait for it to finish.
An example problem would be two threads needing to read the same memory. Thread A starts reading some memory, but can’t finish until thread B sends a value back that thread A needs in order to finish its calculation. Thread B, in order to send the value back, needs to read the memory that Thread A has locked. Both threads are now waiting for each other and are in a deadlock.
Thread Timing
Windows splits up processing time between threads based on priority. As each thread gets a given amount of clock cycles to process. If the thread finishes processing before the time is up, it returns the processing time to the OS. The extra time is then added to the pool of time and split among the other threads. If after every thread has processed there is extra time left over, the time is allocated to the System Idle Process that can be seen with Windows Task Manager.
Multi Threading Considerations
Problems such as the deadlock and race conditions described are some of the challenges that face multi thread applications. Taking advantage of powerful systems with multiple cores capable of running multiple threads at once increases processing speed, but developers have a lot more of challenges to work with. The timing or threads, the work they do, and the memory they share are all thrown into the equation when designing multi threaded applications.
HAPPY CODING!
