Concurrency in Java – Part 2, Object behavior heuristics

This post is the part 2 story in the Concurrency in Java – Series that touches the core concepts of concurrency in Java and provides a balanced view from the JVM memory model specifications as well as from the programmer's perspective. To see the list of posts in this series, please visit here.

----------------********----------------

In the previous post, we saw the following sample code that shows a typical use of synchronized thread safe idiom.

EmployeeManager2 is a simple example about how to make use of thread safe constructs without compromising on concurrency. However as the programs evolve in size, the problem of thread safe also increases exponentially. e.g. assume that we have to calculate an employee’s total CTC package and tax structures also and should be available for reading once the employee is loaded in the list.

One way to do it is to have the individual components of the CTC package and tax formulas within the employee object itself and when the employee is added to the list we do the calculation in the synchronized block and populate the variables with the employee accordingly. However this approach though still preserves the thread safety of the EmployeeManager but creates a problem for us by expanding the scope of the synchronized block and thus the blocking part of the add action and thus the performance of the manager.

Another approach would be to save the CTC and tax structure objects separately in another List in the manager and ensure that the CTC and tax objects are navigable by the employee name/id. This approach gives us a good performance heads up but then would again require us to synchronize on the other two lists. This approach also gives us an added advantage of allowing us to keep the “employee add” operation separate and mutually exclusive of the “CTC package add” and “tax structure add” operations, thus allowing us a provision to induce parallelism between the three.

A more preferred approach to do this, and essentially a variation of the second approach given above would be to keep the CTC and tax information within the employee itself but to just make their calculation actions parallel and mutually exclusive. Since we can retrieve the CTC and tax operations are tied to an employee by his or her name/id so provided an employee id we can calculate and mutate its CTC and tax structures in parallel while the manager serves its clients. This approach is not only thread safe but also performance efficient.

However, there is one small problem induced by our changing requirements (because our clients think they are privileged few and can do anything they want). The client now says that unless the tax structure is calculated and available for introspection, CTC should not be available for viewing even if it has been calculated and vice versa. This essentially means that unless both the operations of CTC calculation and tax structure calculation have finished successfully we cannot disclose the CTC and tax information to the outside world i.e. outside the EmployeeManager.

What our client requirement mandates us to do is convert a singular parallel operation into an atomic operation.

Atomicity

Atomic operations are a real world requirement and essentially a thread safety hazard because of inherent thread nature to incline towards race conditions. Simplest of these problems, in an increment or a decrement operation. E.g. a counter to keep track of number of requests served per day in the web service or a counter to track the number of login failure attempts for a user. A simple i++ operation is essentially composed of a set of 3 operations working together. Though we don’t see that in Java source but a disassemble operation can easily point out this misconception. Allowing us the easy convention of ++ operator does not mean that java internally also treats this as a single operation.

And this is what the above code actually translates into. We can verify that using the javap command tool with the disassemble flag.

Let's walk through this part to see would does this harmless code run.

A. Assembly operation 0. Does the initialization of a variable and pushes the initial value of 10 into the variable. The bipush JVM instruction set takes a single 32 bit length (int) value and pushes it on the operand stack.

B. Assembly operation 2. Pops the latest value in the operand stack and stores the value in a local variable. The variable name is ‘1’ and not i. Note that variable name in Java source code are just human readable denominations and not actually the ones used by assembly JVM machine set. In our example this instruction pops the value of 10 and stores in a memory address offset labeled as ‘1’ for it to read from later.

C. Assembly operation 3. Increments the variable. The iinc operator JVM machine set operator takes 2 parameters. 

• First parameter is the name of the assembly level variable whose value is to be incremented. In our case this variable name is ‘1’, derived from B. above.
• Second parameter is the value by which to increment which in our case is 1.

This operation saves the new value back in the variable after it increments (and is thus atomic in nature in terms of increment and save).

D. Assembly operation 6 and 9. This is the call to our static construct System.out.println followed by the call to load the assembly variable ‘1’ back onto the operand stack so that the System.out.printlnstatement call can read it.

The line number table in the figure actually shows which line in Java code maps to which line in disassembled code. Because of this problem the read, increment and get operations can interleave over each other in a multithreaded setup thus giving wrong results. This 'read, increment and get' instruction set is actually a type of compound operation at a fine grained level (byte code level) that needs to be made as a single atomic operation. Though in some cases this may be acceptable as most software give a fault tolerant guarantee to about 98-99% of operations but sometimes this may not be acceptable. It's far better to understand and fix such issues than leave them to luck. It would be a nightmare if by chance you bought your favorite expensive shopping item from an online checkout and invariably ended up paying for two or three pieces when you only opted for one.

Race Conditions

Compound operations create a situation called race conditions in a multi threaded setup. Such race conditions are not present in a single thread model but can only be seen when many threads work concurrently. Race condition occurs when the correctness of the computation of a Thread depends highly on its relative timing and its interleaving or scheduling characteristics relative to other threads. We may get lucky 95% of times and the 5% of times when it fails would probably be in a production environment.

Race conditions are not exactly the same as data races. Data races involve incorrect data access because of wrong data/coding semantics most of the times. When the programmer forgets to use proper synchronization blocks or leaks the state objects directly we end up with data races. Race conditions on the other hand happen largely because of the context switching choices that the underlying operating system or platform makes. Race conditions is essentially an undesirable by product of Operating Systems going advanced and efficient by employing more finer level parallelism at processor and multi core levels.

To solve our atomic increment problem in the previous section, we can employ the use of AtomicInteger class and fall back on its getAndIncrement() method to safely increment our variable without any further synchronization. AtomicInteger is one of the atomic operation classes introduced in the new java.util.concurrent.atomic package. It uses CAS operation semantics to achieve atomicity and durability in the increment operation. We shall discuss CAS operations and instruction set when we talk through our advanced session on concurrency.

Compound Operations

We saw about one type of race condition in our previous section and also how to fix that problem using an AtomicInteger. But what if we have two such counters or variables?

Consider the following code example.

The ConnectionPoolWatchDog1 has 2 values to keep track of. The number of connections left with it which it can give to its callers and the number of callers who have already borrowed a connection. We could track both of these with a single variable but then what happens if a connection is not borrowed and also not available for leasing out, probably because it’s dependent socket is blocked or non-responsive. So it’s safe to use two counters to track this.

Now these two counters (borrowers and connections) need to be incremented and decremented in one atomic operation. So we resort to an AtomicInteger here also. But then the atomic variable is only mutually exclusive in context of one operation done on it (increment or decrement). So even with using the atomic variables for both the counters we still cannot achieve a thread safe setup because its possible that while one thread is on line 15 decrementing for a borrow operation another thread may decide to return a connection and invoke the statement on line 24.

It is important to note that lines 15 and 26 are mutually exclusive as they both operate on the same variable (borrowers). So are lines 17 and 24. But together they are not, since they can interleave with each other and possibility of a race condition (and even a data race) exists. So to preserver the thread safety of this code we need to make sure that related variables are updated in a single indivisible atomic operation.

To solve this problem we have to rely on plain old locking idioms.

Locking

Locking is a mechanism to explicitly and intentionally make a certain part of code single threaded in nature even when it is run in a multi threaded setup. By employing locking we intentionally restrict multiple threads from accessing some code flow concurrently thus restricting the possibility of data mutation by multiple threads simultaneously. There are 2 types of locking idioms in Java:

• Intrinsic locks: These are also called monitor locks. This locking idiom relies on the JVM guarantee that only 1 thread can ever own a lock on an object. While the object is locked by a thread, all other interested threads wait for the lock to be opened. The se waiting threads are not allowed to pursue some other work while they wait. Until the active thread releases the lock all waiting threads suspend their working.

• Extrinsic locks: These are custom made synchronizers which can be used to influence the locking behavior in code. Java offers a custom java.util.concurrent.locks.Lock interface to implement more extensive locking operations. Extrinsic locking idioms allow more fine grained control over the operations, as against intrinsic locks, since the operation can be spread across and performed until the unlock action is called on the associated Lock object. Extrinsic locks are discussed in more detail in the advanced session.

Intrinsic locks are essentially implemented in java using the synchronized keyword. When used in the method signature it synchronizes the whole method against the lock of the ‘this’ object. When used as a block it synchronized the block code against the lock of the object passed to it as a parameter. ConnectionPoolWatchDog2 solves its thread safety problem by using a synchronized block and locking on the this instance as shown below.

Because the currently active Thread locks out other Threads from gaining access on the object on which synchronized is called out, we say that “the thread has locked the object” and not the other way round. It just the same corollary of booking a hotel room for a night. We book the hotel room and while we stay in that hotel room other customers have to wait it out till we relinquish the lock on the hotel room. A hotel room is like a code block that is synchronized. Only one customer (Thread) can stay in it (run through it) at a given time. We book the room with the hotel, we pay the hotel for the room not the other way round, just as we lock the object for access to synchronized block. When we are done with the room, we relinquish its lock (exit the synchronized block) and its then that others can see the state of the room and if needed book it.

It is important to understand that while a thread holds the lock, other threads waiting for that lock don’t see any changes made to the state by the active thread. This is true even if the active thread is more than the half way through with the synchronized block or even if its on the last executable statement in the synchronized block or method. Its only when the active thread exits the synchronized block or method, that the waiting threads come alive and see what changes actually happened.

However this is not true for threads waiting on different locks. If a Thread T1 is locked on Obj1 and is making changes to a List ‘employees’, threads that are blocking on the lock of Obj1 will not be able to see these changes until the thread T1 exits the lock. However for a Thread T2 locked on some other Obj2 or not locked on anything, can still see these changes as they happen. So while T1 is executing the employees list mutation changes inside the synchronized block T2 can see them all as they happen because its not blocked on the lock which T1 has acquired and if free to do what it wants. 

So if a synchronized block is needed to guard access to a variable or state, then it is important to ensure that all operations to that state everywhere in code must be guarded on the same lock provider or object. When a class has invariants that involves a tight coupling between more than one variable or state holder it is important to ensure that locks are owned consistently on the same object everywhere. Without this thread safety can be compromised.

Reentrancy

Reentrancy is actually an Operating System concept of making the processes and process shared stack reentrant to different processes for effective and block free IPC. Java also borrows this technique from there and implements this on finer grains object intrinsic lock mechanism.

When a thread obtains a lock on an object all other threads requesting that lock wait or essentially block on the object. When the original thread releases the lock other threads can acquire the lock again. This is because intrinsic locks are reentrant. Thus intrinsic locks are by nature acquired on a per thread basis rather than per request basis. This gives rise to a concept of reentrant synchronization. Because a thread can acquire a lock it already owns we call it reentrant synchronization. Without this a Thread would deadlock on itself.

The above code demonstrated the concept of reentrant synchronization. Below figure is the disassembled version of above code.

Lets walk through the sample output of disassembled code.

A. Shows the ACC_SYNCHRONIZED flag. This says that the method in question i.e. hello is a synchronized method. The JVM automatically enters the monitor on the this object when it encounters this flag.

B. This is a monitorenter instruction set. The execution thread enters the monitor on the variable popped at point C. in the diagram which happens to be our object. The LocalVariableTable shows the java object ref/variable mapping with the assembly variable mapping.

D. Shows the monitorexit instruction. After this instruction the monitor is again available for entry by any other thread.

Reentrancy facilitates the java OOP principle of encapsulation but at the locking mechanism level. Without reentrant behaviors threads can deadlock as the waiting threads would never get the change to own the lock once its already taken.

Conclusion

This post covered a lot of major areas of considerations that have a direct impact on how you design and implement concurrent code in Java. Having a clean picture of the internal concepts of how the code works and is transformed to its bytecode representation can help you write unshakable multi threaded apps. 

The next post in the shall discuss State Visibility Guarantees that you get from the JVM specification and JVM implementations. To check the previous post in the series please click here. To see the list of posts in this series, please visit here.

Happy Coding!! 👍