Modern Java Concurrency

• First and foremost... I am very under prepared!

• Consultant

  • Scala

  • Java

  • XQuery, XSLT

• Open Source Hacker

  • Predominantly NoSQL Database Internals

  • e.g. eXist, RocksDB, Shadoop (Scala Hadoop M/R framework)

• W3C Invited Expert for XQuery WG

• Author of  "eXist" book for O'Reilly

• ...Recently moved to Bristol!

• Kind of a whirlwind tour...

  • May introduce many concepts and topics

  • Little detail... lots for you to research later

  • Scare you with JVM and Java SE concurrency

  • Calm you with other concurrency options

1. Concurrency Fundamentals

2. Concurrency on the JVM

3. Modern Java SE Concurrency

4. Better Concurrency Options

1. Do some basic multi-threaded stuff

   • Appears to work

   • Think concurrency is simple ☺

2. Hear other people complaining concurrency is hard

   • Don't really believe them

   • TMy experience was positive... I must know more!?!

   • Blissful ignorance ☺☺

3. Fix some deadlock/livelock bugs

   • Feel like a concurrency ninja ☺☺☺

4. Learn and read more about Java Concurrency in depth

   • The less we feel we know ☹

   • The more insecure we feel ☹☹

   • Concurrency is hard! ☹☹☹

5. Search for higher level abstractions ☯

   • Feel much happier

   • Be more productive

   • Do not be ignorant of the hard stuff!

• What is concurrency?

  • Several computations are executing simultaneously

  • ...and possibly interacting with each other!

• Why do we need concurrency?

  • CPU Clock Speeds are not increasing

    • Often decreasing to save power!

    • Many small cores

  • Need to make use of many compute cores

    • Project Sumatra - GPU from Java

    • Sparc T-5, / Tilera / Azul Vega 3 / Compute clouds etc

  • Not linear speed, rather throughput

  • Not just speed but also isolation and containment

• Why do we need concurrency?

  • CPU Clock Speeds are not increasing

    • Often decreasing to save power!

    • Many small cores

  • Need to make use of many compute cores

    • Tilera (64 cores)

    • Oracle Sparc T-5 (128 cores)

    • Azul Vega 3 (864 cores)

    • Project Sumatra - GPU from Java

    • Compute clouds etc

  • Not linear speed, rather throughput

  • Not just speed... also isolation and containment

• Process vs. Thread

  • Processes

    • Managed by the OS

    • Own resources (file handles, memory, etc)

    • Have their own address space

    • e.g. Java

  
  

  • Threads

    • Within a process

    • Lightweight compared to process

• Executing independent Tasks is simple

• Parallel decomposition of existing linear workloads is hard

• Mutable shared state is hard

• Task/Data access coordination is hard

  • Consistency vs Eventual Consistency

  • Most likely requires synchronisation

• Critical Section

  • Code that accesses a shared resource that must not be concurrently accessed by more than one thread

  • Requires synchronising thread access!

• Mutual Exclusion

  • Only 1 process concurrently within the same critical section

  • Semaphore, Binary or Counting

  • Mutex, has an owner

  • Lock / Spinlock / Reentrant lock / Readers-writers lock

• Caused by interleaved access from multiple threads

• Issues may rarely manifest themselves

• When they do manifest, typically under worst conditions

  • i.e. heavy production load

• Caused by a lack of correct synchronisation

• 2+ threads dependant on a mutable resource

• The result is sometimes not as expected

Consider the function:

private int value = 0;

public int getNext() {
    return value++;
}

A: No! Because....    value++

• Not an atomic operation!

• Sugar for 3 operations: get, add and set

• 2+ threads holding and trying to acquire opposing locks

• Cyclic locking dependency between threads

  • Directed graph: Nodes are threads, Edges are access relationship

Consider the functions:

public void leftRight() {
    synchronized (left) {
        synchronized (right) {
            doSomething();
        }
    }
}
    
public void rightLeft() {
    synchronized (right) {
        synchronized (left) {
            doSomethingElse();
        }
    }
}

• Q: No! Deadlock prone!

• Tools/Solutions

  • Open Calls

    • Calling an (object) method with no locks held

    • Can help by encapsulating locking

  • Lock ordering

  • Deadlock detection, abort one thread (and retry?)

• Often comes from overeager error-recovery code

  • Mistakes Unrecoverable error as Recoverable and retries

• Two threads change their state in response to each other

  • e.g. Two polite people walking down a hallway, try to get out of each others way...and again... and again...

  • Can be mitigated by introducing some entropy into the action or delay

• java.lang.Runnable

  • Something that can be executed

  • Possibly in a Thread

• Threads

  • java.lang.Thread

    • (now) mapped to OS/Kernel threads

    • Daemon or Non-Daemon (user) threads

  • java.lang.ThreadGroup

• Explicit

  • java.util.concurrent.locks

    • ReentrantLock

    • ReentrantReadWriteLock

      • Supports downgrading

      • Manual lock upgrading is deadlock prone!

    • StampedLock (Java 8), Not Reentrant!

• Implicit

  • synchronized keyword

    
    

    • Used on methods, acquires the objects intrinsic lock

      public synchronized setValue(final int value)

      
      

    • On referenced object, acquires intrinsic lock on specified object

      synchronized(thing){ ... }

      
      

    • Is Reentrant!

• What does the volatile keyword do?

• What issues may occur with escaping references in constructors?

• What is the JMM (Java Memory Model)?

  • Why is it important for concurrency?

  • Why did it change for Java 1.5?

• JSR-133 Java Memory Model and Thread Specification Revision

  • Corrected and formalized the operation of synchronized and volatile

  • Detailed how immutable objects work with multithreading

    • Thread-safe provided that references aren't allowed to "escape" while the constructor is being executed

  • Enabled non-blocking coordination among threads through the use of volatile.

• Callable and Future

  • Only really small improvements over java.lang.Runnable

  • Win! Can obtain result from asynchronous computation without shared mutable state

  • Easier to coordinate Threads and their lifetimes

• java.util.concurrent.Callable

  • Similar to java.lang.Runnable, but...

    • allows you to return a result

    • allows you to throw a checked Exception

• java.util.concurrent.Future

  • The result of an asynchronous computation

    • e.g. At some point in the future there MAY be a result

  • ExecutionException wraps possible checked exception of Callable

  • Also provides synchronous (blocking) mechanism for getting result

• java.util.concurrent.ExecutorService

  • The glue between Callable and Future

  • Submit a Callable to an ExecutorService and you are given a Future

• java.util.concurrent.atomic

  • Lock-free

  • Thread-safe

  • Uses the CPUs CAS (Compare and Swap/Set) instructions

  • Provides atomic conditional update of a value/reference

• Fork/Join framework

  • The new old stuff!

    • i.e. really just an ExecutorService plus some glitter

  • TODO What sort of parallelism approach does this provide?

  • TODO Examples...

• java.util.streams

  • Collection.parallelStream

  • A different approach to parallelism, i.e. SIMD approach

  • Inspired by Scala's Parallel Collections???

  • TODO discuss issues around ordering and side-effects...

  • TODO Examples...

• java.util.concurrent

  • CompletableFuture

  • CompletionStage

  • TODO Examples...

• Mutable shared state is the real issue!

• (Correct) synchronised access is really hard!

• Threads and Locks are your primitives!

  • Sadly, Callable and Future offer little more

• Atomics are very helpful... but limited by types

• TODO check --> Fork/Join does not help with shared state!

• java.util.streams can help with SIMD like parallel processing of data

• Higher-level abstractions

• Containment of Shared State

• Correct protected access to Mutable Shared State

• Concurrency where deadlocks are impossible

  • Actually, why should I even have to care about locks?

• I want to: focus on what rather than how!

• It's a Philosophy towards how you compose both your application and larger systems

• Ah! We are going to need more asynchronous concurrency :-/

• Responsive

• Resilient

• Elastic

• Message Driven

p.s. The "Principles of Reactive Programming" online course with Coursera starts again on April 13th 2015

- https://www.coursera.org/course/reactive

So, if we have to do concurrency, what better options exist?

• ReactiveX

  • Based on Observables

  • Like Iterators but for Concurrency

• Akka

  • Based on Actors

• TODO others?