Best of APIs: Non-Blocking Functional Programming!

Recently I took it upon myself to refactor some existing APIs into 100% Non-Blocking, where the CPU never sits around waiting for I/O to finish.  As it turns out, this was only as hard as converting our data access layers into async calls.  From that layer up, I’m essentially converting their results into a Java Future, similar to a Promise in Javascript.  For those unfamiliar with the depths of Functional Programming, Futures and Promises act as Monads.  A monad is a sort of generic wrapper that your functions can operate on, as opposed to the specific data inside the wrapper.  With this in mind, I needed only 3 generic, monad-inspired functions:

join(A, B…)  : Join 2 or more Futures A and B, such that the completed result is a Future containing a list of all inner Futures’ completed results

chain(A, F) : After Future A completes, apply function F to the result, where F returns yet another Future that is a function of A’s result.

map(A, F) : After Future A completes, apply function F to the result, where F runs synchronously and immediately transforms A’s result.

It took a bit of Java multi-threading magic to adapt 3 different MongoDb / Ignite / Spring Future classes to a single monad interface, but with that I was able to mix them all together using just the 3 axiomatic functions. The result was clean, high performance code that does things in parallel when it can, never makes the CPU sit around for net requests, and essentially provides the best performance you could possibly squeeze out of an API, Java or otherwise.  If you force yourself to only use your data layer’s async queries, this also naturally encourages you to write immutable, referentially-transparent Functional code in your biz logic layer.

Simplified Example – Old Blocking API

Let’s examine what our old API implementation does.  For a bit of fun, I’ll take a simplified but non-trivial “Get Valid Recipients for Messaging” endpoint.  You need to get everyone in your friend list, but strip out those who have muted you:

1. Allocate a thread for the request
2. Load my entire friend id list, in batches
3. For every friend, do a O(1) lookup to see if they muted you, remove if so
4. Return remaining friend Ids
5. Release request thread to server pool

Wow, in retrospect, this is pretty bad.  Problem #1 here is that we load the entire friend list before checking mutes, this is a big waste of time.  Problem #2, 99% of the time the 5 steps are running, the request thread is just sitting there, doing nothing but waiting for I/O.

Not only can we easily remedy these problems with the 3 monad functions, but you’ll notice how much cleaner the code looks (maybe not in my pseudocode, but in practice!) given how much faster it’s running.  I often find the most efficient code is uglier, not here!

Simplified Example – New Non-Blocking API

The core of this is an asynchronous recursive function (I’m not just trying to sound cool).  I’ll stay high level and skip over a lot of lower level details.

Let’s define a function F(A), where A is a completed Future containing a set of friend ids:

1. If A is empty, return a completed Future containing an empty list.
2. Fetch M, a Future containing a list of Mute records matching the ids in A.
3. Let V be an empty list to represent the Valid a’s in A that have not muted me.
4. Apply map(M, lambda), where lambda is a closure that loops through and stores all A into V, except any a’s that have muted me in M.  When lambda is created, it captures the reference to the completed A Future but waits for M 🙂 .  Save this lambda output as Future O (for Output).  O will contain a final batch of filtered friend Ids after it’s lambda finishes.
5. Fetch a new Future A2, representing the next page of friend Ids after A.
6. Have F(A) recursively return join(O, F(A2))!!!  <– (Kind of cool IMO)

Now lets see how the whole API runs.  The actual execution order will flow like this, where the green nodes are the inner lambdas running:

1. Allocate a thread for the request
2. Fetch Future A, that will contain the first batch of friend Ids
3. Return chain(A, F)! It will eventually contain a list of Output batches; the entire list of your valid recipients!
4. Throw a map() around #3 to flatten the list of batches into a single flat Id list.
5. Release the request thread before #2 even completes!
6. Whenever a scheduled Future above completes, allocate a thread to run either F or F’s inner lambda, then immediate release the thread again.
7. Once the entire “computation graph” that was built by gluing Futures together completes, allocate a thread to write the response back to the client.

Takeaway

It’s a bit mind-boggling to imagine the countless hours computers have sat in bondage, doing nothing more than waiting for net requests to finish.  If CPU is even remotely a scalability factor for you, non-blocking code will save you oceans of time!  Non-blocking is quite powerful.  Indeed, this is the only real way to build a scalable service in a single-threaded environment like Node.js.  In my humble opinion, if you have a (deep breath) compiled, unit tested, type-safe, multi-threaded, non-blocking API, you’re pretty much at the top of your API game!  My takeaway tip: focus on converting your entire data layer (caches, DBs, etc) into a universal Future monad interface, so that all your biz logic does is mix monads with a handful of utility functions like the 3 above.