Clojure utility functions – part II

Cross-posted from Zolo Labs


Here’s another useful function I keep around:

(defn pmapcat [f batches]
(->> batches
(pmap f)
(apply concat)

view raw
hosted with ❤ by GitHub

Everyone knows what map does, and what concat does. And what mapcat does. 

The function definition for pmapcat above, does what mapcat does, except that by using pmap underneath, it does so in parallel. The semantics are a bit different: first off, the first parameter is called batches (and not, say, coll, for collection). This means that instead of passing in a simple collection of items, you have to pass in a collection of collections, where each is a batch of items. 

Correspondingly, the parameter f is the function that will be applied not to each item, but to each batch of items.

Usage of this might look something like this:

(defn handle-batch [batch]
(blah blah…))
(->> coll
(partition-all n)
(pmapcat handle-batch))

view raw
hosted with ❤ by GitHub

One thing to remember is that pmap uses the Clojure send-off pool to do it’s thing, so the usual caveats will apply wrt to how f should behave.

Clojure utility functions – part I

Cross-posted from Zolo Labs.


I kept using an extra line of code for this, so I decided to create the following function:

(defn domap [function coll]
(->> coll (map function) doall))

view raw
hosted with ❤ by GitHub

Another extra line of code can similarly be removed using this function:

(defn doeach [function coll]
(doseq [c coll]
(function c)))

view raw
hosted with ❤ by GitHub

Obviously, the raw forms (i.e. using doseq or map) can be far more powerful when used with more arguments. Still, these simple versions cover 99.9% of my use-cases.

I keep both these (and a few more) in a handy utils.clojure namespace I created for just such functions.