Clojure Course - Collection Operations

Clojure Course

Part 4 - Collection Operations

The built-in data structures of Clojure: list, vector, set and hash-map are very powerful tools. Together they are sometimes referred to as edn (Extensible Data Notation). Clojurists tend to use them all the time to express data of any complexity, rather than define a new type or class for every possible abstraction, as it is usual in OOP languages.

Manipulating Collections

We have discussed that Clojure data structures are immutable, in fact they are called persistent data structures. To clarify this, it means that we cannot change them in-place, but there are plenty of tools to manipulate them, creating brand new data in the process.

We don't need to be too concerned about running out of memory though, because these collections are pretty smart. They are implemented as trees behind the scenes, and they have a feature called structural sharing. For example, if I add a new element to a vector that already has 100 values, a new vector is created, but the original 100 elements will be shared between the "old" and "new" versions. New memory is only occupied for the single value that is added.

Adding New Values

Now that we understand what it really means to "change" a data structure in Clojure, let's look at some examples.

The conj function (meaning conjoin) adds extra elements to a collection. We can add more than one value at the same time. It works with all collection types, but slightly differently.

(do
  (println (conj '(1 2 3 4)  5 6))
  (println (conj  [1 2 3 4]  5 6))
  (println (conj #{1 2 3 4}  5 6))
  (println (conj  {1 2 3 4} [5 6])))

For lists, the new value is appended to the front (beginning).
For vectors, the new value is appended to the tail (end).
For sets, the order is irrelevant, but the new value is added only if it doesn't exist in the set yet.
For hash-maps, we need to provide each new key-value pair in a vector.

Choosing Specific Elements

We have lots of functions available to us, that can slice and dice the data. Most of them will work with any collection type, sometimes even with native Java or Javascript data types such as ArrayList or Stack.

(let [lost-numbers [4 8 15 16 23 42]]
  (println "first" (first lost-numbers))
  (println "rest" (rest lost-numbers))
  (println "butlast" (butlast lost-numbers))
  (println "last" (last lost-numbers))
  (println "take" (take 3 lost-numbers))
  (println "drop" (drop 3 lost-numbers))
  (println "nth" (nth lost-numbers 3))
  (println "rand-nth" (rand-nth lost-numbers)))

first retrieves the first element, rest skips the first one and keeps everything else.
Same way we can get only the last value, or use butlast to exclude the last one.
take is often used with infinite sequences too, and limits how many elements to use from the collection. Its opposite is drop that ignores a certain number of values from the beginning of the collection, and only keeps the rest.
nth can be used to get one value at a specific index. Indexing starts with 0.
rand-nth just picks one value at random.

These are only a few basic examples, there are much more in the standard library. To see all available built-in functions, check the Clojure Cheat Sheet.

Processing Entire Collections

Concepts from functional programming have already infiltrated most of the mainstream languages. The filter, map and reduce functions are fundamental tools to process all elements of a collection, and these days they can be found in some form in Python, JavaScript, Java (Stream), C# (LINQ) and even in C++.

Filter

Processing all values in a collection, filter will retrieve only those that meet a certain condition, defined by the predicate (test) function. The test is applied to each element, and those where the test returns false, are discarded.

(filter odd? (range 20))

Of course we can use an anonymous function as predicate too.

; Take only the short words from a sentence
(let [sentence "People want to read short words only"]
  (filter
    (fn [word] (< (count word) 5))
    (clojure.string/split sentence #" ")))

Note: we used the split function to turn a string into a sequence of words, by splitting along the whitespaces with the regular expression #" ".

Map

To transform each element of a collection, we use the map function.

(map char '(72 101 108 108 111))

Again we can use anonymous functions.

(map #(rem % 3) (range 20))

Reduce

If you have never used reduce, it may take a while to appreciate its full power. Essentially we need to provide 3 things:

a function that has two arguments, to combine the elements,
an accumulator or initial value - this is actually optional, if we don't provide one, then the first element of the collection will be used instead,
a collection whose elements need to be reduced.

Typically, reduce will create a single value by combining all the elements of a collection. For example we can add all numbers from 0 to 9 like this:

(reduce + (range 10))

There are more tricky usages as well. We might end up with another collection that may have less or more elements than the original one. If we remember how conj works, like it adds new elements to the front of a list, we can use this fact to reverse the order of a collection, by starting with an empty list:

(reduce conj '() (range 10))

Similarly we can conj into a set, to de-duplicate a collection:

(reduce conj #{} [1 3 2 2 5 1 4 1 5 2 3 3 3])

As a more advanced example, this snippet, featured on clojuredocs.org, uses reduce to calculate all prime numbers below 100:

(reduce
  (fn [primes number]
    (if (some zero? (map (partial mod number) primes))
      primes
      (conj primes number)))
  [2]
  (range 3 100))

Eager vs Lazy

Most of the collection functions featured in clojure.core are lazy, including filter and map too. This means that the calculation is delayed until the time when it is really needed. They also produce a lazy sequence, that is a special type of collection. This is usually more efficient for memory usage, and saves some processing time.

On the other hand, eager operations consume the collection right away, for example reduce works this way.

When we compose these functions, especially if we deal with large collections, the order can be very important. It would be beneficial to eliminate some elements from the collections with filter, before we apply any transformation on them with map.

Consider the difference between these two codes that calculate the square of odd numbers. The second one is more efficient, because it eliminates half of the numbers as the first step.

; Example 1: Less efficient
(filter odd? (map #(* % %) (range 20)))

; Example 2: More efficient
(map #(* % %) (filter odd? (range 20)))

In the next chapter we will delve deeper in the topic of function signatures, looking at variadic functions, multi-arity functions and argument destructuring.

[Clojure] 04 Collections