Programming: Meander: The answer to map fatigue

Map fatigue. You may already have it. You might know somebody who does. That debilitating depression that causes otherwise normal programmers to roll their eyes and sigh. It starts out innocuous enough. I’ll just reach into this widget entity value and pull out the description I need. Oh and I’ll grab the count from the inventory entity. Hmmm what’s this argument being passed in here, I wonder if it has the prices I need? Soon things escalate. The afflicted quickly proceed to the next stage; designing in-memory databases, writing style guides, building frameworks and stoically specing their specs. The final phase manifests as full-blown depression. Incapable of writing the simplest function, the programmer sits with their hands on brow muttering nonsensical chants about the shape of data.

But new research has found the answer! Well, old research really. But now there is a library available to deliver it to you. The library is called Meander by Joel “The Falcon” Holdbrooks. It is a pattern matching and expression rewriting library. Let’s jump right into an example to show how this library can be used to treat map fatigue.

The scenario: I have been assigned the task of integrating data from several widget manufacturers. I need to take data about widgets from multiple sources in various shapes and produce a normalized version of it. The end result should be normalized as manufacturer-code, widget-code, description.

Here is the data format of the first manufacturer:


(def skynet-widgets
  [{:basic-info {:producer-code "Cyberdyne"}
    :widgets [{:widget-code "Model-101"
               :widget-type-code "t800"}
              {:widget-code "Model-102"
               :widget-type-code "t800"}
              {:widget-code "Model-201"
               :widget-type-code "t1000"}]
    :widget-types [{:widget-type-code "t800"
                    :description "Resistance Infiltrator"}
                   {:widget-type-code "t1000"
                    :description "Mimetic polyalloy"}]}
   {:basic-info {:producer-code "ACME"}
    :widgets [{:widget-code "Dynamite"
               :widget-type-code "c40"}]
    :widget-types [{:widget-type-code "c40"
                    :description "Boom!"}]}])

I’ll start off with with a stock standard hand rolled bit of code to process the data into the shape I want. Warning: Proceed with caution. High risk of map fatigue ahead.


  (for [{:keys [widgets widget-types basic-info]} skynet-widgets
        :let [{:keys [producer-code]} basic-info
              descriptions (into {} (for [{:keys [widget-type-code description]} widget-types]
                                                  [widget-type-code description]))]
        {:keys [widget-code widget-type-code]} widgets
        :let [description (get descriptions widget-type-code)]
        :when description]
    [producer-code widget-code description])

Well, that’s one way to do it. You might implement it slightly differently. Maybe you prefer to use `map` instead of `for`, or transducers, or different destructuring, or no destructuring at all. One of the subtle barbs of map fatigue is indecision over implementation choices.

Now here is the awesome solution using the Meander library:


  (require '[meander.match.alpha :as m])
  (m/search skynet-widgets
            (scan {:basic-info {:producer-code ?producer-code}
                   :widgets (scan {:widget-code ?widget-code
                                   :widget-type-code ?widget-type-code})
                   :widget-types (scan {:widget-type-code ?widget-type-code
                                        :description ?description})})
            [?producer-code ?widget-code ?description])

Structure! You can clearly see the shape of the data in this solution.

Logic! Those unusual symbols prefixed with `?` are logic variables. The `scan` expression indicates that meander should attempt to match any items in a sequence. A logical join occurs automatically on `widget-type-code` because it appears twice in the overall expression; once in each of the sibling scan expressions. We do not need to aggregate a map of `widget-type-code` to descriptions; the relationship is implicit in the expression. This is the magic sauce of the Meander library; combining pattern matching with logic solving.

Lack of choice! The declarative nature of meander expressions leaves me with no implementation choices to make. I specify the shape of my data and the shape of my desired result, and hey presto it just works! The declarative approach produces compact expressions and elides transformation code.

Based on science! Joel didn’t create this library in a vacuum. He drew inspiration from existing research on expression rewriting. One of his main inspirations was Stratego -- Strategies for Program Transformation. The techniques used here are very effective for code manipulation.

In Clojure, where code is data, techniques for manipulating code translate well to manipulating data. Expression rewriting is a more general problem definition of data rewriting. Meander is well suited to become a pervasive glue for describing data extraction and recombination because it goes beyond destructuring and pattern matching to providing logic solving.

If you or a loved one suffers from map fatigue, take a dose of Meander™ and suffer no more.

Pages

Meander: The answer to map fatigue