The Clojure
This year the Conj was held in Austin, which was an excellent venue. It’s warm! I only needed a light jacket and jeans outside. Austin has great eats, especially the barbecue. Everybody was chowing down on some good ol Texas ribs. There were plenty of nearby eating options for quick meals or longer sit ins. The airport is small and easy, as was land transport.
I spoke to about 50 people out of an attendance of about 500. Below are my notes, I hope they provide some insight into what is happening in the Clojure community.
Tomas: Math professor teaching computer science using Firestone (Online fantasy battle card game) in Clojure. Likes using inline tests attached as metadata to the function. Uses a custom macro is= to avoid empty tests.
Len: Using Clojure for military readiness assessment.
Milt: Yet Analytics recently hired some new recruits. Is using Web Workers as the owner of Reframe state. This avoids UI freezes when processing large communications to and from the server. Web Workers from ClojureScript has been quite difficult, but he has some changes to ClojureScript and Figwheel on the boil which will make this very smooth. Web Workers will be very useful to me in a couple of projects. He outlined his approach as follows:
- Bootstrap: sets up the main <-> worker handlers
- Sente events -> reframe (transit)
- Separate build (to not access document which is not present in WebWorkers)
- Reframe middleware (inteceptor) to check data shape (goes away when using Spec function instrumentation)
- Interceptor to send back to the main process.
Client <-> Worker <-> Reframe <-> Sente <-> Server
I’m really excited about this… it fits beautifully with ‘the way’... keeps the UI responsive… sweet.
Fashion: Rich was rockin a purple shirt, shades and regulation black dancing shoes. Alex Miller surprised nobody, going with his standard plaid shortsleeve. Beards are still in. Longer beards especially. Also popular are sharp undercuts.
Paula: Building Naga; a rules engine for RDF/Datomic used at Cisco (and other places).
Stu: Datomic ETL - Build a function pipeline. The steps don’t matter. They will change.
Clojure provides system generality. Spec gives specificity when you want it.
Presented a useful conform! helper and halt-when for errors.
DB -> Extractor -> Entities -> Spec -> Transform -> Transactions -> Loader -> Datomic
Sivram: Using Clojure for clinical rules/data
Nick Bailey: Developer happiness, maintainability. Clojure fits our needs, is a recruitment tool, inspires passion, and is picked up quickly. Does Java interop between Jython and Clojure
Vijay - Using ClojureScript for HBO client side work.
Most people I met at this Conj were already using Clojure/ClojureScript professionally.
Schmüdde: Composing a small number of primitive colors (select a pallet) and allow some mixing. What are my constraints? Experiment within the constraints using generative approach. Adjust the constraints.
Jason: Proto REPL is a new Atom Clojure integration. Atom can bring up Chrome dev tools access to the DOM (modifiable for visualizations). Use (save x) to collect values (or Sayid). Reformat data for display. Has chart mappings and graphs (what is the underlying lib? looks slick). Made by bringing data centric components together.
Jacob: Using Clojure for ride share and private car rental in Denmark.
lvh: Use JNR for high speed low level library wrapping. Interface as data to support various type combos. Macro elides repeatitiion.
Bill: Sayid is an omniscient debugger. Tracing that collects the values and net time in the entire call graph. Defined println debugging as “sprinkling”. Requires some thought about what to trace. Inspires us to realize a better debugging story. Looks quite editor/REPL integration heavy, I wonder if there is a more Refresh based approach that works. Might be as easy as having a subordinate ProtoREPL running? You should watch this talk for sure.
Rich: Spec-ulation. This is what we are all here for, and Rich delivered! Absolutely can’t miss this talk, watch it now. Breaking changes are broken. Don’t break!! Don’t! Our community is a positive one. Accret. Spec: you can use it, it’s here to stay. The problem is the code is going to execute against some version, but if you have transitive dependencies on multiple versions you can not control which version it will be running. Dependencies are really call dependencies. Level violation. fns call by name. ns requires. artifact context. MAGIC. Coarse grained false security.
What is required? fn args, ns var names, artifacts ns/package names/paths
Provided? fn returns, ns vars/fns, artifacts ns/packages
Accretion, provide more. Relaxation, require less. Fixation, bash bugs. Flip any => Break. Either grew or broke. Nuance: does not apply to private development). Adding is growth, removing is breakage.
Code for growth. Open specs and data formats. Define what you can do, not what you can’t. Prohibition turns growth into break, so don’t prohibit. Presume you may get more. During development Alpha is O.K. or artifact id. Not an excuse to not promise (release it at some point please).
The only truth is the runtime. Think twice about requiring more or providing less.
Same story for web services.
What if we never broke? Can alway use latest. Spec can help - diff check for removing provides or additional requiring. SHA tested (per fn) Turn what would have been breaking into accretion.
Let’s be the community to make it great.
Sean: Recommended 1st Robotics for children wanting to learn to program, and lego Audrino.
A recurring question that came up (originally in the unsession):
How can programmers help scientists? Lack of basic software engineering prevalent in scientific work (change log, version control, tests, documentation, bus factor). Root cause is a continuation problem.
Scientist/programmer pair matchmaking? Book for scientists?
“You are in the Software business!”
Rich: We can’t really solve the continuation problem for science because we haven’t solved it for software engineering either; we don’t know how to pass software ownership onward (especially true for short time spans). We can build better tooling for scientists (and ourselves).
- Build better tools for science
- Clojure and Datomic offer reproducibility [this is huge for science; like git for science]
- Core.matrix is wonderful and can handle the numeric needs of scientists
- Scientists tool needs aren’t specific to themselves
- Can’t fix a cultural problem with tech
- There are no incentives for good software practices in science
- Convert PySci to Clojure?
- Build cool stuff and show scientists! Show them how Clojure can help them
Happy to see so many happy Clojure users.
I don’t make uberjars. For deployment, create 2 artifacts; code and runtime. Use IFS .m2, services all point to the same one. Git repo of company lib-> version which is updated when CI passes. Depend on the name not the version. Depend on code. Services depend on the lib not a specific version. I’d rather have to test that all my company code can go to a new version than have no idea what code is executing in a whole bunch of sub contexts. CI enables this. Test per git SHA can save build times and enable generative testing. AWS CodeDeploy works well for managing the flow of change, test, deploy of the code.
We’ve forgotten how make works, let’s get some of that back. Slow build times are due to getting artifacts you don’t even need, want, and might not even be what your code asked for… and running tests you’ve already run. Stop doing that. [Rich described the build and deploy system that I want consisting of simple and effective tooling choices. I hope someone from Cognitect will blog about their build system soon, it would be a valuable reference path forward along the themes of his talk.]
Simulation of national exit poll has tonnes of data. The whole election runs in one night. Have to do lots of pretend elections. Very bursty data. Was a greenfield rewrite.
Dorian: Making a custom flash cards mobile app in Clojure to learn Clojure while he learns Clojure. Cool idea! I'm going to copy this idea and do the same :)
Jacob: Using Clojure to optimize third party web service access. Measuring the effects of parallelism and chunking. Brainstormed about what sort of problems are good for learning things like core.async. Game min/max, solve the same problem using different tech; make a parachute that doesn’t break the egg.
Benson: The secret to understanding macros is remembering that they happen between the R and E of R^EPL. I found this tip very insightful, thanks for sharing it with me.
Conor: Is building tools for collective intelligence in Clojure. Came to Clojure from Datomic. Runs a research group of people interested in a variety of topics. Found people interested in research groups for different topics like parenting, homeschooling, and writing.
Currently working on a framework for knowledge mapping in Clojure/Datomic. Thinking about how to structure a debate style way to build arguments for and against the truth of statements. Also working on a different approach to time management: schedule your fun time first. Block out time for things that do take time. How much time do you want to allocate to what? Reverse scheduling works better than planning goals.
People in Clojure are self selecting to be helpful enthusiasts. Being helpful trumps skills/knowledge, and our community is brimming with helpfulness. Clojure has core values, a story, a nationhood. People choose to come to the Conj, to be part of a beautiful autocracy. A humble intelligence to choose to remove incidental complexity. Our uniting enemy is mutability. The constitution of nation Clojure is strong. There is trust in libraries, the thought behind them, the language, and ecosystem. This sort of nationalism can dissolve our differences. Recruit how Christians recruit. Put yourself out there, in places you are uncomfortable, work with new people for their strengths.
Eve has shown that you get a more friendly UX when you allow junk in between.
Chris (aka Oakmac): Using ClojureScript for threat detection at Cisco. It’s critical to solve the right problem. You and your team can do a fantastic job, build a polished feature, and it doesn’t mean anything if the user needed bulk load but you build form input validation. Discussed the pros and cons of kata coding interviews.
Adam: Interested in the effect of communication of requirements as teams scale, and how it breaks down as the distance of the programmer from the user grows. Sees much potential in Eve.
Colin: Business is doing great. Sustained growth. Working on many very useful Cursive integrations. Look out for figwheel click through to source soon! Is also taking more interest in refresh based workflows, which I am pleased to hear :) Thinking about how to provide a WallabyJS experience for ClojureScript in Cursive. Interested in taking the Sayid debug/trace approach. life is complicated trying to support clojurescript and spec. Working with Bruce to get figwheel integration (click through line numbers to source, serve out of Cursive, detect dependency changes from the code not the files).
Pretty much everyone was hiring and lots of people were looking. But there wasn’t a recruiting frenzy, quite the opposite. Most people were there for the Conj, to share ideas, build contacts, and enjoy Austin.
Quite a few startups are betting on Clojure: Jack (Ladder), Nikko & Claire (SparkFund), Kim (Reify Health), Rusty (Outpace).
Adam and Matt: Using Datomic to build wormbase.org for curation and query of papers about genome differences in worms. An over abundance of oversight is preventing extension of the solution to wider genomic databases (other animals). Alliance of Genomics is 10k researchers, and they need this tool. They also need to be able to annotate changes online with attribution. Scientists need a social network for collaboration. A gardening tool for building their own content and suggesting edits to others.
Paul: Pamela is a Lisp for space exploration with machine learning capabilities. A modeling language. In space you need to handle the unexpected. ML with real world interfaces. Useful for discovering the real parameters of a system, and handling transitions (bounciness when you stop a car). Learn the bounds of weird stuff.
Chris (aka Chouser): Can we apply Pamela to modeling our software systems to learn the bounds of our timeouts, performance, etc that we log? Working on codename Murkwood; an alternative to Communicating Sequential Processes and Continuation Passing more similar to Sagas… Run the code multiple times idempotently. Identify side-effecty, pausy, errory stuff that has happened or not yet.
Jarret: Pointed me to @adamtanknow sketches of the conj speakers and notes, very cool! I hope to emulate your artistic layouts and flair one day :)
Christophe: Built Powderkeg; a data glue approach to Spark in Clojure.
Eric: You can’t refactor Aristotle into Newton. Recommended reading: Conal Elliot’s Denotational Design. His approach to modeling is physical metaphor (know your domain) -> construct meaning (know your constructs) -> implementation (know your refactoring). Has been making instructional Clojure videos for three years.
Mike: Clara Rules if/then in a declarative fashion. Declared with bindings. Implicit storage of shared intermediate state. Information passing. General. Light weight. Rules have a first class data model. Dispatch is plugable. Can have a durability layer. See Ryan Bush videos. Supporting facts can be chained in metadata.
There were lots of healthcare companies at the Conj this year (Devin - Healthfinch, Kim - Reify Health, Mike - Cerner, Peter - Diligent, Jack - Ladder) and rules were a core competency for most of them.
Devin: Thinking about ways to filter rule information effectively to provide the answer to “Why did you recommend this?” in the context of 5k rules. Need to flag important ones and save the decisions for attribution and explanation.
Great to see and have personal conversations with colleagues past and present… Jeff, Tanya, Chris, Paul, Dan, Rusty, Vincent, Matt, Devin, Alex. I really wish I had spent more time with you instead of mingling. My philosophy on conferences is your there to get ideas and meet new people so I intentionally sit next to someone I've never met/seen before as much as possible. But the downside is I don't get to hang out with the gang, which I regret now. I hope to catch up with you again soon.
Nikko: Wants to make a difference in improving our world. Contracts are crazy, there is no precise meaning, it is just two parties agreeing that they each separately think they know what it means. If there is a dispute it goes to a judge to flip a coin. We can do better! We can make funding better. We can build a better financial foundation for our world. Building SparkFund with Clojure and Datomic.
@apogsasis: Composing music with Clojure Spec. Lots of overlap with Steve’s Juggling. Juggling throws look very much like drum beats. Numeric encoding and rules. Computer assisted constraint and solution space exploration frees us up to focus on more nuanced and skilled tasks.
Gladfly: McDonalds beef patties turned me vegan!
Andrea: Working on bootstrap Clojure (Clojure in Clojure) and it’s many applications such as ClojureScript browser REPLs.
Alex Miller: Was fielding a tonne of spec questions.
Jack: Proud of the build tool Ladder build because it was painfully missing. https://github.com/ladderlife/loonie Solved a real friction point for his team. Making a big bet on Clojure being the best way to run a startup. Pleased with the language and people, but build/deploy was a sore point (so they addressed it).
Chris: Using Clojure at Rackspace. Maintains a Clojure security lib.
Brett: Using Clojure in the mortgage industry (Guaranteed Rate). Found it easy to adapt logic to what a loan officer would do, but the trick is seeing if the end effect is the desired one. Works remotely (most people I talked to worked remotely). Enjoyed the Austin food.
William: Barliman 4.3 is out! Write the test, generate the code, “prove me wrong”. We’ve seen this concept developing, it is reaching a really exciting stage where it can do quite a lot, and is compelling as a programming tool. Can fill in the last X % of code. Can detect “no function can pass this test with the current structure” on partial code. Proof checking. Find logical flaws before you even code.
Seems perfect for data transformations (and we write many of those reshaping fns in Clojure). I would rather write the spec than the code. Also useful for refactoring (discovering better solutions). I highly recommend you watch this talk.
Kim: Is using Clojure to facilitate Clinical trials (Reify Health). Good groups have Talkers, followers, mediators, leaders. +ve influences, -ve influence. Need a balance. Effect is contextual. Multipliers.
Milli: Teaching bootcamps. Gave me some very helpful insights on group formation techniques she uses. Does a personality test and finds that helpful in giving people some context on how to work with each other.
Riley: Studying, learning functional programming.
Rangel : Building Native ClojureScript app. Looks fantastic and it is fast.
Mark: Lots of Aussies getting into Clojure.
Below are some fun sketches I made during the talks, I'm still a beginner but thought they might give you a laugh to see some amateur caricaturization:
The main speaking venue:
Disclaimer: I’m working off notes and memory of a whirl-storm of conversations. I may have misheard/misunderstood something you said, if so I am sorry about that; please let me know and I will promptly retract/update. The information herein is provided as a peek into the community activity going on.