Refx: Origins

In this blog post, I am describing my journey of migrating a medium-sized project from re-frame and Reagent to Helix, creating refx on the way.

A pet project of mine is a mobile-first web application, that I also use as a sandbox for experimenting with interesting pieces of technology.

I’ve built it with my favorite tech stack at that time: The backend is written in Clojure (using init, among others), and the frontend in ClojureScript, using re-frame and Material UI.

The Stack

re-frame

Re-frame is a terrific framework, allowing a nice separation of concerns. You write event handlers as pure functions that describe side-effects, effect handlers to apply these, and subscription handlers to extract and preprocess data from a central database to be displayed by UI. This allows for beautifully functional and reactive programming: Whenever your application state changes, the UI reacts and updates the DOM.

It integrates Reagent, probably the most popular and widely adopted ClojureScript wrapper library for React. With Reagent, you can write React components as pure ClojureScript functions that return Hiccup, a HTML representation using Clojure data structures:

(defn reagent-list []
  [:ul
   [:li.first "First item"]
   [:li [:a {:href "/example"} "Second item"]]
   [:li "Third item"]])

With Hiccup, you can write components as pure Clojure functions, using vectors and keywords and maps. Zero dependencies, zero JavaScript interop. Beautiful.

Material UI

MUI is a JavaScript library providing React components, and integrating those with Reagent requires some React interop with the :> special construct:

(ns my-app.views
  (:require ["@mui/material/List$default" :as List]
            ["@mui/material/ListItem$default" :as ListItem]
            ["@mui/material/ListItemText$default" :as ListItemText]))

(defn mui-list []
  [:> List
   [:> ListItem
    [:> ListItemText {:primary "First item"}]]
   [:> ListItem
    [:> ListItemText {:primary "First item"}]]
   [:> ListItem
     [:> ListItemText {:primary "First item"}]]])

Since this feels ugly to many Clojure developers, there are adapter libraries that attempt to hide the React/JavaScript interop behind a layer of ClojureScript. I originally created my own such project and named it mui-bien, before I discovered an existing and arguably better maintained library called reagent-mui. With these, the same example becomes:

(ns my-app.views
  (:require [reagent-mui.material.list :refer [list]]
            [reagent-mui.material.list-item :refer [list-item]]
            [reagent-mui.material.list-item-text :refer [list-item-text]]))

(defn mui-list []
  [list
   [list-item
    [list-item-text {:primary "First item"}]]
   [list-item
    [list-item-text {:primary "First item"}]]
   [list-item
     [list-item-text {:primary "First item"}]]])

This extra layer comes with the downside that we now depend on the library’s manager to update whenever a new MUI version gets released, but the code looks much more natural.

Leaky abstractions

Many MUI components take React elements as properties. For example, a ListItem can take a component (more precisely: an element) as a secondaryAction. In JavaScript with JSX, you could write this as:

<ListItem
  secondaryAction={
    <IconButton edge="end">
      <DeleteIcon />
    </IconButton>
  }
>
  ...
</ListItem>

Using Reagent, you need to explicitly convert Hiccup to a React element, so the example above could be written as:

[list-item {:secondary-action (reagent.core/as-element
                               [icon-button {:edge :end}
                                [delete-icon]])}
 ,,,
 ]

Note how we can use kebab-case keyword attributes such as :secondary-action and keyword attribute values such as :end and they will be converted to strings for us. Nice!

Another source of pain is the conversion between ClojureScript maps and JavaScript objects. While Reagent and reagent-mui go to great lengths to do the conversion for you, you will eventually encounter places where you are on your own. For example, when you want to access MUI’s theme in a :sx prop, you will need to deal with JavaScript interop all of a sudden:

[box {:sx {:transition (fn [theme]
                        (.. theme -transitions (create "transform")))}}]

Performance impact

It should not be surprising that all these extra layers of abstraction come with a cost. On every React render, our components need to be converted from Hiccup to React elements. Material UI components need to be treated specially, as they already are React elements. Hiccup wrapped in as-element seems extra wasteful, as we often embed Material UI components in Hiccup, only to convert them back to React.

In my application, more complex screens started to feel less responsive, especially on my 5 years old mobile phone. Switching tabs did not happen immediately, but happened after a noticeable delay.

Alternatives to Reagent

I recently learned about two exciting new Clojure projects, that claim to bring ClojureScript to "modern" React. React has evolved a lot since it was first wrapped by Reagent, and today’s JavaScript React developers prefer using functions and "hooks" over old-school class components.

Creating function components and using hooks requires yet another special ceremony with Reagent: You need to use your components with the special :f> operator:

[:f> my-function-component {:attr "value"}]

Not only is this ugly, it also puts the burden of knowing which component is a function component on the caller!

UIx

The first Reagent alternative I learned about was UIx. It follows a similar path like Reagent, defining components using Hiccup and providing an as-element wrapper, but creates function components by default and provide ClojureScript wrappers for React’s built-in hooks. It also claims to be 3x faster than Reagent. Migrating Reagent components to UIx should be rather straight-forward: Most components will work out-of-the-box, only those that use as-element or Reagent’s atom will need to switch to UIx' as-element and use-state.

Helix

The second library I found is Helix. Unlike UIx, Helix does not provide a Hiccup parser, and tries to be a very thin and performant wrapper, using macros to move computation to compile-time where possible.

For a Reagent developer, the API will look a bit weird at first, as Hiccup forms become function calls using Helix' DOM API, and you will use a special variant of defn for defining your components:

(defnc reagent-list []
  (d/ul
   (d/li {:class "first"} "First item")
   (d/li (d/a {:href "/example"} "Second item"))
   (d/li "Third item")))

For a project using MUI, however, a nice property of Helix is that your own components will be real React components, and there is no difference in usage between them and "native" components:

(defnc mui-list []
  ($ List
   ($ ListItem
    ($ ListItemText {:primary "First item"}))
   ($ ListItem
    ($ ListItemText {:primary "First item"}))
   ($ ListItem
     ($ ListItemText {:primary "First item"}))))

(defnc container []
  ($ mui-list))

Notice how the $ macro is used for the custom component mui-list in the same way as for MUI’s "native" components.

With two layers ob abstraction gone, you will find yourself more often using JavaScript interop, with the benefit that no special treatment is required for passing ClojureScript components to JavaScript and vice versa:

($ ListItem {:secondaryAction ($ IconButton {:edge "end"} ($ DeleteIcon))})

Also, with less back and forth conversion required, our rendering performance should be much closer to pure JavaScript implementations. And indeed, after migrating my code base from Reagent to Helix, the UI felt much snappier.

What about re-frame?

The biggest obstacle for me migrating to UIx or Helix was re-frame. I really like re-frame and had a considerable amount of code written for it. I clearly did not want to follow the React approach of "complecting" views with logic, but wanted my logic to stay in re-frame’s event handlers and keep the views pure. In other words: I wanted to continue using re-frame.

However, re-frame is built on top of Reagent, and I did not want to carry this extra baggage. Ideally, I wanted re-frame with Reagent stripped from it.

The birth of refx

When I first thought about porting re-frame’s ideas to a new library I did not know how deep its Reagent integration was, and therefore how big the effort would be to untangle them. At the same time, I thought this was a great opportunity to experiment and learn how re-frame actually works under the hood, so I accepted the challenge.

Reagent in re-frame

Roughly speaking, re-frame consists of the following parts:

  • An event router that implements dispatch

  • A registrar of event, effect, coeffect and subscription handlers

  • A lightweight interceptor implementation

  • A central application database (a Reagent atom)

  • A small number of built-in interceptors, effects and coeffects

  • A subscription system creating Reagent’s "reactions" to react on changes in the application database, and using a cache to reuse these reactions

I noticed that most of re-frame’s code is actually independent of Reagent! While Hiccup views are an essential part of re-frame’s "domino" story, the library itself does not concern itself with rendering, but leaves this entirely to Reagent. Its interface for views is limited to subscribe and dispatch, and dispatching events itself does not involve Reagent until the application database is updated by a coeffect handler.

With this realisation, it felt pretty doable to carve out the Reagent bits and opening re-frame up to different rendering libraries.

Mike Thompson, the author of re-frame, expressed no interest in modifying re-frame in this way, but encourages ports for other libraries [1], so I decided to do just that.

A re-frame port

I named this project refx ("reactive effects"), to stay close to the "re-" naming scheme and make its relationship with re-frame somewhat obvious. Migrating from re-frame to refx should be straight-forward, and I wanted refx to expose a very similar API like re-frame.

At the same time, I did see an opportunity to get rid of some legacy in re-frame’s code base, and to bring in some ideas of my own as well. At the time of writing, I am not sure if this will make refx deviate further from re-frame in the future. I am planning to keep a "compatibility" layer for easy migration though.

Unlike re-frame, refx is agnostic to the rendering library. It is biased towards React and provides hooks to integrate with React apps, and will play nice with any library that support React hooks.

Signals

The core difference between re-frame and refx, and the only original work of refx' first version, is the subscription system, which I based on a "signal" abstraction.

In refx, a signal is an entity whose state changes over time. At any point in time, one can read a signal’s current state as an immutable value, and one can register listener functions that will be called whenever the signal state changes.

This is very much in line with Clojure’s Epochal Time Model, and refx indeed implements its ISignal protocol for ClojureScript atoms. The global application state refx.db/app-db is an ordinary atom.

Subscriptions in refx are implementations of ISignal that will compute their value whenever one of their input signals change. As in re-frame, subscriptions form a signal graph, and React components subscribing to nodes in this graph will automatically re-render whenever a node’s state changes. Unlike re-frame, refx' subscriptions are not based on Reagent’s "reactions", but on a custom implementation, and the connection with React is done using a use-sub hook that uses React’s new useSyncExternalStore hook.

This design is more flexible than re-frame’s, as subscriptions can depend on any signal and are not restricted to app-db changes. For example, it is possible to model the current time or mouse position as a signal, or integrate with other libraries such as DataScript.

Dynamic subscriptions

Refx supports "dynamic subscriptions", that allow subscription vectors to contain signals themselves. Let’s consider an example to see why this is useful.

Let’s assume that your app allows the user to open a document, and the database contains all loaded documents and the ID of the currently active document. There are two subscriptions :document-by-id and :current-document-id. In the UI, you want to display the title of the current document.

In re-frame, there are two ways to solve this:

  1. Create a subscription :current-title that looks up the current document in the database itself. This subscription will now depend on the whole database, and will recompute more often than if it would depend on the :document-by-id and :current-document-id signals.

  2. Create a parameterized subscription [:document-title doc-id] that takes the document ID. This requires the component tree to be structured in a way that a parent component subscribes to the :current-document-id and passes it to the one that will show the title. This will therefore couple your database layout and your component tree to some extend.

Dynamic subscriptions in refx allow a third option: Your subscription :current-title can depend on [:document-by-id (sub [:current-document-id])], allowing for optimal updates while hiding the details of your database structure from your views.

Conclusion

With refx, I was able to migrate my pet project’s UI from Reagent to Helix successfully. While I did not do any benchmarks, I achieved a considerable performance boost, which is easy to explain given that I got rid of two layers of abstraction (reagent-mui and Hiccup).

While migrating the views from Reagent’s Hiccup to Helix macros and hooks was quite some work, the changes needed to substitute refx for re-frame turned out to be minimal: Just change the required namespace from re-frame.core to refx.alpha and replace re-frame’s subscribe calls and any deref-s of the return value with refx' use-sub.

I believe that switching from Reagent to UIx will be the easier migration path, when compared with Helix, trading in some performance for Hiccup convenience. I hope to see more interest and adoption of refx and consider refx + UIx already the better alternative to re-frame for new projects as it should be more "future-proof" and allows for more flexibility in extending and changing parts later on.

Since both UIx and Helix work with thin wrappers around React hooks, their respective hook implementations should be interchangeable, and it should be possible with very little effort to mix and match these two libraries. For example, on could use UIx for most of the UI, and sprinkle in some Helix components for performance optimisation.

1. He stated that in Clojurian Slack’s #re-frame channel when I asked if he would be interested in a PR.