For a long time, I wrote Datomic-based applications in a style where a
value was globally available in my core “db” namespace. It just seamed easy; no
futzing about connecting to a URI (
connect) or creating a database
create-database) just to get started interacting with a database from the
It looks a little something like this:
(ns my.app.db (:require [datomic.api :as d/) (def uri "...") (def conn (d/connect uri)) ;; and you're on with your day
After toiling with this approach for some time, I’ve come to realize this is an anti-pattern, for one big reason:
Early connections bloat application launch time
Connecting to a database in Datomic realizes a number of costs:
- Your peer must handshake with and connect to storage (time).
- Your peer draws portions of the index into local memory (time).
- To connect, you must have an existing database.
- Your application launch immediately consumes a peer slot.
- You’ll be tempted to bake-in bootstrapping/schema-migrations to your connection logic (more time).
All of this adds up; applications, more specifically, your REPL, will take longer to launch, and decrease Clojure’s already tenuous launch times.
Doing it right
The solution is simple, both conceptually and in practice: defer connecting to and setting up a Datomic database until it is actually needed.
In web applications, this means setting up a connection at service launch,
rather than process launch. I generally call this function
within it I wrap up all aspects of database connection and initialization.
The following sample creates a database, connects to it, and transacts all of the relevant schema (using conformity):
(defn bootstrap! "Bootstrap schema into the database." [uri] ;; Create the database (d/create-database uri) ;; an idempotent call ;; Connect to it (let [conn (d/connect uri)] ;; and transact our application's schema... (doseq [rsc ["bouncy-castle.edn"]] (let [norms (c/load-schema-rsc rsc)] (c/ensure-conforms conn norms)))))
Once you have such a function in hand, wrap it up in the code that launches your application. (Ensuring, of course, that launching a REPL doesn’t actually fully launch your application.)
Stuart Sierra’s component makes this rather easy, but it’s just as well to do this closer to your application’s lifecycle:
;; By defining a component... (defrecord DatomicDatabase [uri] component/Lifecycle (start [component] (bootstrap! uri) (let [conn (d/connect uri)] (assoc component :conn conn)) (stop [component] (dissoc component :conn)))) ;; Or, more directly (a Pedestal example)... (defn start [service] (db/bootstrap! db/uri) (alter-var-root #'server (fn [_] (http/create-server service))) (http/start server))
A Conn in Every House
Once your application is up and running, the last thing you need to worry about is providing an active connection to your application.
Despite my warnings, you shouldn’t be afraid to code your APIs to Datomic URIs (rather
than connections). Although
connect is a relatively expensive call, it is
only expensive once. Datomic caches connections internally, thus each
successive call to
connect is essentially inert.
That said, I find the above a little ugly, so I opt instead to inject a
variable via middleware. The following Pedestal interceptor injects a
and most-recent database value into every request:
(defbefore insert-db [context] (let [conn (d/connect uri)] (assoc context :conn conn :db (d/db conn))))
And the REPL…
I’ve side-stepped one last important detail. While my REPLs launch quickly, I
don’t have direct access to a
conn. Unfortunately, for this, I don’t yet have
an ideal solution.
For the time being, I get around this by defining a comment block (as below)
that defines a
conn var in my database namespace. Whenever I launch a REPL,
I invoke this
def, and gain access to a more immediate
conn than connecting
(comment (def conn (doto uri bootstrap! d/connect)))
How about yourself? Have you noticed connecting to a Datomic database bogging down your REPL launches? How do you pass around connections in your own applications?