Fast Clojure Startup with CRaC

Clojure is the best programming language, but one of its downsides can be startup time. There are a variety of existing techniques you can use to speed things up but even with those tools, initialization isn’t free.

While trying to speed up a CI pipeline, I discovered that Clojure works quite well with CRaC/CRIU on Linux. The idea is that you can start a Clojure REPL, initialize the slow parts of your application, and then checkpoint it. Later, you can restore your application from the checkpoint and skip the initialization process.

Clojure’s REPL works great as an restoration point of a process because you don’t need to write any additional post-restoration code to have your entire codebase available to run. Once the REPL is available after restore, it’s a REPL just like any other.

Note: The documentation for CRaC on Azul’s site is quite good. I recommend reading it.

Warning

There are a few caveats to be aware of with this technique.

  1. CRIU is a feature available on Linux. It works in Rancher Desktop on my MacOS laptop but according to Azul’s documentation, Linux in WSL and Parallels may not work. Azul’s documentation suggests that there are builds available for Windows and Mac for development purposes, but I haven’t found them.
  2. Files, Sockets, and Native Resources cannot be open at checkpoint time.
  3. At the time of this writing, this is not available in very many OpenJDK distributions. The build that I had success with was the Azul Zulu JDK CRaC Java package.

Example

This example is published as a git repository at https://github.com/ryfow/clojure-crac-example The easiest way to try it out is to install some combination of Docker and docker-compose, clone the repository, and then run the following commands:

Checkpoint

$ docker-compose run checkpoint-example ./checkpoint.sh

The checkpoint.sh script will run the clj command, initialize the application, and then checkpoint it. The output will look like the following:

Clojure 1.11.1
Simulating doing some initializatiaon
Done with simulation
user=> Feb 25, 2024 2:18:44 PM jdk.internal.crac.LoggerContainer info
INFO: Starting checkpoint
Feb 25, 2024 2:18:44 PM jdk.internal.crac.LoggerContainer info
INFO: /root/.m2/repository/org/clojure/spec.alpha/0.3.218/spec.alpha-0.3.218.jar is recorded as always available on restore
Feb 25, 2024 2:18:44 PM jdk.internal.crac.LoggerContainer info
INFO: /root/.m2/repository/org/clojure/core.specs.alpha/0.2.62/core.specs.alpha-0.2.62.jar is recorded as always available on restore
Feb 25, 2024 2:18:44 PM jdk.internal.crac.LoggerContainer info
INFO: /root/.m2/repository/org/crac/crac/1.4.0/crac-1.4.0.jar is recorded as always available on restore
Feb 25, 2024 2:18:44 PM jdk.internal.crac.LoggerContainer info
INFO: /root/.m2/repository/org/clojure/clojure/1.11.1/clojure-1.11.1.jar is recorded as always available on restore
CR: Checkpoint ...
./checkpoint.sh: line 3:     8 Done                    echo '(org.crac.Core/checkpointRestore)'
         9 Killed                  | clj -M:create-checkpoint -i src/clojure_crac_example/init.clj -r

Restore

To restore the JVM, you can run the restore.sh script:

$ docker-compose run checkpoint-example ./restore.sh

Restoration should happen quickly. You can then use a the restored REPL to tell Clojure to do things.

nil
user=> (foo)
Thanks for calling foo.
nil
user=>

Note: The restored repl is only a restoration of the JVM, so unfortunately readline support won’t work here.

Example Explanation

There are a handful of parts here, I’ll explain each of them briefly.

Dockerfile

The Dockerfile sets up a Linux environment with the Zulu JDK including CRaC support and Clojure. Nothing particularly tricky going on here.

FROM public.ecr.aws/docker/library/debian:bookworm-slim
RUN apt-get update && apt-get install -y curl rlwrap
RUN curl -o zulu.tar.gz -L https://cdn.azul.com/zulu/bin/zulu21.32.17-ca-crac-jdk21.0.2-linux_$(uname -m | sed -e 's/x86_64/x64/').tar.gz
RUN tar -zxvf zulu.tar.gz && rm -rf zulu.tar.gz
RUN ln -s /zulu* /jdk

ENV JAVA_HOME=/jdk
ENV PATH=${JAVA_HOME}/bin:$PATH

RUN curl -L -O https://github.com/clojure/brew-install/releases/latest/download/linux-install.sh && chmod +x linux-install.sh && ./linux-install.sh

docker-compose.yml

The docker-compose.yml just specifies a single service with required settings. Those requirements are:

  1. The Docker container needs to be started in privileged mode
  2. The scripts are mounted to /app
  3. There’s a volume set up for /root/.m2 so the same jar files used during checkpoint are available at restore time.
version: '2.4'

services:
  checkpoint-example:
    privileged: true
    build:
      context: .
    volumes:
      - .:/app
      - m2:/root/.m2
    working_dir: /app
   
volumes:
  m2:

deps.edn

deps.edn adds the CRaC library as a dependency and specifies an alias called create-checkpoint that tells the JVM where to store checkpoints with the -XX:CRaCCheckpointTo=./crac-image JVM option.

{:paths ["src"]
 :deps {org.crac/crac {:mvn/version "1.4.0"}}
 :aliases {
   :create-checkpoint {
     :jvm-opts ["-XX:CRaCCheckpointTo=./crac-image"]}}}

init.clj

init.clj initializes the application. In this example, initialization consists of sleeping for ten seconds and defining a function. In the real world, this would load dependencies and application code and depending on the use case, possibly instrument code with something like Cloverage.

;; Simulate loading a bunch of libraries
(println "Simulating doing some initializatiaon")
(Thread/sleep 10000)

(defn foo []
  (println "Thanks for calling foo."))

(println "Done with simulation");

checkpoint.sh

checkpoint.sh runs Clojure with the previously mentioned create-checkpoint alias, runs the init.clj file, and sends (org.crac.Core/checkpointRestore) to the Clojure process via STDIN. This allows Clojure to initialize and start the REPL before checkpointing.

Once the checkpoint finishes, the clojure process exits and the checkpoint image will be available in ./crac-image as specified by deps.edn.

#!/usr/bin/env bash

echo '(org.crac.Core/checkpointRestore)' | clj -M:create-checkpoint -i src/clojure_crac_example/init.clj -r

# checkpointRestore exits the process with a non-zero exit status, even when it works

exit

restore.sh

restore.sh restores the Clojure process from the ./crac-image directory that checkpoint.sh created.

#!/usr/bin/env bash

java -XX:CRaCRestoreFrom=./crac-image

Conclusion

CRaC helped me shave several minutes off a parallel build pipeline due to the slow initialization of tests, I’m guessing it can help others as well.

Clojure starts up fairly quickly in a simple environment but when you have a lot of code, a lot of dependencies and Cloverage code coverage instrumentation, initialization takes a noticable amount of time. CraC lets you do that initialization less frequently, thereby saving time.