Caching Files in a Devcontainer
In a devcontainer, system files are ephemeral and subject to being reset any time the container is rebuilt. This can can be a problem for development package dependencies. In this video, we’ll learn a simple way to cache Ruby/Rails project gem packages across container rebuilds.
Video transcript & code
So far we have a development container that we can start up with a
docker-compose up command.
$ docker-compose up
We can open a terminal into it
$ docker-compose exec app /bin/bash -l
We can run
bundle install to get our development package dependencies.
$ bundle install
And then we can run Rails commands like
db:migrate to get our development database ready for action.
$ bundle exec rails db:migrate == 20200502233927 CreateIslanders: migrating ================================== -- create_table(:islanders) -> 0.0022s == 20200502233927 CreateIslanders: migrated (0.0024s) ========================= == 20200502234606 RenameTimezzone: migrating ================================== -- rename_column(:islanders, :timezzone, :timezone) -> 0.0103s == 20200502234606 RenameTimezzone: migrated (0.0106s) =========================
But there's a problem.
Let's say we shut down the container...
And then we make a small change to our
docker-compose.yml file. Maybe we add a port mapping, which is something we'll talk about more in an upcoming video.
version: "3.2" services: app: build: context: . dockerfile: Dockerfile volumes: - type: bind source: .. target: /workspace working_dir: /workspace command: sleep infinity ports: - "3000:3000"
And then bring the container up again...
And we open a terminal...
And then run another Rails command...
$ bundle exec rails s bundler: command not found: rails Install missing gem executables with `bundle install`
This time it acts like we never ran
We have to run
bundle install all over again. Which takes a while!
This is not ideal.
The problem here is that our development machine's filesystem is ephemeral, and subject to being reset anytime we update our devcontainer configuration. In many ways this is a good thing, because it forces us to explicitly record every bit of configuration necessary to run our app! But it turns out that there are certain filesystem changes we would like to cache from one machine recreation to the next. For this app, that includes the Ruby Gem packages that Bundler installs.
Fortunately, we do have a place we can keep files that will last longer than our dev container filesystem.
We have our project's source code directory, which is connected into the devcontainer every time we run it!
If we can convince
bundler to put the files it installs somewhere in this directory, they will stick around in our project directory instead of getting wiped out the next time the container is rebuilt.
How we get development tools to change where they put their files varies from tool to tool. In the case of Bundler, a bit of research turns reveals that we can control where it puts files with an environment variable.
We add an
environment: section to our
In this section we set the
BUNDLE_PATH variable to
vendor/bundle, which is a conventional location for Bundler packages when keeping them within the project directory.
version: "3.2" services: app: build: context: . dockerfile: Dockerfile volumes: - type: bind source: .. target: /workspace working_dir: /workspace command: sleep infinity environment: BUNDLE_PATH: vendor/bundle
We re-start our devcontainer environment.
And open a new terminal into it.
Now we can see that
BUNDLE_PATH is set inside this container to
$ echo $BUNDLE_PATH vendor/bundle
bundle install to fetch our project dependencies.
$ bundle install
Once this is done, we can see that the packages have been installed inside the project workspace.
$ ls vendor/bundle/ruby/2.7.0/gems/ actioncable-18.104.22.168 dotenv-rails-2.7.6 mini_portile2-2.4.0 rb-fsevent-0.10.4 actionmailbox-22.214.171.124 e2mmap-0.1.0 minitest-5.14.2 rb-inotify-0.10.1 actionmailer-126.96.36.199 erubi-1.9.0 msgpack-1.3.3 regexp_parser-1.8.2 ...
Let's make another trivial change to the
environment: BUNDLE_PATH: vendor/bundle FOO: bar
And then shut down and re-start the devcontainer.
When we launch a new terminal inside it...
We can run Rails commands without re-installing dependencies!
$ bundle exec rails server => Booting Puma => Rails 188.8.131.52 application starting in development => Run `rails server --help` for more startup options Libhoney::Client: no writekey configured, disabling sending events Puma starting in single mode... * Version 4.3.6 (ruby 2.7.2-p137), codename: Mysterious Traveller * Min threads: 5, max threads: 5 * Environment: development * Listening on tcp://127.0.0.1:3000 Use Ctrl-C to stop
One last thing. We're using the
vendor/bundle subdirectory as a cache, but we don't want to commit the files there to our git repo.
So we should make sure our
.gitignore file includes this path.
# .gitignore # ... /vendor/bundle
And that's how we can keep our development dependencies cached even across container rebuilds. Happy hacking!