In Progress
Unit 1, Lesson 21
In Progress

Caching Files in a Devcontainer

In a devcontainer, system files are ephemeral and subject to being reset any time the container is rebuilt. This can can be a problem for development package dependencies. In this video, we’ll learn a simple way to cache Ruby/Rails project gem packages across container rebuilds.

Video transcript & code

Caching Files

So far we have a development container that we can start up with a docker-compose up command.

$ docker-compose up

We can open a terminal into it

$ docker-compose exec app /bin/bash -l

We can run bundle install to get our development package dependencies.

$ bundle install

And then we can run Rails commands like db:migrate to get our development database ready for action.

$ bundle exec rails db:migrate
== 20200502233927 CreateIslanders: migrating ==================================
-- create_table(:islanders)
   -> 0.0022s
== 20200502233927 CreateIslanders: migrated (0.0024s) =========================

== 20200502234606 RenameTimezzone: migrating ==================================
-- rename_column(:islanders, :timezzone, :timezone)
   -> 0.0103s
== 20200502234606 RenameTimezzone: migrated (0.0106s) =========================

But there's a problem.

Let's say we shut down the container...

And then we make a small change to our docker-compose.yml file. Maybe we add a port mapping, which is something we'll talk about more in an upcoming video.

version: "3.2"
      context: .
      dockerfile: Dockerfile
      - type: bind
        source: ..
        target: /workspace
    working_dir: /workspace
    command: sleep infinity
      - "3000:3000"

And then bring the container up again...

And we open a terminal...

And then run another Rails command...

$ bundle exec rails s
bundler: command not found: rails
Install missing gem executables with `bundle install`

This time it acts like we never ran bundle install!

We have to run bundle install all over again. Which takes a while!

This is not ideal.

The problem here is that our development machine's filesystem is ephemeral, and subject to being reset anytime we update our devcontainer configuration. In many ways this is a good thing, because it forces us to explicitly record every bit of configuration necessary to run our app! But it turns out that there are certain filesystem changes we would like to cache from one machine recreation to the next. For this app, that includes the Ruby Gem packages that Bundler installs.

Fortunately, we do have a place we can keep files that will last longer than our dev container filesystem.

We have our project's source code directory, which is connected into the devcontainer every time we run it!

If we can convince bundler to put the files it installs somewhere in this directory, they will stick around in our project directory instead of getting wiped out the next time the container is rebuilt.

How we get development tools to change where they put their files varies from tool to tool. In the case of Bundler, a bit of research turns reveals that we can control where it puts files with an environment variable.

We add an environment: section to our docker-compose.yml file.

In this section we set the BUNDLE_PATH variable to vendor/bundle, which is a conventional location for Bundler packages when keeping them within the project directory.

version: "3.2"
      context: .
      dockerfile: Dockerfile
      - type: bind
        source: ..
        target: /workspace
    working_dir: /workspace
    command: sleep infinity
      BUNDLE_PATH: vendor/bundle

We re-start our devcontainer environment.

And open a new terminal into it.

Now we can see that BUNDLE_PATH is set inside this container to vendor/bundle.


We run bundle install to fetch our project dependencies.

$ bundle install

Once this is done, we can see that the packages have been installed inside the project workspace.

$ ls vendor/bundle/ruby/2.7.0/gems/
actioncable-              dotenv-rails-2.7.6       mini_portile2-2.4.0           rb-fsevent-0.10.4
actionmailbox-            e2mmap-0.1.0             minitest-5.14.2               rb-inotify-0.10.1
actionmailer-             erubi-1.9.0              msgpack-1.3.3                 regexp_parser-1.8.2

Let's make another trivial change to the docker-compose.yml

      BUNDLE_PATH: vendor/bundle
      FOO: bar

And then shut down and re-start the devcontainer.

When we launch a new terminal inside it...

We can run Rails commands without re-installing dependencies!

$ bundle exec rails server
=> Booting Puma
=> Rails application starting in development
=> Run `rails server --help` for more startup options
Libhoney::Client: no writekey configured, disabling sending events
Puma starting in single mode...
* Version 4.3.6 (ruby 2.7.2-p137), codename: Mysterious Traveller
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://
Use Ctrl-C to stop

One last thing. We're using the vendor/bundle subdirectory as a cache, but we don't want to commit the files there to our git repo.

So we should make sure our .gitignore file includes this path.

# .gitignore
# ...

And that's how we can keep our development dependencies cached even across container rebuilds. Happy hacking!