Use Justfiles to Work with MongoDB

May 21, 2021 - development mongodb beginner

I have started to write Justfiles for all my new projects. There's a good chance you haven't heard of Just or Justfiles. They're very similar to Make and Makefiles. Whereas Make is designed to build artifacts from source files, and combine them in potentially complex ways, Just is "just" a task runner.

Just can be installed with a bunch of different package managers, or you can just download a prebuilt binary.

A short Justfile, containing a single recipe to import some data into a MongoDB cluster looks like this:

import:
    mongoimport $MDB_URI --collection recipes recipe_data.json

The file is literally called Justfile, with no file extension. You would usually put this file at the root of your project directory. This will all seem familiar, if you've ever worked with Makefiles!

This specifies a recipe called "import". When I ask Just to run the import target, it will run the mongoimport command I've written on the line below. The environment variable $MDB_URI specifies the cluster (and database) that the data stored in recipe_data.json will be loaded into. As long as I've set the $MDB_URI environment variable correctly, then if I go to the directory that holds the Justfile and run the following, it will run the mongoimport command and load the data into my MongoDB database.

just import

Back to that $MDB_URI variable. Just has this great feature – it automatically loads environment variables from a .env file you put in the same directory. So if I create a file called .env containing the following, then when I run the just command, I'll be able to refer to the $MDB_URI variable, like I did in the Justfile above.

# .env

MDB_URI=mongodb+srv://username:password@timeseries.abcde.mongodb.com/cocktails?retryWrites=true&w=majority

The idea in this case is that local configuration, like the connection string for connecting to a MongoDB cluster, is stored in the .env file. The .env file is not committed to revision control, whereas the Justfile can be committed and then reused by other team members.

I'll often have a handful of targets for setting up my database in any MongoDB project that I'm working on:

# Justfile

# Recreate the database with sample_data.
all: init import

# Drop all the collections, and recreate them with associated indexes.
init:
    mongosh $MDB_URI init_database.js

# Import recipe and review data into the cluster at $MDB_URI.
import:
    # Load recipe_data.json into the recipes collection:
    mongoimport $MDB_URI --collection recipes recipe_data.json
    # Load review_data.json into the reviews collection:
    mongoimport $MDB_URI --collection reviews review_data.json

Let's break this down:

Lines beginning with # are comments. They're (mostly) ignored by Just.
The first target is the default target, so it will be run if you execute just or just all.
The items after all: are the targets this target depends upon. They'll be executed before any commands associated with the all target. (All doesn't have any commands associated with it. It's just there to collect together the other targets and run them in order.) In this case, executing the all target will first run the init target, and then the import target.
A target is often followed by a list of commands to be run when the target is executed. The commands should be indented, but it doesn't matter if they're indented by tabs or spaces (unlike Make, which is notoriously picky and weird!). Those commands will be run in order when Just executes the target, and if any of them fail, Just will exit with an error.

So with this combination of Justfile and .env files, I've gained two things:

I've got a quick and easy way to run commands related to my project.
I've got documented commands related to my project, for easy reference.

Speaking of documentation, now that I added some comments to my Justfile, if I run just --list I get the following output:

❯ just --list
Available recipes:
    all    # Recreate the database with sample_data.
    import # Import recipe and review data into the cluster at $MDB_URI.
    init   # Drop all the collections, and recreate them with associated indexes.

It prints out each target, along with the comment above each one. Self-documenting code! I ❤ it.

Why Don't You Use Make?

I used to use Makefiles for this kind of thing in the past. I continued to use Make even after I became aware of Just; I couldn't quite see the benefit of using a less ubiquitous tool. But since I switched I've become a total convert, mostly for the following reasons:

The syntax is much less weird and tricky than Make. (Make differentiates between indentation by tabs and spaces, and you end up using both in the same file for different reasons!)
There's no need to declare a .PHONY target.
Just automatically loads variables from a .env file, if one is available.
Just allows you to pass arguments to a command on the command-line.
Just allows you to write commands in any language you like, not just shell!

Some Useful Snippets

Here are a few useful targets that I've found myself using across different projects:

Connect To Your Database

I have the following short recipe in every Justfile! It runs mongosh against the MDB_URI that's stored in the local .env configuration, so you'll get an interactive connection to your database you can use to run ad-hoc commands against your data.

# Connect to the configured database
connect:
    mongosh $MDB_URI

Initialize Your Database For Development

I usually have a JavaScript file that can be run against a new database, with mongosh, to initialize collections and indexes in a reproduceable way. An example init script looks like this:

// init.js

// Drop the collection if it already exists:
db.scores.drop();
// Create a unique index on "username":
db.scores.createIndex({ "username": 1 }, { unique: true });

// Drop the collection if it already exists:
db.stock_exchange_data.drop();
// Time series collections need to be created ahead-of-time.
// Have you tried MongoDB's new time series collections? They're awesome!
db.createCollection("stock_exchange_data", {
    timeseries: {
        timeField: "ts",
        metaField: "source",
        granularity: "hours"
    }
});

Then I'll set up an "init" recipe to run it:

# Initialize the configured database
init:
    mongosh $MDB_URI init.js

Now I can run just init to run the "init.js" script against my database.

A Parameterized Target To Run Mongosh Scripts

If I have a bunch of scripts I may want to run against my database, I'll add a "run" recipe that can run any of the scripts.

The following target makes use of recipe parameters. In the example below, "script" is a variable that is passed on the command-line, and then referred to using {{script}}

# Run a script on the configured database
run script:
    mongosh $MDB_URI {{script}}

With the recipe above in your Justfile, you can execute a mongosh script called "create_view.js" like this:

just run create_view.js

Other Features

The documentation for Just is very long – a good indication of quite how many features it has! Nevertheless, it remains relatively straightforward to get up and running. Some features I've found useful:

You can write recipes in other languages, like Python or JavaScript.
Create a User Justfile that contains recipes you want to be available everywhere.
Just will run, without any dependencies on Linux, MacOS, and Windows. (Although you will need a shell, if you don't have a version of sh installed.). It even includes some handy path manipulation functions for cross-platform path manipulation.

There are a bunch more features. I recommend you scan the documentation to get a better picture of all it can do.

In Conclusion

I've seen a bunch of benefits since I started to add Justfiles to all my projects. It speeds up my day-to-day operations when I'm working with MongoDB (and other tools). When I come back to a project after a while, I can see the commands needed to create and work with the database, by running just --list or just scanning the Justfile. The ability to document each recipe and easily list them is super-helpful, and it also provides a way to help others work with the projects I've worked on.

The examples in this post are relatively straightforward, but Just really comes into its own when you want a recipe to run multiple commands, each with their own list of flags that you would otherwise have to remember. They're an excellent alternative to writing shell scripts!