Authorizing Google Cloud Platform Service Accounts from a Docker Container Running in Heroku

Recently we implemented a system using Google Cloud's Pub/Sub messaging service and My Business API to capture updates to Google Places businesses with a NodeJS server. Google Cloud requires applications to authenticate as service accounts in order to interact with their APIs. Service accounts allow a non-human user to authenticate and access Google's APIs.

The Google Pub/Sub client library loads service account credentials into a GOOGLE_APPLICATION_CREDENTIALS environment variable from an absolute path to a .json file in the filesystem. This meant that we needed access to a service-account.json file in our application's environment in order to authenticate and authorize our service account.

One of our other main requirements for this application was that we deploy the app to Heroku. We also deploy most of our projects in Docker containers, so we needed to figure out how to add the service account's credentials file to the Docker container's filesystem during Heroku's build process.

First (Unsuccessful) Attempts

The first solution we tried was adding the contents of the service-account.json file as a Heroku environment variable. Heroku accepted and saved the value, but it was not available within our Docker container. When we tried to read the value, we got a parsing error at the first " in the file. It seemed as though Heroku was unable to parse raw JSON stored in an environment variable.

Another solution we tried was using one of the "off-the-shelf" Heroku buildpacks that claim to help with this issue. They're both essentially just shell scripts that echo the value stored in a GOOGLE_CREDENTIALS environment variable into a file in the .profile.d/* directory. This directory is similar to /etc/profile.d in standard Unix filesystems, where shell startup files are typically stored.

Unfortunately, one of the current limitations of Heroku's "container" stack is that it does not run .profile or .profile.d/* scripts when booting a dyno. So neither of the Heroku buildpacks worked for us.

Solution

We're still dealing with a Linux filesystem in Docker though, and the filesystem has access to Heroku environment variables during the build process. So the question became: how can we output the contents of a Heroku environment variable to a .json file in the container's filesystem during the build process?

We ended up writing a small shell script that executes in our production Dockerfile to do just this.

First we added the contents of the service-account.json file to a GOOGLE_CREDENTIALS environment variable in Heroku.

We then added the following script ( add-google-credentials.sh ) to the root of our project:

#!/bin/sh

echo "Generating google-credentials.json from Heroku environment variable"

echo $GOOGLE_CREDENTIALS > google-credentials.json

exec "$@"

This script takes the value stored in the GOOGLE_CREDENTIALS environment variable and echoes it into a new .json file in the Docker container's filesystem, creating a fresh copy each time we build the container.

The exec "$@" line replaces the parent process with the current child process. This is important in Docker containers for signals to be proxied correctly, otherwise we might end up with data loss or orphan processes. More information about this can be found in this interesting Unix Stack Overflow discussion.

Finally we needed to actually call this script during the Docker build process, so we added the following two commands to our Dockerfile :

COPY add-google-credentials.sh /app/add-google-credentials.sh
ENTRYPOINT ["sh", "/app/add-google-credentials.sh"]

The first command copies the shell script from our project's root directory into the container's root directory.

The second command basically says "every time this container builds, run the add-google-credentials.sh script."

These changes allowed us to store a GOOGLE_APPLICATION_CREDENTIALS environment variable in Heroku with the value google-credentials.json, which is an absolute path to the service account credentials inside our container. After putting this together we were finally able to interact with the Google Cloud APIs that we needed for this project! 🎉

‍