When trying to run gsutil in a kubernetes Job as nobody to backup stuff to Google Cloud Storage(GCS), I encountered simple error messages like

OSError: Permission denied

But it didn’t say where or how the permission was denied! It worked fine if the container was run as root user so the problem is not with Google Cloud. I searched around and there are 2 occasions that gsutil needs disk access

The first one is the gcloud profile on local file system. Before using gsutil I have to authenticate with

gcloud auth activate-service-account --key-file=service-account-key.json

The command above will create a CloudSDK configuration directory in current user’s home directory. Obviously nobody doesn’t have a home so this will fail. To fix this, an environment variable can be given like

export CLOUDSDK_CONFIG=/tmp/.gcloud

The next one is harder to find, I suspected an option called state_dir is a place to look at and it turned out I was right. From its source code, the state_dir defaults to .gsutil directory in user’s home directory which is also a problem for nobody user. The fix is to override the option in the gsutil command like this

gsutil -o "GSUtil:state_dir=/tmp/.gsutil" cp $backup_file gs://$gcs_bucket

Now it works as expected. 🙂