How to get your own API credentials

Here we describe how to obtain different types of credentials that can be important when working with a Google API:

  • API key (not relevant to all APIs)
  • OAuth 2.0 client ID and secret
  • Service account token
  • External account token (“workload identity federation”)

This can be important for both users and developers:

  • Package authors: If you are writing a package to wrap a Google API, you may provide some built-in auth assets so that things “just work” for your users. Regardless, you will need credentials to use during package development and in testing.
  • Package users: Wrapper packages may or may not provide default auth assets. If they don’t, you are required to provide your own. Even if they do, you may prefer to bring your own, to have more control over your own destiny. With your own credentials, you avoid sharing quota with other users, which can reduce time-consuming errors and retries. If you use your own credentials, you are no longer at the mercy of someone else choosing to roll the credentials when you least expect it.
  • Everyone: The best method for auth in non-interactive settings is to use a service account token or workload identity federation, which require some advance setup.

Note that most users of gargle-using packages do not need to read this and can just enjoy the automatic token flow. This article is for people who have a specific reason to be more proactive about auth.

Get a Google Cloud Platform project

You will need a Google Cloud Platform (GCP) project to hold your credentials.

Go to the Google Cloud Platform Console:

  • https://console.cloud.google.com
  • This may involve logging in or selecting your preferred Google identity.
  • This may involve selecting the relevant organization.

This console is your general destination for inspecting and modifying your GCP projects.

Create a new project here, if necessary. Otherwise, select the project of interest, if you have more than one.

Enable API(s)

Enable the relevant APIs(s) for your GCP project.

In the left sidebar, navigate to APIs & Services > Library.

Identify the API of interest. Click Enable.

If you get this wrong, i.e. need to enable more APIs later, you can always come back and do this then.

Think about billing

For some APIs, you won’t be able to do anything interesting with the credentials hosted in your project unless you have also linked a billing account. This is true, for example, for BigQuery and anything that has to do with Maps. This is NOT true, for example, for Drive or Sheets or Gmail.

If your target API requires a billing account, that obviously raises the stakes for how you manage any API keys, OAuth clients, or service account tokens. Plan accordingly.

If you’re new to Google Cloud Platform, you’ll get to enjoy GCP Free Tier. At the time of writing, this means you get a $300 credit and no additional billing will happen without your express consent. So there is a low-stress way to experiment with APIs, with a billing account enabled, without putting actual money on the line.

API Key

Some APIs accept requests to read public resources, in which case the request can be sent with an API key in lieu of a token. If this is possible, it’s a good idea to expose this workflow in a wrapper package, because then your users can decide to go into a “de-authed” mode. When using the package in a non-interactive or indirect fashion (e.g. a scheduled job on a remote server or via Shiny), it is wonderful to NOT have to manage a token, if the work can be done with an API key instead.

Some APIs aren’t really usable without a token, in which case an API key may not be relevant and you can ignore this section.

  • From the GCP Console, in the target GCP Project, go to APIs & Services > Credentials.
  • Do Create credentials > API key.
  • You can capture the new API key via clipboard right away or close this pop-up and copy it later from the Credentials page.
  • In any case, I suggest you take the opportunity to edit the API key from the Credentials page and give it a nickname.

Package maintainers might want to build an API key in as a fallback, possibly taking some measures to obfuscate the key and limit its use to your package.

What does a user do with an API key?

Package users could register an API key for use with a wrapper package. For example, in googlesheets4, one would use googlesheets4::gs4_auth_configure() to store a key for use in downstream requests, i.e. after a call to googlesheets4::gs4_deauth():

library(googlesheets4)

gs4_auth_configure(api_key = "YOUR_API_KEY_GOES_HERE")
gs4_deauth()

# now you can read public resources, such as official example Sheets,
# without any need for auth
gs4_example("gapminder") |>
  read_sheet()

OAuth client ID and secret

Most APIs are used to create and modify resources on behalf of the user and these requests must include the user’s token. A regular user will generally need to send an OAuth2 token, which is obtained under the auspices of an OAuth “app” or “client”. This is called three-legged OAuth, where the 3 legs are the client, the user, and Google.

The basic steps are described in the Prerequisites section for doing Google OAuth 2.0 for Mobile & Desktop Apps:

  • From the GCP Console, in the target GCP Project, go to APIs & Services > Credentials.
  • Do Create credentials > OAuth client ID.
  • Select Application type, either “Desktop app” (the most common type used with gargle) or “Web application” (useful for the pseudo-OOB flow).
  • You can capture the client ID and secret via clipboard at this point or, as we recommend, download the full information as JSON. We recommend using the JSON, as this conveys the client type (desktop vs. web) and any redirect URIs (important for the web type).
  • At any time, you can navigate to a particular client ID and click “Download JSON”.

Two ways to package this info for use with gargle:

  1. Use gargle::gargle_oauth_client_from_json(). This is the preferred workflow, because the JSON conveys the client type (desktop vs. web) and any redirect URIs (important for the web type).
    • Provide the path to the downloaded JSON file.
  2. Use gargle::gargle_oauth_client().

In both cases, I suggest you devise a nickname for each OAuth credential and use it as the credential’s name in GCP Console and as the name argument to gargle::gargle_oauth_client_from_json() or gargle::gargle_oauth_client().

Package maintainers might want to build this client in as a fallback, possibly taking some measures to obfuscate the client ID and secret and limit its use to your package.

  • Note that three-legged OAuth always requires the involvement of a user, so the word “secret” here can be somewhat confusing. It is not a secret in the same sense as a password or token. But you probably still want to store it in an opaque way, so that someone else cannot easily “borrow” it and present an OAuth consent screen that impersonates your package.

What does a user do with an OAuth client?

Package users could register their own client for use with a wrapper package. For example, in googledrive, one would use googledrive::drive_auth_configure() to do this:

library(googledrive)

google_client <- gargle::gargle_oauth_client_from_json(
  path = "/path/to/the/JSON/that/was/downloaded/from/gcp/console.json",
  name = "acme-corp-google-client"
)
drive_auth_configure(client = google_client)

# now any new OAuth tokens are obtained with the configured client

Service account token

For a long time, the recommended way to make authorized requests to an API in a non-interactive context was to use a service account token. An alternative exists – workload identity federation – which is available to applications running on specific non-Google Cloud platforms, such as AWS or GitHub Actions. If you can use workload identity federation, you probably should (see the next section). But for those who can’t, here we outline the use of a conventional service account.

An official overview of service accounts is given in this official documentation by Google. But note that it’s not necessary to understand all of that in order to use a service account token.

  • From the GCP Console, in the target GCP Project, go to IAM & Admin > Service accounts.
  • Give it a decent name and description.
    • For example, the service account used to create the googledrive docs has name “googledrive-docs” and description “Used when generating googledrive documentation”.
  • Service account permissions. Whether you need to do anything here depends on the API(s) you are targetting. You can also modify roles later and iteratively sort this out.
    • For example, the service account used to create the googledrive docs does not have any explicit roles.
    • The service account used to test bigrquery has roles BigQuery Admin and Storage Admin.
  • Grant users access to this service account? So far, I have not done this, so feel free to do nothing here. Or if you know this is useful to you, then by all means do so.
  • Do Create key and download as JSON. This file is what we mean when we talk about a “service account token” in the documentation of gargle and packages that use gargle. gargle::credentials_service_account() expects the path to this file.
  • Appreciate that this JSON file holds sensitive information. Treat it like a username & password combo! This file holds credentials that potentially have a lot of power and that don’t expire.
  • Consider storing this file in such a way that it will be automatically discovered by the Application Default Credentials search strategy. See credentials_app_default() for details.
  • You will notice the downloaded JSON file has an awful name, so sometimes I create a symlink that uses the service account’s name, to make it easier to tell what this file is.
  • Remember to grant this service account the necessary permissions on any resources you plan to access, e.g., read or write permission on a specific Google Sheet. The service account has no formal relationship to you as a Google user and won’t automatically inherit permissions.

Authors of wrapper packages can use the symmetric encryption strategy described in Managing tokens securely to use this token on remote servers, such as continuous integration services like GitHub Actions. Note that workload identity federation may also be an option on GitHub Actions, if you are able and willing to do the necessary setup (see below).

What does a user do with a service account token?

You could provide the token’s filepath to a wrapper package’s main auth function, e.g.:

# googledrive
drive_auth(path = "/path/to/your/service-account-token.json")

Alternatively, you could put the token somewhere (or store its location in an environment variable) so that it is auto-discovered by the Application Default Credentials search strategy.

Workload identity federation

Workload identity federation lets applications running outside Google Cloud, such as on AWS or GitHub Actions, access Google Cloud resources by using a federated identity instead of a service account key. This eliminates the dilemma of how to safely manage service account credential files.

Unlike a service account key, the configuration for workload identity federation is not a long-lived secret that you need to store and protect. On AWS, it’s a file of non-sensitive metadata that you download once and keep on the instance; the sensitive data is obtained “on-the-fly” at run time. On GitHub Actions, the configuration file is written fresh for each job and contains only a short-lived credential. The configuration is used for a token exchange that ultimately yields a short-lived GCP access token. This access token allows the external application to impersonate a service account and inherit the permissions of the service account to access GCP resources.

So what’s not to love? Well, first, this auth flow is only available if your code is running on AWS, Azure, GitHub Actions, or another platform that supports the OpenID Connect protocol. Second, there’s a non-trivial amount of configuration to do on the GCP side. But once that is done, auth can work automagically with gargle: on AWS, you download a configuration file; on GitHub Actions, the google-github-actions/auth action writes one for you.

This feature is still experimental in gargle. It supports AWS, as well as configurations where the subject token comes from a URL or a file. GitHub Actions is an important example of the URL case: the google-github-actions/auth action writes a configuration file and points the GOOGLE_APPLICATION_CREDENTIALS environment variable at it, so gargle can discover it automatically. gargle currently requires that the configuration uses service account impersonation. In GitHub Actions, this means you must give the action a service_account. For more, see the documentation for credentials_external_account(). Like conventional service account tokens, workload identity federation is a great fit for the Application Default Credentials strategy for discovering credentials. See credentials_app_default() for more about that.

These two links provide, respectively, a high-level overview and step-by-step instructions for this flow:

Further reading

Learn more in Google’s documentation: