8 posts tagged with "api keys"

Many public APIs choose to use API keys as their authentication mechanism, and with good reason. In this article, we’ll discuss how to approach API key security for your API, including:

• why you should consider API key security
• design options and tradeoffs
• best practices of API key authentication
• technical details of a sound implementation

This article is language agnostic and doesn't provide a particular solution for PHP, Python, TypeScript, C# etc but every language should afford the capabilities that would allow you to build an appropriate solution.

There is an accompanying video presentation of this content: API Key Authentication Best Practices

Why API Keys? Why not?​

I talked about this in more detail in Wait, you’re not using API Keys? but in summary, API keys are a great choice because they are plenty secure, easier for developers to use vs JWT tokens, are opaque strings that don’t give away any clues to your claims structure, and are used by some of the best API-first companies in the world like Stripe, Twilio, and SendGrid.

Perhaps the most legitimate complaint against API keys is that they are not standardized, which is true, but — thanks to programs like GitHub’s secret scanning program — some patterns are starting to emerge.

If you’re building a public API, API-key authentication is much easier for a developer to configure and learn. They work great in curl and, provided you follow some of the best practices outlined here, are plenty secure.

The main case where I would not advocate for using API keys is for operations that are on behalf of an individual user. For this, OAuth and JWT is a much better fit. Examples of APIs that do and should use OAuth are Twitter and Facebook. However, if you’re Stripe and the callee of your API is an ‘organization’ and not a user, API keys are a great choice. Perhaps the best example of this is the GitHub API, which uses both: API-keys for organization-level interactions and JWT for on-behalf of users.

Decisions to make​

The best practices for API key authentication are becoming somewhat recognizable now, but there is a dimension where we still see some variability in the implementation of API keys: to make the key retrievable or irretrievable.

The world of API-key implementations is divided into two groups. The first will show you your API key only once. You'll need to copy it and save it somewhere safe before leaving the console. This is irretrievable. Typically the keys are unrecoverable because they are not actually stored in the key database, only a hash of the key is stored in the database. This means, if lost, the keys can genuinely never be recovered. Of course, in the case of a loss you can usually regenerate a new key and view it once.

The other group allows you to go back to your developer portal and retrieve your key at any time. These keys are typically stored encrypted in the database. Meaning if the database is stolen, the thief would also need the encryption codes to access the API keys.

The tradeoffs here are tricky, and there are two schools of thought

1. Irretrievable is better because it’s more secure. The keys are stored via a one-way encryption process so they can never be retrieved, or stolen from the database in a worse case scenario.
2. Retrievable offers good-enough security with some advantages, and it’s easier to use. The keys are stored encrypted via reversible encryption. One potential security advantage is that users are less likely to feel pressured to quickly store the key somewhere to avoid losing it. A person that follows best practices will use a vault or service like 1password. However, some users will take the convenient path and paste it into a .txt file for a few minutes thinking, “I’ll delete that later.”

So what are some examples of APIs that support recoverable vs. unrecoverable today??

Irretrievable: Stripe, Amazon AWS

Retrievable: Twilio, AirTable, RapidAPI

There is some correlation between services that protect sensitive information and services seem more likely to use irretrievable, while services that are less sensitive choose retrievable for ease of use and good-enough security.

The choice is yours. Personally, I lean a little toward retrievable because I know that I personally have made the mistake of quickly pasting a newly generated irretrievable key into notepad and forgetting about it. You may come to a different conclusion for your own API key authentication.

Best Practices of API Key Authentication​

The bit of API key authentication advice you’ve been waiting for… the best practices of API key auth based on the patterns observed in the API world and our experience building our own API key authentication service for Zuplo.

1/ Secure storage for the keys

Depending on your choice of retrievable vs. irretrievable, you’ll need to take a different path. For irretrievable keys, it’s best to store them as a hash, probably using sha256 and ensure that they are stored as primary key (this will help avoid hash collisions, unlikely but possible so you should build in a retry on create and insert).

For retrievable, you’ll need to use encryption so that the values can be read from the database to show to the user at a later date. You have a few choices here, like storing the keys in a secure vault, or using encryption programmatically to store in a standard database and manage the keys yourself.

2/ Support a rolling transition period

It’s critical that you allow your users to roll their API keys in case they accidentally expose it, or just have a practice of periodically changing them out. It’s important that this ‘roll’ function either allows for multiple keys to exist at the same time or allows the setting of an expiry period on the previous key, otherwise rolling the key will cause downtime for any consumers that didn’t get the chance to plug-in the new key before the last one expired. Here’s Stripe’s roll dialog:

In Zuplo, we allow folks to have multiple keys so they can temporarily add another key and delete the old one as soon as they’re done with the transition.

3/ Show the key creation date

It’s important to show developers when the key was created so they can compare the date to any potential incidents. This is especially important if you support multiple keys so that users can differentiate between old and new.

4/ Checksum validation

Since checking the API key will be on the critical path of every API call, you want to minimize latency. This is one of the reasons that you’ll want to add a checksum to your API key. Here’s an example API key from Zuplo:

zpka_a5c5e56x54c4437fbd6ce7dee9185e_631238

The last section _631238 is a checksum that we can use to verify in the request pipeline whether this even looks like a valid key. If not, we can simply reject the request and avoid putting load on the API key store.

5/ Support secret scanning

One of the reason we have the unusual beginning of the key "zpka_" is so that we could participate in programs like GitHub’s secret scanning. This allows us to create a regular expression that allows GitHub to inform us if an API key is accidentally checked into a repo. Then we can automatically revoke the key and inform its owner of the event. We also use the checksum to double-check that it’s one of our keys before locating the service and owner.

At Zuplo, we participate in the GitHub secret scanning program so that we can offer these services to any customer using our API-key policy.

(Aside: Yes, the example key above triggered our secret scanning when I checked this article into GitHub and we got notified about a token leak 👏😆)

6/ Minimize latency and load on your API Key storage

To reduce the latency on every API request, you might consider using an in-memory cache to store keys (and any metadata read from the API Key Store). If you have a globally distributed system, you’ll want multiple caches, one in each location. Since Zuplo runs at the edge, we use a high-performance cache in every data center. To increase security it's important to consider only caching the one-way hashed version of the API-key (taking care to avoid hash-collisions by doing a pre-hash collision check at the point of key-creation, using the same hash algorithm).

You’ll need to choose an appropriate TTL (time-to-live) for your cache entries which has some tradeoffs. The longer the cache, the faster your average response time will be and less load will be placed on your API Key store - however, it will also take longer for any revocations or changes to key metadata to work.

We recommend just a couple of minutes maximum; that’s usually plenty to keep your latency low, a manageable load on your API Key Store, and be able to revoke keys quickly.

If this is important to you, you might want to design a way to actively flush a key from your cache.

7/ Hide keys until needed

Today, everybody has a high-quality camera in their pocket. Don’t show an API key on-screen unless explicitly requested. Avoid the need to show keys at all by providing a copy button. Here’s the supabase console, which almost gets full marks, but would be even better if it provided a copy option without needing me to reveal the key visually.

8/ Attention to detail — keys need to be copied and pasted

Sometimes it’s the little things in life; for example, try double-clicking on key-1 below to select it. Then try key-2.

key-1: zpka-83fff45-1639-4e8d-be-122621fcd4d1

key-2: zpka_a5c5e56x54c4437fbd6ce7dee9185e_631238

Note how much easier it is to select the API key in snake_case?

9/ Consider labeling your keys

A number of services are increasingly doing this to help their customers, but mostly to help their support teams. For example, in Stripe they have a convention as follows:

• sk_live_ - Secret Key, Live version
• pk_test_ - Publishable Key, Test version

This not only supports GitHub secret key scanning above, but it can also be invaluable to your support team when they can easily check if the customer is using the right key in the right place. Your SDK can even enforce this to prevent confusion.

The downside to key labeling is that if the key is found without context by a malicious user - they can discover which services to attack. This is one advantage of using a managed API key service that dissociates the key from any specific API. GitHub have a great article - Behind GitHub’s new authentication token formats.

A canonical flow through an API key check​

Stacking all of that together, here’s a flow chart showing the canonical implementation of the API key check using all these practices above.

Conclusion​

API keys are a great approach if you want to maximize the developer experience of those using your API, but there are quite a few things to think about when it comes to API key security. An alternative to building this yourself is to use an API Management product with a gateway that does all the work for you and includes a self-serve developer portal. Examples include Apigee, Kong, and — of course — Zuplo.

The Author​

Before founding Zuplo, Josh led Product for Stripe’s Payment Methods team (responsible for the majority of Stripes payment APIs) and worked at Facebook and Microsoft, where he founded a number of services, including Azure API Management.

Updated December 4 2022 to update recommended hashing algo to sha256 based on community feedback.

Today we’re announcing that Zuplo offers API Key Scanning on GitHub for API keys generated in Zuplo.

According to the most recent GitGuardian report, in 2021 over 6 million secrets were leaked, which was 2x 2020’s total and 3 in every 1,000 commits exposed at least one secret. The massive Heroku security incident in April 2022 was caused by API Keys checked into source control. It’s no surprise then that since we opened Zuplo up publicly we’ve seen a lot of excitement about our API Key Management capabilities. We’ve written why we think API Keys are the best way to secure your API here and now we make it effortless to secure both you and your customers with API Key Scanning.

"Heroku determined that the unidentified threat actor gained access to the machine account from an archived private GitHub repository containing Heroku source code."

Respecting the developer workflow is one of our central tenets at Zuplo, which is why we designed it with GitOps in mind. Starting today, if one of the API keys for one of your APIs in Zuplo shows up in a public repo on GitHub you’ll receive an alert from Zuplo notifying you of the token and the URL where the match was found. You can also choose to have Zuplo notify your customer on your behalf.

Zuplo API Key management includes:

• secure storage and management of keys and metadata - with an admin UI and API to manage consumers.
• integrated developer portal with self-serve key management for your customers.

If you've already built your own API Key solution we can easily integrate Zuplo authentication with custom policies or even help you API key to Zuplo for even greater protection. It's never too late to make hosting your API much easier.

API Key Leak Prevention is part of our Business and Enterprise subscriptions.

We recently discussed some best practices for versioning your API with (spoiler) a strong recommendation that you should require that clients indicate the version of the API they were designed for. Go read the article for more details.

I’ve been giving a talk at a few conferences recently about how the world’s fastest-growing companies develop their public API, which included a piece on versioning. One question I commonly get from attendees after the talk is, “How do I move clients off the old version of my API?”, where clients might be internal depts, long-term customers or even teammates who don’t want to change that UI code that is dependent on v1.

In most cases, you can’t just turn off the old version. In the cases above you’ll cause business harm by breaking the marketing department or taking down one of your loyal customers. These folks have priorities and updating to /v2/ of your API is probably not at the top of the list. So how might you create more urgency?

Creating Urgency​

We do not recommend just turning off the API permanently. Instead, you can take the approach of scheduling a ‘brownout’ or timed, temporary downtime. This is where you take the API down during a low-impact period for a short amount of time, maybe at 3 am in the morning, for 2 minutes. This is probably enough to trigger a bunch of alarms and service alerts that make the impact of the upcoming breaking change clear to the consumer.

We’d recommend sharing the schedule of the planned outages so people aren’t surprised at all and know what the glideslope to actual deprecation looks like. Some businesses share that the version of the API they are using will be fully deprecated on a specific date but, knowing that some clients will not upgrade in time, activate the first brownout at this time and notify clients that they have one more week to complete their upgrade. This can you make you appear like you’re being generous, you’re giving them more time than they were told they would have, but they still get the shock of alerts firing.

Another technique to encourage folks to move off an old API, and combines well with brownouts, is to start to add deliberate latency to the old version of the API. You can use an API gateway to do this and, as time progresses, you can increase the latency to multiple seconds even - depending on the use case.

Again, this is a business decision but once you’ve decided you need to create urgency amongst consumers of your soon-to-be-deprecated, old version of your API. This is a better approach to just going dark on active customers on the scheduled date.

We introduced a sleep and brownout policy to Zuplo to make this even easier. If you want to try it out for yourself and schedule a brownout, go sign up for Zuplo.

Weekly Zoom Chat

Are you starting work on, or in the middle of building, a public customer or partner API? Register for our "Building a Customer API" weekly chat on Zoom every Thursday at 3pm ET/12pm PT and learn from featured guests and other developers building public APIs.

Planning to build a new public customer or partner API but not sure where to start your research? Connect with fellow developers and nail development of your new API by joining our meetup on Zoom, "Building a Customer API".

On June 9, 2022 at 3pm ET/12pm PT we'll be joined by Utsav Shah of fast growing startup Vanta (and host of the Software at Scale podcast). He'll share his experience building a customer API and challenges when implementing rate limiting. Then we'll have an open discussion on implementing rate limiting and exposing APIs to customers and partners.

Register Here

About Utsav Shah

Utsav Shah is a software engineer at Vanta and host of the Software at Scale podcast. Before joining Vanta, Utsav was responsible for enabling product velocity and ensuring the reliability of Dropbox’s monolith Python web application and large async systems like Cape.

About Vanta

Our mission at Vanta is to be a layer of trust on top of cloud services, to secure the internet, increase trust in software companies, and keep consumer data safe. Think of us as your automated security and compliance expert.

Building a Customer API​

At 3pm ET/12pm PT every Thursday, Zuplo hosts a 1-hour round-table discussion virtually (Zoom) to help developers plan for and build their public customer or partner APIs. We feature one attendee each week who has recently built a public customer or partner API and who will share how they approached the process and what they learned, then stick around to answer questions and discuss. This chat is limited to 25 spaces per week to keep the conversation flowing and fruitful. We'll start with a Q&A with the featured guest and then have an open discussion immediately following.

Have more questions, check out our event FAQ.

At some point, you’re going to make an update to your API that would break existing clients if they don’t change their code. That’s OK, change happens.

However, it is critical that you give yourself the option to do two things when this situation arises:

1. Support multiple versions of your API simultaneously (so that you can give older clients the opportunity to migrate to your latest version).
2. Inform a client that the version they have coded for is no longer supported

Life is much easier if you think about versioning from the beginning of the lifecycle of your API. A key decision to make is how you want to design versioning into your API; that is, how should the client communicate the version of the API they are coded to work with?

There are two primary options:

1. URL-based versioning - where the version is encoded directly in the URL, e.g. /v1/charges. This is the most common approach and is used by most large API-first companies like Stripe, Twilio, SendGrid and Airtable.
2. Header-based versioning - where the version is in a header; either a customer header like api-version: v3 or as part of the accept header, e.g. accept: application/vnd.example+json;version=1.0.

Our recommendations​

1/ Keep those options open

First and foremost, we strongly recommend that you make the version a mandatory part of all requests received by your API. So any request that doesn’t include the version should receive a 4xx error code (400 if it’s a required header, 404 if it is missing from the URL).

This is the most important decision because it means you always have both options outlined at the opening of this post.

2/ Keep it simple

After this, we recommend URL-based versioning. There is plenty of precedent in the market and it’s the easiest for developers to use - it’s easier to test with CURL, call with fetch, test in a browser if you support GET requests. It’s just easier.

3/ Use headers if you’re passionate about building a pure REST implementation

The primary reason to use headers over URL-based versioning is to avoid violating a REST principle that states a URI should refer to a unique resource (v1/foo and v2/foo could theoretically point to the same resource). However, such pure implementations of REST APIs have not proven popular and are trickier for developers to use.

4/ Don’t break rule 1

There are examples of APIs in the public domain that have a default version if the client doesn’t specify a version. Here’s GitHub’s documentation on their API:

Even though it encourages developers to use the version header, the API still works without it and just assumes v3. We think this is a mistake; if GitHub upgrades to v4 and that becomes the new default, all of those old clients that didn’t follow the best practice will experience unpredictable behavior and strange errors.

The day comes for most startups, even those that aren’t API-first SaaS businesses. When a large partner or customer — who can’t use your UI at the scale they need — requests an API. This is a high-quality problem - a customer that integrates with your API has higher switching costs and is more likely to be retained.

Sharing an API is a non-trivial exercise that can eat a surprising amount of your eng team’s time and there are three pillars that you need as a minimum bar:

1/ Authentication​

How will the partner authenticate securely with your API?

Most startups go with API-key authentication because it’s secure and the easiest to use for developers (more on this here) - this is the right choice in my experience. There’s a lot to consider when building a secure API-key solution:

• Where do I store the keys and do so securely?
• How do I let partners self-serve?
• Can partners easily roll keys to ensure best practice security?
• How do I implement read-once key infrastructure for best-practice security?

This can take even the best engineering teams multiple weeks to ship, and be an ongoing burden to maintain and scale that will reduce your team's agility.

2/ Documentation​

How will the developer learn how to use your API?

Your partner’s developers will need documentation to learn how to use your API. Maybe a shared google doc is enough? But, your engineering team will spend much less time helping partner eng folks if they have real API docs - generated using open standards like Open API - that have integrated test clients and API keys. This will save your team time and your partner’s eng team - they’ll thank you for it!

3/ Protection​

How do you stop a rogue for-loop in your partners’ code from taking down your whole business?

A partner hitting your API with a Denial of Service attack is rarely a deliberate, malicious act. Rather, it’s probably a simple coding error that results in an infinite loop that - without protection - can take down your API, or your whole business. That’s why rate-limiting is an essential part of any shared API.

Wait... there are more than three pillars?​

Those three pillars are just the basics - Ideally, you have a strategy around versioning, analytics, composition, routing, caching, and have the right abstraction to deal with unexpected needs from new partners.

Any API program eventually runs into a customer that needs JWT or mTLS security - will your solution easily allow the layering of different security options? Can you easily maintain both versions of your API? Can you quickly implement a brown-out to push partners onto v2?

We recently shared some reasoning on why we think API keys are the best authentication approach for your public API.

We think this is so important that we built it as a feature of the Zuplo gateway. Our API Key management includes:

• secure storage and management of keys and metadata - with an admin UI and API to manage consumers.
• integrated developer portal with self-serve key management for your customers.

| Note, if you've already built your own API Key solution, and have a database store with your keys and users, we can easily integrate zuplo authentication with custom policies. It's never too late to make hosting your API much easier.

See it all in action in this 2-minute video:

Try it out now, for free at portal.zuplo.com

Have you noticed something the best API companies have in common?

Folks like Stripe, Twilio, Airtable, SendGrid, and many more?

Yep, they all use API Keys as the primary authentication method for their APIs. Why is this?

There are two primary reasons

1/ Security​

While there is no formal protocol for API Keys and most implementations have some level of variability - at least compared to standards like OAuth and OpenID Connect - they offer a good level of security, arguably greater than using JWT tokens for a few reasons.

• Revocability - API Keys can be quickly revoked for any reason, whereas JWT tokens are hard to revoke and often require the reset of an entire certificate or tenant.
• Opaqueness - unlike JWT tokens, which can be easily decoded using services like jwt.io, API keys are completely opaque and don’t reveal any hint of your internal authorization mechanism.
• Self-management - a good API program with API keys allows consumers to manage API keys themselves and, in the event of a leak (or accidental push to a GitHub repo), the consumer can quickly revoke and roll their keys. If the same mishap occurs with a JWT token, it is typically harder for the consumer to self-serve and revoke the validity of the JWT token.

2/ Optimizing Time to First Call (TTFC)​

Great API companies focus on developer experience and obsess about metrics like time-to-first-call. That is, how long does it take for a developer to find your documentation and get everything set up, and successfully invoke your API.

If you choose an authentication option that has some complexity (token acquisition, token refresh, etc) you are automatically increasing that Time to First Call and thus reducing the conversion rate of developers that get to know and adopt your platform.

Summary​

Of course, there are valid reasons to use a more complex authentication method for your API, particularly if the consumer is doing work on behalf of another identity (e.g. accessing the GitHub API on behalf of a GitHub user) then OAuth makes the most sense.

However, if you’re primarily identifying the B2B business partners, API keys are a secure choice that is easy to use and will optimize the conversion funnel for winning new developers.