Clinton Blackburn recently published A rant about APIs, wherein he listed various frustrations he experienced recently while integrating his product with some third parties.

Reading it brought to mind some of my own reoccuring gripes on the topic. I've spent much of my career operating large ecommerce websites, all of which depended on services hosted apart from the customer-facing infrastructure. Here are a few issues that only become a problem after your new integration has been successfully deployed to Production.

No Sandbox / Test account

Some services only provide a single, production endpoint to build against. These days, that is mercifully uncommon for services with side effects (eg payment processing, outbound email) but it's far from universal.

When you are first implementing the integration, this isn't a big deal - the "production" service isn't in use yet anyway. The problem comes later in the application's lifecycle, when some kind of migration (of code or infrastructure) is required. At that point, the new implementation has no choice but to validate against the production endpoint.

Sandbox service does not effectively match production

Although most critical APIs do now offer sandbox services, many diverge from their production counterparts in significant, non-functional ways. For example, hosting the sandbox service with a different DNS domain, on a different IP network, with weaker access controls than production.

Such differences typically won't trouble a developer - even one reimplementing their application's client while the initial implementation is operating in production - but the operations team may have a very different experience.

Consider an operations team migrating an application onto new infrastructure. They need to validate that the existing application, running on new infrastructure, can continue to work without problems. Differences in sandbox service behaviour can stymie that validation. In one memorable case, I have seen an infrastructure migration aborted hours into a downtime window when it was discovered that a critical service with a wide-open sandbox had strict access controls in production - and a 24hr turnaround time for changes.

Immediate credential rotation / single secret per account

My most common grumble is with services that support only a single secret per account - a password, or a single access key. This limitation makes it impossible to rotate credentials without some disruption to the calling service, and it's particularly pronounced when the issuing service generates the new secret.

The bad time process looks like this:

Click buttons to rotate the secret. (Service calls start failing here)

Store the new secret in a management tool.

Deploy updated config (if required) and restart the calling service.

Confirm that service calls are now succeeding.

Better services will issue a new secret while the existing one remains valid. This allows operators to update calling services with the new secret, and verify that the old one is no longer in use before disabling it.

Zachary Stevens

Other articles

Amazon Linux, And The Perils Of Continuous Release

Dry Run CloudFormation Updates Using Ansible

Increasing Jenkins Executors With Chef

Five Things I Hate About Chef

Gonzo: Increasing Agility by Understanding Risk

Scale Summit 2014

cfgmgmtcamp.eu 2014

Cooking With Jenkins (and Test-Kitchen, and Docker)

Vagrant, VMware, and Berkshelf

Other articles

social