Download Free Install Free

Using Fundamental UI Principles to Design Outstanding APIs

Flávio Juvenal
October 29, 2018

Table of Contents

It doesn’t take a lot of experience to recognize solid library APIs from less-than-functional ones. When dealing with third-party libraries, programmers can usually quickly grasp if they’ll have a hard time using and integrating with them. Most of the time, the difference lies within the API design – at the end of the day, even the most complex problems can easily be solved with a well-designed API.

Take this classic API comparison of urllib2 vs. Requests. To make an authenticated request with urllib2, the following code is required:

import urllib2
gh_url = 'https://api.github.com'
req = urllib2.Request(gh_url)
password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, gh_url, 'user', 'pass')
auth_manager = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_manager)
urllib2.install_opener(opener)
handler = urllib2.urlopen(req)

In comparison, using Requests streamlines the process:

import requests
r = requests.get('https://api.github.com', auth=('user', 'pass'))

This is a pretty drastic difference, which probably explains why most developers choose to use Requests, even though urllib2 is in the standard library (for Python 2).

However, when you ask a programmer what exactly makes an API library stand out, chances are they won’t have a clear-cut answer. This is because it’s quite challenging to narrow down best practices for APIs in a straightforward and actionable way. While saying that an API should be “intuitive” or “simple” is an obvious response, it’s not nearly descriptive enough to guide a developer toward a successful API design.

In this blog post, we’ll try to overcome this challenge by using a few practical concepts along with examples inherited from User Interface (UI) design.

Recognize Your API Is a User Interface

Before introducing the two concepts that will guide you toward successful API design, let’s discuss what the acronym API actually means: an Application Programming Interface implies that someone will use it. Technically, APIs are used to communicate pieces of software, but it’s reasonable to say that humans are the actual API end users – since humans write the code that interacts with APIs. This means we can – and should – consider User Interface principles when designing APIs.

Follow the Principle of Least Astonishment to Find the Right Default Behaviors

The Principle of Least Astonishment (POLA) states that a User Interface behavior should not astonish users. If astonishment is the end result for your users, you might be looking at a potential need for a redesign. That applies to APIs just as well: if the default behavior is strange to users, it’s not appropriate. Surprises aren’t good on APIs: when integrating with APIs, programmers write code according to behaviors they expect. If those expectations don’t match the real API behavior, the integration code will break, which is frustrating for programmers.

The behavior that programmers expect is based on analogies, familiarity, context, etc. In any software with a GUI, for example, you’ll expect CTRL+C/CMD+C to mean copy. But on a Unix terminal, you’ll expect CTRL+C to send a SIGINT to the running program. APIs are the same way: context matters.

A real-world example where the POLA could have prevented a bad API is the old behavior of parseInt in JavaScript. Prior to the EcmaScript 5 standard, when no radix parameter was passed to parseInt, the function returned the integer parsed in octal:

parseInt('010')
// output: 8

While that may seem reasonable as the integer literal 010 means 8 inside JavaScript code, that behavior violates the POLA from an API point-of-view. The most common use case for parseInt is to convert an integer string inputted by the program end-user.

Therefore, the context that matters most here is the layman context where leading zeros aren’t actually significative. For that reason, parseInt was fixed in EcmaScript 5 to ignore leading zeros and parse as decimal when no radix parameter is passed.

Understand how language conventions affect context

You’ve probably heard compliments about great APIs being idiomatic. When discussing Python, the word most used is Pythonic. That’s a fancy way of saying the API successfully follows the patterns and good practices of the underlying programming language. For example, imagine you’re porting a Java class that does standard matrix operations such as multiplication. That Java class has a method multiply that accepts another matrix as its parameter, like this:

class Matrix {
public Matrix multiply(Matrix other) {
// …
}
}

If you (naively) convert that Java class to Python, on the other hand, you would end up with:

class Matrix:
def multiply(other): ...

But there’s actually a much more common way of expressing the multiply method in Python: the multiplication operator __mul__. Using operator overloading, you can write matrix_a * matrix_b in Python, which is much more Pythonic than matrix_a.multiply(matrix_b).

Thus, the best Python port of the Java code would be this one:

class Matrix:
def __mul__(other): ...

There’s a caveat here, though. It’s not enough to just use the syntax of __mul__. It’s also critical to follow __mul__ semantics. In the Python standard library and popular third-party libraries, __mul__ returns a new value, while keeping the original values unmodified. In other words, __mul__ has no side effects. If an API implements __mul__ but breaks that contract, the POLA is violated. To make an idiomatic API, you must not only use familiar syntax, but also follow familiar semantics.

It’s worth noting that what’s idiomatic in a programming language can change over time, especially in rapidly developing languages like JavaScript. For example, it used to be common to pass callbacks all around to write asynchronous code, such as AJAX with XMLHttpRequest. Then, JS APIs started to use Promises instead of callbacks to handle async code. For that reason, an AJAX replacement that uses Promises was introduced, called Fetch. JS is still evolving fast and the next step is to use async/await keywords with Promises as a way of writing more readable, asynchronous code.

Consider POLA for finding what’s safe-by-default

The POLA is also helpful when it comes to figuring out reliable best practices: good APIs prevent mistakes by avoiding dangerous situations by default. For example, before Django 1.8, if someone created a ModelForm without specifying which fields it had, that form would accept all model fields. Ultimately that would lead to security issues, as the form would accept any field of the model and someone probably wouldn’t notice that when adding a sensitive field to the model. The unsecure code before Django 1.8 went like this:

class UserForm(ModelForm):
class Meta:
model = User

After the change on Django 1.8, the unsecure code becomes much more explicit:

class UserForm(ModelForm):
class Meta:
model = User
fields = '__all__'

The same safe-by-default principle similarly follows the whitelisting is better than blacklisting and the Zen of Python’s “explicit is better than implicit” principles.

Balance Simplicity and Completeness with Progressive Disclosure

A common mistake programmers make when building an API is trying to address all use-cases with a single product. It’s the same issue designers run into when building a digital product without a specific focus: they’ll design something that’s ultimately hard to use for everyone across expertise levels. When designing an interface, be it for a product or an API, there’s always a tradeoff between simplicity and completeness.

The solution for finding balance on that tradeoff is following the UI principle of Progressive Disclosure.

Take a look at Google’s homepage in the screenshot above. Most people who navigate to Google’s homepage want to do a textual search. So even though Google is a huge company with hundreds of services, its homepage is entirely focused on textual search, because that’s what the majority of users are coming to the service for. However, textual search isn’t the only service you can access from the homepage. You can go to gmail, image search, other Google services, etc.

This is called Progressive Disclosure. The highest priority use case is front and center — there’s no clutter, and you put in minimum effort to reach that function. The more advanced features require further interaction, but that’s okay. The tradeoff is worth it to preserve simplicity for the most common use case (in this case, textual search).

It’s true that if programmers expect an API to deal with special cases, they’ll get frustrated when it ends up preventing them from performing customizations on attributes, changes in behaviors, etc. On the other hand, it’s even more frustrating for a developer when an API demands they write a lot of code for something that the program should support with minimal effort. The priority there is to figure out what most end users expect. In other words, what are the majority of use cases your API has to deal with?

At the end of the day, your users want an API to solve their problem by just calling a function and passing some parameters. Conversely, users who want to solve unusual problems already expect to have a harder time. What a good API achieves is something like the following table:

% of Users Expectations on how to solve their problem
80% Use high-level functions or classes
15% Override behavior by inheriting classes, calling more granular lower-level functions, modifying defaults, etc.
4% Change private attributes
1% Fork! And give back a PR

That’s like the Pareto Principle of APIs – to deal with 80% of the use cases your users should use only 20% of your API: the very straightforward, high-level classes and functions. But don’t forget to let the other 20% use the remaining 80% of your API’s functionality: the more complex, granular, lower-level classes and functions are just as important to them. Essentially, a good API will progressively disclose its lower-level constructs as users move from basic to complex usage.

Let’s take a look at an example of Progressive Disclosure for APIs in practice by looking at Requests, a very well-built API. What’s the most basic way to authenticate an HTTP request? Certainly basic authentication with just username and password. Thus, the Requests library handles this type of authentication in the simplest way possible, with a tuple containing username and password:

requests.get('https://api.github.com', auth=('user', 'pass'))

However, there are other methods of HTTP authentication one can use. To support that, Requests accepts instances classes like OAuth1 on the auth parameter:

from requests_oauthlib import OAuth1

url = 'https://api.twitter.com/1.1/account/verify_credentials.json'
auth = OAuth1('YOUR_APP_KEY', 'YOUR_APP_SECRET',
'USER_OAUTH_TOKEN', 'USER_OAUTH_TOKEN_SECRET')
requests.get(url, auth=auth)

Authenticating with OAuth1 is slightly more complex than simply passing a tuple parameter, but users won’t be frustrated by that. They want to do something a bit less common, so they expect the process to be a bit more complex. The important thing is that they’ll actually be able to do it.

Moving on to a more specialized case, imagine if the user needs to use a completely custom authentication method. For that use case, Requests allow you to inherit from the AuthBase class and pass an instance of your custom class to the auth parameter:

from requests.auth import AuthBase

class PizzaAuth(AuthBase):
def __init__(self, username):
self.username = username

def __call__(self, r):
r.headers['X-Pizza'] = self.username
return r

requests.get('http://pizzabin.org/admin', auth=PizzaAuth('kenneth'))

The key takeaway here is that Requests never gets in your way when you need to perform less common tasks, but the implementation complexity grows only as the exceptionality grows. On Requests, common use cases are easily built with high-level constructs, but rarer use cases are still possible with lower-level constructs.

To achieve this balance, well-developed APIs pay attention to the opportunities for extension you might be missing. Imagine a function called print_formatted that prints a string with colors on the local terminal – that function doesn’t have a single responsibility. It actually does two things: format and print. An API with a function like print_formatted is losing use cases: what if someone wants to format the string to send it via a socket to a remote terminal? The same problem could happen if your API doesn’t accept some parameter, or doesn’t support configuration over an attribute, or even doesn’t return an internal resource that the user needs to handle. If you know your API users, you’ll know what they need. We aren’t suggesting you should remove print_formatted and have only print and format functions. If print_formatted is what your 80% of users want to do, keep it! Good APIs have layers: they progressively disclose lower-level constructs for niche applications but default to high-level solutions to common problems.

It’s true that you’ll have to write some extra code that’s flexible enough to handle the different use cases your API might have to support. However, what’s more difficult than writing this code is figuring out what use cases your API users need, and determining the 80% vs. 20% of use cases. Remember, your API is a UI, and it’s not possible to build a functional UI without talking to and knowing its users. Keep in mind that you’ll need to actually reach your users: understanding the problems they have and what they expect from a solution is a crucial step.

It’s safe to say that on great APIs trivial things are simple to do, while unusual things aren’t simple but still possible. For the 80% of use cases, your API should be simple. For the remaining 20%, it should be flexible.

Summary and More Resources

To summarize this blog post into one tweet, we can say that great APIs make simple tasks easy, complex use cases possible, and mistakes difficult.

If you wish to learn more about good practices for APIs, check the following resources:

If you have any questions or comments, feel free to reach me on Twitter: @flaviojuvenal. Thanks!

Flávio Juvenal is a software engineer from Brazil and partner at Vinta Software. At Vinta, Flávio builds high-quality products for US companies with Django and React. When not coding, he’s trying to find the perfect coffee beans to use at his company’s Gaggia espresso machine.