Therefore, it’s especially important that interfaces have a good design. Currently, HTTP APIs are predominantly used. Anyone working with these on a regular basis will notice that many APIs aren’t very successful. Sooner or later, this will present some challenges to their operators. Because of this, it’s very important to perform an API design review before the interface goes live.
What are the reasons for poor API design? Often, this is due to a lack of experience. But in many cases, development teams simply aren’t given enough time to work meticulously on API design. At the project management level, there’s also the perception that experienced developers should be able to implement an HTTP-based interface in no time with the help of modern tools and frameworks. That’s basically correct. But in the first iteration, the design is usually not very well thought out. Good API design takes time, and ideally a review from an experienced API expert.
Why is good API design so important? One of the most important reasons is that interfaces developed for system integration generally have a very long, useful life. Therefore, they must be operated, maintained, and often also developed further over a long period of time. New requirements are added with regard to the data exchanged over interfaces, use cases, or the end devices that will be connected. Additionally, the number of users or clients typically increases over time. Once an API is in operation and used by clients, you can’t change it at will anymore. If you don’t want to annoy your users, you have to make sure that compatibility between previous clients is maintained when the interface is changed. This is precisely why API design needs to be high-quality when the interface is initially put into operation. Then, you can accomplish all of this with manageable effort. But in less successful designs, challenges will soon arise. It’s important to consider the environment where the API is used. The application areas of HTTP-based APIs can be roughly divided into five categories.
- Backends for web-based applications
- Communication within microservices architectures
- Integrating systems in the same company
- Integrating systems of a limited number of known companies (suppliers, business partners)
- Public APIs
Application areas differ, significantly in some cases, in terms of the number of API clients and familiarity or proximity to their development teams. Backends for web-based applications usually have only one client, which is the respective front end. Their developers are usually familiar and belong to the same team. Necessary API changes can be agreed “on demand” without having to take other clients into consideration. API design can be tailored exactly to the application’s use cases. The situation is completely different with public APIs. These can potentially have a large number of clients, which often cannot be known when developing the interface. To some extent, you must anticipate its desired operations, data fields, and targeted use cases. Or, the API design must be generic and extensible enough to allow for random uses. Incompatible changes to public APIs are usually very difficult to make once the interface is up and running in use. Additionally, for many companies, public API products can have a significant impact on revenue and success. Sometimes the API can be a company’s core service, or becomes one over time.
The first API design absolutely must succeed. The other three categories form different grey areas between these extremes. Depending on the environment where a specific API is used, the importance of its design quality can vary. This should serve as a decision-making aid about if you need to carry out an API review at all before the interface goes live. For web-based application backends, this usually isn’t necessary, for instance. But it can be very useful in microservice applications.
An API design review usually starts with some basic questions. What is the purpose of the API? Who is going to use it (what is the target audience)? In what usage scenarios? The answers to these questions have a significant impact on how the API should be designed. For example, if it’s a public API, we must assume that non-corporate users might not know some abbreviations or terms common in the API operator’s company. We need to think about if individual data structures should be specifically translated to the outside world in generally understood terms. If we know the API’s future users, desired data structures and operations can be coordinated with them and API design can be iteratively developed together. But if we don’t know the future users yet, API design must be as generic as possible and basically enable any use case.
Another question that needs to be addressed early on is the intended strategy for versioning. Should the API have explicit version numbers? If so, this implies some follow-up questions, all of which aren’t easy to answer. How will the respective version be made visible (URL, header attribute, content type)? How many versions (max) should be operated simultaneously? How long will older versions continue to be supported? An alternative to explicit API versions could be simply not having any. Only the latest API version would be in operation at any given time. It would simply be put into operation at certain points in time. Of course, this can only succeed if each new API version is backwards compatible with the previous version and doesn’t have any “breaking changes”. This procedure is much more convenient for the API operator, but it requires an API design that’s easily extensible even in the very first version.
At this point, it should be clear why it’s so important to think carefully about API design and review it before the first version goes live. During the review, many aspects are considered like consistency, comprehensibility, usability, standards and best practices compliance, and security. Depending on if it’s an internal or public API, where it’s used, and how many users it has, individual aspects of the review can vary in importance.
One of the most important points is consistency, both in an API and across a project or entire company’s APIs. If all APIs, operations, behaviors, data models, and everything that makes up API design follow the same approach, these APIs are easier to use and understand. Once you’ve learned how to use one of the operator’s APIs, transitioning to the next API is automatic as it follows the same design and behavior. This leaves a good impression on non-company API users like customers or suppliers, facilitates cross-company system integration, and ensures a low burden of support requests. API design consistency also has tremendous benefits internally. Developers can easily move between projects without having to learn a new API approach every time. Searching for or filtering data should always work the same, whether you’re looking for people, products, orders, or suppliers. The same goes for querying, updating and deleting data. Data formats (date, time, metric units, and decimal places) should also be used consistently, as should labels, attribute names and formats.
Another aspect of API review is the interface’s understandability. This concerns data structures, their attribute names, semantics, data formats, abbreviations, and code. The naming and structure of URL paths to individual API endpoints should also be considered. Are these understandable and as expected? There’s often a natural hierarchy in the domain, which ideally is also found in the API design. For example, let’s look at the domain of a fictional map or address service. There are several continents on the earth and they contain countries, which then contain cities. Cities have multiple streets, etc. If an API reproduces this structure, this significantly increases their comprehensibility. API client developers can already predict certain structures of the API based on their background knowledge before they study the documentation. API reviewers try to put themselves into the shoes of client developers. What might they want to accomplish (create a new street, for instance)? How would they assume this would work with the API? And is this really possible, or does the API provide for a completely different, unexpected way to create a new street? Of course, API comprehensibility is greatly influenced by its documentation. An API specification in OpenAPI format is the least that an API operator should offer. But as a rule, this isn’t enough. For example, you need further information about the meaning of attributes in data formats, or the use of search filters and operations. Documentation must be written to make it easy for the API’s users to understand. For public APIs, documentation should be free of company- or industry-specific technical jargons and abbreviations.
Usability considerations are also an important part of any API review. There are several operations that sooner or later, nearly all APIs need. Ideally, these should therefore be part of the API design from the outset. This includes the aforementioned searching and filtering (“Find all orders from the year 2022 with an order value of at least €1,000”). The ability to specify a limit on the results returned is also important for API clients. If necessary, you also must be able to navigate through a result set page by page (pagination). It’s rarely useful or efficient to transport thousands or even tens of thousands of records to the client all at once. Another useful operation for API clients is the ability to specify the delivered results’ level of detail. For example, when searching for customers, all known customer information could be requested (address, order history, etc.). Or, it might be enough to only request the names and customer numbers. As previously explained, these operations should work the same everywhere and be implemented consistently. They may seem generic, but this is what lets the API operator provide an interface that isn’t designed for specific use cases and allows each individual user to implement their own use cases with generic operations. Evaluating an interface’s usability also involves looking at data structures and the question of how many individual requests a client would have to send in order to implement typical use cases. If the number of requests is too high, this indicates potential for improvements in the API design.
Compliance with standards and best practices
API reviewers should also keep an eye on standards and best practices compliance. It rarely makes sense to deviate from these and implement your own proprietary solutions. As a rule, it only complicates users integrating the API. For public APIs, it can also be a distribution obstacle and impact their success. You must consider the (correct) usage of the HTTP protocol, its request types (GET, PUT, POST, PATCH, DELETE) and status codes. URL path and query parameter naming and generally known best practices should also be followed. In the security realm, it’s essential to use proven standards like OAuth, OpenID Connect, or JWT.
Ideally, the enterprise has an API design policy that establishes a design standard for all APIs. There are many good (and not so good) ways to design HTTP-based APIs, but enterprises should opt for a consistent approach. Guidelines define the look and feel of API design. When done well, they align with common practices in the outside world, making APIs even easier to use. But when these guidelines exist, there are still plenty of opportunities to introduce unintended inconsistencies. Uncovering these is one task of API reviews.
Naturally, security is also a very important topic. Therefore, it’s recommended that you commission a security expert in a separate review. The API reviewer can also already look at some security-relevant aspects. For example, this includes exchanged data structures and the API operations possible on them. What data and attributes are readable? Which ought to not be delivered? What data can be manipulated, and by whom?
One sensible strategy is to initially make only a small amount of data available, possibly even only the minimum amount of meaningfully usable attributes. An API operator can wait and see what other attributes the API users request and decide which should be accessible on a case-by-case basis. This way, it gradually creates an interface that really only delivers the necessary data. In terms of backwards compatibility, this procedure is more sensible than removing attributes from the interface afterwards. The API must also deliver IDs for accessing and referencing individual resources or entities. For security reasons, it’s highly advisable not to make internal IDs (like primary database keys) known over the interface. Instead, all resources should also have external IDs that aren’t countable and difficult to guess. For example, UUIDs are suitable here. These must be translated into internal IDs at the transition between API and internal system. The OWASP API Security Project provides further information about typical security vulnerabilities.
Public APIs can significantly contribute to a company’s revenue and success. Therefore, they should be understood as a product. For some companies, their API is even their core service. APIs should be given the same care and diligence as any other product. This also applies to the design. Even with internal, non-public APIs, design is a decisive factor for the interface’s success or failure. Poor design has consequences in the medium term, like extra work, support requests, poor extensibility, and usability. An API design review from an experienced API expert is an important part of API development. Of course, the review should be performed before the API goes live and is used by clients for the first time. Important review aspects include consistency, comprehensibility, usability, compliance with standards and best practices, and of course, security. These aspects can be weighted differently depending on the environment the API is used in.