The topic of APIs is even more important today than it was in the past, because the mistakes of earlier times are not wanted or allowed to be made again today. Fortunately, direct access to the database of another service is no longer on the list of project managers or software architects. Instead, more and more interfaces are emerging as APIs according to Richardson’s REST Maturity Model. At the same time, there is a growing desire for coordinated, controlled, and managed access to APIs. Which is not very surprising considering the increasing number of APIs.

The consideration of using an API gateway to manage these interfaces in an initial response is perfectly understandable. But on closer inspection, the question arises as to which functions of the API gateway should be used. Often, the requirements for managing and operating APIs are less than the feature set that API gateways provide. On the other hand, to manage the service mesh, one should think about using a suitable service mesh tool. From a certain size of the service mesh or a certain complexity of the communication behavior of the services, there is no way around it. Istio would be a representative of the service mesh tools that specialize in operation on a cloud platform.

The trend towards operating services in the cloud has also been taken up by API gateway manufacturers. They are currently changing their systems away from monolithic gateways to so-called micro-gateways, which are more in line with cloud philosophy.

Thus, the question arises whether one wants to fulfill the requirements for the operation and management of the APIs with a more or less classic API gateway or whether it is better to switch to a service mesh tool. Or is a mix of these two tools the better solution approach?

Functional scope of “classic” API gateways

This is not the place for a product comparison of different API gateways, but all common representatives of this guild, such as Google Cloud Apigee, Red Hat 3scale, MuleSoft, Kong or WSO2, generally offer the following functionalities.

API applications: This refers to the logical grouping of different APIs into a common application that can then be administered and operated as a whole.

Rate Limiting and Throttling: This defines how many API calls may be made within a certain time interval. If this limit is exceeded, the call is rejected with an error message. In most cases, an HTTP status code 429 (“Too Many Requests”) is sent to the caller. The limit is of a technical nature in order not to overload the called system. Throttling occurs when no more requests are allowed within the specified time interval after the limit has been exceeded. Only after this interval has elapsed further requests are allowed again.

API Quota: This functionality is focusing the commercial aspect of API access. As a rule, the call limit is agreed upon here for a longer period than is the case with rate limiting. For example, one specifies that the API may only be called a thousand times per month. In addition, this limit is set separately for each consumer and then often also leads to the possibility of separate billing of call costs per consumer. Some API gateways allow defining these quotas on API applications.

Load balancing with failover: Since every request to an API is routed through the API gateway, they are also able to offer more or less extensive load balancing with integrated failover. It should be analyzed in detail whether the API gateway is able to react dynamically to changes in the runtime availability of the services or whether it only enables a static configuration of the existing API endpoints.

Access Control: Today, no productive system can do without security and access control. It goes without saying that API gateways also offer this. Different security settings can be defined for different API applications in order to regulate the access options of the various user groups to the APIs. A connection to common OAuth or OpenID Connect servers is now state of the art. Setting or manipulating the HTTP header is also part of the gateway’s basic equipment.

Logging: Every request that is controlled by the gateways must of course also be logged. Logging with a higher or lower level of detail is possible in every API gateway.

Management GUI: For managing the APIs, the common API gateways all offer a graphical user interface. However, a technical interface for automated configuration of the APIs is not available in each of the API gateways. Deployment of new or modified APIs, triggered by an event in the CI/CD pipeline, is therefore not always easy to implement.

Monitoring with analytics: Some of the gateways offer extensions that can be used to further evaluate the collected call data. The spectrum of applications ranges from simple monitoring to complex analyses of the use of the APIs. Some business models of API operators can no longer do without such analytics results.

Developer portal: The API gateways offer so-called developer portals to initiate external API partners or to simplify API client development. There, it is often possible to access released APIs for testing one’s own client via self-registration. These test environments are then operated in a sandbox. Some APIs also offer SDKs for the common programming languages to simplify entry into API development.

API gateway functionality in a service mesh tool

In the following, the functionality of the API gateways will be compared on the basis of the service mesh tool Istio. For an explanation of Istio, please refer to the article “Don’t be afraid of the service mesh” (p. 26).

A few remarks beforehand: Istio only covers the technical aspects of the service mesh. For the commercially oriented functionalities, such as billing, Istio currently has no corresponding capabilities. Istio also does not have analytics capabilities. Anyone who wants to analyze the calls to their APIs by customers even more precisely in order to draw more conclusions about their behavior will, unfortunately, come up empty with Istio. This is where API gateways offer add-on components that better handle the analytics space. Istio offers extension points for the technical aspect of rate limiting and throttling. API Quota functionality that goes beyond pure rate limiting, such as limiting API calls for a specific partner in a period of one month, is possible with the same extension points for rate limiting. These extension points have to be connected with additional services, installed and managed separately. Nevertheless, Istio offers functionalities that are quite comparable to those of API gateways.

Ingress Gateway and Virtual Service

An adequate way to logically group APIs into applications can be done with Istio through various Istio Ingress Gateways in combination with Istio’s Virtual Service. Ingress Gateways control the entry into the service mesh in Istio. They can be defined with different routings in the Virtual Services. From a technical point of view, these routings are thus managed together.

The following Istio rules define an ingress gateway on port 80 for HTTP accesses and connects it to a Virtual Service that redirects to the myservice service in version v1 for request URLs with the /status or /delay prefix (Listings 1 and 2). This combination of Istio rules is arbitrarily extensible and thus covers all requirements that allow common control of API access.

apiVersion: "networking.istio.io/v1alpha3"
kind: Gateway
metadata:
  name: mesh-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    # this gateway is for requests coming from all other hosts
    - "*"

apiVersion: "networking.istio.io/v1alpha3"
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  # this VS is for requests coming from all other hosts
  - "*"
  gateways:
  # and is bound to the following istio ingress gateway
  - mesh-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: myservice
        # this subset must be defined in a DR
        subset: v1

Manage complex distributed systems

Explore the API Management Track

Learn more

Istio’s Rate Limits

A prerequisite for the dynamic limitation of requests to a service by Istio is the activation of an additional backend and some configurations to connect to this backend. The first approach of Istio to offer rate limits was a so-called Policy Enforcement rule. This rule was declared deprecated with Istio 1.6. Starting from Istio 1.9 there are two new alternatives to define rate limits: EnvoyFilter and WebAssembly. Both alternatives enhance the Envoy proxy and delegate the service requests to a rate limiting backend to check for allowing this request. A reference implementation for this backend service, written in Go with a Redis backend, exists. The open source community maintaining the Envoy proxy is also responsible for the Envoy RateLimit service. A Redis server is needed to manage the quota values inside the service mesh.

Envoy checks every HTTP request inside the service mesh against all settings of the HTTP Filter. This HTTP filter can be enhanced with a special Istio Rule EnvoyFilter. In this rule all necessary settings must be defined to connect and delegate the checks to a special rate limiting backend service. Based on the response of this service, Envoy proxy decides what to do with the request. When a predefined limit is reached, the Envoy proxy interrupts the access with an HTTP status code 429 (“Too Many Requests”).

WebAssembly on the other hand is a sandboxing technology to also enhance the Envoy proxy. A WebAssembly plugin can be developed in several programming languages and gets executed in a special WebAssembly Runtime embedded in the Envoy proxy. It is planned to have a growing ecosystem of WebAssemblies. Only the creativity of a programmer limits this open programming model and rate limiting can be one reason to use this upcoming technology.

Inside the Envoy RateLimit service, the evaluation window in which the analysis takes place can be selected as fixed or rolling. With a fixed evaluation window, the limits apply to the period from, for example, 9:00 to 18:00. The rolling window refers to the period of the last 10 minutes, for example. To establish more complex rate limits, multiple limits can be defined, which are then evaluated and monitored in the specified order. It is also possible to distinguish whether the request comes from a logged-in user or not by validating an existing JSON web token (JWT). Other HTTP request headers can also be evaluated. In addition, special limits can be established if the request is from a specific IP address. These are just some of the possibilities Envoy RateLimit service offers.

Istio has clearly focused on the technical limitation of the requests. This also makes it possible to avoid denial-of-service attacks (DoS).

A so-called API quota, as with an API gateway, which is basically used for a billing model, can also be implemented with Istio’s rate limit extensions, but the API gateways offer these possibilities out of the box. On the other hand the flexibility of Istio’s rate limit is determined by the possibilities of the rate limiting backend.

Load balancing and resilience

Istio, which is built on Kubernetes (other platforms are also possible), works closely with Kubernetes when it comes to load balancing. The runtime information of the available Kubernetes pods that are accessed via a Kubernetes service is also available to the Envoy proxy. This enables the Envoy proxy to establish a client-side load balancing. Istio’s Control Plane regularly informs itself about the currently available pods of a service and forwards this information to the sidecar. The sidecar can then intelligently distribute the load among the available pods. Together with the resilience rules (timeout, retry, circuit breaker, and bulkhead), which are also evaluated in the sidecar, problems in the calls can be compensated. Thus Istio has capabilities that go far beyond those of an API gateway.

Security

Istio provides its own security module (Citadel), which takes care of certificates and mutual TLS. In addition, Istio can be provided with Role-based Access Control (RBAC) settings and an integration with JWT based authentication is possible out of the box. Furthermore, the evaluation or manipulation of request header values has been possible in Istio for a long time.

Starting with mTLS, as one aspect of the wide range of security, it is very easy to define how traffic will be encrypted or not. By defining an Istio rule PeerAuthentication mTLS can be defined for the whole mesh, only for a Kubernetes namespace and even only for some services. To establish a migration path, mTLS can be defined in different modes: PERMISSIVE or STRICT. A PERMISSIVE connection can be either plaintext or mTLS tunnel. STRICT forces a mTLS connection. Istio manages all necessary SSL certificates out of the box and frees the admin from this annoying activity. The following listing shows an Istio rule which forces mTLS tunnels on all workloads in namespace foo:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: foo
spec:
  mtls:
    mode: STRICT

To enable access control on workloads in the service mesh two different Istio rules must be defined. The first rule, a RequestAuthentication defines what request authentication methods are supported. It validates the JWT in the authorization header and checks whether it was issued by the corresponding OpenId Connect server. In listing 4 all requests to httpbin workload in namespace foo will be authenticated with credentials derived from a JWT issued by issuer-foo coming from OpenId Connect server example.com:

apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: httpbin
  namespace: foo
spec:
  selector:
    matchLabels:
      app: httpbin
  jwtRules:
  - issuer: "issuer-foo"
    jwksUri: https://example.com/.well-known/jwks.json

The second rule, an AuthorizationPolicy, enables all access controls based on the JWT claim values validated by the previous rule. The scope of this rule also ranges from a complete service mesh to a single workload. Multiple rules of this type will be evaluated in a predefined order and if one rule matches, the request will be forwarded to the service. Listing 5 defines an AuthorizationPolicy which allows requests from namespace test to HTTP Endpoint /data in namespace foo only if the issuer of the JWT is https://accounts.google.com:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: httpbin
 namespace: foo
spec:
 action: ALLOW
 rules:
 - from:
   - source:
       namespaces: ["test"]
   to:
   - operation:
       paths: ["/data"]
   when:
   - key: request.auth.claims[iss]
     values: ["https://accounts.google.com"]

An AuthorizationPolicy can also be defined as a DENY-Rule. Listing 6 shows a rule to stop all HTTP POST requests from namespace dev to namespace foo:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: httpbin
 namespace: foo
spec:
 action: DENY
 rules:
 - from:
   - source:
       namespaces: ["dev"]
   to:
   - operation:
       methods: ["POST"]

The previous security listings show only a small range of Istio’s security capabilities. Thus, also in terms of security requirements, the functional scope of Istio is comparable to that of an API gateway, if Istio does not even surpass the possibilities of API gateways here.

Logging and tracing in the service mesh

Without sufficient logging, no reasonable operation of a service mesh is possible, as this usually requires a high number of services. For this, Istio relies on the possibilities offered by Kubernetes or the Docker containers. Istio’s homepage describes how logging can be set up with Fluentd. Creating a so-called EFK logging stack (Elasticsearch, Fluentd, Kibana) is thus very easy. Istio’s own components also use this logging stack.

With the possibility of distributed tracing based on Jaeger or other components following the OpenTracing standard, Istio offers a functionality that is naturally not included in API gateways.

Kiali as a management GUI for Istio

Meanwhile, a GUI also exists in Istio to view your Istio rules and other information important for managing the service mesh. Kiali already provides help to get an overview of the service mesh. Since Istio was first started with a set of rules based on YAML files, it can be configured very well with scripts. The execution of the scripts can be integrated into an existing CI/CD pipeline. This gives Istio a plus point, as scripting in API gateways is not as prominent everywhere.

Developer portal

Istio does not offer the possibility to generate an SDK for the client developer. Self-registration must also be done with other systems. Only when it comes to sandboxing Istio is as good or bad as the API gateways. Usually, the biggest effort of sandboxing is to establish the appropriate test or simulation environment. Once that is accomplished, managing the sandboxes is only a much smaller effort. Deleting, restoring, and assigning the sandbox can be done very easily thanks to Kubernetes and Istio’s scripting capabilities.

Conclusion

At the end of this article, let’s return to the beginning and the question: How do you want to provide other systems with coordinated access to your APIs? In order to find a way out of the typical consultant answer “it depends”, one should consider exactly which functions of an API gateway are desired. Are the possibilities of Istio or the other service mesh tools sufficient – especially under the aspect that from a certain number of services onwards a service mesh tool can no longer be dispensed with? The consequence of this would be to operate a pure API gateway as an additional component.

Because of the trend toward self-responsibility in projects and the associated self-responsible operation of all components, project managers should consider carefully whether additional systems are necessary to achieve the project requirements. However, these considerations should not be turned into the opposite by implementing missing functionality oneself at great expense.

For smaller infrastructures where it is not yet necessary to use a service mesh tool, it may make sense to use an API gateway. The same applies to infrastructures that have to get by without Kubernetes. It is often sufficient to start with a small gateway solution and only later switch to the full range of functions of an API gateway. With growing service landscapes, it will probably not be possible to do without the further advantages of Istio. But then the question arises again whether you can manage with Istio alone and whether you can or want to do without the functions of the API gateway.

It depends!