In distributed systems such as microservices or cloud-based applications, the central management of policies is playing an increasingly important role. Whether access controls, security specifications or regulatory requirements – without a flexible and scalable solution, they quickly reach their limits. The challenges lie not only in the management of policies, but also in their consistency and adaptability across different systems. Different technologies, decentralized responsibilities and individual implementations make it difficult to enforce rules uniformly and efficiently.

This is where the Open Policy Agent (OPA) comes into play. It allows policies to be defined centrally and enforced independently of the underlying technology. A very typical application scenario is the authorization of requests to API gateways [1]. Common gateways such as Kong, Apache APISIX or Envoy provide plug-ins for the integration of the OPA as standard.

Open Policy Agent explained in practice

The Open Policy Agent can either be provided as a standalone instance in a Docker container or integrated directly into an existing application like a library. As a standalone component, OPA is typically accessed via a REST API to validate requests based on defined rules and enforce policies. For this blog post, we have chosen a scenario in which the API Gateway and the Open Policy Agent operate on the same machine, which minimizes latency. This corresponds to the proven sidecar pattern, which enables a scalable and robust architecture.

To illustrate how the Open Policy Agent works, let’s look at a typical example with an API gateway that controls access to multiple endpoints. Here, the API gateway serves as the central interface for requests and forwards them to the underlying services – but only after successful authorization by the OPA. Figure 1 provides an overview of the communication between the components [2].

Fig. 1: Authorization at the API Gateway with OPA

The interesting part here is the OPA plug-in. The requests to the API Gateway are processed via the plug-in. It breaks down the request into its components: URL, header, body etc. and passes this information on to the Open Policy Agent in a separate request. This evaluates the request using the rules from the Rego scripts and returns either true or false. The plug-in now compiles the response for the actual caller. If false, a response with an HTTP status code 403 is usually generated [2].

Rules for secure data access

In our scenario, we want to ensure that the user can only access their own data and cannot see the data of other users.

• In our architecture, the user authenticates himself via OAuth2 and sends a bearer token (JWT) in the header, which contains the user ID.
• The endpoint that we want to secure uses the user ID in the URL, e.g. https://mydomain.de/profile/<userId>.

As both the header and the URL are passed to the Open Policy Agent, we can check them against each other:

1. We extract and decrypt the JSON Web Token (JWT).
2. We verify the signature of the JWT using Rego functions.
3. We extract the user ID from the URL and we extract the user ID from the JWT
4. We return true if the user IDs match, otherwise false.

Listing 1: Example authorization rules in Rego

package javamag.exmaple

import rego.v1




default allow := false




# 1. read userId from the URL path

# path: mydomain.de/profile/<userId>

userId_from_url := input.parsed_url[2]




# 2.1 Read bearer token from the header

bearer_token := token if {

  auth_header := input.headers.authorization

  startswith(auth_header, "Bearer ")

  token := substring(auth_header, count("Bearer "), -1)

}




# 2.2 Extract and verify JSON Web Token

jwt_payload := payload if {

  io.jwt.verify_hs256(bearer_token, "<secret>")

  [_, payload, _] := io.jwt.decode(bearer_token)

}




# 3. the actual authorization rule: We compare the UserIds

# from the JWT payload and from the URL

allow if {

  jwt_payload.userId == userId_from_url

}

At the beginning, the variable allow is set to false. It must be specified in the plug-in configuration in order to be evaluated. Under step one, the user ID is read from the URL path. The input object contains all the data transferred from the plug-in to OPA, including the HTTP method, header, body, URL and path segments of a URL, as seen in the code example [3]. In the second step, the JSON web token is read, verified and extracted. In the last step, the user ID from the JWT is compared with the user ID from the URL path. The result is recorded in the allow variable and evaluated via the OPA plug-in.

Production scenario

The example shown above is a good start, but authorization rules are often much more complex than “Only you may access your data”. The Rego Playground offers a variety of examples such as Attribute-based Access Control (ABAC) or Hierarchy-based Access Control (HBAC) [4]. Such approaches are helpful for implementing typical requirements in practice.

A common scenario concerns additional user groups that require access to customer data. For example, there are often clerks who are assigned to specific customers. However, this assignment is rarely based on a fixed logic that could be modeled in Rego, but rather on external lists that are regularly updated. Such lists regulate which administrator has access to the data of a particular customer.

This is where OPA comes into its own. Data can be integrated directly into the agent, for example as JSON data, which is managed together with the rules. This is a practical solution for smaller, static data volumes that only change infrequently. For larger amounts of data or if the data is updated regularly, however, it is advisable to connect external data sources, for example via REST APIs or databases. The separation of data and logic offers significant advantages:

• Maintainability: changes to the data do not require any adjustments to the Rego rules. Guidelines and data remain cleanly separated.
• Scalability: Large amounts of data can be better managed and efficiently queried in dedicated systems, such as in a distributed Redis cache.
• Flexibility: By connecting external data sources, OPA can be seamlessly integrated into existing company systems.

An example of the process is as follows:

1. OPA Policy uses a rule to check whether a clerk may have access to a particular customer’s data.
2. To do this, the assignments are retrieved in real time from an external data source, for example via a REST API.
3. The decision is made based on the current data – regardless of how often it changes.

This separation of responsibilities means that OPA remains clear and flexible, which is particularly beneficial in productive scenarios. To achieve this, we add a rule to the existing Rego script (Listing 2).

Listing 2: Extension of authorization rules in Rego

# 1. read userId from the URL path

# 2. extract JWT

...

# 3. rule: Check administrator authorization

is_sachbearbeiter_allowed := true if {

  # REST API call to check the mapping clerk -> customer

  mapping_result := http.send({

    "method": "GET",

    "url": sprintf("https://example.com/sachbearbeiter/%s/kunden/%s", [jwt_payload.userId, customerId_from_url])

  })

  mapping_result.status_code == 200

}




# 4. access is allowed if the user is either:

# - Is the customer himself (JWT userId == customerId from the URL) or

# - Is a clerk and has access according to the mapping

allow if {

  jwt_payload.userId == customerId_from_url

}

allow if {

  is_clerk_allowed

}

Here, in the third step, we execute a REST call against an external system to check whether a clerk has access to a customer and add another allow-if block in which we check exactly this condition.

External data sources

Our example shows that additional data is often required to validate and implement OPA policies, which varies depending on the application and policy. Typical data sources include information about users and resources, context and configuration data, real-time information and external system data. To ensure efficiency and performance, only the data that is necessary should be transferred to the OPA. Regular synchronization is crucial to enable the agent to make context-aware and accurate decisions.

The diverse data requirements demand flexible and efficient data management. For this, OPA provides five options that enable optimal integration and use according to the complexity and use case [5]. Let’s take a brief look at them.

JWTs

JSON Web Tokens are an essential component for securely transmitting user information. Once a user has been successfully authenticated, the system generates a JWT that contains relevant user information such as group memberships, first name, last name and other attributes [6]. The token serves as input for OPA, is decoded and the extracted data is used for decision-making in the policy.

Overload input

The information provided by authentication systems is often not sufficient for complex policy decisions. Such systems primarily provide attributes such as roles or access restrictions. Requirements such as the identification of a resource owner do not fall within their scope. External systems can step in here by providing OPA with additional relevant metadata such as owner, creation date or authorizations.

However, dependence on external data poses challenges. If specific attributes are transferred, there is a dependency on the structure of the source system. To increase flexibility, all relevant metadata should be provided instead, making policies more universal and robust.

Bundle API (polling)

The Bundle API enables the integration of static data and policies through downloads from a central server at defined intervals. This is particularly suitable for smaller amounts of data that can be stored in memory. Handling large external data sources requires special attention. Data management held in memory can lead to resource bottlenecks without careful planning. Therefore, well-designed data architecture is essential to comply with storage restrictions and ensure system performance.

Push Data

By using a replicator, data can be efficiently retrieved from external sources and seamlessly integrated via the OPA API. In contrast to the Bundle API, which synchronizes complete data packages, the replicator enables the use of delta updates. With this approach, only changed data is synchronized, which reduces both the network load and the processing time. This makes this method particularly suitable for small to medium-sized data volumes that change frequently and can be stored completely in memory [5].

Pull data during evaluation

This approach allows policies to request data directly from external sources instead of the caller providing it in advance. This ensures up-to-date and flexible data but carries the risk that OPA cannot decide if the data source is unavailable, as there are no retry mechanisms. In contrast to the overload input approach, the policy requests the required information during the evaluation. This approach is particularly suitable for dynamic data whose timeliness is crucial or for data volumes that cannot be kept in the working memory. Despite its flexibility, this approach requires precise planning to minimize risks such as dependencies on external systems and to ensure system reliability.

Overview

There are therefore several approaches for effectively integrating external data into the decision-making process, with each method having its specific advantages and disadvantages (Table 1). The combination of different methods makes it possible to select the optimum strategy for the specific application. The use of multiple mechanisms promotes the flexibility and adaptability of the system to future changes or expansions. However, the administration of such a hybrid approach entails an increase in technical and organizational complexity, including complex requirements in the areas of implementation, documentation and maintenance.

The use of different data sources harbors potential risks such as inconsistencies or contradictory information, which can lead to different results. This makes it more difficult to identify and analyze problems in policy evaluation, as several sources can exert influence at the same time. In addition, dependence on external systems increases the vulnerability of the entire ecosystem, which can affect overall reliability. A methodically thought-out mix tailored to the specific requirements of the system nevertheless offers significant advantages. Each option used should have clearly defined areas of responsibility to minimize redundancies and potential conflicts. However, without systematic planning and stringent management, mixed approaches can generate unnecessary complexity and additional risks [5].

Table 1: Overview of the various options for connecting external data [5]

Extension of existing applications

With the Open Policy Agent, the authorization logic is no longer located in the applications themselves, but centrally on the agent. The key question is whether applications that previously had none, or inadequate authorization logic can be expanded accordingly and thus secured. This is exactly what we examined and implemented for a customer.

A third-party system for customer management, which was accessible via a REST API, unfortunately had inadequate authorization mechanisms. There was only one user with unrestricted access to all endpoints. To resolve this situation, we implemented the Kong API Gateway and the Open Policy Agent for the customer.

Fig. 2: Production scenario

The Kong API Gateway was deployed here in hybrid mode. The Kong Konnect Control Plane is accessible via the Internet and provides an interface for configuring the API Gateway. The Data Plane was installed on the customer’s server and receives the requests. These are first forwarded to the Open Policy Agent and then transmitted to the third-party system if authorization is successful. For the requests to the third-party system to be successful, an authentication token is required, which must first be requested from the third-party system using a secret key. In our case, we have used a separate Kong plug-in for this. It performs the authentication, caches the token and automatically adds it to the header. In this way, we ensured that requests from the API gateway to the third-party system are not rejected due to a lack of authentication. In addition, we implemented a small replicator that loads data from a data source and compiles and updates it for OPA. This enabled the customer to establish its own authorization concept, in which administrators were allowed access to certain endpoints of the third-party system as well as insights into the data of selected end customers.

Conclusion

In our scenario, we have shown that the Open Policy Agent can be used flexibly and can even add complex authorization requirements to third-party systems. In our scenario, we quickly reached the limits of API gateway integration. For example, endpoints that allow end customers to search cannot be used. This is because, depending on the search criteria, end customers would be returned that are not assigned to the agent.

Overall, the Open Policy Agent is a powerful tool for authorization decisions, which is impressive due to its centralization with an API gateway. This enables a clear and uniform authorization logic across all endpoints. Security and compliance requirements are met more efficiently as authorization is defined and managed centrally.

Outlook

One crucial aspect that has not yet been considered in detail concerns the efficient and secure provision of policies in different environments. This raises the question of how to ensure that policies function correctly and that potential side effects are identified at an early stage. Another focus will be the development of a CI/CD pipeline for the delivery of policies. These exciting topics await you in the next blog post, we will show you how to test policies efficiently and deploy them successfully.

Links & literature

[1] Open Policy Agent: https://www.openpolicyagent.org/docs/latest/

[2] How to Manage Your API Policies with OPA (Open Policy Agent): https://konghq.com/blog/engineering/how-to-manage-your-api-policies-with-opa-open-policy-agent

[3] Kong OPA plug-in code: https://github.com/open-policy-agent/contrib/blob/main/kong_api_authz/src/kong/plugins/opa/access.lua

[4] The Rego Playground: https://play.openpolicyagent.org/

[5] External Data: https://www.openpolicyagent.org/docs/latest/external-data/

[6] Introduction to JSON Web Tokens: https://jwt.io/introduction

🔍 Frequently Asked Questions (FAQ)

1. What is the Open Policy Agent (OPA)?
OPA is a general-purpose policy engine that enables centralized decision-making for authorization across APIs, cloud, and Kubernetes environments.

2. How does OPA handle JWT-based authorization?
OPA can decode, verify, and extract claims from JWTs in Rego policies to enforce user-specific access rules.

3. Can OPA connect to external data sources?
Yes. OPA supports static data bundles, REST-based queries, and dynamic push/pull integrations for real-time decision-making.

4. What is the role of Rego in OPA?
Rego is the policy language used by OPA to express complex authorization logic in a structured, programmable format.

5. How is OPA used with API gateways like Kong or Envoy?
OPA can run as a sidecar or plugin with API gateways to evaluate access requests and enforce rules before routing traffic.

6. What’s the benefit of separating policy logic and data in OPA?
It enhances maintainability, scalability, and performance by allowing logic and data to evolve independently.