Since the existence of Java, there’s been a need to access libraries and third-party memory written in other programming languages. This is particularly true for those developed with C/C++. The Java platform offers the Java Native Interface (JNI) for these accesses. With JNI, applications can contain native code written in programming languages like C/C++ and in the Java programming language.

From the perspective of the Java platform, the JNI API enables the integration of legacy code and resolves the interoperability issues between Java code and natively compiled code (Fig. 1). But the memory operations are not without their problems. Since native code is executed, used objects must be freed again. Between native calls, it is directly returned to the Java application without being able to detect potential errors in the native code.

Fig. 1: Java application call from native C/C++ code with JNI

Project Panama

Project Panama was created to meet the demand for better developer support in solving this programming task. It is used to make the interactions between JVM and native code easier. For this, connections between the JVM and the well-defined non-Java interfaces (foreign APIs) have been revised—in particular, the APIs usually used by C programmers.

Project Panama [1] includes the JDK Enhancement Proposals to the Foreign-Memory Access API (JEP-370, JEP-383), the Foreign Linker API (JEP-389), the Vector API (JEP-338), and the following components:

native JVM function call
native data access of the JVM or in the JVM heap
new data layouts in the JVM heap
native metadata definition for the JVM
API extraction tools for header files (jextract)
native library management APIs
native oriented interpreter and runtime hooks
class and method resolution hooks
native oriented JIT optimizations
tools or wrappers for compliance with safety features
work progress with hard-to-integrate native libraries

STAY TUNED!

Learn more about API Conference

With Early Access Build 17-panama+3-167, Project Panama [2] aims to test a prototype implementation with foreign-memory support, foreign-function support, and a native extraction tool from the foreign-jextract branch of the openjdk/panama-foreign Panama repository. The panama-foreign Foreign repository [3] contains two main branches:

The foreign-memaccess+abi development branch contains the Foreign-Memory Access API (JEP 393) and the Foreign Linker API (JEP 389). The former can be used to interact with different types of memory resources, including off-heap memory or native memory. The latter can be used to call native code in a .dll/.so/.dylib or to create a native function pointer to a Java method which can be passed to code in a native library.
The other development branch—foreign-jextract—includes an API for parsing native headers to create an abstract representation (declarations) from a C header file. The branch includes the Java extraction tool jextract, which builds upon the API and generates the Java bindings that access functions and/or structures in a native library described by a defined header file. Therefore, the Java tool jextract will generate the Java bindings from a native library header.

The development tool jextract isn’t included in JDK 17. But it comes shipped with the Panama Early Access Builds project (OpenJDK version 17-panama) [5] and can be found in the path ~/jdk-17/bin. The jextract tool—which, strictly speaking, is not part of the runtime-related Foreign Linker API—simplifies development considerably by generating Java classes from the C header files (Fig. 2). By doing this, the underlying details are hidden from the Panama Foreign Linker API by jextract. The incubator module jdk.incubator.foreign is used when running jextract (module jdk.incubator.jextract). Module and package names are the same [4].

Fig. 2: Java application call from native C/C++ code with jextract

The JDK 17 includes JDK Enhancement Proposal (JEP-412) [5], Foreign Function, and Memory API, which are in incubator status. The API is intended to allow Java programs interoperate with code and data from outside the Java Virtual Machine (JVM). With the foreign-functions calling of code from outside the JVM and secure foreign-memory access not managed by the JVM, the API allows Java programs to call native libraries and perform native data processing without the brittleness and danger of JNI.

The approach so far with JNI

Since early JDK versions like Java 1.1, there has been a standardized procedure for connecting native libraries in the form of the Java Native Interface (JNI). In contrast to the progressive innovation of the JDK from version to version (Coin, Lambda, Java Module System), working with the JNI hasn’t evolved in software terms. The programmer needs to have knowledge of C and C++ with the associated tools. This high barrier for entry and the reduction of the error level in JNI usage led to the creation of Foreign-Memory Access API and Java Foreign Linker API.

With JNI, the following steps have been implemented so far:

Write and compile Java program
Generate a header file from Java class
Writing a C program
Generate a C program as a dynamic or shared library file
Load a dynamic library or shared library
Run a Java program that executes the C program with JNI

Manage complex distributed systems

Explore the API Management Track

Learn more

Using the jextract tool

jextract is a simple, yet practical tool that generates a Java API from one or more native C headers. Interaction with the jextract tool usually involves two steps:

Generating the Java interface for the C header files with jextract
Writing a Java program that calls the wrapper API points generated by jextract

jextract example

The Hello World example [6] in Listings 1 through 7 was built on an Ubuntu Linux kernel release 4.15.0-163-generic with x86_64 processor using OpenJDK 17-Panama.

openjdk version "17-panama" 2021-09-14
OpenJDK Runtime Environment (build 17-panama+3-167)
OpenJDK 64-Bit Server VM (build 17-panama+3-167, mixed mode, sharing)

#ifndef helloworld_h
#define helloworld_h
 
extern void helloworld(void);
 
#endif /* helloworld_h */

#include <stdio.h>
 
#include "helloworld.h"
 
void helloworld(void) {
  printf("Hello World!\n");

gcc -shared -o libhelloworld.so helloworld.c
 
 
Shared-Library-Pfad java.library.path review: libhelloworld.so /usr/lib

~/jdk-17/bin/jextract -t org.hello -lhelloworld helloworld.h
 
WARNING: Using incubator modules: jdk.incubator.jextract, jdk.incubator.foreign

'constants$0.class'
  helloworld_h.class
  RuntimeHelper.class
'RuntimeHelper$VarargsInvoker.class'

import static org.hello.helloworld_h.*;
 
public class HelloWorld {
  public static void main(String[] args) {
    helloworld();
  }
}

~/jdk-17/bin/java --enable-native-access=ALL-UNNAMED --add-modules jdk.incubator.foreign HelloWorld.java
 
WARNING: Using incubator modules: jdk.incubator.foreign
1 warning
Hello World!

STAY TUNED!

Learn more about API Conference

Conclusion and outlook

The long-term planned implementation of Project Panama—with the Foreign-Memory Access API, the Foreign Linker API in JDK 16, and the Foreign Function & Memory API (Incubator) in JDK 17—is taking shape. It continues in JDK 18 with JEP 419, Foreign Function & Memory API (Second Incubator) [7]. The Incubator API with the module jdk.incubator.foreign will be transferred to a Preview API in java.base with its own package name. These interfaces enable and strongly simplify type-safe access to native libraries. Additionally, there’s is the convenient usage of familiar abstraction classes like ByteBuffer. Compared to JNI, development times are decreased by reducing access barriers. The use of native add-on functions can potentially lead to even more security than before. The new APIs are on a higher level, offering better possibilities for integrating external libraries into Java development projects with the jextract tool. After a transitional period, the jextract tool will be shipped separately and then further developed in a standalone GitHub repository. More early access binaries for jextract will follow. This makes it much easier for developers to integrate a machine learning library into an existing Java project—for instance, with the Tribuo Java library, or with the TensorFlow library (C/C++, Python). Still, after the incubator status, the Foreign Function & Memory API needs the maturity and stability to replace the JNI calls in the long run and will need to be able to use arbitrary third-party code.

Links & Literature

[1] https://openjdk.java.net/projects/panama/

[2] https://jdk.java.net/panama/

[3] https://github.com/openjdk/panama-foreign

[4] https://download.java.net/java/early_access/panama/docs/api/jdk.incubator.jextract/module-summary.html

[5] https://openjdk.java.net/jeps/412

[6] https://github.com/openjdk/panama-foreign/blob/foreign-jextract/doc/panama_jextract.md

[7] https://openjdk.java.net/jeps/419