Monday, February 1, 2021

Multilingual Engine: Executing JavaScript in Oracle Database

Oracle Database Exam Prep, Database Preparation, Oracle Database Guides, Oracle Database Learning, Oracle Database Certification

Starting with 21c, Oracle Database can now execute JavaScript, powered by GraalVM. In this blog post, we peek under the hood of this new feature; we will also have a follow-up blog post showing JavaScript as a server-side language in Oracle APEX.

◉ Using JavaScript in Oracle Database

◉ Multilingual Engine: Embedding GraalVM in the Oracle Database

◉ A Vision for the Future

Recently, Oracle released the next version of the world’s most advanced database, Oracle Database 21c. This release adds the ability to execute JavaScript code directly in the database. Oracle Database has long had support for server-side programming in PL/SQL, Java and C. Server-side business logic plays an important role in many enterprise applications: first, the logic gets executed where the data resides, rather than having to bring the data to the logic. This removes unnecessary data transfer over the network and can significantly improve performance of data-intensive operations, especially if applied to terabytes of data or more. Second, storing and executing, for example, business rules inside the database guarantees that the rules are followed by every application, as well as users accessing the data, which can drastically simplify the implementation of security and compliance requirements. Last, but not least, commonly-used functionality can be stored in a central place and executed as simple user-defined functions on top of ordinary SQL statements, avoiding the need to replicate the code in every application. This can be especially practical when the logic is more complex and/or tends to change frequently.

Read More: 1Z0-497: Oracle Database 12c Essentials

With 21c, the set of supported languages was expanded to include JavaScript, one of the most widespread and popular programming languages today. Developers can now use this popular language for database programming and tap into the rich ecosystem of tools and libraries available for JavaScript. Similar to PL/SQL, server-side JavaScript execution is tightly integrated with the database with code executing within the process of a database session, close to the data and SQL execution. This tight integration enables efficient data exchange between JavaScript on the one side and SQL and PL/SQL on the other.

In the 21c release, JavaScript support in Oracle Database focuses on the low-code application framework Oracle Application Express (APEX). Starting with Oracle Database 21c and APEX 20.2, developers can implement server-side logic in APEX applications (e.g. dynamic actions and processes) in JavaScript and are no longer restricted to just PL/SQL. We believe this is an exciting new feature for APEX developers, that has the potential to increase productivity and enable cool new features in APEX applications. We’ll take a more detailed look at the new JavaScript feature in APEX in a follow-up blog post.

Besides APEX, server-side JavaScript execution in Oracle Database is available through a general-purpose PL/SQL API. In this blog post, we give a short tour through this API and some of the features available in 21c. Furthermore, we peek under the hood and discuss some details about how server-side JavaScript execution is implemented.

We call the component that powers JavaScript execution in Oracle Database Multilingual Engine (short: MLE). The name might give away that MLE is much more than just a JavaScript engine. Under the hood, the major ingredient of MLE is GraalVM — a polyglot runtime that can execute several programming languages with high performance. We built MLE on the GraalVM foundation using the unique embedding capabilities of GraalVM that we believe are interesting for other scenarios as well. But first, let’s have a look at the user-facing side.

Using JavaScript in Oracle Database

Oracle Database provides the PL/SQL’s quote syntax to avoid any escaping issues in code snippets.

        DECLARE

           ctx DBMS_MLE.context_handle_t := DBMS_MLE.create_context();

        BEGIN

           DBMS_MLE.eval(ctx, 'JAVASCRIPT', q'~console.log("Hello, World!");~');

           DBMS_MLE.drop_context(ctx);

        END;

        /

The DBMS_MLE package offers additional procedures, for example to exchange values between JavaScript and PL/SQL.

By default, the JavaScript function console.log() writes to the buffer of the PL/SQL package DBMS_OUTPUT. If this snippet is executed through a client like SQL Developer Web, SQLcl or SQL*Plus, you’ll see the output generated by console.log() on the console of those tools. Depending on the client, it might be necessary to enable retrieval of DBMS_OUTPUT in the client. For example, in SQL*Plus, the printing of the output generated by DBMS_OUTPUT is controlled by the SET SERVEROUTPUT setting. The output can also be retrieved manually using the DBMS_OUTPUT package.

DBMS_MLE is not limited to a single context within a session. Using DBMS_MLE.create_context(), we can create multiple contexts within a session, each of which represents a completely independent JavaScript runtime. Granting fine-grained control over contexts to developers is a powerful feature: it enables isolation of different applications in separate contexts within the same session and prevents any interference between the applications. Note, however, that contexts are not free. Each context consumes memory within the current database session. To minimize the impact on database resources, applications should not create more contexts at a time than are strictly required, and drop contexts as soon as they are no longer referenced. This is no different to resources like SQL cursors.

To be really useful for server-side programming — for example in APEX applications — JavaScript code needs to be able to interact with the database. MLE provides the mle-js-oracledb JavaScript module that supports the execution of SQL and PL/SQL statements. In the next example, we use this module to execute a simple SQL query that returns the current time.

        DECLARE

           ctx DBMS_MLE.context_handle_t := DBMS_MLE.create_context();

           user_code clob := q'~

              const oracledb = require("mle-js-oracledb");

              const sql = "SELECT SYSTIMESTAMP as ts FROM dual"

             // execute query

             const result = oracledb.defaultConnection().execute(sql);

             console.log(JSON.stringify(result.rows));

           ~';

        BEGIN

           DBMS_MLE.eval(ctx, 'JAVASCRIPT', user_code);

           DBMS_MLE.drop_context(ctx);

        END;

        /

The mle-js-oracledb API closely follows the regular client-side Oracle Database driver for node.js. Using mle-js-oracledb, applications can execute arbitrary SQL statements (e.g. queries, DML statements, anonymous PL/SQL blocks), bind values to placeholders, and fetch query results. Existing JavaScript code that uses the node-oracledb API can usually be adapted to mle-js-oracledb with little effort. Compared to node-oracledb, we don’t have to actually connect to the database here. All SQL statements executed from JavaScript code in MLE are executed within the current database session. The oracledb.defaultConnection method returns a Connection object that represents the current session. Note that server-side JavaScript code executed in APEX can conveniently access the connection object via the apex.conn property.

When fetching query results and binding placeholders in SQL statements, mle-js-oracledb translates between PL/SQL types on the one side, and JavaScript types on the other side. Just as with node-oracledb, PL/SQL types are mapped to the closest respective JavaScript type by default. In the example above, the result column ts has the PL/SQL type TIMESTAMP WITH TIME ZONE, and is fetched as a JavaScript Date value. 

Converting between PL/SQL types and native JavaScript types is not always appropriate, though. Data type conversions can lead to a loss of precision. In the next example, we execute a SQL query that returns a NUMBER value. We truncate the result of exp(4) to 3 three decimal places, expecting the result 54.598.

        DECLARE

           ctx DBMS_MLE.context_handle_t := DBMS_MLE.create_context();

           user_code clob := q'~

              const oracledb = require("mle-js-oracledb");

              // SQL NUMBER result: 54.598

              const sql = "SELECT trunc(exp(4), 3) AS n FROM dual";

              const result = oracledb.defaultConnection().execute(sql);

              // fetch floating-point JavaScript number value

              // Output: 54.598000000000006

              console.log(result.rows[0][0].toString());

           ~';

        BEGIN

           DBMS_MLE.eval(ctx, 'JAVASCRIPT', user_code);

           DBMS_MLE.drop_context(ctx);

        END;

        /

By default, Oracle NUMBER values are fetched as JavaScript number values. Conversion from the more precise Oracle NUMBER format to the less precise JavaScript number format can lead to unexpected results. In our example, the returned JavaScript number 54.598000000000006 differs slightly from the actual result of the SQL query due to the floating-point conversion. Although this effect might not seem too relevant in this particular example, this changes drastically once we start dealing with currency values, for example — here, numerical precision is paramount.

To improve on this situation, the mle-js-oracledb API enables developers to choose the data representation that is appropriate for a particular application scenario. As in the two preceding examples, SQL values can be fetched as native JavaScript types, which integrates conveniently with existing JavaScript code. As an alternative, MLE offers JavaScript APIs for selected PL/SQL types that eliminate the need for conversion to a native JavaScript type, and avoids any loss of precision. In the following example, we execute the same SQL query as before and fetch the resulting NUMBER column. However, this time we instruct mle-js-oracledb to fetch the column as OracleNumber objects.

        DECLARE

           ctx DBMS_MLE.context_handle_t := DBMS_MLE.create_context();

           user_code clob := q'~

              const oracledb = require("mle-js-oracledb");

              const sql = "SELECT trunc(exp(4), 3) AS n FROM dual";

              // fetch NUMBER column as OracleNumber

              const options = { fetchInfo: { N: { type: oracledb.ORACLE_NUMBER } } };

              const result = oracledb.defaultConnection().execute(sql, [], options);

              // print decimal OracleNumber value

              // Output: 54.598

              console.log(result.rows[0][0].toString());

           ~'

        BEGIN

           DBMS_MLE.eval(ctx, 'JAVASCRIPT', user_code);

           DBMS_MLE.drop_context(ctx);

        END;

        /

OracleNumber objects represent the exact same decimal value as the original SQL value. Furthermore, OracleNumber objects provide methods for decimal precision arithmetics with the same semantics as the corresponding PL/SQL operations on the NUMBER type.

Multilingual Engine: Embedding GraalVM in the Oracle Database

So far, we’ve seen how Oracle Database 21c can execute JavaScript and enable APEX applications to benefit from server-side JavaScript logic. But how does this actually work under the hood? In the introduction of this article, we mentioned that the Multilingual Engine (MLE) at its core is an embedding of GraalVM in Oracle Database. Not only is GraalVM a great stand-alone runtime for Java and other languages, but it is also embeddable into native applications. MLE is an interesting example of how GraalVM can enrich existing applications with programming language capabilities, for example JavaScript execution. In the rest of this article, we shine a spotlight on some aspects of the architecture of MLE and discuss the GraalVM features that enable this embedding.

The following diagram shows a simplified version of MLE’s architecture. Let’s go through some of these boxes and arrows and see what they entail.

Oracle Database Exam Prep, Database Preparation, Oracle Database Guides, Oracle Database Learning, Oracle Database Certification
MLE Architecture
 
We’ll start at the lowest layer: How does Oracle Database — a native application written in C — actually call into GraalVM to execute JavaScript code? After all, GraalVM is implemented in Java. The answer is GraalVM Native Image, which can compile Java applications into stand-alone executables that can run without a JVM. This feature is typically used for Java microservices that have low memory requirements and start almost instantaneously. But along with standalone executables, Native Image can also generate shared libraries that can be loaded into an existing application. This is the core technology that enables the Multilingual Engine. Native Image compiles the MLE runtime and all required GraalVM components like the JavaScript runtime — all implemented in Java — into a shared library that is loaded on-demand into a database process.

After loading the Multilingual Engine, a database process calls functions in the MLE Native Image to manage contexts and actually execute JavaScript code. Likewise, MLE calls native database functions in order to provide services like SQL execution to JavaScript code. Some of these call paths are sketched in the diagram above — all arrows that cross from red (native database code) to blue (MLE Native Image) or vice versa. Because such calls occur frequently, they are critical for performance. Fortunately, GraalVM implements calls between C code and functions in a Native Image very efficiently, with little overhead over regular calls between C functions.

So far, the embedding of GraalVM in Oracle Database appears similar to at image build time, minimizing the startup effort when the MLE Native Image is loaded. Moreover, areas of the Native Image heap that have been initialized at image build time with read-only access at runtime are transparently shared between database processes. This reduces the memory impact of MLE in scenarios with multiple concurrent database sessions that execute JavaScript code with MLE.

In an embedding scenario like Node.js, memory for the JavaScript runtime is allocated directly from the operating system. For the integration into Oracle Database, MLE has to reach deeper than that. Oracle Database includes sophisticated management of OS resources like CPU and memory to optimize resource usage specifically for database workloads. Furthermore, Oracle Database is a multi-tenant system which ensures that individual tenants cannot exceed the imposed resources limits. To ensure that JavaScript execution plays by the same rules, MLE is integrated with the Oracle Database Resource Manager. MLE allocates all memory required for the JavaScript runtime through database services, never from the operating system directly. All such allocations are subject to Resource Manager policies. In the same way, CPU limits imposed by the Resource Manager are effective for JavaScript code, and JavaScript code can be cancelled reliably when requested by the database. This ensures that the MLE component is a good citizen in a database instance.

Moving further up the stack, let’s discuss the management of MLE contexts that can be created using the DBMS_MLE API. As described earlier, MLE gives control over context management in a database session to developers, providing a flexible programming model. This is not a special feature of MLE though. The ability to use multiple contexts that encapsulate application and runtime state and are logically independent is built deeply into GraalVM. The GraalVM Polyglot API for embedders supports the creation of multiple independent contexts. MLE’s API for context management is implemented directly on top of the Polyglot API.

Recalling that MLE allows creating several contexts in the same database session, let’s have a look at the following diagram which depicts how the context handles directly map to Polyglot contexts managed in the MLE runtime:

Oracle Database Exam Prep, Database Preparation, Oracle Database Guides, Oracle Database Learning, Oracle Database Certification
MLE Context Management

While contexts encapsulate the application state, GraalVM manages other program resources within Polyglot engines. Throughout a database session, MLE uses the same polyglot engine underneath multiple contexts. If the same JavaScript application code is evaluated in multiple contexts in the same session that share an engine, the underlying engine can re-use the executable representation of JavaScript code across those contexts. This can reduce the memory footprint as well as the time required to parse JavaScript code if multiple contexts are used.

A powerful feature of GraalVM is language interoperability. JavaScript and other programming languages implemented on top of GraalVM’s Truffle language implementation framework can interact with each other by calling methods and accessing each other’s data with little to no overhead. MLE uses GraalVM’s language interoperability to provide JavaScript code with access to database functionality in a controlled and secure manner. In the previous section, we had a look at the JavaScript API for SQL execution provided by the mle-js-oracledb module. This JavaScript API is actually implemented on top of a component called the MLE SQL Driver. This component supports the execution of SQL statements within the current database session using a safe API. The JavaScript implementation of the mle-js-oracledb module calls the MLE SQL Driver through Truffle’s language interoperability protocol to execute SQL statements and fetch query results. Thanks to Truffle’s efficient implementation of cross-language calls, this interaction has practically no overhead compared to calls between two JavaScript functions.

This concludes our tour through some of the internals of the Multilingual Engine. To summarize, the Oracle Database Multilingual Engine allows developers to write server-side logic in JavaScript, a modern, widely used programming language. MLE is an example of how GraalVM’s unique features enable its embedding into highly complex systems like Oracle Database.

A Vision for the Future


With JavaScript support in Oracle Database 21c, we took a first step at bringing GraalVM to the Oracle Database. We are looking forward to improving the Multilingual Engine in future releases and believe there are many great features that can further improve the developer experience for server-side programming with modern languages and enable great applications on top of Oracle Database. One such feature is support for JavaScript modules as first class citizens and providing integration and tooling for NPM packages. We also aim to make MLE truly multilingual by adding support for more languages.

Source: oracle.com

Related Posts

0 comments:

Post a Comment