Introduction to Oracle TimesTen In-Memory Database

Introduction to Oracle TimesTen In-Memory Database

TimesTen is a relational in-memory database. It can be used as a standalone database of record, as a cache, or as a combination of both. The query language is SQL, and it supports many standard APIs (ODBC, JDBC, OCI, ODP.NET, …) as well as PL/SQL for stored procedures and functions. TimesTen has a high degree of compatibility with Oracle Database in terms of its SQL dialect, PL/SQL, and APIs.

What do we mean by an 'in-memory' database?

In a traditional, disk-based database, the data managed by the database primarily resides on persistent storage such as disk, SSD or flash storage. The database management system maintains a memory cache (in Oracle database this is called the SGA) and, in most cases, for data to be read (queries) or written (DML) it must first be brought from persistent storage into this RAM cache.

A lot of the work that the DBMS does involves managing the contents of this cache to try to ensure that data is available in the cache when it is needed, and that modified data gets written back to persistent storage in a timely fashion.

TimesTen is based on a different approach. The entire dataset that is managed by TimesTen (the TimesTen database) is loaded into RAM, from persistent storage, at the time that the TimesTen database starts up.

The data then remains entirely in RAM until the database is shut down. This design enables the use of different storage formats for data and simpler, faster data access methods with much less DBMS kernel overhead. This in turn provides a major speed up in data access, for both read and write operations.

Typically, the performance improvement is at least an order of magnitude (hence the origin of the name TimesTen), but in some cases may be a lot more.

What kind of performance might I expect from a TimesTen database?

TimesTen speeds up access to data by eliminating a lot of the overhead found in traditional database systems. This results in much lower, and more consistent, response times for SQL operations. Typical SQL response times for TimesTen are measured in microseconds rather than milliseconds.

This reduction in operation latency also leads to greatly increased query and transaction throughput. In fact, performance is typically improved by an order of magnitude compared to traditional, disk-based databases and in many cases the speedup can be much larger.

If the entire database is permanently in memory, isn't it vulnerable if a problem occurs, such as a power failure?

If TimesTen only kept data in RAM then yes, it would indeed be vulnerable in the event of software/hardware failures, power outages etc. TimesTen has a built-in persistence mechanism that ensures data stored in the TimesTen database is protected against such eventualities. This persistence mechanism is based on two cooperating mechanisms: checkpointing and transaction logging.

Checkpointing

Every TimesTen database has a pair of checkpoint files located on persistent storage. Each of these files is a snapshot of the entire in-memory database as of some moment in time. The checkpoint files provide a means of physical database recovery after an unexpected database shutdown.

During database operation, TimesTen periodically refreshes these files, in a round robin fashion, from the in-memory database. This mechanism is known as checkpointing and there are various parameters that can be tuned to optimize it based on the workload and the performance characteristics of the persistent storage. The checkpointing mechanism is a background operation that does not block query or DML execution within the database.

Transaction logging

TimesTen also maintains a change journal, known as the transaction log, on persistent storage. Whenever an operation modifies persistent data within the database it results in one or more log records which describe the change.

These log records are posted into an in-memory buffer (the log buffer) from which they are written to the transaction log. The transaction log stream is divided up on storage into multiple files, each of a predefined maximum size. The transaction log provides a mechanism for both transaction rollback (undo) and logical database recovery (redo).

By default, transaction logging occurs outside the application transaction path so does not impact the performance of DML operations.

The combination of checkpointing and transaction logging makes the database persistent across planned shutdown / startup events . It also allows TimesTen to recover gracefully after an unexpected shutdown, without losing any data.

How do applications connect to TimesTen?

TimesTen supports a wide variety of database APIs including ODBC, JDBC, Oracle Call Interface (OCI) and ODP.NET. In addition, connectivity for other languages such as Python, Node.js, Go etc. is available via Open Source components.

Regardless of the API chosen, there are two connectivity mechanism available: direct mode and client-server mode.

Direct mode is a low latency, high throughput local connectivity mechanism available to applications running on the same host as the TimesTen database. By eliminating inter-process communication and context switching overheads, direct mode provides the highest possible levels of responsiveness and throughput.

Client-server mode is a traditional TCP/IP based connectivity mechanism and can be used by applications executing anywhere where there is network connectivity to the TimesTen database host. Clients communicate with the TimesTen server component, which is itself a direct mode application.

Can TimesTen be used as a cache?

TimesTen has built-in functionality that allows it to be used as a read-only or read-write cache for Oracle Database. Because TimesTen is relational, and supports standard SQL, PL/SQL and industry standard database APIs, it is often easier from an application perspective to use TimesTen as a cache for Oracle Database versus introducing an object, or key-value, based cache.

TimesTen's cache functionality provides SQL based syntax to declare the Oracle tables that are to be cached, the caching mode (read-only, read-write, static or dynamic caching etc.) and to perform the initial data population of the cache. Once the cache is populated, TimesTen's cache refresh / change propagation mechanisms keep the data synchronized between the TimesTen cache and the Oracle database.

In addition to TimesTen's native caching functionality, you can also use Oracle GoldenGate to replicate data from any source supported by GoldenGate to a TimesTen database. This provides an alternative method for implementing read-only caching and allows caching of data sourced from databases other than Oracle.

It is important to remember that even when being used as a cache, TimesTen still provides full database functionality including local persistence and high availability. It is not uncommon to see hybrid use cases, where a single TimesTen database may act as the database of record for some tables while also containing tables which are cached copies of data from a non-TimesTen database.

See also the related blog Is TimesTen an Oracle Database Cache or a System of Record?

Does TimesTen provide any High Availability capabilities?

TimesTen can be deployed in several different ways:

1. As a single standalone database.

2. As a high-availability pair of databases (active-standby), with optional additional read-only copies.

3. As a distributed, shared nothing, highly available, elastically scalable cluster database.

Options (1) and (2) are referred to as 'TimesTen Classic' while option (3) is known as 'TimesTen Scaleout'. All three deployment modes support TimesTen's cache functionality and GoldenGate. TimesTen HA is integrated with the Cache functionality so you can implement a highly available cache as well as a highly available database of record.

TimesTen Classic provides high availability by way of TimesTen's integrated active-standby pair replication. Such a deployment consists of a pair of databases with explicit active (reads and writes allowed) and standby (only reads allowed) roles. High performance parallel data replication replicates changes from the active to the standby.

If the active fails then the standby can be very quickly promoted to become active and the workload moved to the new active. The failed active is then recovered and resynchronised to become the new standby.

In addition, the core active-standby pair can support additional read-only copies. These read-only replicas can be located in a different data center to the core active-standby pair and this provides a Disaster Recovery capability. In the event of a total loss of the primary data center, a remote read-only copy can be very quickly promoted to become the active member of a new active-standby pair at the remote site.

TimesTen Scaleout provides automatic high-availability through K-safety. When you deploy a Scaleout database you specify how many copies of the data to maintain (1 - 5). With a single copy there is no HA (ideal for dev and test setups) but with more than one copy the database becomes highly available. Commonly 2 or 3 copies tend to be used for production. TimesTen Scaleout handles all aspects of HA making this an easy-to-use configuration for DBAs and developers alike.

What about the Cloud, Containers, Kubernetes etc.?

TimesTen is easy to deploy in private or public clouds. The simplicity of TimesTen means it can be installed easily and quickly and operated with a minimum of oversight. The most common licence model for cloud deployments is bring your own licence (BYOL): customers use new or existing licenses to deploy in the cloud in a similar fasion to on-premises deployments.

Similarly, TimesTen is well suited to deployment in containers, provided that some form of persistent storage is available (after all it is a database). When container-based deployment is being used, usually a software layer that will orchestrate the containers is used; by far the most popular such layer is Kubernetes.

TimesTen is fully supported in Kubernetes environments and provides a richly functional Kubernetes operator. The TimesTen Kubernetes Operator provides full, automated lifecycle management for TimesTen in an extremely simple to use fashion. All you need to do is create a few small, simple configuration files, issue a few commands and your TimesTen HA configuration is deployed and will be automatically managed.

Source: oracle.com