Industrialization of artificial intelligence and machine learning for SaaS applications

This article provides an overview of how machine learning (ML) applications, a new capability of Oracle Cloud Infrastructure (OCI) Data Science, help software-as-a-service (SaaS) organizations industrialize artificial intelligence (AI) and machine learning and embed AI and ML functionality into their product experiences. It discusses our experience and the challenges we had to overcome when we faced the problem of delivering hundreds of AI and ML features to tens of thousands of customers. It describes key capabilities of ML applications that shorten the time-to-market for AI and ML features by standardizing implementation, packaging, and delivery. The article also provides an outlook into other areas beyond SaaS that can benefit from using ML applications.

It has been widely recognized that building an AI and ML solution isn’t the same as building a general software solution. Today, businesses understand that moving from a data science proof-of-concept to production deployment is a challenging discipline.

Many known success stories of AI and ML solution productizations exist, not only among high-tech IT companies but across many industries like healthcare, retail, energy, agriculture, and more. Thanks to AI and ML frameworks, toolkits, and especially services provided by major hyper-scale cloud vendors like Oracle, it is becoming easier to develop, deploy and operate AI and ML solutions. The rate of successful projects is also positively influenced by the fact that companies are adopting practices like MLOps that streamline the ML lifecycle.

However, with SaaS applications, the percentage of successful projects is lower. For example, let’s imagine you want to enrich your SaaS application with a new AI and ML feature. It’s on a completely different level! You need to develop and run thousands of pipelines on thousands of varied customer datasets, training and deploying thousands of models. To avoid this management nightmare, you must automate these tasks as much as possible.

The success of these projects depends on the efforts and knowledge of large, professional teams of experienced software engineers, data engineers, and data scientists. This investment can be expensive for an organization, and not every attempt gets a happy ending. AI and ML projects tend to go over budget in general. According to a McKinsey & Co. survey, the cost of 51% of AI projects was higher than expected when AI high performers were excluded. Another serious issue can be delays. Teams building AI and ML features for SaaS on their own can experience setbacks that extend the project by a year or more.

Some problems and challenges associated with delivery and operations of AI and ML feature for SaaS applications are too complex for every team or organization to address repeatedly. A better strategy relies on ML applications to solve them for you, enabling your development and data science teams to better focus on business problems.

Our experience: Building bespoke solutions for SaaS

At Oracle, we have been delivering market-leading SaaS applications for decades and taking the ML applications approach for years through working with SaaS teams to help them add new AI and ML capabilities to their SaaS applications. This work enabled us to gain a deep understanding of both the needs of SaaS products and challenges related to delivering AI and ML features within SaaS applications.

To stay competitive, SaaS organizations like Oracle Fusion Applications or Oracle NetSuite need the ability to introduce new AI and ML use cases as intelligent features for their customers. They need to rapidly develop, deploy, and operate AI and ML features. The development lifecycle needs to be shortened from months to weeks. This goal poses a challenge because of the large number of SaaS customers and the size of SaaS organizations. To give a sense of the scale, with thousands of customers who each have hundreds of AI and ML features, Oracle runs millions of ML pipelines and models in production!

We also must cover the entire lifecycle of AI and ML features. To accomplish this goal, implementing business requirements, debug, test, and deploy solutions to production must be simple for data scientists and engineers. Then these organizations must be able to monitor and troubleshoot the production fleet of solution instances provisioned for thousands of customers, used by millions of users.

Taking into consideration the scale of AI and ML development and operations, SaaS organizations need to standardize the implementation and development of AI and ML features to efficiently manage and evolve them.

ML application origins: Adaptive intelligent applications

The roots of ML Applications go back to our work with Adaptive Intelligent Applications (AIA). Under the umbrella of AIA organization, we have been building several generations of a framework that helps SaaS teams build AI and ML features for SaaS applications. We focused primarily on Fusion applications like enterprise resource planning (ERP), human capital management (HCM), and customer experience (CX). However, other non-Fusion applications and teams were involved even in the early days.

To make a long story short, we helped AIA to move most of their applications to production. If you are a Fusion user, you have likely already interacted with AIA functionality. To find out more about AI Apps for finance, human resources, sales, service, and procurement, visit AI Apps Embedded in Oracle Cloud Applications.

For further explanation, we provide a brief description for three examples of successfully productized AIA features.

ERP intelligent account combination defaulting

This AIA feature assists the payables function by using AI and ML to create default code combination segments when processing invoices don’t match purchase orders (POs). Predicting and defaulting code combination segments reduces manual keystroke effort, reduces human errors, saves invoice processing time, and reduces costs.

AI-UX suggested actions

News feed suggestions make Oracle customers more productive by recommending important tasks to them in a timely fashion. Fusion apps have a wide range of functionality and knowing what you have access to across those apps at a particular time and how to navigate to important tasks can be challenging.

News Feed Suggestions track the navigation behavior of a user and users like them to make recommendations for the tasks they are most likely to perform at that time. For example, if a group of users historically submits timecards each Friday, they see a suggestion for that task on that day. If tasks related to closing at the end of each month exist, the users with roles associated with those tasks see suggestions for those tasks. With news feed suggestions, Oracle helps your users get to the tasks that matter faster.

CX sales lead conversion probability

Within CX sales, AI and ML models analyze data on profiles, sales, and the interactions you have with company prospects and customers to create a score to indicate the propensity of each lead to progress to a sale close. This functionality enables sales teams to have more effective lead management with better prioritization, which leads to more incremental sales, more efficient processes through a reduced need to wrangle data, and improved marketing effectiveness and return on investment driven by tighter targeting and personalized content.

ML applications: Basics

ML applications can help SaaS teams address most of the challenges they face when trying to add AI and ML features to their SaaS applications. Let’s look to a few basic properties of ML applications to help understand how ML applications work and their benefits.

Provider and consumer roles

ML applications distinguish between users providing or authoring ML applications (Providers) and the users consuming or using ML applications (Consumers). These roles are even reflected in the Oracle Cloud Console. ML applications provide two UI areas, as shown in the following image.

ML application resource

An ML application resource, or just ML app, is a self-contained representation of an AI and ML use case. It has well-defined boundaries defined by contracts and components that define its implementation.

A typical ML app consists of the following features:

◉ Data pipelines responsible for the preparation of training data

◉ ML pipelines that train and deploy models

◉ Triggers and schedules that define workflows

◉ Storage for the data used by other components

◉ Model deployments that serve predictions

◉ AI services that can be used instead of or in addition to custom pipelines and models

You can think of an ML app as a blueprint that defines how an AI and ML use case is implemented for a customer. This blueprint is used to instantiate a solution for each SaaS customer.

ML application instance resource

An ML application instance resource, or ML app instance, represents an ML app prepared or instantiated for a specific customer. An ML app Instance is created for a customer during provisioning, according to the blueprint defined by the ML app for the end-to-end solution. The solution typically trains customer-specific models with customer data sets. SaaS applications then use the prediction services provided by ML app Instances that were created for them.

As-a-service delivery across tenancies

A key characteristic and benefit provided by ML applications is the standardization of implementation, packaging, and the delivery of AI and ML use cases, which allows ML application providers to offer prediction services that they implement as-a-service. Consumers can see only the surface of the provided ML apps. They interact with ML apps through defined contracts. They can’t see the implementations.

On the other hand, providers are responsible for the management and operations of the implementations. They ensure that prediction services implemented in the ML apps are up and running. They monitor them, react to outages, and roll out fixes and new updates.

Unlike other OCI services that operate in a single OCI tenancy, ML applications support the interactions between consumers and providers across OCI tenancies. Providers and consumers can work independently in their own separate OCI tenancies. It allows for loose coupling between providers and consumers. This separation is a huge advantage because teams of engineers and data scientists developing ML apps don’t need to ask for access to the tenancy of the SaaS application. This configuration simplifies and improves security for the overall solution. Compute and storage resources used by ML apps also don’t interfere with resource consumption and service limits in SaaS tenancy.

You might ask how providers can support consumers without access to the SaaS tenancy. ML applications guarantee observability to providers. For every ML app instance created, ML app instance view resource is created for providers in their tenancies. Instance views mirror ML app instances from consumer tenancies and link all related resources, such as buckets or model deployments. This setup benefits operations and troubleshooting. Providers can easily navigate to the resources used by a specific customer and review monitoring metrics and logs.

Democratization of AI: ML applications and AI services

Although ML applications drastically simplify the development of AI and ML features for SaaS applications, you still need data scientists to build ML pipelines and models for your ML apps.

We found a way to further accelerate the development of AI and ML features. In many cases, you don’t need to develop new pipelines and models and can use a generic AI and ML solution. For example, when you need sentiment analysis, anomaly detection, or forecasting, you can use AI Services. AI Services provide the best-in-class implementation of common AI and ML problems. You can use AI Services to implement the machine learning part of your ML apps.

By adopting AI Services in conjunction with ML applications, you don’t need a large data science department. Instead, your citizen data scientists can enrich your SaaS applications with cutting-edge AI features. Under the hood, ML applications and AI services apply transfer learning and tailor AI and ML models to the specific data sets used by your customers.

Best of all, this shortcut doesn’t close the door to future evolution. You can update your implementation later and provide your own ML pipelines and models if you choose. Thanks to the versioning capabilities, you can roll out even such a substantial change without affecting clients using your initial implementation.

ML applications: Advanced features

The delivery and operations of AI and ML features for SaaS applications  too complex for every team or organization to repeatedly address have problems and challenges associated with. A better strategy relies on ML applications to solve them for you, enabling your development and data science teams to focus on business problems. The following sections can give you insight into these problems and solutions.

Versioning: Evolving the fleet

Versioning is one of the most important benefits of ML applications for providers. Typically, teams developing AI and ML use cases don’t want to solve this type of problem on their own. They’re aware of or soon realize how complex it can get, and they start looking for tools or services that solve these problems.

When you decide on a custom solution, think through how to support versioning and consider the following questions:

 

◉ When and how is a new version provisioned for new customers?

◉ How can you roll out a new version to all the customers who have been already using an older version?

◉ How can you update implementations for existing customers when a pipeline is changed? What if the change isn’t backwards compatible?

◉ What if the interface of your prediction service needs to be changed?

◉ What if the change of your prediction service is backwards incompatible?

◉ How do you deal with migrating customers from old to new versions?

◉ How do you migrate without downtime?

◉ What if your SaaS application has been updated only for some customers and your prediction services are receiving different versions of incompatible requests?

You can imagine that engineers must answer hundreds of tough questions.

On the other hand, consider usability and user experience. You don’t want your data scientists to spend days and weeks following a complex change management process, filling in forms, and asking for approvals. Imagine that a data scientist fixes a typo in an ML pipeline. It must be easy to implement the change, test it, and roll it out to production without needless delay.

ML applications allow providers to release changes to production within minutes independently of the SaaS application release process. Still, ML application’s strong versioning capabilities give providers confidence that the validated changes are delivered to customers without them noticing it. Providers must only update the implementation of their ML apps. ML applications ensure that the whole fleet of existing instances used by customers is updated without outages.

Finally, versioning guarantees reproducibility and traceability. Providers will always know which version, even code revision and line of code, is used by a particular customer in an environment and which changes were applied to the customer’s implementation. Without this feature, troubleshooting problems can be challenging. You also struggle with answering questions from auditors. For every change ever implemented, you must track who introduced the change and when.

Fleet monitoring and management

ML Applications are a powerful tool that allows for the deployment of AI and ML functionality at a mass scale. This brings the question of how to monitor and manage the fleet of millions of AI and ML models.

When you have numbers of customers in the low hundreds and a few ML apps, the basic features of ML applications with OCI Monitoring and Logging services can be sufficient for you to monitor and troubleshoot customers’ ML app instances. However, when the adoption grows and you operate dozens of ML apps for thousands of customers, you need more sophisticated tools. The situation usually gets even more complicated because of the necessity to provide the applications in multiple regions and environments.

Different roles in your organization can benefit from an aggregated view that collects information across teams, regions, and environments. Your managers can use fleet monitoring to answer the following questions:

◉ How many customers are affected by a failure or defect?

◉ Are sudden failures of prediction services limited to a region?

◉ How many instances of an ML app are provisioned within a region?

◉ How many ML apps are used by a major customer?

Your product managers can analyze trends around customer adoption and usage. They also control the exposure of new features to customers by defining customer segments, such as early adopters. Without aggregated monitoring metrics, they’re flying blind.

Data scientists can get great assistance from fleet monitoring. Typically, data scientists face a difficult problem. They need to generalize their models across diverse data sets that customers have. They need to ensure that the performance metrics of models meet business requirements for most customers, while managing the tradeoffs between accuracy and operational performance and efficiency. Using fleet monitoring, they can evaluate model performance metrics and take corrective actions.

Reusable components and patterns

ML applications reflect the observation that machine learning implementations tend to follow a relatively small number of patterns. We provide a library of common patterns that you can use as a foundation for the implementation of new ML applications. Providers can pick a pattern and build a new ML application on top of the pattern by adding more business logic instead of building the complete end-to-end implementation from scratch. The pattern ensures that the implementation follows best practices, is efficient and scalable, addresses all corner cases, and covers nonfunctional aspects like security and monitoring.

Patterns define extensibility points that allow providers to add customization into the overall end-to-end implementation of the ML application. For example, a pattern can introduce a transformation extensibility point that runs your Spark SQL in the transformation step within a data pipeline that’s part of the pattern’s implementation.

Demonstrating the benefits of this approach is easy by considering, for example, a pattern that ingests data from Fusion and serves predictions with a REST endpoint. The pattern implements a data pipeline using OCI Data Integration that incrementally ingests Fusion data and transforms them using OCI Data Flow. Next, an ML pipeline reads transformed training data, trains a new model, and finally deploys the newly trained model as a model deployment. Then, active monitoring triggers when something unusual happens, such as a data ingestion failure or an out-of-memory error in training. Providers are notified and can fix the problem or take proactive action to avoid failures.

You can imagine how inefficient and error-prone it would be to let every team implement their own ML apps from scratch instead of applying the pattern. Better to let engineers and data scientists focus on their business problems instead of challenging them with questions like, “What if the data connection is interrupted while a data increment is downloaded? Is the ingestion going to resume without data loss? What if the training pipeline fails?”

Another essential feature of ML Applications is that Providers can customize patterns. Providers can customize provided patterns and even build new greenfield patterns that they can reuse across their organization.

Vision of ML applications

ML Applications have already been a game changer for Oracle SaaS organizations for years. Now, as part of OCI Data Science, we’re expanding the core functionality and addressing new business problems so all our customers can take advantage.

Helping OCI customers: Independent software vendors building SaaS applications

The obvious extension to the internal usage of ML applications by Oracle teams is to open this capability to you: OCI customers building your own SaaS applications. You can add AI and ML features to your SaaS applications with the same efficiency as the Oracle SaaS organizations.

These days, it can take several years before teams mature their AI and ML development and manage to integrate AI and ML features into their SaaS applications. Most of those teams understand the challenges and expect to get the required functionality as a service instead of implementing everything from scratch on their own. A typical example of functionality that no one wants to build is versioning. Neither engineers nor data scientists want to think through all the corner cases and define how to introduce backward compatible or even incompatible changes into various parts of their AI and ML implementation. They don’t want to define update strategies for models and services used by SaaS customers.

Similarly, ML applications can help you when you need to start AI and ML features for multiple lines of businesses or geographic locations. For example, financial institutions might want to instantiate ML features with different configurations for AMER, EMEA, and APAC regions. Healthcare providers might need to provide an independent instance of an ML App to each hospital.

SaaS application customization: Bring your own implementation (BYOI)

Large companies using SaaS applications with ML application-based features, such as customers of Fusion ERP, might need to customize the out-of-the-box provided AI and ML capabilities and utilize extra data they have outside of the SaaS application or use their private intellectual property. ML applications are designed to enable those companies to build custom ML applications and replace out-of-the-box available functionality provided by Oracle.

Third-party partners who provide specialized functionality for business verticals can even provide such customizations, as described in the next section.

Beyond SaaS: ML application marketplace

One of the key benefits of ML Applications is the ability to offer prediction services as-a-service to the SaaS applications consuming them. SaaS applications using the prediction services don’t need to worry about their implementations and operations. They simply use the provided functionality. On the other hand, the implementers of prediction services are responsible for the operations of their ML app deployments.

We plan to allow anyone who builds any applications on OCI to benefit from this capability. Anyone can build ML apps on OCI, becoming a provider of ML applications. Provided ML apps can be registered in a marketplace allowing any OCI customer to discover them and use them.

Anyone who needs a specific AI and ML functionality can consume ML applications registered in the marketplace. They can use prediction services designed for a certain business problem without needing to invest in the development of their own machine learning pipelines and models. The marketplace can also assist customers and suggest ML apps that a customer can use with data sets they own.

Source: oracle.com

Continue reading

Custom models and text translation come to OCI Language

Part of Oracle Cloud Infrastructure (OCI) AI services, OCI Language service allows you to perform sophisticated text analysis at scale without any machine learning (ML) knowledge. It provides pretrained models for sentiment analysis, entity extraction, language detection, and many other natural language tasks. Today, we’re excited to announce two exciting capabilities in the Language service: Customizable models and automatic text translation. These features are now in limited availability in all OCI’s commercial regions and can be accessed through OCI software developer kits (SDKs) and REST APIs.

Build your own custom language models

OCI Language enables you to train customized language models, even if you’re not a natural language processing (NLP) expert. You can train custom classification and custom named entity recognition (NER) models. The OCI Language service learns from previously labeled data, such as classified records and samples of entities extracted from text, to train the models. After you train a custom model, you can deploy dedicated endpoints to serve your requests.

Custom classification

Imagine that you’re responsible for the support tickets that come into your company. Each day, you receive thousands of tickets that need to be routed to specific departments. In the past, you had humans perform this task. With this tedious work, keeping your employees excited about their work is difficult. Over the last couple of months, your employees have routed thousands of tickets to the right department. Wouldn’t it be great if AI could help them?

Using OCI Language, you can create a custom model that learns from all the work the humans have done in the past.

When the model is trained on previous conversations, it can classify any new requests automatically, freeing humans to perform less tedious tasks.

OCI Language allows you to host the custom models on dedicated endpoints. Depending on the expected throughput you need to handle, you can assign a larger or smaller number of inference units (a unit of compute) to your endpoint.

We described a support ticket classification scenario, but the same principles apply to other use cases, such as document classification, clause classification, and intent recognition.

Custom named entity recognition (NER)

OCI also supports the ability to identify terms that are unique to your domain, such as product part codes, manufacturing terms, and specific financial entities. You provide sample data with labeled entities to train a custom NER model that can then be used to automatically identify the entities in text.

To illustrate the types of problems that using custom NER can solve, let’s continue our support ticket use case. Imagine that many of the support tickets you receive deal with shipment issues. You want to extract critical information from each ticket, such as the order ID, the shipment date, and the name of the recipient.

Using OCI Language, you can automate this process, but first you must gather the training data. OCI Data Labeling, a service for labeling datasets, can help you label the data to train such a model. It allows you to define the custom entities, and then mark the location of those entities in the text, as shown for the following examples:

With the labeled data, you can train your own custom NER model. OCI Language provides an intuitive workflow to create and organize models. You can also evaluate global and entity-specific metrics to help you identify other data that you need to further improve your model.

When you’re pleased with the quality of the model, you can create a dedicated endpoint to automatically perform the entity extraction for you. So, you to convert unstructured data (language prose) into structured data, enabling you to automate downstream processes!

Text translation

OCI Language now allows you to automatically translate text across 21 languages. This exciting feature uses state-of-the-art AI neural machine translation techniques to translate text at scale with high accuracy. Sample use cases of automatic text translation include building multilingual chatbots, automatic application localization, translation of support tickets, article translation and any kind of application that helps you understand others globally.

We’re always expanding our language coverage. At the time of this blog post, the following languages are supported:

◉ Arabic (ar)

◉ Brazilian Portuguese (pt-BR)

◉ Czech (cs)

◉ Danish (da)

◉ Dutch (nl)

◉ English (en)

◉ Finnish (fi)

◉ French (fr)

◉ Canadian French (fr-CA)

◉ German (de)

◉ Italian (it)

◉ Japanese (ja)

◉ Korean (ko)

◉ Norwegian (no)

◉ Polish (pl)

◉ Romanian (ro)

◉ Simplified Chinese(zh-CN)

◉ Spanish (es-ww)

◉ Swedish (sv)

◉ Traditional Chinese (zh-TW)

◉ Turkish (tr)

You can experience the text translation capabilities from the Oracle Cloud Console.

Source: oracle.com

Continue reading

Fourth Quarterly update on Oracle Graph (2022)

Oracle Graph Server and Client 22.4 is available for download for use with databases in the Cloud (OCI Marketplace image is available) and for databases on-premises. Note that the new features described here are not yet available in Graph Studio, we will update this post when they are available.

Oracle Graph Server and Client version 22.4 has been released and contains a number of feature updates. Graph Server (PGX) now supports a graph loading progress reporting API, subgraph loading enhancements, and synchronization for published graphs. Oracle Graph Server and Client 22.4 is the first release to fully implement PGQL 1.5. You can find this specification here:  https://pgql-lang.org/spec/1.5/. These PGQL updates include the ability to use IS as an alternative for a colon, support for scalar subqueries, and support for EXISTS and NOT EXISTS subqueries. This release also includes some packaging enhancements and desupported features, which can be found at the end of this post.

Graph Server (PGX) Features

Graph reading progress reporting API

Oracle Graph Server 22.4 includes a new API added to PgxFuture that retrieves loading progress. This feature works with PG view graphs. 

This is an example of how the progress reporting API can be used:

PgxFuture<PgxGraph> future = session.readGraphByNameAsync("MYGRAPH", GraphSource.PG_VIEW)

FutureProgress progress = future.getProgress() // new API

Optional<GraphLoadingProgress> loadingProgress = progress.asGraphLoadingProgress()

long numLoadedVertices = loadingProgress.get().getLoadedVertices()

Subgraph loading enhancements

Previously, a subgraph was given the same name as the PG view plus a number. However, the load() function now accepts a name as a parameter. Additionally, the query for the subgraph now allows for any-directed edge patterns.

This is an example of how these features can be used when creating a subgraph:

PgxGraph subgraph = session.readSubgraph()

    .fromPgView("MYGRAPH")

    .queryPgql("MATCH (s)-[e]-(d)") // NEW: any-directed pattern

    .load("MYSUBGRAPH") // NEW: name the resulting subgraph

Synchronize published graphs

Published graphs in the Graph Server (an application can ‘publish’ graphs to share with other sessions connected to the Graph Server) can now be synchronized with updates in the database. Previously only graphs within a session could be synchronized.   This is enabled by using the new API to specify the graph configuration object when creating a synchronizer object.

This is an example of how the graph configuration can be included in the synchronizer call:

Synchronizer synchronizer = new Synchronizer.Builder<FlashbackSynchronizer>()

    .setType(FlashbackSynchronizer.class)

    .setGraph(graph)

    .setGraphConfig(graphConfig) // NEW

    .setConnection(connection)

    .build()

PGQL Features

IS as an alternative for colon (:)

In previous versions of Oracle Graph, a colon (:) was used to indicate label predicates. To include syntax in the SQL/PGQ standard (the draft ISO standard for SQL to query property graphs), the newest version of Oracle Graph now supports “IS” as a label predicate. However, this feature is additive, and does not remove the ability to use a colon as a label predicate.

This is an example of the label predicate options in 22.4:

//Before

FROM MATCH (s:Friend) -[e:knows]-> (d:Person)

//Starting 22.4 the following is supported

GROM MATCH (s IS Friend) -[e IS knows]-> (d IS Person)

PGQL on PG Views: Scalar Subqueries

Oracle Graph Server and Client 22.4 allows you to use scalar subqueries in PG views.  You can use them as part of an expression. A scalar query is a query that returns a scalar value, exactly one row and exactly one column. In this latest release of Oracle Graph, you can use scalar queries as part of an expression in a SELECT, WHERE, GROUP BY, HAVING or ORDER BY clause.

For example, you could write a query such as the following:

SELECT p.name AS name

    , ( SELECT SUM(t.amount)

        FROM MATCH (a) <-[t:transaction]- (:Account)

      ) AS sum_incoming

    , (SELECT SUM(t.amount)

        FROM MATCH (a) -[t:transaction]-> (:Account)

      ) AS sum_outgoing

    , (SELECT COUNT(DISTINCT p2)

        FROM MATCH (a) -[t:transaction]-> (:Account) -[:owner]-> (p2:Person)

        WHERE p2 <> p 

      ) AS num_persons_transacted_with

    , (SELECT COUNT(DISTINCT c)

        FROM MATCH (a) -[t:transaction]-> (:Account) -[:owner]-> (c:Company)

      ) AS num_companies_transacted_with

        FROM MATCH (p:Person) <-[:owner]-> (a:Account) 

ORDER BY sum_outgoing + sum_incoming DESC

PGQL on PG Views: EXISTS and NOT EXISTS subqueries

EXISTS and NOT EXISTS return true or false depending on whether the subquery produces at least one result, given the bindings obtained from the outer query.

In the following example, we can query to find friends of friends, and for each friend of friend, return the number of common friends. Assuming we have a graph that shows relationships among a group of people, we can do this simply with a NOT EXISTS subquery:

SELECT fof.name, COUNT(friend) AS num_common_friends

    FROM MATCH (p:Person) -[knows]-> (friend:Person)

                  -[knows]-> (fof:Person)

    WHERE NOT EXISTS (

        SELECT * FROM MATCH (p) -[:knows]-> (fof)

    )

Packaging Enhancements

◉ There is a new Oracle Cloud Infrastructure Marketplace image for RDF Server that uses Apache Tomcat. Users can choose between the image that uses Apache Tomcat or the one that uses Oracle WebLogic Server when deploying RDF Server.

◉ The Graph Server RPM now declares libfortran as a dependency. This dependency is required by PGX.ML.

◉ The Graph Server RPM installation now generates a self-signed certificate into /etc/oracle/graph/server_keystore.jks

Source: oracle.com

Continue reading

JSON Relational Duality: The Revolutionary Convergence of Document, Object, and Relational Models

JSON Relational Duality is a landmark capability in Oracle Database 23c that provides game-changing flexibility and simplicity for Oracle Database developers. This breakthrough innovation overcomes the historical challenges that developers have faced when building applications, either when using the relational model or when using the document model.

JSON Relational Duality delivers a solution that provides the benefits of both relational tables and JSON documents, without the tradeoffs of either model.

Limitations using relational and document models for app dev

The relational approach is very powerful but not always the easiest for app dev 

The relational model is very powerful and efficient since it uses data normalization to ensure data integrity while avoiding data duplication. Relational operations make modeling and accessing data very flexible, however, in some cases, it is not always the easiest for developers:

◉ Developers typically build apps in terms of app-tier language objects, while relational databases store data as tables, rows, and columns. Constructing individual application-tier objects often requires accessing multiple tables. 

◉ To get around these difficulties, developers often use Object Relational Mapping (ORM) frameworks. While ORMs can simplify app-dev, they also introduce significant overheads: They usually require multiple database round-trips to manipulate a single app-tier object, they are inefficient because they do not take full advantage of the capabilities of the database engine, they do not manage concurrency control very well, and applications need to use different ORM frameworks for different languages. They are also extremely poor at batch or bulk operations that must insert or modify many app-tier objects. 

◉ Application-tier ORM frameworks also introduce the possibility of divergent semantics across modules and microservices unless all of them share exactly the same mapping information.

The relational model is therefore a very efficient data storage format but sometimes poses challenges for developers when used as a data access format and ORMs introduce inefficiencies and other trade-offs.

 

JSON document databases have their own shortcomings

Document databases are popular with developers because they make it easy to retrieve and store hierarchically organized data corresponding to app-tier language objects. The JSON document model allows apps to directly map objects into a hierarchical JSON format, avoiding the need for decomposition or reconstitution, and the associated complexities. However, the JSON document model is far from ideal as a storage format because:

◉ Documents often need to store overlapping data. For example, different Order documents may store the same Customer information redundantly. Data duplication leads to inefficiency and potential inconsistency, since an update to shared information (such as a customer phone number) may require updating many Order documents atomically. 

◉ To get around this problem, some document databases recommend normalizing documents using references: Instead of including the Customer document within the Order document, an Order document may simply include an ID for that Customer document. However, normalizing documents completely defeats the simplicity of the document model, and results in a model that is actually the worst of both worlds!

◉ It is also very difficult to model many-to-many relationships using the document model. Attempts to model the relationships lead to even greater data duplication and the potential for additional inconsistencies. 

Documents/JSON are therefore a developer-friendly data access format and make it easy for developers to get started, but have significant limitations as a data storage format, especially as the complexity of an app increases.

How Oracle Database 23c JSON Relational Duality revolutionizes app dev

Oracle Database 23c JSON Relational Duality converges the benefits of the Relational and Document worlds within a single database without any of the tradeoffs discussed earlier. The new capability in Oracle Database 23c that enables this convergence is referred to as a JSON Relational Duality View. 

Figure 1: JSON Relational Duality: Best of both worlds

 

Using Duality Views, data is still stored in relational tables in a highly efficient normalized format but is accessed by apps in the form of JSON documents (figure 2). Developers can thus think in terms of JSON documents for data access while using the highly efficient relational model for data storage, without having to compromise simplicity or efficiency. In addition to this, Duality Views hide all the complexities of database level concurrency control from the user, providing document level serializability.

Figure 2: Stored as rows - Accessed as JSON documents

Duality Views can be declared over any number of tables using intuitive GraphQL syntax. For example, the following Duality View renders the relational data available in the order, orderitem and customer tables as a JSON document corresponding to an app-tier Order object:

Figure 3: Declaring a Duality View

Developers can easily define different Duality Views on the same or overlapping set of relational tables, making it easy to support multiple use cases on the same data (such as OrderObj and ShipmentObj Duality views that share common tables such as orderitem and customer). 

Using Duality Views, developers now have much greater flexibility: 

◉ SQL access to all data, including data in JSON columns, using SQL JSON extensions 

◉ JSON document access to all data, including access to data stored in relational tables, using Duality Views

Developers can manipulate JSON documents produced by Duality Views in ways they are used to, using their usual drivers, frameworks, tools, and development methods.

Extreme simplicity and flexibility for developers

Developers greatly benefit from the simplicity of being able to retrieve and store all the data needed for a single app-tier object in a single database operation. Applications using Duality Views can now simply read a document from the view, make any changes they need, and write the document back without having to worry about the underlying relational structure:

◉ Duality Views eliminate the need for ORM frameworks

◉ Reads and writes of Duality Views can use familiar HTTP operations such as GET, PUT, and POST. 

◉ Applications that prefer an API over HTTP can use the Simple Oracle Document Access API (SODA), Oracle Database API for MongoDB, or ORDS. 

◉ Application operations using Duality Views are optimally executed inside the database since they enable fetching and storing all rows needed for an app-tier object use case in a single database access, in contrast with the often inefficient database access code generated by ORMs.

JSON Relational Duality therefore provides the storage, consistency and efficiency benefits of the relational model while also providing the simplicity and flexibility of the JSON document model.

Lock-Free Concurrency Control with Oracle Database 23c 

Duality Views also benefit from a novel lock-free or optimistic concurrency control architecture in Oracle Database 23c that enables developers to manage their data consistently across stateless operations.

◉ Traditional locking does not work with stateless operations such as REST GET and PUT since locks are stateful and cannot be held across stateless calls.

◉ A new lock-free concurrency control algorithm in Oracle Database 23c allows for consistent updates across stateless operations.

◉ The lock-free scheme extends the Entity Tag (ETAG) concept from the HTTP protocol into the core database, an ETAG being a hash or a signature for the contents of a retrieved web page.

◉ When a GET is performed on a Duality View, the returned JSON document also contains the ETAG of the set of rows used in constructing the document.

◉ When that document is modified and later PUT back into the database, the supplied ETAG is compared with the current ETAG of the rows. If the ETAG differs, the object must have been modified between the GET and the PUT and the PUT is rejected

◉ The application can then re-GET the document with the new ETAG and retry the PUT

◉ If the PUT is successful, we are guaranteed that no intervening changes have occurred to the object and ensuring document-level atomicity and consistency. 

◉ Document-level serializability using lock-free concurrency control allows developers to focus on their app instead of implementing debugging concurrency control and data consistency issues within the application-tier.

Source: oracle.com

Continue reading

Ensuring Data Consistency in Microservice Based Applications

In my previous post I described Oracle’s newly announced product, Oracle Transaction Manager for Microservices (MicroTx). In this post I’ll cover why distributed transactions are needed in a microservice based application and the various distributed transaction protocols supported by MicroTx.

As application architecture moves from monoliths, through SOA, to today’s microservices, issues crop up that monoliths and some SOA based applications don’t need to worry about. One area in particular is around data consistency. In a monolithic application, typically all data is stored in a single database. Consistency across tables is managed by local transactions to ensure the data remains consistent, i.e., an update to two tables either both succeed or neither of them succeed. Moving to microservices where each microservice maintains its own database, local transactions are no longer sufficient to provide consistency.  This is where distributed transactions become a requirement.

Distributed Transactions

To help ensure data consistency across microservices, a distributed transaction is often used. Distributed transactions try to move a system from one consistent state to another consistent state.  hey are often utilized to handle the various failure scenarios that can occur in distributed systems, ideally without burdening the application developer with too much work. One of the earliest distributed transaction protocols is the XA two phase commit protocol defined by The Open Group. Using XA, applications can ensure that the updates to multiple data sources can be done while still adhering to the ACID guarantees of a transaction.

ACID

Transaction protocols typically try to provide ACID guarantees, where ACID is an acronym for:

◉ Atomicity – All changes occur or none of the changes occur – prevents partial updates.

◉ Consistency – The system as a whole move from one consistent state to another consistent state.

◉ Isolation – Changes by one transaction aren’t seen by any other transaction until the transaction is complete - sometimes referred to as serializability meaning the results are the same whether transactions execute in parallel or are serialized.

◉ Durability – Once the outcome of the transaction has been determined, the outcome is durably recorded and will take place even in the presence of temporary failures.

Not all transaction models ensure all these ACID properties. The popular Saga model of eventual consistency for example typically gives up Isolation, which can lead to inconsistent outcomes that are difficult to compensate as a transaction may make a decision based upon potentially dirty or stale data.

XA

The XA standard for Distributed Transaction Processing defines the model and the protocol that occur between the Application Program (microservice), the Transaction Manager, and the Resource Managers (databases). The basic flow is:

1. Application Program asks Transaction Manager to start a transaction

2. Application Program updates one or more Resource Managers

3. Application Program asks Transaction Manager to commit the transaction

4. Transaction Manager asks each Resource Manager to prepare, meaning be ready to commit when asked

5. If all Resource Managers successfully prepare, then the Transaction Manager tells all Resource Managers to commit

6. If a failure occurs before the decision to commit by the Transaction Manager is made, all Resource Managers will be asked to rollback

While the above is greatly simplified, it shows the general flow. From the XA Specification:

If we replace the monolithic application program with a set of microservices, we will see something like the following picture. A client or other microservice calls A, which starts the XA transaction by calling the Transaction Manager. A’s business logic updates its resource manager and calls B which also calls the transaction manager to enlist in the transaction. B’s business logic does some updates to its resource manager and calls C. Likewise, C calls the transaction manager to enlist in the transaction and its business logic updates its resource manager. Finally, A calls the transaction manager to commit or rollback the transaction. The transaction manager then prepares and commits A, B, and C’s resource mangers or rolls them back.

Which is a little complicated and we’ll explain how this can be simplified using Oracle Transaction Manager for Microservices (MicroTx) in my next post.

Sagas

Sagas are a distributed transaction model that relies on the idea of eventual consistency. This means that during the execution of the transaction, some microservices are in an inconsistent state with respect to the other microservices involved in the transaction. Sagas provide the advantage that each microservice uses local transactions to maintain consistency within the microservice. This reduces the time locks are held to just the duration of the local transaction instead of for the entire duration of the distributed transaction as is done in XA.

In this picture we can see Sagas look a lot like XA transactions, but with some significant differences. First in Sagas, there isn’t any notion of a resource manager. Another difference is that all the participants use local transactions during the Saga execution instead of a distributed transaction. However, the most significant difference is that the microservice must provide application logic to complete or compensate its part of the Saga. Compensating the microservice’s part of the Saga can become quite complicated as the microservice’s state may have changed due to other Sagas by the time it comes to compensate. How and what completing or compensating means and performed is completely up to the application. This in contrast to XA transactions where the infrastructure takes care of committing or rolling back the state of the involved resource managers.

The basic flow for Saga is:

1. Initiator calls the transaction coordinator to begin a Saga

2. Initiator calls one or more participant microservices

3. The participant microservices enlist in the saga by calling the transaction coordinator to provide their complete and compensate URIs.

4. The initiator then calls the transaction coordinator to complete or compensate the saga

5. The transaction coordinator calls each participant’s complete or compensate URI

Try-Confirm/Cancel

The Try-Confirm/Cancel transaction model relies on a specific application pattern where microservices provide reservations of resources that will later either be confirmed or canceled. Confirming a reservation means that the resource is now owned by the caller, whereas canceling the reservation means the resource is put back into inventory. It is a two-phase transaction model that doesn’t require locking and the risk of deadlocks or performance issues. However, it is only suitable for application that use a reservation of resources model.

The basic transaction flow for Try-Confirm/Cancel is as follows:

1. Initiator calls the transaction coordinator to begin the Try-Confirm/Cancel transaction

2. Initiator calls one or more participants to make reservations

3. Participants make the reservation and return a URI representing the reservation

4. Initiator calls the transaction coordinator to either confirm or cancel the transaction

5. The transaction coordinator then confirms or cancels all of the reservations using PUT or DELETE

Later in this series of blog posts, I’ll cover how MicroTx can simplify the use of distributed transactions for each of these models.

Select the Right Protocol for the Required Consistency

By providing multiple distributed transaction protocols, MicroTx allows application developers to choose the level of consistency needed for their microservices. For very strong consistency, developers can choose the XA protocol and get all the ACID properties of a transaction that spans their microservices. One of the benefits of using XA is that it doesn’t require any application logic to ensure consistency. All that’s required is to bracket the beginning and end of a transaction.

Some applications may be better suited for Sagas where the overall transaction time may be quite long, say over the interaction with a user. In these sorts of applications, Sagas in the form of Eclipse MicroProfile Long Running Actions (LRAs) provide a solution. LRAs provide a form of eventual consistency where systems may be inconsistent with one another but will eventually be consistent. One issue with LRAs is that unlike XA, there is no isolation, so other requests may see this temporary inconsistent state across the microservices involved. Also, LRAs require application specific logic to complete as well as compensate a microservice’s involvement in the transaction.

The third supported transaction protocol is the Try-Confirm/Cancel protocol. It relies completely on HTTP verbs to manage a distributed transaction. During the first part of the protocol, the initiator makes POST requests to reserve resources. Once all the resources are reserved, it asks the transaction coordinator to confirm all the reservations. The transaction coordinator then calls PUT on all the resources to confirm the reservation. If all the reservations can’t be made or for some other reason, the initiator can ask the transaction coordinator to cancel all the reservations. The transaction coordinator then calls DELETE on all the reserved resources.

Source: oracle.com

Continue reading

Announcing Oracle Tuxedo 22c

Oracle is pleased to announce a new major release of Oracle Tuxedo, version 22c (22.1.0.0.0).  This release contains a number of enhancements and new features.  This blog post briefly describes the various changes in this release.

Kubernetes and Cloud Based Deployments

Tuxedo 22c simplifies deployment in Kubernetes and cloud environments and enables non-Tuxedo specialists to more easily deploy and manage Tuxedo. Along with this release, Oracle is providing Dockerfiles and Helm charts in Github to help run Tuxedo applications in container-based environments such as Docker and Kubernetes. Using these files as a starting point, developers can quickly create an image that has Tuxedo and their application already installed and ready to run.  Once the image has been created, containers in Docker can easily be started with the docker run command.  To deploy the image in Kubernetes, Oracle is providing helm charts to take care of installation and running the image.  These charts have been tested on minikube, Oracle Kubernetes Engine (OKE) in Oracle Cloud, and should work with other Kubernetes distributions including Red Hat OpenShift. Sample applications along with their Dockerfiles and Helm charts are also in Github.

We plan to continue providing additional tools and integrations for deploying and running Tuxedo applications in cloud-native environment, including integration with native Kubernetes observability and management tools to make it easier to run Tuxedo in Kubernetes and Cloud environments.

Figure 1. Pre-built Tuxedo containers in Oracle Container Registry enable faster and simpler Kubernetes and Cloud deployments

Integrating Tuxedo Applications into a Microservices Based Application

This new release of Tuxedo fosters re-use of existing Tuxedo services (native or re-hosted from mainframe) in Microservices applications.  It includes an enhanced Service Architecture Leveraging Tuxedo (SALT) release that enables Tuxedo services written in C/C++, COBOL, or Java to participate in an XA distributed transaction managed by Oracle Transaction Manager for Microservices (MicroTx).  Using SALT and MicroTx, new microservices developed in Java or TypeScript can use existing Tuxedo services exposed as REST end-points in SALT as part of a distributed XA transaction.  This allows Tuxedo services to be included in an XA transaction coordinated by MicroTx that spans multiple databases, ORDS/APEX applications, Java and TypeScript based microservices, and Oracle Blockchain Platform smart contracts. The result is a broader transaction orchestration across polyglot application components, enabling strong data consistency, reducing development, and simplifying testing and troubleshooting.

Figure 2. Transaction orchestration of Tuxedo services with polyglot Microservices enables re-use and speeds up adoption of more flexible and modular application architectures

Oracle Database Application Continuity Support

Application Continuity (AC) is a feature available with Oracle Real Application Clusters (RAC), Oracle RAC One Node, and Oracle Active Data Guard options that masks temporary database outages from applications. AC masks these outages by recording the database session locally and replaying the recording should the connection to the database be lost. This is all done in the database driver and transparent to the application so that the outage appears to the application as a slightly delayed execution.

In order to support Application Continuity in all recoverable failure scenarios, it is necessary for the client application to demarcate session boundaries, i.e., when a database session starts and when it ends.  Tuxedo 22c provides an option to automatically call the Oracle Client Interface (OCI) start session API prior to starting to process a service request and then call the OCI end session API when the service completes.  This allows non-XA Tuxedo services to transparently leverage Application Continuity  and hide most database disruptions from the Tuxedo service without any code changes.  Note this only works for servers not currently involved in an XA transaction.  XA transactions are not supported by Application Continuity.

Figure 3. Close integration with Oracle Database Application Continuity shields Tuxedo applications from errors caused by recoverable database disruptions

Secure Deployments

Oracle security policies as defined by Oracle Software Security Assurance program require that Oracle products when deployed are secure by default.  Secure in this context means they only use encrypted communication, they use approved authentication, authorization, and cryptographic algorithms.  Deploying a product in an insecure fashion requires explicit configuration by the customer.  As a result, Tuxedo 22c by default requires that authentication and authorization be enabled, and that all network communication uses TLS with approved ciphers.  This may be difficult to implement or not needed for some Tuxedo customers, so Tuxedo 22c provides options to allow disabling certain security features such as TLS communication, and disabling authentication and authorization.  If customers choose to deploy in an insecure fashion by setting these configuration options, they will be warned in the Tuxedo ULOG that they’re deploying in an insecure fashion.  Other security enhancements include updates to open-source packages used by Tuxedo and security bug fixes.

Source: oracle.com

Continue reading