Dashboards: Dead or Alive? The evolution from data graveyards into data gold mines

Recently I’ve seen the subject of analytics dashboards reaching the end of their usefulness raised on more than one occasion, whether in a live in-person event or in a social post. But is this really the case? Are dashboards indeed an outdated form of presenting data to stakeholders and making decisions?

Are dashboards really dead?

What are traditional BI dashboards?

Classic dashboards have been around for decades, intended to empower business users with data-driven insights without the need for technical skills. When I began my analytics career nearly 20 years ago, "dashboards" were already prevalent. They served to simplify data access, letting executives explore key metrics without knowledge of the underlying data architecture.

Building those early dashboards was no simple task; it required deep expertise across data visualization tools, storage systems, and accessing and transforming data into business concepts. While the creation was complex, the consumption was simplified. Users could understand and action insights.

Traditional dashboards removed the complexity of data for businesspeople. But creating them still demanded technical specialization. These static, legacy dashboards remain common today despite lacking modern flexibility and intelligence. The next generation of analytics must balance technical depth with intuitive business exploration. Insights for all, complexity for none.

Figure 1: A traditional dashboard example in Oracle Business Intelligence Enterprise Edition

What's the problem with traditional BI dashboards?

Legacy dashboards often rely on dated technologies and methods, lacking capabilities such as natural language, ML predictions, and built-in augmented data preparation. Developers are resistant to adopt (or adapt to) emerging innovations. The old "if it ain't broke, don't fix it" mentality has many organizations still using old approaches that lag behind current best practices.

For example, traditional dashboards struggled with real-time analytics, taking minutes or hours to refresh. Interactivity was limited too. Some allowed basic filtering or drill-down within preset data sources, but venturing outside the developer's predefined lanes was impossible.

When people say things such as “dashboards are where good data goes to die,” they're typically referring to the traditional type of dashboards. The data presented on these dashboards often becomes stagnant and isn't actively used for decision-making, gradually diminishing its value over time. These dashboards are generally constructed and maintained centrally by IT. Making changes can be challenging, and the process of creating dashboards is often rigorous and time-consuming. As business needs evolved, many dashboards were created but rarely removed. Consequently, organizations have been amassing huge repositories of thousands of inactive dashboards. Business users perceive little value in them, often turning to alternate visualization tools or even resorting back to spreadsheet-based solutions instead. Sound familiar?

This type of traditional dashboard is what Gartner refers to as “mode 1” business intelligence (BI). BI that was created centrally by a dedicated team, posted to a repository for business users and left there for the rest of eternity.

How are modern dashboards different?

A modern analytics dashboard (which leverages cloud technology) offers enhanced accessibility, real-time interactivity, user-accessible data integration, and collaborative capabilities that distinguish it from the older style of IT-led BI dashboards prevalent in legacy on-premises tools. Modern dashboards excel on two fronts compared to traditional, IT-led approaches. First, embedded AI/ML augments human analysis with machine intelligence. Second, self-service analytics empowers business users with autonomy.

AI and ML boost productivity by automating tedious data preparation and enrichment tasks. They also detect hidden insights such as anomalies, unseen patterns, and future predictions. This enhances data stories with machine learning, minus the manual effort.

However, technology alone isn't the answer. True enablement comes by putting analytics directly into the hands of business users by fostering a culture of data literacy and by providing intuitive tools for self-service exploration and decision-making. Self-service removes reliance on IT or specialists to build reports. Now cross-functional users can freely access, analyze, and share data on their own. With IT empowering through governance, not limiting through control, the possibilities are endless. People ask their own questions, find their own answers, and tell impactful data stories that positively affect the business without bottlenecks.

The future of dashboards is democratization, not restriction. Blending embedded AI/ML with self-service exploration makes modern dashboards a portal into limitless insights.

How has Oracle Analytics evolved?

Previous versions of Oracle Analytics (such as OBIEE) were designed for centralized, IT-controlled business intelligence. The semantic model simplified access to complex data architectures, transforming them into business metrics.  But there were drawbacks, such as IT becoming a bottleneck trying to keep up with business requests. Now with the option to upgrade to Oracle Analytics Server (OAS) on-premises or hosted in cloud, OBIEE customers gain the opportunity to give businesspeople the modern analytics tools and interfaces that their users demand while still retaining their investment in the corporate semantic model.

Today's Oracle Analytics platform (both Oracle Analytics Cloud and Oracle Analytics Server) has expanded into self-service for business users. Now cross-functional teams can easily obtain insights without relying on specialists. AI and ML augment human analysis with recommendations and productivity enhancers such as generative AI chat interfaces.

Figure 2: Oracle Analytics platform YouTube product tour

IT retains governance for compliance and reliability. But the focus has shifted from centralized reporting to decentralized self-sufficiency. User autonomy, not technical complexity, now sits at the core. By combining governance with flexibility, Oracle enables actionable analytics. People ask their own questions and find answers quickly. Backed by machine learning, human intelligence scales further. Oracle Analytics allows organizations to become truly insights-driven by making analytics accessible, intelligent, and empower users to be self-sufficient.

Figure 3: Oracle Analtyics Cloud with live interactive data visualizations embedded into a slide. Live visualizations allow people to focus on the results and making the right decision.

Call to action

Sure, there are most certainly many examples of BI or analytics projects that still have traditional style centrally created static dashboards where data indeed goes to die. But hopefully it’s clearer that all dashboards aren’t created equal, and the same fate doesn't apply to modern dashboards that are designed for self-service and that use AI/ML throughout.

For clarity, data doesn’t die and dashboards aren't dead. But traditional mode 1 style BI can stagnate data, making it less useful and untrustworthy. Business users and executive stakeholders that aren’t savvy enough to connect to and retrieve the data that they need will always rely on some form of dashboard. In the absence of a good solution, they'll resort to spreadsheets – and no one wants that.

My recommendation is to stop building old-school centralized dashboards (also known as mode 1) that are used briefly and then forgotten. Instead empower users with self-service and modern technologies (such as AI/ML) to answer their own questions, tell their own stories, and make better more analytics-driven decisions (also known as mode 2).

Source: oracle.com

Continue reading

New AI capabilities with Oracle Analytics

Oracle Analytics delivers ongoing innovation through the year, and offers a wide range of AI capabilities. It provides users capabilities augmented with AI, Machine Learning, and Data Science, as well as leveraging Oracle Cloud Infrastructure (OCI) integrated AI Services.

Oracle Analytics offers 3 types of AI capabilities:

1. Built-in, AI-augmented capabilities.

2. Integrated ML capabilities.

3. Integrated AI capabilities.

AI capabilities of Oracle Analytics - Author: Philippe Lions, VP Product Management, Oracle.

 

Here's a list to help you discover some of the key AI features for Oracle Analytics:

Built-in AI Augmented capabilities

AI Auto-Insights: This feature generates data visualizations automatically from a specific dataset. It will create an optimal visualization for the data elements selected.

AI Auto Insights, Explain and Segmentation

Explain: This feature uses Machine Learning to automatically explain a metric or attribute - like “Attrition” in the example below - with a single click. It calculates the correlations, drivers, segmentations and anomalies for the dataset.

1-Click ML/AI feature: this feature enables you to create statistics and ML-generated insights, like clustering with ML or forecasting, using algorithms like ARIMA or ETS with a single click.

ML Integrated Capabilities

Drag and Drop ML and Data Science: this feature allows you to train models using numeric prediction, multi-classifier, binary classifier, or clustering with different built-in algorithms. These ML algorithms can be customized, trained, tuned, and then published to the wider analytics user community.

ML Data Flow with Prediction by Low Code No Code

 

Integrated AI capabilities

Oracle Cloud offers multiple AI services: https://www.oracle.com/artificial-intelligence/ai-services/

Currently, Oracle Analytics is integrated with 3 of them and more are planned in our Product Roadmap. Integration means that you can create an AI service in OCI, register the AI model, and ingest the results into Oracle Analytics as a dataset to create visual data stories.

Below are some key OCI AI services currently integrated with Oracle Analytics:

1. AI Vision

2. AI Language

3. AI Document Understanding

Below you can review these AI services and discover data visualizations built using them.

AI Vision Integration

AI Vision provides Object Detection and Image Classification. There are live demos for each in the public OAC instance.

A blog and demo showing object detection is available here:

Data Visualization showcasing the use of an OCI AI Vision - Author: Benjamin Arnulf

A blog and demo showing image classification is available here:

Data Visualization showcasing the use of an OCI AI Image Classification trained model with 3,000 public MRI - Author: Benjamin Arnulf

AI language Integration

Oracle Analytics has the capability to analyze language using AI:    

Oracle Analytics data disualization showing the use of an OCI AI Language with a pre-trained model - Author: Philippe Lions

AI Document Understanding

Oracle Analytics is integrated with OCI AI Document Understanding and can recognize and extract values from a passport, receipts, resumes (aka CVs), invoices and more. See a blog and demo below:

Oracle Analytics Data Visualization showing the use of OCI AI Document Understanding with a pre-trained model for key value extraction. Authors: Benjamin Arnulf and Philippe Lions

AI Digital Assistant

This will be integrated into Oracle Analytics, as announced at the keynote of T.K. Anand at OCW23. In summary, you will be able to ask questions like “What is the root cause for attrition and the trend for the past 3 quarters?”, and the integration will leverage Cohere or other Large Language Models (LLM) to create an answer and generate relevant data visualizations. 

Oracle Analytics and the OCI AI Digital Assistant using large language models to generate answers and data visualizations.

Source: oracle.com

Continue reading

Introducing Manufacturing Analytics in Oracle Fusion SCM Analytics

The world of modern manufacturing is highly dynamic and presents numerous challenges, including fluctuating supply and demand, supply chain disruptions, unexpected production inefficiencies, and the ongoing need to identify optimal suppliers to manage order backlogs. The agility to make swift decisions and pivot strategies is crucial. But the lack of integrated insights integrated across operational data and key business systems hinders agility, making it hard to streamline the manufacturing ecosystem and obscuring both potential opportunities and threats.

Oracle Fusion SCM Analytics introduces the Manufacturing Analytics capability that provides manufacturers with timely insights into work order performance, shop floor efficiency, and plan-to-production processes. Connecting supply chain and manufacturing data all in one place, this new feature helps drive better access, actionable insights, and increased productivity.

Timely insights into work order performance

Analyzing work order data is very complex, and this makes it difficult to unearth trends or meaningful insights about performance, such as analyzing if cycle times are excessive or if manufacturers are meeting their order targets. By leveraging the comprehensive array of ready-to-use key performance indicators (KPIs) in Oracle Fusion SCM Analytics, manufacturers get a concise overview of work order execution, and they can easily discern patterns in manufacturing cycle times, as well as pinpoint the root causes behind deviations that influence overall output quality. Intuitive dashboards help answer questions like:

1. How much time does it take to produce my products?
2. How does actual time to produce compare against planned time?
3. What is the production yield for my plant?

This allows manufacturers to easily detect bottlenecks across manufacturing stages, helping preserve work order schedules and fostering a culture of analytics-driven continuous improvement.

Manufacturing Analytics Overview

Detect anomalies in product quality to take fast corrective action

Gathering accurate and timely data from various sources on the shop floor can be difficult, and inaccurate or incomplete data can contribute to misleading insights, hindering the effectiveness of analytics-driven improvements.

With prebuilt Manufacturing Analytics, plant managers can have a single shop floor overview to quickly identify patterns in scrapped, rejected, canceled, and reworked items. They can also easily deduce the reasons for waste, then quickly implement corrective measures. With easy-to-use self-service visualizations, shop floor managers can swiftly identify anomalies that impact product quality, answering questions like:

1. How is my quality of production?

2. What percent of items is being reworked?

3. How much do I scrap during production?

4. Is my plant reaching its targeted production output?

5. How many work orders are completed on time?

Product Quality Overview

 

See production impact across the entire plan-to-produce process

A significant challenge is the lack of consolidated insights across vital departments, from procurement and inventory to order fulfilment and sales. Such fragmentation makes it difficult to identify threats and opportunities that have a direct bearing on production.

Oracle Fusion SCM Analytics provides prebuilt unified data and analytics across manufacturing, procurement, inventory, financials, fulfillment, and sales, with the ability to easily include other data sources to get interconnected insights. With holistic supply chain analytics, production leaders can:

1. Predict how supplier performance, such as delayed deliveries and product defects, will affect production goals.

2. Get a clear understanding of how material and inventory costs influence margins.

3. Continuously monitor inventory patterns and stockouts to mitigate risks associated with part backlogs.

4. Proactively forecast potential disruptions stemming from supplier performance, helping to ensure production goals remain intact.

Plan-to-produce cross-departmental process 

 

Create a system of insights from Oracle Cloud Application data and external data sources

Monitoring work order statuses is challenging because the necessary data comes from various sources like equipment sensors, maintenance records, and different software systems. This diversity in data sources also complicates the organization and analysis of work order information, making it challenging for production managers to identify patterns and gain valuable insights about things like cycle time overruns or performance against ideal order goals. 

With Oracle Fusion SCM Analytics, plant managers can easily blend data from various sources to get a unified view of both IT and OT data. With a single view, they can understand the cause and effect of machine downtime on product quality or other critical production metrics, and they can see the root cause of variances in production processes to mitigate future risks. 

Source: oracle.com

Continue reading

Oracle Fusion HCM Analytics analyzes data to reveal Inferred Skills

You're a visionary people leader or HR professional in a trending industry, entrusted with securing top talent to sustain your competitive edge. Do you have the workforce insights you need to make appropriate investments across your team, including swift reskilling/upskilling, aligning with business goals, and responding to the dynamic industry landscape? Can your team run day-to-day operations, recruit talent, set up project teams, and successfully guide career paths? Can they answer the following critical questions with confidence?

• What's the difference between a software engineer and a front-end engineer?
• Is it a good idea to hire a sales engineer as a developer?
• Can a business analyst be promoted to an associate product manager?
• What's the path to success for an up-and-coming marketing manager?
• Does a team have the required skills for the project?

If the answer is “maybe,” you may not be making optimal decisions for your organization's growth. 

Innovation in technology means it's easier than ever to make data-driven decisions across various business functions. However, people leaders and HR managers continue to face challenges in recruiting, deploying, and developing their workforce. These challenges persist for several reasons:

• Skills data remains outdated: Many intranets and HCM systems offer a way for employees to highlight their skills. However, very few employees are proactive and understand the value of sharing their talents and skills or seeking opportunities to utilize them.
• Job titles are getting blurred: With changing industry trends, jobs are evolving fast, and employees with the same titles in the same industry and region might have expanded their skills beyond their initial responsibilities. Titles are changing too - a title may have different meanings in different industries and geographies, and creative titles are often used for the same job. This makes it tough to rely on titles as a clear indication of responsibilities.
• A surplus of applications but a shortage of jobs: The pool of resumes for a job requisition is often much higher than the capacity of the recruiting teams to review them. Additionally, the quality and fit of each resume varies, and finding the right candidate is challenging. As a result, recruiters now need to dedicate additional time to review applications to ensure hiring equity.

Recent technological advancements, especially in unstructured data analysis, machine learning, and artificial intelligence, can help overcome these enterprise-wide challenges. 

Introducing Inferred Skills, a new addition to Oracle Fusion HCM Analytics driven by machine learning

Talent is a prime organizational investment, and skills form the core of talent analysis. Fusion HCM Analytics’ Inferred Skills capability delivers dynamic skill management to employees and managers. It utilizes machine learning (ML) to identify, enrich, and augment existing skills from diverse sources, automatically uncovering and highlighting new possibilities not just for individual employees but for teams and organizations as a whole. These are skills that are not found on employees’ talent profiles but they should be a part of the profiles according to the ML model. Employees get new perspectives on career paths, while managers can effectively make informed decisions for recruiting, internal mobility, retention, and addressing skill gaps.

Inferred Skills can offer significant new value

Skills Dashboard

Maximize employee potential with better skills utilization

Inferred Skills enables employees to discover new skills to help grow their careers. Managers are better equipped to assess individual and team readiness for a project, connect skilled individuals with the right opportunities, and quickly leverage internal talent to fulfill urgent needs.

Drive strategic workforce planning with better skills-to-title mapping

Skills can span varying business requirements. Inferred Skills refines skills-to-titles alignment, thereby narrowing the pool of applicants for specific job needs. For instance, granular roles such as Recruiting might share skills with broader functions such as Human Resources. Inferred Skills furnishes a detailed recruiting skillset, serving as a reference to distinguish HR generalists.

Adapt to industry trends with agile upskilling/reskilling

With Inferred Skills, leaders can track and analyze employee skills and address skill gaps at any time. They can gain insights into skill penetration at every level, from the entire organization to specific employees, and can make insight-driven decisions regarding the skills they need to develop or invest in. This, in turn, aids workforce planning, enabling organizations to better recruit, train, and retain the right talent. 

Hearst pilots Inferred Skills

Hearst is one of the country’s largest media and information services companies and operates over 360 businesses globally across the health, transportation, television, newspaper, and magazine industries. Hearst sees the value of Inferred Skills and is excited to pilot this new capability and its potential to enhance HR and business operations.

Hearst continuously evaluates new technology to improve analytics and operations.  Over the past year, the company has embarked on a mission to profile talent and gather skill-related data, including conducting numerous training sessions. These sessions emphasized the importance of complete talent profiles for improved employee development and internal mobility. While monitoring the data collection through initial reports, it became evident that although operational measures can encourage completion, a significant gap remained due to incomplete skills information.

“Being able to intelligently supplement data we have about employee skills could help us make more strategic decisions in employee development programs, ensure we have the right people for the work that needs to get done, and ultimately help with key HR outcomes like boosting employee engagement and retention.” Christin Friley, Director of HCM Analytics, Hearst

Inferred Skills emerged as a way to meet Hearst’s needs to supplement the data gathered from employees, with the following potential benefits:

• More comprehensive employee data
• The ability to conduct more robust analytics
• An enhanced ability to support employee development and retention
• Better insights for internal mobility

Hearst will continue piloting Inferred Skills data and hopes to use it to enhance reporting and analytics in pursuit of better supporting employees and business outcomes. 

Source: oracle.com

Continue reading

Generative AI Chatbot using LLaMA-2, Qdrant, RAG, LangChain & Streamlit

In the evolving landscape of conversational artificial intelligence (AI), the Retrieval-Augmented Generation (RAG) framework has emerged as a pivotal innovation, particularly in enhancing the capabilities of chatbots. RAG addresses a fundamental challenge in traditional chatbot technology: The limitation of relying solely on pretrained language models, which often leads to responses that lack current, specific, or contextually nuanced information. By seamlessly integrating a retrieval mechanism with advanced language generation techniques, RAG-based systems can dynamically pull in relevant and up-to-date content from external sources. This ability not only significantly enriches the quality and accuracy of chatbot responses but also ensures that they remain adaptable and informed by the latest information.

In an era where users expect highly intelligent and responsive digital interactions, the need for RAG-based systems in chatbots has become increasingly critical, marking a transformative step in realizing truly dynamic and knowledgeable conversational agents. Traditional chatbots, constrained by the scope of their training data, often struggle to provide up-to-date, specific, and contextually relevant responses. RAG overcomes this issue by integrating a retrieval mechanism with language generation, allowing chatbots to access and incorporate external, current information in real time. This approach not only improves the accuracy and relevance of responses but also enables chatbots to handle niche or specialized queries more effectively. Furthermore, RAG’s dynamic learning capability ensures that chatbot responses remain fresh and adapt to new trends and data.

By providing more detailed and reliable information, RAG significantly enhances user engagement and trust in chatbot interactions, marking a substantial advancement in the field of conversational AI. This technique is particularly useful in the context of chatbots for the following reasons:

• Enhanced knowledge and information retrieval: RAG enables a chatbot to pull in relevant information from a large body of documents or a database. This feature is particularly useful for chatbots that need to provide accurate, up-to-date, or detailed information that isn’t contained within the model’s pretrained knowledge base.
• Improved answer quality: By retrieving relevant documents or snippets of text as context, RAG can help a chatbot generate more accurate, detailed, and contextually appropriate responses. This capability is especially important for complex queries where the answer might not be straightforward or requires synthesis of information from multiple sources.
• Balancing generative and retrieval capabilities: Traditional chatbots are either generative (creating responses from scratch) or retrieval-based (finding the best match from a set of predefined responses). RAG allows for a hybrid approach, where the generative model can create more nuanced and varied responses based on the information retrieved, leading to more natural and informative conversations.
• Handling long-tail queries: In situations where a chatbot encounters rare or unusual queries (known as long-tail queries), RAG can be particularly useful. It can retrieve relevant information even for these less common questions, allowing the generative model to craft appropriate responses.
• Continuous learning and adaptation: Because RAG-based systems can pull in the latest information from external sources, they can remain up-to-date and adapt to new information or trends without requiring a full retraining of the model. This ability is crucial for maintaining the relevance and accuracy of a chatbot over time.
• Customization and specialization: For chatbots designed for specific domains, such as medical, legal, or technical support, RAG can be tailored to retrieve information from specialized databases or documents, making the chatbot more effective in its specific context.

The need for vector databases and embeddings

When we investigate the retrieval augmentation generation systems, we must grasp the nuanced, semantic relationships inherent in human language and complex data patterns. But traditional databases, which are intended to be structured around exact keyword matches, often fall short in this regard. However, vector databases use embeddings—dense, multidimensional representations of text, images, or other data types—to capture these subtleties. By converting data into vectors in a high-dimensional space, these databases enable more sophisticated, context-aware searches. This capability is crucial in retrieval-augmentation-generation tasks, where the goal is not just to find the most directly relevant information but to understand and generate responses or content that are semantically aligned with the query. Trained on large datasets, embeddings can encapsulate a vast array of relationships and concepts, allowing for more intuitive, accurate, and efficient retrieval and generation of information, thereby significantly enhancing user experience and the effectiveness of data-driven applications.

In this post, we use the Llama2 model and deploy an endpoint using Oracle Cloud Infrastructure (OCI) Data Science Model Deployment. We create a question and answering application using Streamlit, which takes a question and responds with an appropriate answer.

High-level solution overview

Deployment of the solution uses the following steps:

1. The user provides a question through the Streamlit web application.
2. The Streamlit application invokes the predict call API to the model deployment.
3. The model deployment invokes Langchain to convert user questions into embeddings.
4. The function invokes an Qdrant Service API to send the request to the vector database to find the top k similar documents.
5. The function creates a prompt with the user query and the similar documents as context and asks the large language model (LLM) to generate a response.
6. The response is provided from the function to the API gateway, which is sent to the Streamlit server.
7. The user can view the response on the Streamlit application.

Getting started

This post walks you through the following steps:

1. Setting up the Qdrant database instance
2. Building Qdrant with Langchain
3. Setting up RAG
4. Deploying a Streamlit server

To implement this solution, you need an OCI account with familiarity with LLMs, access to OCI OpenSearch, and OCI Data Science Model Deployment. We also need access to GPU instances, preferably A10.2. We used the following GPU instances to get started.

The workflow diagram moves through the following steps:

1. Pass the query to the embedding model to semantically represent it as an embedded query vector.
2. Pass the embedded query vector to our vector database.
3. Retrieve the top-k relevant contexts, measured by k-nearest neighbors (KNN) between the query embedding and all the embedded chunks in our knowledge base.
4. Pass the query text and retrieved context text to our LLM.
5. The LLM generates a response using the provided content.

Hardware requirements

For the deployment of our models, we use a distinct OCI setup that uses the NVIDIA A10 GPU. In our scenario, we deployed 7b parameter model using NVIDIA A10.2 instance. We suggest using the Llama 7b model with the VM.GPU.A10.2 shape (24-GB RAM per GPU, two A10).

Prerequisites

Set up the key prerequisites before you can proceed to run the distributed fine-tuning process on OCI Data Science:

• Configure a custom subnet with a security list to allow ingress into any port from the IPs originating within the CIDR block of the subnet to ensure that the hosts on the subnet can connect to each other during distributed training.
• Create an Object Storage bucket to save the documents which are provided at time of ingestion in the vector database.
• Set the policies to allow OCI Data Science resources to access OCI Object Storage buckets, networking, and others.
• Access the token from HuggingFace to download the Llama2 model. To fine-tune the model, you first must access the pretrained model. Obtain the pretrained model from Meta or HuggingFace. In this example, we use the HuggingFace access token to download the pretrained model from HuggingFace by setting the HUGGING_FACE_HUB_TOKEN environment variable.
• Log group and log from logging service to monitor the progress of the training.
• Go to OCI Logging and select Log Groups.
• Select an existing log groups or create one.
• In the log group, create one predict log and one access log.
• Select Create custom log.
• Specify a name (predict|access) and select the log group you want to use.
• Under "Create agent configuration," select Add configuration later.
• Select Create agent configuration.
• Notebook session: Used to initiate the distributed training and to access the fine-tuned model
• Install the latest version of Oracle Accelerated Data Science (ADS) with the command, pip install oracle-ads[opctl] -U

Deploying the Llama2 model

Refer to the blog, Deploy Llama 2 in OCI Data Science, where we depicted on how to deploy a Llama2 model on an A10.2 instance.

To estimate model memory needs, Hugging Face offers a Model Memory Calculator. FurtherFurthermore, for insights into the fundamental calculations of memory requirements for transformers, Eleuther has published an informative article on the subject. Use the custom egress functionality while setting up the model deployment to access the Qdrant database.

Setting up the Qdrant database

To set up the Qdrant database, you can use the following options:

- Create a Docker container instance

- Use a Python client

Initialize Qdrant with Langchain

Qdrant integrates smoothly with LangChain, and you can use Qdrant within LangChain with the VectorDBQA class. The first step is to compile all the documents that act as the foundational knowledge for our LLM. Imagine that we place these in a list called docs. Each item in this list is a string containing segments of paragraphs.

Qdrant initialization

The next task is to produce embeddings from these documents. To illustrate, we use a compact model from the sentence-transformers package:

from langchain.vectorstores import Qdrant

from langchain.embeddings import LlamaCppEmbeddings

import qdrant_client

  

#Load the embeddings model

embedding = LlamaCppEmbeddings(model_path=model_folder_directory,n_gpu_layers=1000)

# Get your Qdrant URL and API Key

url = 

api_key = 

# Setting up Qdrant

client = qdrant_client.QdrantClient(

    url,

    api_key=api_key

)

qdrant = Qdrant(

    client=client, collection_name="my_documents",

    embeddings=embeddings

)

Qdrant upload to vector database

# If adding for the first time, this method recreate the collection

qdrant = Qdrant.from_texts(

                texts, # texts is a list of documents to convert in embeddings and store to vector DB

                embedding,

                url=url,

                api_key=api_key,

                collection_name="my_documents"

            )

# Adding following texts to the vector DB by calling the same object

qdrant.add_texts(texts) # texts is a list of documents to convert in embeddings and store to vector DB

Qdrant retrieval from vector database

Qdrant provides retrieval options in similarity search methods, such as batch search, range search, geospatial search, and distance metrics. Here, we use similarity search based on the prompt question.

qdrant = Qdrant(

    client=client, collection_name="my_documents",

    embeddings=embeddings

)

# Similarity search

docs = qdrant.similarity_search(prompt)

Setting up RAG

We use the prompt template and QA chain provided by Langchain to make the chatbot, which helps pass the context and question directly to the Llama2-based model.

from langchain.chains.question_answering import load_qa_chain

from langchain.prompts.prompt import PromptTemplate

template = """You are an assistant to the user, you are given some context below, please answer the query of the user with as detail as possible

Context:\"""

{context}

\"""

Question:\"

{question}

\"""

Answer:"""

chain = load_qa_chain(llm, chain_type="stuff", prompt=qa_prompt)

## Retrieve docs from Qdrant Vector DB based upon user prompt

docs = qdrant.similarity_search(user_prompt)

answer = chain({"input_documents": docs, "question": question,"context": docs}, return_only_outputs=True)['output_text']

Hosting Streamlit application

To set up Compute instances and host the Streamlit application, follow the readme on Github.

Source: oracle.com

Continue reading

Mitratech avoids SLA breaches with OCI Application Performance Monitoring

Measuring how an application meets its service level agreement (SLA) is a critical IT Operations (ITOps) responsibility. I had the pleasure of co-presenting a session on this topic with Sunny Kathuria of Mitratech Corporation at Oracle CloudWorld 2023 (LRN2384). This blog summarizes key points from the session including the techniques Oracle itself uses to track its SLA using Oracle Cloud Infrastructure Application Performance Monitoring service (OCI APM) and how Mitratech ensures their services meet SLAs, using OCI APM tracing, profiling, synthetic monitoring, alarms, and dashboards.

SLAs, service availability, and Synthetic Monitoring OCI APM techniques 

An SLA defines the expected availability of a service – the percentage of the time the service is available in a set period (e.g., the service will be available 99.5% of the time in any month). The SLA also defines the availability metric – a yes/no test that can be checked periodically (e.g., the service is available if a user can access its UI and perform transactions).

Synthetic monitoring is utilized to execute tests that determine the availability metric value and to calculate and present the service availability based on the result of all the tests.

Figure 1:  Synthetic monitors enable service availability as well as location, network, and backend performance reporting

Monitoring SLA challenges

At Oracle, upwards of 10,000,000 Synthetic test runs are executed every day and these are some of the challenges our Ops teams encounter and how OCI APM is used to address them.

High test volume

A mid-size organization may have tens of applications and services while a large enterprise or global service provider may have hundreds and even thousands. Each one requires multiple availability tests, executed from multiple internal and external locations, repeated several times an hour. Operation teams need to set, maintain, and analyze results for a very large number of test executions. This leads to another challenge - handling false positives like tests that fail due to intermittent network problems. While percentage-wise, these are small numbers, when running many scripts it results in too many false positives to handle manually.

OCI APM automation (terraform, CLI, SDKs), helps with the creation and maintenance of the tests. That includes an option to set a test to maintenance mode, excluding it from the availability calculation. Executing the test from multiple vantage points provides built-in redundancy, increasing the reliability of the tests. Flexible availability calculation options (e.g., at least 2 out of at least 5 test executions need to fail before a test is considered down), can be used to address the false positive problem.

Figure 2:  Public Vantage Points available in OCI and other cloud providers' regions

Security

Applications and services have to be protected to prevent unauthorized access with one or more authorization schemes. Synthetic Monitors need to be able to execute the required authorization to be able to progress with the test flow, as well as to test the different authorization options, as they are part of the application that needs to be tested. A second concern is the need to protect passwords, tokens, usernames, etc., that are used in test scripts.

OCI APM service provides support for multi-factor authentication (MFA) login, and resource principal authentication, commonly used to control access to REST services. Integration with OCI Vault service is used to protect sensitive content (like usernames and passwords) used in the scripts.

Scheduling

Some applications allow a limited number of concurrent authentication sessions, in some cases more concurrent sessions means a higher bill from the authorization provider. Running tests from multiple vantage points at the same time will increase the number of used concurrent sessions.

OCI APM supports several scheduling schemes, allowing configuration that will limit the number of tests executed concurrently. A second benefit can be the use of a round-robin scheme to enable wider geographical coverage while keeping a lower total test count.

Triage and diagnostics

Network test and data explorer, in-script markers (custom metrics), HAR files for each browser tests execution, manual and automatic screenshot capturing, and built-in integration with real user monitoring (RUM) and application tracing are powerful capabilities that quickly identify network, infrastructure, and application issues as they are proactively exposed by the service. 

Figure 3: Network Data Explorer – diagnose network issues on the path from vantage points to the monitored application/service

Figure 4: Service topology and tracing used to identify slow or problematic steps in a transaction

Mitratech’s journey featuring OCI APM

Mitratech is a service provider and a proven global technology partner for corporate legal departments, risk & compliance teams, and HR professionals seeking to raise productivity, control expenses, and mitigate risk by deepening organizational alignment, increasing visibility, and spurring collaboration across the enterprise.

Mitratech was looking to consolidate multiple application performance monitoring solutions to reduce costs and streamline application monitoring for their DevOps users. Mitratech evaluated OCI APM as part of their move to run their hosted environment on OCI. Today, OCI APM is used to monitor availability, performance, and diagnostics for their Java-based microservices application.

Mitratech OCI Application Performance monitoring setup

• 8 APM domains configured to monitor applications
• 10+ applications configured with OCI APM’s Synthetic Monitoring
• 350+ Java applications configured with OCI APM
• Integrations with PagerDuty and Grafana

Mitratech’s application troubleshooting workflow

When troubleshooting issues, tracing multi-span interactions and being able to drill down into the server side (including databases, hosts, and network) can be quite a challenge. Something that many have had to acquire multiple tools to do. Miratech found all those requirements could be met using OCI APM’s Tracing capability.

• Trace Explorer is used to view traces and spans and identify performance issues and bottlenecks in the monitored application, from browser to database
• Diagnostics features like CPU consumption, allocated memory, and GC impact per thread, as well as a collection of thread stack snapshots including state and lock information

Integration with PagerDuty

OCI APM is auto-discovered by PagerDuty making integration across the two solutions easy. From the service directory page in PagerDuty, search for “OCI” and select “OCI-APM.”

Integration with Grafana

    

Mitratech uses the OCI APM Service dashboard for all internal performance and availability needs. As a service provider, they also need to provide visibility to some of this data to their customers, without providing them access to the OCI console. Using the OCI APM Service API, Mitratech enabled Grafana dashboards with the required data, utilizing an external authentication system.

Figure 5:  OCI APM data accessed in a Grafana dashboard

To wrap up, we reviewed key points covered in session LRN2384 at Oracle CloudWorld 2023  for monitoring SLAs with examples of how Oracle and Mitratech use modern observability solutions natively available from the Oracle Cloud like OCI APM. To learn more about OCI Observability and Management including OCI APM, visit the links below.

Source: oracle.com

Continue reading

Is the Oracle 1Z0-084 Certification Worth It? Exploring the Benefits

Oracle certifications have long been recognized as valuable assets in the realm of IT and database management. Among the array of certifications Oracle offers, the 1Z0-084 exam stands out, focusing on Oracle Database 19c - Performance Management and Tuning.

Before delving into the benefits, let's uncover the essence of the 1Z0-084 exam. This certification concentrates on honing skills related to Oracle Database 19c, emphasizing performance management and tuning - an essential aspect in today's dynamic technological landscape.

The Allure of Oracle 1Z0-084 Certification

Elevating Your Career Trajectory

In an era where businesses hinge on robust database systems, professionals with the 1Z0-084 certification command attention. It is a testament to your proficiency in improving database performance, positioning you as a valuable asset for employers searching for top-tier talent.

Whether you are eyeing a promotion within your current organization or seeking to make a mark in a new venture, the 1Z0-084 certification can be the catalyst that propels your career to new heights.

Unparalleled Expertise in Performance Management

Obtaining the 1Z0-084 certification goes beyond showcasing theoretical knowledge - it demonstrates your mastery of the intricate art of performance management and tuning within Oracle Database 19c. This level of expertise can be a game-changer, opening doors to challenging and rewarding projects that demand a nuanced understanding of database optimization.

The certification validates your hands-on skills, assuring employers that you comprehend the principles and can effectively apply them in real-world scenarios.

Industry Recognition and Credibility with Oracle 1Z0-084 Certification

Oracle certifications carry weight globally, and the 1Z0-084 is no exception. Holding this certification is akin to having a stamp of approval from one of the industry leaders. It adds a layer of credibility to your professional profile, making you a sought-after expert in database optimization.

Imagine the benefit of having a certification that is recognized and respected worldwide. It increases your credibility and reflects positively on the organizations you associate with, further solidifying your position as a reliable and knowledgeable professional.

Navigating the Oracle 1Z0-084 Certification Journey

Resources Galore

The path to 1Z0-084 success is paved with ample resources provided by Oracle. From comprehensive study materials to practice exams and detailed documentation, Oracle ensures that aspirants can confidently tackle the certification journey.

One of the key benefits here is the structured approach to learning. The resources provided are curated to cover all aspects of the exam syllabus, ensuring that you have a holistic understanding of the concepts and skills needed to ace the certification.

Community Support

Joining the Oracle community is not just about preparing for an exam; it's about becoming part of a network of professionals who share a common goal. The community delivers a platform to connect with fellow certification seekers and experienced professionals, creating a collaborative space where knowledge and experiences are freely shared.

Engaging with the community can seriously enhance your preparation. You gain insights, tips, and the assurance that you are not alone in your journey. The community's collective wisdom becomes valuable in your quest for 1Z0-084 excellence.

Making the Decision: Is Oracle 1Z0-084 Certification Truly Worth It?

The Return on Investment

Investing time and effort into obtaining the 1Z0-084 certification is not just a commitment; it's an investment in your professional future. Regarding career opportunities and enhanced expertise, the return far outweighs the initial commitment.

Consider this certification as a strategic move in your career chessboard. It's not just about immediate advancement; it's about setting yourself up for long-term success. Organizations value professionals who continually invest in their skill sets, and the 1Z0-084 certification is a testament to your commitment to excellence.

Adapting to Industry Trends

In the fast-paced and ever-evolving landscape of IT, staying ahead of the curve is not a luxury; it's a necessity. The 1Z0-084 certification ensures you keep pace with industry trends and lead in performance management and tuning.

Technology waits for no one, and today's relevant skills must be improved tomorrow. The 1Z0-084 certification equips you with the latest knowledge and practices, making you a proactive player in the tech arena. It's a commitment to staying relevant and being a driving force in the face of technological advancements.

Conclusion

In conclusion, the Oracle 1Z0-084 certification transcends being a mere credential, a gateway to opportunities and professional growth. As you venture on this certification journey, remember that the benefits extend beyond the immediate. It's not just about passing an exam; it's about honing skills that will shape your career trajectory.

Whether you are an aspiring database professional eager to make your mark or a seasoned expert looking to stay ahead, the Oracle 1Z0-084 certification is undoubtedly worthwhile. It goes beyond the checkboxes on a resume; it's a testament to your dedication to excellence in database management.

In a world of ever-shifting fierce competition and technological landscapes, having the Oracle 1Z0-084 certification on your resume is not just an asset - it's a strategic advantage. It signals to the industry that you are not just adapting to change; you are driving it.

So, is the Oracle 1Z0-084 Certification worth it? The resounding answer is yes. It's an investment, a validation, and a ticket to a future where your performance management and tuning expertise sets you apart in the vast sea of IT professionals.

Continue reading

OCI Observability and Management for Multi-Cloud Database: Amazon RDS

Multi-cloud has gained importance in recent years as companies distribute their workloads across multiple cloud providers per their unique preferences and convenience.

This blog explains how OCI Observability and Management services, such as Database Management, Stack Monitoring, Operations Insights, and Logging Analytics are used for monitoring RDS database flavors running on Amazon AWS.

Figure 1:  Multicloud Architecture: OCI - AWS

Steps to monitor the RDS database on Amazon AWS

1. IPsec connectivity between OCI and AWS Cloud
2. Provisioning of AWS Native RDS and RDS Custom Databases on Amazon
3. Installing Management Gateway and Agent on RDS instances
4. Discover the RDS database as an External Database
5. Enable Database Management, Stack Monitoring and Operations Insights
6. Discover Database System on Database Management

IPsec Connectivity between OCI and AWS

Multiple options are available to establish connectivity between OCI and AWS, we are using an IPsec VPN tunnel using a Site-to-Site VPN with static routing.

Prerequisites for connectivity between OCI and AWS:

1. VPC and the related subnets are created already on AWS.

2. VNC and subnets are already provisioned on OCI.

Once the connectivity is done, OCI resources will be able to connect to AWS resources via a private network.

Figure 2:  OCI Site-to-Site tunnel status

Figure 3:  AWS Site-to-Site tunnel status

Provision RDS and RDS Custom

Once the connectivity is established, create the desired databases on AWS. There are three options available to provision the databases on AWS.

1. Amazon RDS Database for Oracle.
2. Amazon RDS Custom for Oracle.
3. Database on AWS EC2 instance.

Follow the documentation to create the appropriate database instance.

Install Management Gateway and Management Agent

Use Management Gateway and Management Agent to discover the RDS databases in OCI O&M. Install Management Gateway on the AWS EC2 host which has the public IP address and Management Agent (MACS) on the RDS Custom EC2 instance which is in the private-only subnet.

Figure 4:  RDS Database Management with MACS Architecture

• Download and Install the Management Gateway on the EC2 instance which is on the public subnet.
• Install the Management Agent on the RDS EC2 instance.
• Check the status of the Management Gateway and Management Agent on the OCI console under Observability and Management.

Figure 5:  Management Agent list on O&M

• Install the required O&M service plugins on the Management Agent.

Discover the RDS database as an External Database

After enabling the required plugins, navigate to OCI Menu -> Database Management and Administration. Select the deployment type as External under filters. Select Register External Database to register the RDS database. Refer to the documentation for the steps to discover the Databases.

Figure 6:  External Database Discovery Flow

Follow the steps to discover the CDB and PDB (RDS Custom for Oracle) and non-Container (RDS for Oracle) databases.

Enable Database Management, Operations Insights, and Stack Monitoring

Navigate to the External Database, select the container Database, and Enable Database management, Operations Insights, and Stack Monitoring Services

Figure 7:  Enable Database Management and other O&M services for CDB

Figure 8:  Enable DBM and other O&M Services for PDB

Similarly enable all the services for Non-Container Databases.

Here we discovered CDB, PDB, and Non-Containers running on Amazon RDS.

Figure 9: Database Fleet for Non CDB, CDB and PDB

Similarly, enable those databases for Operations Insights capacity planning to uncover performance issues, forecast consumption, and plan capacity using machine-learning-based analytics on historical and SQL data.

Figure 10:  RDS Databases enabled on Operations Insights (PDB & Non-CDB)

Now, "Add hosts" for Operations insights capacity planning.

Figure 11:  Added hosts for Operations insights capacity planning.

Monitor the RDS databases on Stack Monitoring to get full-stack visibility into the health of the Oracle Database. 

Figure 12:  RDS Databases discovered on O&M Stack Monitoring

Discover Database System on Database Management

Use Database Management services to monitor the database ecosystem such as Oracle Listeners, Automatic Storage Management (ASM), and Clusterware.

Navigate to Database Management -> Administration -> Managed Databases. Select the deployment type as External. Select Discover System on the top menu, then Select External Database System.

Review the Database system details and select Create. The progress can be monitored via the Work Request.

Figure 13:  RDS Database System Discovery Summary

Post discovery, navigate to the Managed database and select the database system associated with the database.

This will discover the listener, cluster, and ASM (if available).

Source: oracle.com

Continue reading