NeoAnd/Docling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Support AsciiDoc and Markdown input format (#168)

Browse Source 
* updated the base-model and added the asciidoc_backend

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated the asciidoc backend

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* Ensure all models work only on valid pages (#158)

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* ci: run ci also on forks (#160)


---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com>

* fix: fix legacy doc ref (#162)

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* docs: typo fix (#155)

* Docs: Typo fix

- Corrected spelling of invidual to automatic

Signed-off-by: ABHISHEK FADAKE <31249309+fadkeabhi@users.noreply.github.com>

* add synchronize event for forks

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: ABHISHEK FADAKE <31249309+fadkeabhi@users.noreply.github.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>

* feat: add coverage_threshold to skip OCR for small images (#161)

* feat: add coverage_threshold to skip OCR for small images

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* filter individual boxes

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* rename option

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* chore: bump version to 2.1.0 [skip ci]

* adding tests for asciidocs

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* first working asciidoc parser

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* reformatted the code

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the mypy

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* adding test_02.asciidoc

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* Drafting Markdown backend via Marko library

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* work in progress on MD backend

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* md_backend produces docling document with headers, paragraphs, lists

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Improvements in md parsing

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Detecting and assembling tables in markdown in temporary buffers

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Added initial docling table support to md_backend

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Cleaned code, improved logging for MD

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Fixes MyPy requirements, and rest of pre-commit

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Fixed example run_md, added origin info to md_backend

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* working on asciidocs, struggling with ImageRef

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* able to parse the captions and image uri's

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the mypy

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* Update all backends with proper filename in DocumentOrigin

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Update to docling-core v2.1.0

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fixes for MD Backend, to avoid duplicated text inserts into docling doc

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Fix styling

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Added support for code blocks and fenced code in MD

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* cleaned prints

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Added proper processing of in-line textual elements for MD backend

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Fixed issues with duplicated paragraphs and incorrect lists in pptx

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Fixed issue with group ordeering in pptx backend, added gebug log into run with formats

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com>
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Signed-off-by: ABHISHEK FADAKE <31249309+fadkeabhi@users.noreply.github.com>
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>
Co-authored-by: Peter Staar <taa@zurich.ibm.com>
Co-authored-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com>
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Co-authored-by: ABHISHEK FADAKE <31249309+fadkeabhi@users.noreply.github.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com>
This commit is contained in:
Christoph Auer
2024-10-23 16:14:26 +02:00
committed by GitHub
parent 3496b4838f
commit 3023f18ba0
52 changed files with 3731 additions and 3517 deletions
Download Patch File Download Diff File Expand all files Collapse all files

303
tests/data/groundtruth/docling_v2/redp5695.md
View File

@@ -36,17 +36,12 @@ Ever wonder how many transactions a bank processes per day? What about the pace
The most recent platform for IBM Z is IBM z16™. The IBM z16 supports the following features:
GLYPH<SM590000> On-chip AI acceleration
GLYPH<SM590000> Quantum-safe crypto discovery
GLYPH<SM590000> Simplified compliance
GLYPH<SM590000> Flexible capacity
GLYPH<SM590000> Modernization of applications
GLYPH<SM590000> Sustainability
- GLYPH<SM590000> On-chip AI acceleration
- GLYPH<SM590000> Quantum-safe crypto discovery
- GLYPH<SM590000> Simplified compliance
- GLYPH<SM590000> Flexible capacity
- GLYPH<SM590000> Modernization of applications
- GLYPH<SM590000> Sustainability
With these features, enterprises can upgrade applications while preserving secure and resilient data.
@@ -56,7 +51,6 @@ Figure 1 on page 3 shows a picture of the IBM z16 mainframe.
Figure 1 IBM z16
<!-- image -->
## IBM z16 and IBM LinuxONE Emperor 4 features
@@ -67,7 +61,6 @@ Figure 2 provides a snapshot of the IBM Z processor roadmap, which depicts the j
Figure 2 IBM Z: Processor roadmap
<!-- image -->
The IBM z16 and IBM LinuxONE Emperor 4 are the latest of the IBM Z, and they are developed with a 'built to build' focus to provide a powerful, cyberresilient, open, and secure platform for business with an extra focus on sustainability to help build sustainable data centers. Although the z16 server can host both IBM z/OSfi and Linux workloads, LinuxONE Emperor 4 is built to host Linux only workloads with a focus on consolidation and resiliency. Depending on the workload, consolidation from numerous x86 servers into a LinuxONE Emperor 4 can help reduce energy consumption by 75% and data center floor space by 50%, which helps to achieve the sustainability goals of the organization.
@@ -76,7 +69,6 @@ Figure 3 on page 5 shows a summary of the system design of IBM LinuxONE Emperor
Figure 3 System design of IBM z16 LinuxONE Emperor 4
<!-- image -->
The IBM z16 and IBM LinuxONE Emperor 4 servers are built with 7-nm technology at a 5.2 GHz speed. They consist of four dual-chip modules (DCMs) per central processor complex (CPC) drawer, each of which is built with two 8-core Telum processor chips that has "first in the industry" on-chip acceleration for mid-transaction, real-time AI inferencing, which supports many different use cases, including fraud detection.
@@ -87,7 +79,6 @@ Figure 4 provides more information about the features of AI Accelerator integrat
Figure 4 IBM z16 on-chip AI Accelerator integration with IBM Z processor cores
<!-- image -->
The IBM z16 and IBM LinuxONE Emperor 4 server platforms are built with the hardware features that are shown in Figure 4 with addressing data and AI workloads in mind. Regardless of where the ML and deep learning (DL) frameworks are used to build and train data and AI models, the inferencing on existing enterprise application data can happen along currently running enterprise business applications. CP4D 4.6 supports Tensorflow and IBM Snap ML frameworks, which are optimized to use the on-chip AI Accelerator during inferencing. Support for various other frameworks is planned for future releases.
@@ -96,7 +87,6 @@ Figure 5 on page 7 shows the seamless integration of AI into existing enterprise
Figure 5 Seamless integration
<!-- image -->
## What is Cloud Pak for Data on IBM Z
@@ -109,18 +99,14 @@ Figure 6 shows a solution overview of CP4D. The infrastructure alternatives are
Figure 6 Solution overview of Cloud Pak for Data
<!-- image -->
We highlight the four main pillars that make IBM Z the correct infrastructure for CP4D:
GLYPH<SM590000> Performance and Scale
GLYPH<SM590000> Embedded Accelerators
GLYPH<SM590000> Reliability and Availability
GLYPH<SM590000> Security and Governance.
- GLYPH<SM590000> Performance and Scale
- GLYPH<SM590000> Embedded Accelerators
- GLYPH<SM590000> Reliability and Availability
- GLYPH<SM590000> Security and Governance.
From a performance perspective, CP4D on IBM Z provides your data and AI with high transaction processing and a powerful infrastructure. From the embedded accelerators perspective, CP4D on IBM Z can investigate each transaction thanks to a cutting-edge DL inference technology even in the most demanding, sensitive, and latency-prone real-time workloads. From a reliability perspective, CP4D on IBM Z provides high availability and resiliency. Lastly from the security perspective, CP4D on IBM Z is suitable for protecting sensitive data and AI models for enterprises in highly regulated industries or those industries that are worried about security.
@@ -128,17 +114,12 @@ From a performance perspective, CP4D on IBM Z provides your data and AI with hig
With CP4D on IBM Z and IBM LinuxONE, users can develop, train, and deploy AI and ML models. Users can accomplish this task by using the CP4D IBM Watsonfi Studio and IBM Watson Machine Learning (WLM) services. By using these two fundamental services, users can accomplish the following tasks:
GLYPH<SM590000> Provision various containerized databases.
GLYPH<SM590000> Explore, clean, shape, and alter data by using Data Refinery.
GLYPH<SM590000> Use project-specific data that is uploaded, or connect to distant data.
GLYPH<SM590000> Create Spark run times and applications.
GLYPH<SM590000> Create, build, evaluate, and deploy analytics and ML models with trust and transparency.
GLYPH<SM590000> Leverage the AI Integrated Accelerator for TensorFlow 2.7.2 and Snap ML 1.9.
- GLYPH<SM590000> Provision various containerized databases.
- GLYPH<SM590000> Explore, clean, shape, and alter data by using Data Refinery.
- GLYPH<SM590000> Use project-specific data that is uploaded, or connect to distant data.
- GLYPH<SM590000> Create Spark run times and applications.
- GLYPH<SM590000> Create, build, evaluate, and deploy analytics and ML models with trust and transparency.
- GLYPH<SM590000> Leverage the AI Integrated Accelerator for TensorFlow 2.7.2 and Snap ML 1.9.
For more information about the specifics of these capabilities, see Capabilities on Linux on IBM Z and IBM LinuxONE.
@@ -172,22 +153,16 @@ Figure 7 on page 11 provides an overview of the components that are supported on
Figure 7 Developing, training, and deploying an AI model on Cloud Pak for Data on IBM Z and IBM LinuxONE
<!-- image -->
In summary, here are some of the reasons why you should choose AI on IBM Z:
GLYPH<SM590000> World-class AI inference platform for enterprise workloads:
-Embedded accelerators: A centralized on-chip AI accelerator that is shared by all cores.
-Industry standard AI ecosystem: Many industry open-source data science frameworks are available on the platform.
-Seamlessly integrate AI into existing enterprise workload stacks: Train anywhere, and then deploy on IBM Z.
GLYPH<SM590000> Security: Encrypted memory, and improved trusted execution environments.
GLYPH<SM590000> Sustainability: Reduce your energy consumption with real-time monitoring tools about the energy consumption of the system.
- GLYPH<SM590000> World-class AI inference platform for enterprise workloads:
- -Embedded accelerators: A centralized on-chip AI accelerator that is shared by all cores.
- -Industry standard AI ecosystem: Many industry open-source data science frameworks are available on the platform.
- -Seamlessly integrate AI into existing enterprise workload stacks: Train anywhere, and then deploy on IBM Z.
- GLYPH<SM590000> Security: Encrypted memory, and improved trusted execution environments.
- GLYPH<SM590000> Sustainability: Reduce your energy consumption with real-time monitoring tools about the energy consumption of the system.
## AI use cases
@@ -203,23 +178,15 @@ For the airline industry, processes such as air traffic management, flight manag
In the following sections, we describe the following use cases:
GLYPH<SM590000> "Use case 1: Responsible AI augmented with risk and regulatory compliance" on page 12 AI model lifecycle governance, risk management, and regulatory compliance are key to the success of the enterprises. It is imperative to adopt a typical AI model lifecycle to protect new end-to-end risks.
GLYPH<SM590000> "Use case 2: Credit default risk assessment" on page 22
Core banking solutions running on IBM Z that are involved in processing inbound transactions need real-time fraud detection to prevent fraud. Other types of possible use cases might be credit risk analysis, anti-money laundering, loan approval, fraud detection in payments, and instant payments.
GLYPH<SM590000> "Use case 3: Clearing and settlement" on page 25
The use of AI can help to predict which trades or transactions have high risk exposures, and propose solutions for a more efficient settlement process.
GLYPH<SM590000> "Use case 4: Remaining Useful Life of an aircraft engine" on page 27
We describe how AI can help to avoid unplanned aircraft downtime by determining the remaining time or cycles that an aircraft engine is likely to operate before failure.
GLYPH<SM590000> "Use case 5: AI-powered video analytics on an infant's motions for health prediction" on page 30
In this section, we describe how AI can predict an infant's health conditions by monitoring real-time body movements.
- GLYPH<SM590000> "Use case 1: Responsible AI augmented with risk and regulatory compliance" on page 12 AI model lifecycle governance, risk management, and regulatory compliance are key to the success of the enterprises. It is imperative to adopt a typical AI model lifecycle to protect new end-to-end risks.
- GLYPH<SM590000> "Use case 2: Credit default risk assessment" on page 22
- Core banking solutions running on IBM Z that are involved in processing inbound transactions need real-time fraud detection to prevent fraud. Other types of possible use cases might be credit risk analysis, anti-money laundering, loan approval, fraud detection in payments, and instant payments.
- GLYPH<SM590000> "Use case 3: Clearing and settlement" on page 25
- The use of AI can help to predict which trades or transactions have high risk exposures, and propose solutions for a more efficient settlement process.
- GLYPH<SM590000> "Use case 4: Remaining Useful Life of an aircraft engine" on page 27
- We describe how AI can help to avoid unplanned aircraft downtime by determining the remaining time or cycles that an aircraft engine is likely to operate before failure.
- GLYPH<SM590000> "Use case 5: AI-powered video analytics on an infant's motions for health prediction" on page 30
- In this section, we describe how AI can predict an infant's health conditions by monitoring real-time body movements.
## Use case 1: Responsible AI augmented with risk and regulatory compliance
@@ -231,11 +198,9 @@ How mature is your AI governance? In this section, we provide a use case demonst
Here are the three main reasons why organizations struggle with the adoption of AI:
GLYPH<SM590000> Scaling with growing regulations
GLYPH<SM590000> Lack of confidence in operationalized AI (making responsible AI)
GLYPH<SM590000> Challenges around managing the risk throughout the entire AI workflow
- GLYPH<SM590000> Scaling with growing regulations
- GLYPH<SM590000> Lack of confidence in operationalized AI (making responsible AI)
- GLYPH<SM590000> Challenges around managing the risk throughout the entire AI workflow
## Scaling with growing regulations
@@ -249,17 +214,12 @@ Responsible AI protects against loss of data privacy, and reduced customer loyal
Organizations need to mitigate risk of the following items:
GLYPH<SM590000> Deciding not to use certain technologies or practices
GLYPH<SM590000> Using personal information when needed and with a user's consent
GLYPH<SM590000> Ensuring automated decisions are free from bias
GLYPH<SM590000> Customer confidence by providing explanations for business decisions
GLYPH<SM590000> Fraud to the organization and to customer's accounts
GLYPH<SM590000> Delays in putting models into production
- GLYPH<SM590000> Deciding not to use certain technologies or practices
- GLYPH<SM590000> Using personal information when needed and with a user's consent
- GLYPH<SM590000> Ensuring automated decisions are free from bias
- GLYPH<SM590000> Customer confidence by providing explanations for business decisions
- GLYPH<SM590000> Fraud to the organization and to customer's accounts
- GLYPH<SM590000> Delays in putting models into production
In fact, in a recent survey, these concerns were echoed by real AI adopters when asked what aspects of trust are most important to them. Although explaining how AI decides is the primary concern, all of these concerns are important.
@@ -269,7 +229,6 @@ For example, a business can start testing a model before production for fairness
Figure 8 Typical AI model lifecycle
<!-- image -->
Due to regulations, more stakeholders adopt the typical AI model lifecycle to protect their brand from new end-to-end risks. To ensure various aspects of both regulatory compliance and security, the personas that must be involved include the chief financial officer (CFO), chief marketing officer (CMO), chief data officer (CDO), HR, and chief regulatory officer (CRO), along with the data engineers, data scientists, and business analysts, who build AI workflows.
@@ -286,110 +245,92 @@ In a world where trust, transparency and explainable AI matters, every organizat
Lifecycle governance helps you manage your business information throughout its lifecycle, that is, from creation to deletion. IBM AI governance addresses the problems that challenge records managements:
GLYPH<SM590000> Monitor, catalog, and govern AI models from anywhere throughout the AI lifecycle.
GLYPH<SM590000> Automate the capture of model metadata for report generation.
GLYPH<SM590000> Drive transparent and explainable AI at scale.
GLYPH<SM590000> Increase accuracy of predictions by identifying how AI is used and where it is lagging.
- GLYPH<SM590000> Monitor, catalog, and govern AI models from anywhere throughout the AI lifecycle.
- GLYPH<SM590000> Automate the capture of model metadata for report generation.
- GLYPH<SM590000> Drive transparent and explainable AI at scale.
- GLYPH<SM590000> Increase accuracy of predictions by identifying how AI is used and where it is lagging.
## Risk management
Risk management is used in IBM AI governance to identify, manage, monitor, and report on risk and compliance initiatives at scale:
GLYPH<SM590000> Automate facts and workflow management to comply with business standards.
GLYPH<SM590000> Use dynamic dashboards for clear and concise customizable results.
GLYPH<SM590000> Enhanced collaboration across multiple regions and geographies.
- GLYPH<SM590000> Automate facts and workflow management to comply with business standards.
- GLYPH<SM590000> Use dynamic dashboards for clear and concise customizable results.
- GLYPH<SM590000> Enhanced collaboration across multiple regions and geographies.
## Regulatory compliance
Regulatory compliance is a set of rules that organizations must follow to protect sensitive information and ensure human safety. Any business that works with digital assets, consumer data, health regulations, employee safety, and private communications is subject to regulatory compliance.$^{3}$ The IBM AI governance solution for IBM Z includes the following tasks:
GLYPH<SM590000> Help adhere to external AI regulations for audit and compliance.
GLYPH<SM590000> Convert external AI regulations into policies for automatic enforcement.
GLYPH<SM590000> Use dynamic dashboards for compliance status across policies and regulations.
- GLYPH<SM590000> Help adhere to external AI regulations for audit and compliance.
- GLYPH<SM590000> Convert external AI regulations into policies for automatic enforcement.
- GLYPH<SM590000> Use dynamic dashboards for compliance status across policies and regulations.
Enterprises can develop AI models and deploy them by using IBM Watson Studio or WML on CP4D on Red Hat OpenShift on a virtual machine that is based on IBM z/VM or Red Hat Enterprise Linux KVM on IBM Z. AI governance on IBM LinuxONE is supported in the following two ways:
GLYPH<SM590000> Monitor the AI models with Watson OpenScale on CP4D on Red Hat OpenShift on a virtual machine on IBM Z.
GLYPH<SM590000> Enterprises can develop AI models by creating and training models by using Watson Studio and development tools such as Jupyter Notebook or JupyterLab, and then deploying the model onto WML on CP4D on Red Hat OpenShift on a virtual machine on IBM Z. Then, these enterprises can achieve end-end AI governance by running AI Factsheets, IBM Watson OpenScale, and IBM Watson OpenPagesfi on CP4D on x86.
- GLYPH<SM590000> Monitor the AI models with Watson OpenScale on CP4D on Red Hat OpenShift on a virtual machine on IBM Z.
- GLYPH<SM590000> Enterprises can develop AI models by creating and training models by using Watson Studio and development tools such as Jupyter Notebook or JupyterLab, and then deploying the model onto WML on CP4D on Red Hat OpenShift on a virtual machine on IBM Z. Then, these enterprises can achieve end-end AI governance by running AI Factsheets, IBM Watson OpenScale, and IBM Watson OpenPagesfi on CP4D on x86.
Figure 9 on page 16 shows the end-to-end flow for a remote AI governance solution.
Figure 9 Remote AI governance solution end-to-end flow
<!-- image -->
To achieve end-to-end AI governance, complete the following steps:
1. Create a model entry in IBM OpenPages by using CP4D on a x86 platform, as shown in Figure 10.
- 1. Create a model entry in IBM OpenPages by using CP4D on a x86 platform, as shown in Figure 10.
Figure 10 Creating a model entry in IBM OpenPages
<!-- image -->
2. Train a model by using Watson Studio and by using development tools such as Jupyter Notebook or JupyterLab on CP4D on Red Hat OpenShift on a virtual machine on IBM Z, as shown in Figure 11.
- 2. Train a model by using Watson Studio and by using development tools such as Jupyter Notebook or JupyterLab on CP4D on Red Hat OpenShift on a virtual machine on IBM Z, as shown in Figure 11.
Figure 11 Training an AI model by using Watson Studio
<!-- image -->
3. Deploy the model by using WML on CP4D on Red Hat OpenShift on a virtual machine on IBM Z, as shown in Figure 12.
- 3. Deploy the model by using WML on CP4D on Red Hat OpenShift on a virtual machine on IBM Z, as shown in Figure 12.
Figure 12 Deploying an AI model by using WML on Cloud Pak for Data
<!-- image -->
4. Track the external model lifecycle by browsing through the Catalogs/Platform assets catalog by using AI Factsheets and OpenPages while using CP4D on an x86 platform, as shown in Figure 13. The external model (deployed on CP4D on Red Hat OpenShift on a virtual machine on IBM Z) is saved as a platform asset catalog on the x86 platform.
- 4. Track the external model lifecycle by browsing through the Catalogs/Platform assets catalog by using AI Factsheets and OpenPages while using CP4D on an x86 platform, as shown in Figure 13. The external model (deployed on CP4D on Red Hat OpenShift on a virtual machine on IBM Z) is saved as a platform asset catalog on the x86 platform.
Figure 13 External model
<!-- image -->
You can track the model through each stage of the model lifecycle, as shown in Figure 14, by using AI Factsheets and OpenPages.
Figure 14 Tracking the model
<!-- image -->
You can see that the model facts are tracked and synchronized to IBM OpenPages for risk management, as shown in Figure 15.
Figure 15 Model facts that are tracked and synchronized to IBM OpenPages on an x86 platform
<!-- image -->
5. Create an external model by using IBM OpenScale on the x86 platform, as shown in Figure 16.
- 5. Create an external model by using IBM OpenScale on the x86 platform, as shown in Figure 16.
Figure 16 Creating an external model on an x86 platform
<!-- image -->
IBM OpenScale provides a comprehensive dashboard that tracks fairness, quality monitoring, drift, and explainability of a model. Fairness determines whether your model produces biased outcomes. Quality determines how well your model predicts outcomes. Drift is the degradation of predictive performance over time. A sample is shown in Figure 17 on page 21.
Figure 17 IBM OpenScale dashboard that is used to monitor the external model
<!-- image -->
You developed and deployed the AI model by using Watson Studio, WML on CP4D on Red Hat OpenShift on a virtual machine on IBM Z, and end-to-end AI model governance by leveraging AI Factsheets, OpenScale, and OpenPages on CP4D on a x86 platform. Figure 18 shows end-to-end AI governance when using IBM OpenPages, AI Factsheets, and OpenScale.
Figure 18 Final result: End-to-end AI governance when using IBM OpenPages, AI Factsheets, and OpenScale
<!-- image -->
## Use case 2: Credit default risk assessment
@@ -410,7 +351,6 @@ Figure 19 on page 23 shows a sample architecture about how to design and develop
Figure 19 Architecture for credit risk prediction by using an ML AI model on IBM Z
<!-- image -->
A data scientist can leverage Watson Studio to develop and train an AI model and WML to deploy and score the model. In this sample architecture, the WML Python run time leverages the ML framework, IBM Snap Machine Learning (Snap ML), for scoring, can leverage an integrated AI accelerator at the time of model import.
@@ -427,22 +367,17 @@ Figure 20 shows an architecture for predicting credit risk by using DL on IBM Z.
Figure 20 Architecture for credit risk prediction by using DL on IBM Z
<!-- image -->
Data scientists can start creating and training a DL AI model by using a Jupyter Notebook instance and Watson Studio. Then, they can deploy the model by using WML on CP4D running on IBM Z, which provides an endpoint. Other applications, including the IBM WebSphere server, can produce credit risk results by using the model's endpoint.
In summary, here are some considerations for developing real-time AI models, such as credit risk assessment:
GLYPH<SM590000> A preference for in-platform run times of the model, such as faster execution results.
GLYPH<SM590000> Less overhead in the end-to-end flows might improve scoring time.
GLYPH<SM590000> If you are using models that are not deployable, CP4D offers a custom Python run time to build your own stack if they are not available on the platform.
GLYPH<SM590000> AI inferencing based on ML or DL models can increase the accuracy of better credit risk assessment.
GLYPH<SM590000> Using IBM z16 and on-chip AI acceleration with the Telum chip that is embedded with regular Integrated Facility for Linux (IFLs) provides an execution speed for your transactions that cannot be achieved by other means.
- GLYPH<SM590000> A preference for in-platform run times of the model, such as faster execution results.
- GLYPH<SM590000> Less overhead in the end-to-end flows might improve scoring time.
- GLYPH<SM590000> If you are using models that are not deployable, CP4D offers a custom Python run time to build your own stack if they are not available on the platform.
- GLYPH<SM590000> AI inferencing based on ML or DL models can increase the accuracy of better credit risk assessment.
- GLYPH<SM590000> Using IBM z16 and on-chip AI acceleration with the Telum chip that is embedded with regular Integrated Facility for Linux (IFLs) provides an execution speed for your transactions that cannot be achieved by other means.
## Use case 3: Clearing and settlement
@@ -466,54 +401,35 @@ Figure 21 provides a high-level diagram of a clearing and settlement use case fo
Figure 21 Clearing and settlement use case for financial transactions by using Cloud Pak for Data
<!-- image -->
Here are the steps of the high-level process flow:
1. Create a connection to a database (for example, an IBM Db2fi database) where the historical data will be used for ML model building.
2. Read the data from the database and prepare the data for AI by using the Data Refinery tool in CP4D.
3. A Jupyter Notebook or JupyterLab IDE that is provided by the Watson Studio component in CP4D helps us build and train the AI model. The trained model can be saved into a WML repository.
4. Deploy the saved model into a deployment space for batch deployment.
5. Create a batch deployment by using any of these interfaces:
a. Watson Studio user interface from an Analytics deployment space.
b. WML Python client.
c. WML REST APIs.
6. A hardware configuration can be chosen for the deployment.
7. A batch deployment processes input data from a file, data connection, or connected data in a storage bucket, and writes the output to a selected destination.
8. One way to run batch deployment to predict or score is to create and run a batch deployment job.
9. Provide an input data type:
a. Inline data for entering a JSON format payload.
b. Select Data asset , click Select data source , and then specify your asset.
10.The output data type can be a new output file or a connected data asset.
11.A Kubernetes admin can change the maximum number of concurrent batch jobs that can be run.
12.Get the deployment endpoint URL. For more information, see Getting the deployment endpoint URL.
- 1. Create a connection to a database (for example, an IBM Db2fi database) where the historical data will be used for ML model building.
- 2. Read the data from the database and prepare the data for AI by using the Data Refinery tool in CP4D.
- 3. A Jupyter Notebook or JupyterLab IDE that is provided by the Watson Studio component in CP4D helps us build and train the AI model. The trained model can be saved into a WML repository.
- 4. Deploy the saved model into a deployment space for batch deployment.
- 5. Create a batch deployment by using any of these interfaces:
- a. Watson Studio user interface from an Analytics deployment space.
- b. WML Python client.
- c. WML REST APIs.
- 6. A hardware configuration can be chosen for the deployment.
- 7. A batch deployment processes input data from a file, data connection, or connected data in a storage bucket, and writes the output to a selected destination.
- 8. One way to run batch deployment to predict or score is to create and run a batch deployment job.
- 9. Provide an input data type:
- a. Inline data for entering a JSON format payload.
- b. Select Data asset , click Select data source , and then specify your asset.
- 10.The output data type can be a new output file or a connected data asset.
- 11.A Kubernetes admin can change the maximum number of concurrent batch jobs that can be run.
- 12.Get the deployment endpoint URL. For more information, see Getting the deployment endpoint URL.
## Summary
With this use case, we attempted to demonstrate how to predict, in real time, whether the transaction that is being processed might be a fraudulent transaction or not. By using the method, you have the following advantages:
GLYPH<SM590000> No Impact to SLAs and the batch process window.
GLYPH<SM590000> Proactively stop losses, and lower operational, regulatory, and compliance costs.
GLYPH<SM590000> The solution is using a DL framework like TensorFlow for high-performing, low latency scoring.
- GLYPH<SM590000> No Impact to SLAs and the batch process window.
- GLYPH<SM590000> Proactively stop losses, and lower operational, regulatory, and compliance costs.
- GLYPH<SM590000> The solution is using a DL framework like TensorFlow for high-performing, low latency scoring.
## Use case 4: Remaining Useful Life of an aircraft engine
@@ -525,7 +441,6 @@ Figure 22 provides an overview of the inferencing architecture for the RUL of an
Figure 22 Inferencing architecture on IBM Z
<!-- image -->
Because we are looking into data-driven model development, the data set of our target is the run-to-failure data of the engine. We are looking into a supervised learning problem, and we use regression techniques to learn from the data. DL techniques such as Long Short-Term Memory (LSTM) or Gated Recurrent Units (GRU) are our choice because we are looking into a time series data set. TensorFlow or PyTorch frameworks are leveraged to create models. AI governance monitors the data and model drift to maintain the model quality throughout the model's life.
@@ -548,20 +463,15 @@ Figure 23 on page 29 provides a more in-depth view of the architecture of an AI-
Figure 23 In-depth architectural view
<!-- image -->
In summary, consider the following points while developing an AI-based predictive maintenance application:
GLYPH<SM590000> CP4D offers a Python run time to build a custom solution stack, but also supports different components like Watson Studio, WML, Db2, Data Refinery, OpenScale, AI Factsheets, and OpenPages.
GLYPH<SM590000> The trustworthiness of the predicted output is important for critical use cases.
GLYPH<SM590000> IBM Z provides high data security and low latency requirements at scale for the critical applications.
GLYPH<SM590000> A data scientist can choose to train the model and deploy it on CP4D seamlessly with the latest tech stack that is available.
GLYPH<SM590000> The AIOps and MLOps supported by CP4D to track AI model and data lifecycle throughout the application lifecycle.
- GLYPH<SM590000> CP4D offers a Python run time to build a custom solution stack, but also supports different components like Watson Studio, WML, Db2, Data Refinery, OpenScale, AI Factsheets, and OpenPages.
- GLYPH<SM590000> The trustworthiness of the predicted output is important for critical use cases.
- GLYPH<SM590000> IBM Z provides high data security and low latency requirements at scale for the critical applications.
- GLYPH<SM590000> A data scientist can choose to train the model and deploy it on CP4D seamlessly with the latest tech stack that is available.
- GLYPH<SM590000> The AIOps and MLOps supported by CP4D to track AI model and data lifecycle throughout the application lifecycle.
## Use case 5: AI-powered video analytics on an infant's motions for health prediction
@@ -593,7 +503,6 @@ Figure 24 shows an architectural diagram about how to design and develop an AI m
Figure 24 Architecture for AI-powered video analytics
<!-- image -->
Live camera feeds or recorded videos of an infant's movement are the inputs for a pose detection model. This video streaming data was stored in IBM Cloudfi Object Storage for image processing. Video data must be transformed into frames so that the infant's body poses can be detected. These post-estimation components of the pipeline predict the location of all 17-person key points with 3 degrees of freedom each (x, y location and visibility) plus two virtual alignment key points. This approach also embraces a compute-intensive heat map prediction of infant body posture.
@@ -602,29 +511,21 @@ When changes in body posture or movement happen, analytics can be performed, and
We can leverage the following AI technology stack for this use case:
GLYPH<SM590000> Convolutional neural network: Build an artificial neural network model on video streaming and images.
GLYPH<SM590000> TensorFlow: A DL back-end framework that is based on TensorFlow.
GLYPH<SM590000> Mediapipe: A library that helps with video streaming processing and prediction of human pose estimation.
GLYPH<SM590000> OpenCV: A real-time computer vision library that helps perform image processing.
- GLYPH<SM590000> Convolutional neural network: Build an artificial neural network model on video streaming and images.
- GLYPH<SM590000> TensorFlow: A DL back-end framework that is based on TensorFlow.
- GLYPH<SM590000> Mediapipe: A library that helps with video streaming processing and prediction of human pose estimation.
- GLYPH<SM590000> OpenCV: A real-time computer vision library that helps perform image processing.
WML was used for deployment of the pose detection model and generated notifications to users with web and mobile applications, and it integrates with Fitbit for push notifications so that hospitals and parents can take preventive actions.
## Additional resources
GLYPH<SM590000> The Cloud Pak for Data 4.5 on IBM Z Overview Demo video provides an overview of some of the more important features of CP4D on IBM Z.
GLYPH<SM590000> IBM Cloud Pak for Data Tutorials.
GLYPH<SM590000> Here are some additional use cases that use the data science frameworks that are available as part of CP4D on IBM Z and IBM LinuxONE:
-Payment Card Fraud Detection by using TensorFlow on CP4D on IBM Z and IBM LinuxONE is a payment card fraud detection use case.
-Fashion-MNIST clothing classification with PyTorch on Cloud Pak for Data on IBM Z and IBM LinuxONE is a Fashion-MNIST clothing classification use case.
-Payment Card Fraud Prevention by using Snap ML on IBM Cloud Pak for Data on Red Hat OpenShift on a virtual machine on IBM Z and IBM LinuxONE, which leverage the z16 integrated AI accelerator describes a use case that uses Snap Machine Learning in Cloud Pak for Data on IBM Z and IBM LinuxONE. It is a Snap ML use case.
- GLYPH<SM590000> The Cloud Pak for Data 4.5 on IBM Z Overview Demo video provides an overview of some of the more important features of CP4D on IBM Z.
- GLYPH<SM590000> IBM Cloud Pak for Data Tutorials.
- GLYPH<SM590000> Here are some additional use cases that use the data science frameworks that are available as part of CP4D on IBM Z and IBM LinuxONE:
- -Payment Card Fraud Detection by using TensorFlow on CP4D on IBM Z and IBM LinuxONE is a payment card fraud detection use case.
- -Fashion-MNIST clothing classification with PyTorch on Cloud Pak for Data on IBM Z and IBM LinuxONE is a Fashion-MNIST clothing classification use case.
- -Payment Card Fraud Prevention by using Snap ML on IBM Cloud Pak for Data on Red Hat OpenShift on a virtual machine on IBM Z and IBM LinuxONE, which leverage the z16 integrated AI accelerator describes a use case that uses Snap Machine Learning in Cloud Pak for Data on IBM Z and IBM LinuxONE. It is a Snap ML use case.
A companion video can be found at Credit Card Fraud Detection by using Snap ML on IBM Cloud Pak for Data on IBM Z and IBM LinuxONE.
@@ -662,15 +563,13 @@ ibm.com /redbooks/residencies.html
## Stay connected to IBM Redbooks
GLYPH<SM590000> Find us on LinkedIn:
- GLYPH<SM590000> Find us on LinkedIn:
http://www.linkedin.com/groups?home=&gid=2130806
GLYPH<SM590000> Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks weekly newsletter:
https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm
GLYPH<SM590000> Stay current on recent Redbooks publications with RSS Feeds:
- GLYPH<SM590000> Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks weekly newsletter:
- https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm
- GLYPH<SM590000> Stay current on recent Redbooks publications with RSS Feeds:
http://www.redbooks.ibm.com/rss.html