BIWA SIG Bijeenkomst op donderdag 26 mei 2016

Donderdag 26 mei 2016 was er een BIWA SIG Bijeenkomst van de OGh. De onderstaande onderwerpen zijn behandeld:

Metadata Management met Oracle Enterprise Metadata Management (OEMM). Eric van Ettekoven van Oracle gaf ons een introductie van deze tool waarmee zowel Data Lineage als Impact Analyse van Informatie systemen een stuk eenvoudiger wordt.

De presentatie van Eric is hier; te downloaden.

Robbert Naastepad van MN gaf ons een inkijk in de Informatie Architectuur op basis van Oracle Data Integrator in combinatie met de Exa-hardware (Exadata & Exalogic) van Oracle. Robbert liet zien tegen welke uitdagingen je in deze betreffende architectuur kunt aanlopen en behandelde mogelijke oplossingen.

Ook de presentatie van Robbert is te downloaden en wel hier;

Over een volgende BIWA SIG Bijeenkomst wordt ook al nagedacht. Deze zal waarschijnlijk ergens na de zomer gehouden worden.

Mogelijke onderwerpen:

Oracle R (klantcase)
Oracle BI Cloud Service (klantcase)
Data Virtualistie
Oracle BI and Spatial

Mochten er andere interessante onderwerpen zijn laat dan even een berichtje achter of neem contact met mij op.

Hopelijk tot een volgende keer!

Daan Bakboord

Oracle Data Integrator 11g Cookbook – Review

Somewhere in the summer of 2010 the first 11g R1 release of Oracle Data Integrator (ODI) became GA. ODI is the leading Data Integration platform within the Oracle Product Family. It seemed like a logical step that a book on ODI 11g would be launched. The Oracle ODI Product Management team have shared their knowledge into a book called; Oracle Data Integrator 11g: Getting Started. This book presents the audience with a step-by-step tutorial. In the end the reader should be familiar, comfortable and successful with (the new features of) ODI 11g R1.

After the first ‘Getting Started’ book a more advanced book on ODI 11g seemed like a very logical next step.  The Oracle Data Integrator 11g Cookbook is a collection of recipes which cover the more advanced topics in ODI 11g.

Oracle Data Integrator 11g CookbookWhere the first book is more an introduction to ODI, the aim of the second book will uncover the more advanced features of ODI. Some basic knowledge of ODI is essential if you want to get the most out of the this Cookbook.

The Oracle Data Integrator 11g Cookbook is nicely structured. In a chronological order, the various components of ODI are covered. From Installation, Security, Variables, (inside) Knowledge Modules, Advanced Coding, Advanced ETL Processing to a Catch-all chapter which reveals the little know secrets and gives answers to several FAQ’s.

Each chapter consists of several (related) recipes. Each recipe is a step-by-step explanation of a specific task. Sometimes you have to perform some actions to get ready for the (next) recipe. Next to that it’s explained how things work. Last but not least, the recipes go beyond the steps necessary to complete a task in the; ‘There’s more’- and the ‘See also’-sections of the recipe. Within the tasks you will find notes which give some additional information to the recipe. Screenshots, code-(snippets) and links to external sites (Documentation, Oracle Support, Blogs, etc.) complete the recipes.

Consider the Oracle Data Integrator 11g Cookbook as a written MasterClass. Probably even for the more experienced ODI consultants, this book will reveal something they didn’t know before. I guess the authors succeeded in writing a book which differs from the first one and is still worth reading.

For me personally, I this book gives me more understanding of the capabilities of Oracle Data Integrator. I just recently (seriously) started exploring ODI. I would say that in this stage the Oracle Data Integrator 11g: Getting Started better fits my knowledge, but the  Oracle Data Integrator 11g Cookbook definitely is next on my reading list.

Enjoy reading this book.

RM BI Forum 2013 – Day One

Check out my RittmanMead BI Forum 2013, Day One Notes
Philipe Lion showing the V305 SampleApp
No Sales, no Demo, Samples –>  how to solve (functional) challanges, reverse engineer
Showcase the latest new and / or improved features
Tighter integration with other products from the Oracle Analytics family
Wait for V305 and start exploring the new functionality
Anthony Heljula – Performance Tuning – Real Customers Stories
Aggregate Persistance in Oracle BI – Agile
Creating and deploy Aggregates in Oracle BI (Both Physical as well as Logical)
Engineerd Systems (Oracle Exalytics, Oracle Database Appliance, Oracle Exadata)
Take everything (All possible slections) into account – Dashboard prompts (Granularity), Column Selectors, etc.
Operational Report Tuning – Dimension Elimination
Pushing down calculations to RPD
Incremental Load with Aggregate Persistance via Notepad (staging table and where clause)
Peak ETA Extract Transform Aggregate
Aggregate Groups – in Parralel, Using existing Aggregates as Source
Compression TimesTen
Parralel Query Database
Caching – Database TimesTen in Memory, Oracle BI Server Cache, Oracle BI Presentation Cache
Check his presentation here.
Marco Klaasens – Secrets of OBIEE Deployment at Liberty Global
Oracle APEX for Data Entry
– Focus and Sequence (Agile)
– Knowledge
– Constraints
– Standards
Uli Bethke, Maciej Kocon – Oracle Data Integrator
ODI Performance
Metadatadriven SQL Code Generator –> Advanced Analytics – Analytic Functions
Code Templates (Knowledge Module)
Java Agents
ODI Scheduling
Stage –> Dim –> Facts
Packages (Scenarios) versus Load Plans
Efficiency Analysis
Dependency Driven Scheduling
ODI Deployment / Release
Source Control
Trunk / Branch
Adam Bloom – New Developments in BI Multi-Tenancy and Cloud
Oracle Cloud – Hardware, Software infrastructure
NIST – National Institute of Standards and Technology
– Per Tenant Analytics
– Per Tenant BI
– Centrally Provisioned, Departemental BI
Toby Potter – Driving BI from Social Data with Datasift
There is more to a tweet then the 140 characters
It’s not the volume, but the value of data
Servicing Oracle via the biggest Hadoop cluster of Europe
Use your brain to use contextually right data- what do you want to know or do with the data
Datasift + Oracle = Social BI Platform
Making decisions on data which is only for 95% correct is whole new way of thinking as opposed to traditional (Financial) DWH-systems
TED Sessions
Jon Mead – Why I want to be in BI in 5 years of time
(Big) Data (Software, Statisitics, Scope)
Development techniques (Tools, Approaches)
Deliver (Cloud, Packaged, Control)
Anthony Heljula – Incrementally loading Exalytics using Notepad
Incremental Load with Aggregate Persistance via Notepad (via an additional staging table and a where clause)
Check his presentation here.
Mike Durran – HA, DR, CFC
How to keep your Exalytics systems online?
HA – Hig Availability
DR – Disaster Recovery
CFC – Cold Failover Cluster (CFC)

RM BI Forum 2013 Notes – Oracle Data Integrator Master Class

Today the RittmanMead BI Forum 2013 started with an Oracle Data Integrator Master Class provided by (in alphabetical order)  Mark Rittman, Michael Rainey and Stewart Bryson.

I made some notes, so I’ll try to highlight some points from the Master Class. It won’t be complete, but it gives an indication. I didn’t know much about ODI, so the start of the Master Class was very welcome to me.

Overview by Stewart Bryson

Just in time for Oracle BIA – everything ODI, bye, bye Informatica and DAC (DAC translated into configuration and packaging and load plans).

Further integration with Oracle Weblogic and the Fusion Middleware

ODI & OBI Jive really good together. Abstraction of physical Sources logic built into the model not the process. Same like OBI. Develop the model, not the query. Metadata. See the the similarity between ODI and OBI.

Scripting becomes Groovy with ODI  (see later on).

Everything you do in ODI goes via Agents
OCI versus JDBC –> ETL versus E-LT (insert into select)

Develop once in the Model instead of in the interface / mapping. Interface inherits from the model, override if needed. Interconnect interfaces via yellow temporary interfaces

Knowledge Modules (KM) –  the physical implementation of the logic
How to load is determined by the KM, E-LT or ETL

Packaging (sequencing) versus Load Plans

Goldengate and ODI by Michael Rainey

Oracle Reference Architecture for Information Management and Big Data – 2013 Whitepaper

Oracle Goldengate – Data Replication

CDC Journalizing in ODI – Journalizing Knowledge Module (JKM)

Goldengate uses the JKM….generated metadata and a readme how to apply. No full integration

RMAN redo log, archive log , cleaning the logs

Parent Child Journalizing two different tables via two interfaces into the same target table

Subscription Views a RM feature to provide a choice for the ETL developer to choose which object to use

Oracle Goldengate and ODI are a Perfect Match

ODI and Big Data by Mark Rittman
If you do not care about a single row you refer to Big Data
Decisions based on your own data vs decisions based on all data relevant to you

Oracle Big Data



ODI (or OBIEE) to connect the Hadoop world to the traditional DWH world via the Hive.

HiveODBC (DocId 1520733.1) and HiveJDBC

ODI and OBIEE als enabelers to use the result out of the Hadoop world
Check Mark’s presentation

The three R’s of ODI – Resuming, Restarting, Restoring by Stewart Bryson

Using database features in ODI to eliminate the “Aftermath scenarios”.


Enable resumability in ODI to prevent a load to fail and just suspend
Monitor the resumable view – dba_resumable view

Load Plans


Flashback Table
Flashback Database

Record the SCN (be careful with row level and block level)


archivelog mode
Block change tracking file
Insert /*APPEND*/

Check the origin of the “Three R’s of Data Warehouse Fault Tolerance” here.

Automating & Scripting Oracle Data Integrator by Michael Rainey

Some great examples of how to make life a little bit easier. Could Oracle have used this to migrate Oracle BIA from Informatica to Oracle Data Integrator earlier?

ODI – 11g Expert Accelerator for Model Creation – David Allen 

All in all it was a very valuable Master Class. I learned a lot and there is lot more to investigate and explore. Hopefully this is the standard for the rest of the BI Forum.