IBM Systems Lab Services and Training A Data Centric ...

IBM Systems Lab Services and Training

Phase 0 - Database Statistical Ana

 The DB2 for i system catalogs provid
following questions:
–Which files are in use?
–Which files are core files? Work fil
–Which files are DDS defined?
–What are the relationships betwee

11 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

alysis

de the answers to the

les?
en core files?

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Example Catalog Analysis Queries

WITH Member_Counts AS ( --
S
SELECT table_schema, table_name, T1.*, C
m
CASE Number_Partitions WHEN 1 THEN 1 ELSE 0 END S
one_member, S
F
CASE WHEN Number_Partitions BETWEEN 2 AND 255 J
THEN 1 ELSE 0 END many_members, U
w
CASE WHEN Number_Partitions > 255 THEN 1 ELSE 0 G
END too_many_members

FROM QSYS2.SYSTABLESTAT T1

JOIN QSYS2.SYSTABLES T2

USING(table_schema, table_name)

where T2.file_type <> 'S') -
SELECT Table_Schema, COUNT (DISTINCT
table_name) total_files, S
SUM(one_member) AS One_Partition, ta
SUM(many_members) AS LT_256_Partitions, m
SUM(too_many_members) AS Too_Many_Partitions M
FROM Member_Counts S
GROUP BY ROLLUP (Table_Schema);
F

J

U

w

G

12 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

s

- Work files
SELECT table_type, Table_Schema,
COUNT (DISTINCT table_name) total_files,
max(number_rows) max_rows, max(data_size) max_size,
SUM(OPEN_OPERATIONS) Opens,
SUM(cLEAR_OPERATIONS) Clears
FROM QSYS2.SYSTABLESTAT T1
JOIN QSYS2.SYSTABLES T2
USING(table_schema, table_name)
where T2.file_type <> 'S' AND clear_operations > 0
GROUP BY ROLLUP (table_type, table_schema);

--Indexing

SELECT table_type, Table_Schema, COUNT (DISTINCT
able_name) total_files, max(number_rows) max_rows,
max(data_size) max_size,
MAX(maintained_temporary_index_size) max_MTI_Size,
SUM(Index_builds) Ix_Builds

FROM QSYS2.SYSTABLESTAT T1

JOIN QSYS2.SYSTABLES T2

USING(table_schema, table_name)

where T2.file_type <> 'S'

GROUP BY ROLLUP (table_type, Table_Schema);

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Agenda

 Getting started
–Why Data Centric, Why SQL?
–Training, Teaming and Tools

–Discovery (Phase 0)

–Isolation (Phase 2)
–Integrity (Phase 3)

13 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Database Core Fundamental Items

 The following core items must be pres
model:
– All database objects are defined u
– All base tables (physical files) hav
– Parent or master tables have a pri
– Dependent files have foreign key
– All key fields are named the same
– Columns (fields) are defined appro
defined as date or time types, long
etc)
– There is a minimal use of work file
– Properly normalized database stru

 Correcting the above issues is a critic
such requirements as flexibility, agil

14 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

s

sent in the current physical data

using SQL DDL.
ve a unique key
imary key
y constraints
e across tables
opriately (i.e. date and time data is
g text fields are defined as varchar,

es
uctures (3NF)

cal success factor to addressing
lity and scalability

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Creating an LDM from an Existing P

Logical Data Add Co
Model Fundame

Items

Transfo
Mod

Reverse Physical Data Automate
Engineer Model tools high
recomme
DDS

15 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

Physical Data Model

ore Logical Data
ental Model
s

orm
del

ed New Physical Generate
hly Data Model DDL
ended

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Phase 1 – Current state

u
s Application
e using
r RLA
s

Business Data
logic access

16 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

Format

PF Data

Database PF
integrity rules

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Phase 1 Reverse Engineering and

u
s Application
e using
r RLA
s

Business Data
logic access

SQL PROC SQL
View
Initial modernization -
Foundation for
leveraging SQL

17 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

Transformation

Original
format

LF

Data
New format

SQL Application Logic SQL

Table Table

Database
integrity rules

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Enhanced DDL TABLE and Surrog

CREATE TABLE CUST_MAST 1 (
CUST_MAST_ID FOR COLUMN 2
CUSTMASTID BIGINT GENERATED BY
DEFAULT AS IDENTITY PRIMARY KEY,
CUSTKEY INTEGER NOT NULL UNIQUE 3,
CUSTOMER CHAR(25) NOT NULL ,
ADDRESS CHAR(40) NOT NULL ,
CITY CHAR(30) NOT NULL ,
STATE CHAR(2) NOT NULL ,
ZIPCODE NUMERIC(10, 0) NOT NULL ,
PHONE CHAR(15) NOT NULL ,
CM_LAST_CHANGED FOR COLUMN
CMLASTCHG TIMESTAMP NOT NULL
FOR EACH ROW ON UPDATE
AS ROW CHANGE TIMESTAMP);

Notes
1. Original PF is now LF and references new SQL table

1.2. New SQL only columns are not part of surrogate file

3. CUSTKEY is now unique key constraint (if appropriat
4. FIELDREF no longer used, R in REF column ignored

18 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

gate DDS LF

CRTLF CUSTMAST

A* REF(FIELDREF) 4

A R CUSTMASTR PFILE(CUST_MAST 1)

A CUSTKEY R

A CUSTOMER R

A ADDRESS R

A CITY R

A STATE R

A ZIPCODE R

A PHONE R

A K CUSTKEY 3

CUST_MAST © 2011 IBM Corporation

te)
for LFs

ices

IBM Systems Lab Services and Training

Updated LFs Explicitly Sharing Sur

CRTLF CUSTMAST2

A* REF(FIELDREF)

A R CUSTMASTR2 PFILE(CUST_MAST 1)

A CUSTKEY R

A CUSTOMER R

A ADDRESS R

A CITY R

A STATE R

A ZIPCODE R

A PHONE R

A K CUSTKEY

Notes
1. All DDS LFs are built over the new
2. The format CUSTMASTR is part o

19 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

rrogate LF Format

CRTLF CUSTMASTL1
A* R CUSTMASTR PFILE(CUSTMAST)
A R CUSTMASTR2 PFILE(CUST_MAST1)
A FORMAT(CUSTMAST2)
A K CUSTOMER

CRTLF CUSTMASTL2
A* R CUSTMASTR PFILE(CUSTMAST)
A R CUSTMASTR2 PFILE(CUST_MAST1)
A FORMAT(CUSTMAST2)
A K STATE
A K CITY
A K CUSTOMER

w SQL table CUST_MAST
of the surrogate LF CUSTMAST

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Updated LFs – No Explicit Format S

CRTLF CUSTMAST2

A* REF(FIELDREF)

A R CUSTMASTR2 PFILE(CUST_MAST 1)

A CUSTKEY R

A CUSTOMER R

A ADDRESS R

A CITY R

A STATE R

A ZIPCODE R

A PHONE R

A K CUSTKEY

Notes
1. All DDS LFs are built over the new
2. The format CUSTMASTR is still us
3. Format IDs are the same

20 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

Sharing

CRTLF CUSTMASTL1
A* R CUSTMASTR PFILE(CUSTMAST)
A R CUSTMASTR2 PFILE(CUST_MAST1)
A CUSTKEY
A CUSTOMER
A ADDRESS
A CITY
A STATE
A ZIPCODE
A PHONE
A K CUSTOMER

w SQL table CUST_MAST © 2011 IBM Corporation
sed for impact analysis

ices

IBM Systems Lab Services and Training

Agenda

 Getting started
–Why Data Centric, Why SQL?
–Training, Teaming and Tools

–Discovery (Phase 0)
–Migration (Phase 1)

–Integrity (Phase 3)

21 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Phase 2 Isolation –View Access

u

s Application I/O
e using Handlers
r RLA

s

Business Data access
logic Commitment

control

SQL PROC SQL
View
Initial modernization -
Foundation for
leveraging SQL

22 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

SQL New format
View
Data
New format

SQL Application Logic SQL

Table Table

Database
integrity rules

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Handlers to the Rescue!!!

Handlers intercept traditional IO
operations

IBM SO

Sk
Su

 Handler transforms data
access to SQL procedure calls
– Utilize modern tools and
advanced database feature
and function

 Start with a few key programs
– No need to change all files

23 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

 Handler transforms 5250 to any UI
device
– 3rd party tools available

OA Foundation  Minimal change to existing
RPG programs
Software – Create or acquire a
handler
kills & – Add at least one line of
upport code to an existing
program

And it’s
free!!!

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Getting Started with SQL Handlers

 Tools and resources are available to
–Visit the RPG Cafe for articles, tuto
RPG, RDi and RTCi
–Visit developerWorks and downloa

http://www.ibm.com/developerworks/ibmi/li
brary/i-roaforsql/index.html?ca=drs-

24 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

help build handlers
orials, podcasts and forums about

ad a sample SQL handler

i
www.ibm.com/rational/cafe

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Handling RPG Using a Statement-b

FDDSFILE1 UF E K DISK No external def
F handler(‘STMHANDLER') PREPARE INTO

//Allocate descriptor. sqlN = nbr of co
SQLDA.sqln = rpgOA.flds.num * number of
SQLDA_p = %Alloc(((SQLDA.sqlN) * 80) +
//PREPARE INTO same as PREPARE and DESC
EXEC SQL PREPARE SQL_Descriptor_S1 INTO
//Varying list dynamic SQL uses row sto
EXEC SQL FETCH NEXT FROM SQL_Statement_

USING DESCRIPTOR :sqlda INTO :

25 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

based Handler

finitions required when using namesValues = ‘1’
O describes table in SQLDA

ols from OA*max nbr of rows to fetch
f Rows;

16);
CRIBE
O :SQLDA FROM :SQLString;
orage area associated with SQLDA
_C1 FOR :numberOf ROWS
:rowStorageArea:sqlIndAry ;

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Phase 2 Isolation – Shared Result

u

s Application I/O
e using Handlers
r RLA

s

Business Data access
logic Commitment

control

All applications share
common SQL
procedures

26 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

Set

SQL PROC Result Set

SQL Data
View New format

SQL Application Logic SQL

Table Table

Database
integrity rules

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Handling RPG Using a Procedure-b

FDDSFILE1 UF E K DISK DESCRIBE
F handler(‘PRCHANDLER') PREPARE i

//Result set consumption (7.1)
EXEC SQL ASSOCIATE RESULT SET LOCATORS
EXEC SQL ALLOCATE C1 CURSOR FOR RESULT
//Get Name Values
For i = 1 to Flds.num; //Flds provided

// set the length and value informati
Flds.field(i).valueLenBytes = sqlvar
Flds.field(i).value = sqlvar.sqldata

EndFor;

27 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

based Handler

TABLE provides view format when
is external

Visual App
Diagram part

of RDP

S (:a) WITH PROCEDURE P1;
T SET :a;

d by RPG using namesValues
ion for this field
r.sqlLen;
a; sqlvar part of SQL descriptor

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Agenda

 Getting started
–Why Data Centric, Why SQL?
–Training, Teaming and Tools

–Discovery (Phase 0)
–Migration (Phase 1)
–Isolation (Phase 2)

28 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Phase 3 Integrity

u

s Application I/O
e using Handlers
r RLA

s

Business Data access
logic Commitment

control

All applications share
common SQL
procedures

29 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

SQL PROC CRUD

SQL Data
Join New format
View
Instead
of

Trigger

SQL SQL Constraints SQL

Table Table

Database
integrity rules

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Instead of Triggers

 Enables updates/deletes or inserts
for SQL views which are not
considered updateable, delete-
able or insert-able

– A join would be one such view

 Allows database changes without
impacting existing Stored
Procedures

– Insert/Update/Delete events are
redirected to appropriate table

 Developer creates IOT logic

30 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Instead of Triggers and Constraints

INSTEAD OF INSERT ON JOIN_V1
REFERENCING NEW AS N FOR EACH ROW MODE

DB2SQL
BEGIN

INSERT INTO Parent
(Col1, Col2…)
VALUES (N.Val1, N.Val2…);
INSERT INTO Child
(Parent_ID, Col1a, Col2a…)
VALUES (IDENTITY_VAL_LOCAL(), N.Val1a,
N.Val2a…);

END;

INSTEAD OF DELETE ON JOIN_V1
REFERENCING OLD AS O FOR EACH ROW MODE

DB2SQL
BEGIN

DELETE FROM Parent
WHERE Parent_Key= O.Parent_Key;
END

31 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

s

 Parent contains PK as IDENTITY
E  IDENTITY_VAL_LOCAL returns generated

IDENTITY value from prior INSERT

 FOREIGN KEY constraint rule
CASCADE

 Child rows deleted by DB2

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Summary

 Migrating legacy databases to moder
and additional skill set
–Understanding query optimization
–Relational database design knowle
–Acquire a good tool

 You should have good business reas
–Need for enhanced feature and fu

 Start small, get some experience
–Identify a pilot application which w

 Performance improvement is a benef
reengineering
–The amount of performance gain w
data centric development
• DDL and DML

32 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

rn DB2 databases requires planning

edge

sons for migrating
unction, lack of skills in marketplace

would benefit from modernization
fit, not the main reason for
will be determined by the level of

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

IBM Education Roadmap for DB2 &

 Database Modernization Workshop
– Understand the reasons for DB reengineering.
– Understand the benefits of using SQL.
– Understand RDMS concepts such as normalizatio
referential integrity, etc.
– Learn the methodology and steps involved for
reengineering legacy databases without impacting
the legacy programs
– Understand the methodology and tools required f
identifying high usage files
– Learn conversion of Record level access to SQL
access through the use of embedded SQL and
external SQL Views
– Understand the concepts of using bridge and
wrapper programs as productivity tools for access
SQL IO modules including RPG OA
– Understand the use of Triggers, Instead of Trigge
User Defined Functions, and Stored Procedures a
database modernization techniques
– Labs are based on Rational Developer for Power
(RDP) and IBM Data Studio

– Source is provided

33 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

& SQL © 2011 IBM Corporation

on,
g
for

sing
ers,
as

ices

IBM Systems Lab Services and Training

Where to Start

 Review the following Redbook:
– Modernizing iSeries Application Data

 You can find more information about Rati
websites:
– http://www.ibm.com/software/rational/
– http://publib.boulder.ibm.com/infocent
expand ibm i 7.1 Information Center->
Languages->RPG

 To learn more about the Rational Develop
create the Visualizer Application Diagram
download an evaluation copy, visit the de
website at: http://www.ibm.com/develope

 To get the most up to date, in depth know
modernization strategy, including hands o
Access, Rational Developer for Power an
IBM i Database Modernization Workshop
http://www.ibm.com/systems/i/software/d
i Modernization Workshop from the list of

34 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

Access (SG24-6393)

ional Open Access at the following

/products/openaccess/
ter/iseries/v7r1m0/index.jsp and then
>Programming->Programming

per for Power Systems tool used to
ms within this presentation, and to
eveloperWorks Rational Tools download
erworks/downloads/r/rdp/

wledge of the DB2 for i Data Access
on experience with Rational Open
nd Infosphere Data Architect, enroll in the
p at the following website:
db2/db2educ_m.html and choose DB2 for
f courses.

ices © 2011 IBM Corporation

IBM Systems Lab Services and Training

Additional Information

 DB2 for i Websites

– Home Page: ibm.com/sy

– DeveloperWorks Zone: ibm.com/dev

– Porting Zone: ibm.com/par

 Newsgroups & Forums
– USENET: comp.sys.ibm.as400.misc
– DevloperWorks:
https://www.ibm.com/developerworks
– System i Network DB2 Forum: http://f

 Education Resources - Classroom & Onl

– ibm.com/systemi/db2/gettingstarted.h
– ibm.com/partnerworld/wps/training/i5o

 DB2 for i Publications

– White Papers: ibm.com/partnerwo
– Online Manuals: ibm.com/systems/i/
– DB2 for i Redbooks (http://ibm.com/re

• Getting Started with DB2 Web Qu
• OnDemand SQL Performance An
• Preparing for and Tuning the SQL

6598)

35 IBM Systems Lab Services and Training – ibm.com/systems/services/labservi

ystems/i/db2
veloperworks/db2/products/db2i5OS
rtnerworld/i/db2porting

c, comp.databases.ibm-db2

s/forums/forum.jspa?forumID=292
forums.systeminetwork.com/isnetforums/

line
html
os/courses

orld/wps/whitepaper/i5os
/db2/books.html
edbooks)
uery for System i (SG24-7214)
nalysis … in V5R4 (SG24-7326)
L Query Engine on DB2 for i5/OS (SG24-

ices © 2011 IBM Corporation