This facility enables you to manipulate the contents of any number of files grouped together in a Warp Case, to create test data projected forward to future dates or backwards to previous ones and to scramble the contents of sensitive data. You can therefore organise, schedule and execute all date warping and data scrambling through a single request whenever it is relevant.
Data Warping works directly on the files that you specify. You should therefore ensure that the data to be warped is not part of a live system and that, if appropriate, you have taken back-up copies.
Data Warping will operate over physical files (PF or PF38). Alternative File Descriptions can be used to concatenate dates or other fields which are held as separate elements into one field for warping.
It should be noted that Data Warping can process many records simultaneously. It is therefore necessary to place an exclusive lock on the file before processing can commence.
If during the execution of a Data Warp instructions exist for fields which are now in error, they will be reported via a warning message in results. Examples of such errors include; fields which no longer exist, are no longer of the correct type and fields which do not contain logically correct date information.
This facility provides both Date Aging and Data Scrambling each of which is covered separately in the following sections.
Date Aging
The objective is to consistently move selected date information forward or backward by a specified period.
Date Elements and Formats
The first stage is to define the format of the dates within OS/400 numeric fields. If you already have this information as the result of a Year 2000 exercise or by the use of a 4GL, it is possible to automatically assign date formats in a User written API. Please contact Original Software for details.
The following elements with a date are supported:
Century (C) 1,0 This is defined as a logical indicator where ‘0’ indicates the 20th century and ‘1’ indicates the 21st century.
Day (D) 2,0 This is the calendar day and any warp processing will respect normal month boundaries, leap years and the Year 2000. Note that if a field that contains the Day is warped in isolation, such boundary information is not available, so the Day number will be returned to 1 each time the 28th day is passed.
Month (M) 2,0 This is the calendar month and moves between 1 and 12.
Short Year (Y) 2,0 This is the calendar year when represented by 2 digits. When warped it will move in the range 0 to 99.
Long Year (L) 4,0 This is the calendar year when represented by 4 digits. When warped it will move in the range 0 to 9999.
Julian Day (J) 3,0 This is the day count within a year which can move between 1 and 366. Warp processing is sensitive to leap years and the Year 2000. It should be noted that OS/400 only supports Julian Days in combination with a Short Year.
Hundred Year Date (H) 5,0 This is a day count which must be used in isolation. When warped it will move in the range 0 to 99999.
Period (P) 2,0 This is a 2 digit business period indicator which can move between 1 and whatever high range is specified as part of the warp factors.
Week (W) 2,0 This is a 2 digit business week indicator which can move between 1 and whatever high range is specified as part of the warp factors.
Date Data Type (DATE) IBM supplied value which can have varying format and length.
Timestamp (TIMESTAMP) IBM supplied value in the format yyyy-mm-dd-hh.mm.ss.mmmmmm
For each field that contains date information to be aged, it is necessary to define the structure of the date information it contains. In the current release date information can only be aged if it is held in packed or signed numeric fields, with zero decimal places and with a length that equates to the total length of the selected elements as defined in the table above, or if it is held in date data type or timestamp fields.
The following date formats are supported (all sequences of elements within a date are supported):
DAY – D, DML, DMY, DMYC
JULIAN – J, JY
HUNDRED – YEAR H
WEEK – W, WL, WY, WYC
PERIOD – P, PL, PY, PYC
MONTH – M, ML, MY, MYC
SHORT YEAR – Y, YC
LONG YEAR – L
CENTURY – C
DATE Data Type – DATE
TIMESTAMP – TIMESTAMP
Dates embedded in longer numeric fields are also supported. This caters for dates held in odd length fields and dates embedded in other information such as Invoice Numbers. A * must be inserted into a valid Date Format for every position to be treated as a null value and hence ignored when Date Warping occurs. For example, Invoice Number 00980725 would be defined as **YMD.
Please note that null fields cannot be warped. Any attempt to warp a file containing a null field will cause the run to end and a warning message to be generated in results.
As of version 8.3.0, for default, uninitialised ISO Dates that contain ‘0001-01-01’
or ‘0001-01-01-00.00.00.000000’, the values will not be incremented, the values will be left ‘as is’.
Warp Aging Factors
Dates are held to different levels of precision within any application. For example a person’s age is held as a Year, a financial reporting unit is often a Year and a Period, and a tax point date will be a Year, a Month and a Day. We have already seen how you can define these different structures and we will now examine how Date Warping allows you to age this information to achieve your desired results.
Sampling
Date Age Sampling is available via the Description option for a Warp Case. It allows the user to see what effect the current warp factors will have upon different date formats given a starting date that they can specify.
Data Scrambling
The objective is to mix the information from user selected fields across the records in a file, so that sensitive ‘live’ information may be protected. The scrambling occurs vertically within a file so that the use of data remains in context. Taking a payroll file as an example, after scrambling has occurred a record might contain the CEO’s Payroll Number with the Janitor’s address and the Sales Manager’s salary.
Given that the scrambling occurs vertically within a file, there is no point scrambling a file containing a single record and little real protection is achieved on files containing only a few records. Within a field, information remains intact.
Full information on the scrambling routine is not published so as not to assist anyone attempting to unscramble the information.
As an alternative to the vertical scrambling it is also possible to generate new values for fields. The use of Synchronised Scrambling allows you to change the data in a field from one value to another consistently across the database, replacing real values with new ones that optionally do not already exist.
