Complex Transforms. CoSort Supports Custom, Field-Level Transformation Functions.

Home » Solutions » Data Transformation » Custom Transforms

Custom Transforms

Complex Transformations at the Field Level through Custom Routines

Challenges:

Many ETL systems need complex transformations. Their custom coding often requires special script components, SQL procedures, or assemblies. In addition to learning and maintenance curves, these objects may not operate at the field level, in the same I/O pass as other transformations, or leverage the APIs of best-of-breed software.

You may have many data reengineering requirements that result from data profiling. But there may not be enough time or expertise available to write enough custom cleansing or standardization routines to address these needs.

Solutions:

CoSort's SortCL integrates and processes data in the staging area. While the tool simultaneously transforms, reports, and secures data at the field level, the opportunity for custom transforms is also at hand. SortCL's field-level function support allows you to plug in complex transforms and run them with your other data processing, presentation, and protection tasks.

You can specify custom transforms on the same field twice in one job script; i.e. during inrec (before sort, join, or report declaration) and outfile (e.g. after sorting). This allows you to transform each field in at least two different ways in that one I/O pass.

Note that the SortCL tool leverages its own field-level transform capability out of the box. In addition to default encryption and de-identification routines, there is a third-party library example. A Melissa Data address standardization library function call is documented in the SortCL manual. You can invoke similar transformation routines from Melissa Data, another vendor, or your own libraries using the C interface template.

CoSort (IRI, Inc.)