How SortCL Users Can Re-map Fields and Reformat Records

Home » Solutions » Data Transformation » Re-Map / Reformat

Re-map/Reformat

Re-map, Reformat, and Report while Transforming!

Challenges:

High volume data transformation is often performed one slow step a time; i.e. sort, to join or aggregate, then stop in a flat file that gets handed off to another program, a data mart, scrubbing or BI tool, etc. Alternatively, pre-processing to filter and format data prior to large transforms may be performed. Either way, multiple passes through the data are necessary.

Solutions:

CoSort's SortCL tool can perform many simultaneous data mappings because it uses symbolic field names to move and format data on output. This allows you to reformat and report on data in the same job script and I/O pass performing the 'heavy lifting' of ETL.

Specifically, as SortCL maps your fixed or variable position fields from input to output, it can re-map (re-position, re-size, align, trim, pad), and type-convert data. Additional custom layout options also include changing fixed-position layouts to variable (floating) and vice versa. You can parse, strip or rewrite header records on output. You can insert special formatting characters and environment variables -- including markup language commands for web reporting.

SortCL performs mathematical expressions (cross-calculation) between field data, as well as on joined and/or aggregated values, to derive and output new detail or summary field values! You can create as many simultaneous output targets and formats as you need in the same job script, and produce them in the same pass through your data.

SortCL also reformats files from one type to another. So you can come in with Micro Focus ISAM and leave with CSV, or go from a huge flat file to XML, and vice versa.

For database indexing, you can format a "sequencer" field to assign an integer value to each sorted output record, and specify its starting value and offset position. This allows the database to cross-reference records over a number of fields.

CoSort (IRI, Inc.)