CoSort PlugIns and Flat File Transforms Speed ETL Tool Operations

Home » Solutions » ETL DB Acceleration » ETL Transforms

Speed ETL Operations and Tools

Why Data Warehouse Architects Choose CoSort

Challenges:

Despite their parallel processing and "push down optimization" options, ETL tools and the database engines they may rely on cannot perform the largest transformations optimally, or without having an adverse impact on concurrent operations. As a result, large sort, join, and aggregation jobs may run slowly, and subsequent tasks like loading, analytics, or BI displays may run slowly or require separate passes or products.

Solutions:

If you already use an ETL tool, CoSort can help you run large jobs faster. If you do not, and you use flat files primarily, consider what CoSort's SortCL tool can do by itself, and in conjunction with CoSort Fast Extract and your database loader if you have high volumes of data in Oracle.

You can directly speed sorting (and downstream joins, aggregates, and loads) within Informatica or DataStage with CoSort's unique plug-in sort transformations. CoSort also links with Kalido, ETI, Software AG Natural, SAS, and IDS TeraStream.

You can also run CoSort's SortCL tool alongside ETL packages, or consider SortCL its own ETL tool for flat files. Using the metadata you have, you can exploit SortCL's ability to perform, combine, and accelerate parallel data transformations and operations, such as:
• Sorts
• Joins
• Aggregates
• Lookups
• Perl-Compatible Regular Expressions
• Data Type and File Format Conversions
• Protections (via field-level encryption, de-ID, etc.)
• Detail, delta and summary reports
• Test data generation

You can call SortCL from the shell (as a batch executble or ETL tool command), API, or Java GUI -- and flow data back and forth through files, pipes, or procedures as needed.

CoSort (IRI, Inc.)