CoSort (IRI, Inc.)

Unfortunately, many data acquisition tools and methods used for comparing files and processing changed data are cumbersome, error-prone, and limited to one database's sources or performance. Regular reads and updates of the entire database is an impractical alternative to refresh the warehouse, especially as data volumes grow.

In other words, the feasibility of doing in-database work for deltas shrink as table sizes grow.

Start with a SELECT query in CoSort's Fast Extract (FACT) tool for example, and dump only those rows generated after a certain timestamp to a flat file. CoSort's SortCL tool can then conditionally join this file to one or more other archive files, and produce multiple output targets. You can present the changed records in custom reports, feed a (pre-sorted) database loader, and populate other applications with the inserts, changes, and/or deletes -- all at the same time.

You can also count the number of new, modified, or missing records in your reports, or consult SortCL's runtime statistics (which count inner and outer matches at each join). Label, log, and analyze changes in the transaction data to spot red flags and assess trends forensically.

SortCL can, at the same time, also:
• re-format
• cross-calculate
• rank
the changed data, and use its field-level commands or functions to help you:
• transform
• cleanse
• protect
• present
the changed data.

Thus, you can cost-effectively capture changes in your data, and leverage simultaneous transformation and reporting opportunities, during high-volume file compare and data refresh operations.