CoSort's Multi-Dimensional Lookup Functionality Substitutes Values Fast
Challenges:
Complex computations -- including the pseudonymization of field data, multi-table joins, and discrete solutions based on business logic -- can be expensive and slow. Where a simple lookup can replace a runtime computation, the performance gain can be significant since retrieving a value in memory can be faster than computing that value.
Lookups typically occur in ETL tools and databases. Lookup transformations must be specially configured, and extra steps may be required to improve their performance in volume.
Meanwhile, analogous functionality has not been available in the file system where more speed, resources, and simultaneous functionality may be available.
Solutions:
Multi-dimensional lookup table functionality is now available in external, flat file environments. CoSort's SortCL tool supports lookup transformations during the inrec and output phases of your job script. Value substitutions can be made through several columns in a delimited set file.
This approach bypasses database tuning and integrity issues to deliver:
• discrete solutions
• a pseudonymization method for data security
• an alternative to joins across many tables
By running lookups in the file system and SortCL's simple 4GL, you may need less preparation and see faster results in the future.
In addition, you can:
• combine lookups with other processes in the same I/O pass
• process resulting values in the same SortCL script
• immediately encrypt or otherwise protect the lookup results
• simultaneously format the lookup results in custom reports