"The Comcast Data Engineering and Management Integration (DEMI) organization works with 10 terabytes of DAP (Directory Access Protocol) data on a daily basis as we work to distribute business-critical information resources to the rest of the company. The fact is we would not be able to pull this off successfully without CoSort. It accurately and quickly processes billions of rows of DAP data and allows us to join and analyze this information in connection with our other data warehouse processes. No other tool gives us this much speed and flexibility and allows the processing and conversion of this volume of flat-file LDIF records. The very talented CoSort team worked directly with us to develop their module and was able to turn it around very quickly. In turn they have developed a long-term customer relationship with America's largest Cable Operator and a large (50 TB) and growing data warehouse."
1.) Development time was under 30 minutes to generate the CoSort SortCL data layouts and job scripts, as well as the KSH script to unzip, process (CoSort) and load (SQLLDR).
2.) The processing of data using CoSort dropped from 2-5 hours under Perl to ~15 minutes (processing 10 million subscriber and 10 million device records). In fact, most of the time in the processing of the data sets was spent unzipping the large data files.