Innovative Routines International (IRI), Inc. (The CoSORT Company) respects your time and privacy. You can stop or start future quarterly mailings at any time: click here or on the links at the bottom of this message. Please forward this newsletter to anyone interested in IRI or our high-performance data manipulation and management  tools, or open systems data processing technology.

 

FAst extraCT Logo       CoSORT Logo       RowGen Logo
The CoSORT Journal: Data Sorting and ETL News
                                                                                  Quarter 3, 2005
In this issue:


CoSORT Exhibit at OpenWorld
CoSORT V8.2.3 On the Way
From Fast ETL to Fast Queries: 
A New Partnership with RightOrder
New CoSORT Agency in the UK
Tech Tip: Sort Work (Disk) Areas
Major CoSORT Exhibition

OOW'05   San Francisco, September 17-22

IRI will exhibit for the 8th year in a row at Oracle OpenWorld. The conference is merging this year with PeopleSoft Connect and Retek World, and will be held once again in San Francisco's Moscone Center. 

IRI staff will present and demonstrate FAst extraCT (FACT) and its direct metadata and command-line ties with SortCL and SQL*Loader for fast, one-pass Oracle reorg and ETL operations. Also featured will be IRI's RowGen random data synthesizer and table/file/report simulator, plus relevant new partner solutions.

Visit Booth 822 or call IRI now to schedule business meetings at the conference, or throughout the Bay Area next month

New UK Agency

BLENHEIM SOFTWARE INTERNATIONAL LTD.

London has more to celebrate than the 2012 Olympics and beating the Australian’s at Cricket ... there is a new CoSORT agent in town! BLENHEIM Software International, located in the Thames Valley close to Heathrow, provides UK marketing and technical support services for systems software in both legacy and open systems environments.  

Most of the active UK ADABAS mainframe market now uses BLENHEIM software products. With a Natural sort replacement for UNIX, and a host of mainframe sort and data migration facilities (like mvs2scl), "The CoSORT package synergizes nicely with our portfolio of data processing tools" commented Managing Director Eric Stannard. "We are excited about the opportunities CoSORT will represent here in the UK." 

Partnership Speeds Oracle ETL and Queries

CoSORT / IRI, Inc. and San Jose, CA-based RightOrder, Inc. are collaborating to speed staging and presentation of Oracle data. 

You already know that FAst extraCT (FACT) combines with CoSORT's SortCL (for transforms, reports, and pre-load sorts) and SQL*Loader for rapid, single-pass ETL operations. Once the tables are populated, RightOrder's QueryEdge engine seamlessly speeds the most complex queries (involving multi-table joins) within Oracle -- up to 100X faster!

Both companies realized that ETL architects and DBAs can co-leverage these solutions to accelerate both sides of the data warehousing continuum -- populating then querying Oracle tables faster by combining these superior methods and algorithms.

Click here to read the press release or here for more information.

About CoSORT and The CoSORT Journal
  • CoSORT solutions serve data warehouse (ETL) architects, very large database (VLDB) administrators, mainframe sort migrators, and developers and independent software vendors (ISVs) building faster sorting and data transformation into their applications.
  • CoSORT delivers the IT industry's fastest UNIX sort engine and one of its most powerful flat-file manipulation and reporting programs: SortCL, which combines: row filtering and conditional selection, sort/merge and joins, drill-down aggregation and cross-row calculation, conversion and generation of more than 100 data types, database sequencing, and multi-target, multi-level output reformatting for reports, hand-offs, and DB load utilities.
  • Other special features include: coroutine sort architecture; fully tunable and scalable parallel sort performance on all multi-CPU UNIX and Windows servers; cross-calculation on aggregated values and aggregation on cross-calculated values, cross-table joins (matching) integrated with data conversion and expression logic; multinational date and timestamp support; a cross-platform Java GUI; and, e-commerce reporting via CSV/CLF and IP Address manipulation, plus ELF input and HTML output.
  • CoSORT also has plug-n-play replacements or parameter converters for sorting in: ACUCOBOL-GT, Amdocs Ensemble (telecom billing) Ascential DataStage; Informatica PowerCenter and PowerMart; Cincom Supra; IBM's DB2 loader and MVS/VSE sorts; MF COBOL Workbench, Net and Server Express; SAS System; Software AG Natural; Sun MRP, and, UNIX SVR4 (/bin/sort), et al.
  • IRI has begun to offer other data manipulation and management solutions like: FACT for fast unloads from Oracle; RowGen for custom data generation and format simulations; netCONVERT for mainframe tape data conversion and reformatting; x-PRESS for fast, and secure data compression and decompression; Logon for controlling and auditing access to UNIX systems; and Permitas for licensing and activating software applications.
  • The CoSORT Journal is a quarterly Email newsletter designed to keep subscribers up-to-date on salient news and events at IRI, Inc. Past editions are archived here.

To remove or add an Email address in future CoSORT Journal mailings, please email news@iri.com. To contact an IRI agent, click here, call 1-800-333-SORT, or email info@iri.com.

Next CoSORT Version: 8.2.3

CoSORT 8.2.3 is the latest update in the V8 release sequence for UNIX and Windows platforms. v8.2.3 is planned for general release by the quarter's end.

New features in CoSORT 8.2.3 (since 8.2.2) include:

  • Micro Focus ISAM (Index) File Handling
    Collation and conversion of 10 sub-types
  • Delimited Field Precision
    Easier numeric specification in floating fields
  • Windows File <Space> Name Support
    Tilde abbreviations are now just optional 
  • Date Interval Calculation
    SortCL measures days between datestamps 
  • mvs2scl Enhancements
    Multiple output files, more INREC support
More Linux Platforms Supported

Fedora Core, Asianux, Gentoo

In addition to CoSORT's x86 and Itanium platform support for Debian, Mandrake, Red Hat Enterprise Linux (RHEL), SuSE (Novell SLES) Linux, and TurboLinux, CoSORT now also  runs on Fedora Core 4 (Red Hat), Asianux, and Gentoo (Linux derivates).

CoSORT also runs on FreeBSD, OS/X, Solaris 10 and Linux for IBM's eServer x, i, p and zSeries, plus of course, all other current, commercially-available UNIX and Windows platforms. Click here for a platform list.

Tech Tip: WORK_AREA Allocation

Maximize Efficiency with this RC Setting

One of the most important -- and often overlooked  -- settings in the cosortrc file (UNIX) or registry (Windows) is for WORK_AREA(s).

Whenever you are sorting more data than will fit in memory* the CoSORT engine will send overflow data to one or more designated WORK_AREA directories.  This outside-of-memory condition triggers an external sort, versus an internal (in-memory) sort. 

Ideally, your system administrator or product installer will configure this and other tuning controls correctly. Unfortunately, many users keep this parameter set to the default ./ (current working) directory, which is rarely optimal in a multi-disk, large-sort environment. It is therefore beneficial to know about WORK_AREAs and how to (re)specify them properly.

Because most high-volume sorts become external and overflow to disk, it is important that you have sufficient disk space to hold the overflow. The CoSORT user must also have access and permission to write to, and read from, the WORK_AREA(s). The speed of the overflow disks can be critical too, so for that reason, do not specify remote (LAN) storage.

IRI also makes other recommendations about the number and location of WORK_AREAs at CoSORT installation -- on-screen, in the installation guide, in the CoSORT manual's Appendix (Sec D), and in the private support area of the CoSORT web site. You can also
obtain specific recommendations on (re)assigning temporary work directories for large sorts from an IRI engineer. Email your questions to support@iri.com.

* When the total input file size(s) exceed the lesser of the CoSORT user's ULIMIT DATA value or the cosortrc MEMORY_PERTHREAD_MAX(01) value), you are no longer performing an in-memory sort; the excess data must be cached in temporary files.


Copyright © 2005 Innovative Routines International (IRI), Inc.
2194 Highway A1A, Suite 303, Melbourne, FL 32937-4932. All rights reserved in the USA and in other jurisdictions.

CoSORT
, SortCL, Permitas, and Rowgen are trademarks of IRI. FACT is a trademark of CoSORT Korea, Ltd. Other product or brand names mentioned herein may be (registered) trademarks or copyrights of their respective owners.