RowGen Generates Safe Test Data in Real File, Table and Report Formats CoSort Products

Home » Products » RowGen

RowGen - Safe Test Data in Real Formats

Do you need fast access to large volumes of intelligent test data? RowGen v2 is a complete test data solution for creating safe, realistic, and referentially correct databases, files, and reports. RowGen uses your data models and metadata -- plus parallel data transformation, formatting, and generation functions -- to build multiple test data targets in multiple formats along with an audit trail for versioning and compliance.

Summary

RowGen v2 creates massive amounts of accurate, relational test data in the same table, file, and report layouts you use in production. RowGen leverages your data model and metadata formats so you can quickly test database and ETL operations, develop better applications, benchmark different platforms, and outsource application formats without exposing personally-identifying information.

Features/Benefits

• High performance in volume
• Easy-to-use GUI and CLI
• Preserves referential integrity
• Uses DDL, DDF investments
• CoSort inside speeds DB loads
• Custom ranges improve apps
• Perfect file/report formatting
• Reduces complexity and risks

Introduction

How can you build and test databases and applications without the right data types, record formats, value ranges, file sizes, or inter-table relationships you require? How can you safely share your reports or file formats with co-workers or solution providers? How can you create differently-structured files for building and stress-testing new applications, or for benchmarking hardware and software? How can you generate test data with complex business rules and massive volume requirements?

Until now, you could either forego adequate testing and make inaccurate suppositions and extrapolations, or, you could spend a lot of time writing custom 3GL or shell programs to build specific test sets with the layouts you want. Then you would have to repeat the process every time you needed different record or file formats, or different data types or value ranges. Alternatively, you can scour a low-end market of test data tools that cannot match RowGen's speed in volume or its functional versatility.

IRI's RowGen software was created by data modeling, integration, and processing experts to save time and energy in the creation of perfect, safe test data sets in modifiable table, file, and report formats. With RowGen, you are immediately ready for development, benchmarking, and outsourcing.

RowGen combines a convenient GUI with a powerful 4GL, giving you the freedom to create, transform, and format your test in table, file, and report layouts that are realistic in their appearance, content, relationships, and volume. RowGen metadata is saved in explicit text scripts; a central repository of data definitions and manipulations that you can easily modify and re-use. RowGen can also produce an audit trail in XML format -- which includes all job metadata -- to help you track project lineage and verify compliance with privacy regulations.

Test Data for Everything and Everyone

RowGen is a complete solution for test data creation. Its core strengths are high data volumes and functional versatility. Consider how you can leverage these special product attributes to your advantage:

Multi-Source	Mix and match real data with randomly generated data, using multiple selection criteria and > 120 different data types
Multi-Target	Create many test (output) files simultaneously, and use them in any database or application
Model-Driven	Leverage your existing .ddl files to populate structurally and referentially correct test databases and data warehouses
Multi-Format	Build test data directly into many different file formats, including: COBOL, CSV, LDIF, web log, XML, structured reports, variable blocked / sequential, etc.
Multi-Function	Don't just create data, manipulate and format it in the same pass with: sorting; aggregation; custom field filters, layouts, and transforms; page/report formatting; EVs, etc.
Multi-Level	Build test data directly into detail and summary (drill-down and roll-up) report formats that include test data value aggregations
Multi-Platform	Run RowGen on any current Unix, Linux, and Windows platform to use the power of your hardware and test data where you need it
Multi-Metadata	Use the data layouts already defined in CSV headers, COBOL copybooks, SQL*Loader control files, ELF web logs, and any other application layout MIMB supports
Multi-Purpose	Use RowGen for application development and stress-testing, DB population, ETL prototyping, outsourcing, benchmarking, etc.
Multi-Partner	Benefit from RowGen's technical alliances with the leading hardware, database, data integration (ETL), BI application, and metadata vendors

Test Database Targets

RowGen users who want to leverage their data models to create test tables can employ a Java front-end developed by RapidACE, LLC. The "RA-RowGen" GUI will read your .DDL files and use their layout information and primary-foreign key dependencies to create and run the RowGen control language (.RCL) job scripts necessary to build referentially correct test data for:

Oracle
Sybase
SQL Server
Teradata
DB2 UDB
Packaged Apps (e.g. PeopleSoft, Informatica)
Interdependent Flat Files

The RA-RowGen GUI allows you to create drag-and-drop categories with generic settings to handle multiple classes of tables from a DDL, and then apply the script creation engine to the category. RA-RowGen also contains a scripting wizard to create test data in custom flat file and report structures. Through the development of the RA-RowGen GUI, RapidACE has also cultivated an expertise in the data definitions and manipulation syntax of the CoSort and RowGen products. It uses these technologies in conjunction with your data models to help you transform large data sets (CoSort), or generate safe test data sets that reflect your tables and respect their referential constraints (RowGen).

Test Column (Field) Targets

Within the test records RowGen generates, you can define any number, size, format (data type), position, and value range for your fields. RowGen can randomly generate the data for each specified field in the type you declare, e.g.:

ASCII & EBCDIC Characters
Numeric, Whole, Currency, IP Address
Alpha & EBCDIC Digits
RM and MF COBOL Numerics
Other Binary Numerics
US, Europe, ISO, Japan Timestamps
Unicode & Multi-byte

Or, you can randomly select field values (with weighted frequencies) from one or more real SET files or literal set value ranges. In the first case, your data elements are as random as possible. In the second, you safely create test data from production data* or stress ranges. Either way, with RowGen, you can combine randomization and realism to fine tune your test data.

Beyond raw data generation and real data selection, RowGen supports your business logic in creating "intelligent test data." RowGen supports field-level manipulations and masking, conditional value filters, and complex transformation routines that you can write in your own language and link in at runtime.

*Actual field elements from real data files. While you would use RowGen as a multi-purpose data generator, you cannot use it as a general-purpose processor of real data -- for that, ask the CoSort data transformation tool, SortCL.

Test File and Report Targets

RowGen gives you complete control over the generation of test set formats through an explicit scripting language you can modify in a text editor or the RA-RowGen GUI. The RowGen Control Language ("RCL") uses the same familiar 4GL reporting syntax of CoSort's sort control language ("SortCL") program to describe the precise layout, size, and content of the target file(s) you want -- down to the size, position, separator, and data type of each field element -- as well as a myriad of conditional selection and set file/value support (for range testing), plus the custom formatting features you need to generate reports. For more information on the 4GL behind RowGen, see: Products > CoSort > SortCL

Test files -- that you can create one or more at once -- can be in any of these formats:

ACUCOBOL Vision
CSV
ELF (W3C Extended Log Format)
LDIF (LDAP)
Line Sequential
Mainframe (Unisys) Variable Block
Micro Focus Variable Length (Small/Large)
Micro Focus I-SAM
Record Sequential (Text)
Variable Sequential
XML

RowGen will generate as many records per file, and as many files, as you specify in one job script and I/O pass! You can format each output file with customized field, record, and report layouts, and specify transformations like sorting, aggregation, and cross-calculation.

Test reports can contain header and footer records, detail and summary data, environment variable and special constants, and use condition logic for record selection and segmentation. RowGen includes the same report generating capability for your test data that CoSort has for real data (for a complete list, see Solutions > Business_Intelligence > Reporting_Functions2).

Test Your Applications

The RowGen test data definition file (.DDF) format is supported by the popular Meta Integration Model Bridge (MIMB) technology ("The Switzerland of Metadata"), so you can leverage the data layouts you already have in your applications (which MIMB supports). This means you can more easily populate your applications with test tables, files, and reports used in production -- and with even broader value ranges for stress testing.

RowGen also uses the same metadata and repository structures as the CoSort SortCL tool so, you can re-use your test file layouts in both RowGen and CoSort. Finally, the RowGen test data definition file (.DDF) format is also created by the Fast Extract (FACT) tool for Oracle and DB2, and by the included converters for COBOL copybooks, CSV file headers, SQL*Loader control files and ELF web logs.

You can also use RowGen's built-in data transformation functions (such as expressions and aggregation) to test the functionality of your applications. For example, you can compare the results of your program's summarization routine with the summary value that RowGen produced while creating the test data.

RowGen targets can be disk files, named or unnamed pipes (stdout), or one or more custom output procedures (that you write and link to RowGen). Direct the output to your database's load utility (pre-sorted!) or other applications for population, processing, or presentation. Again, your outputs can range from simple, one-field flat files to elaborately-formatted HTML reports complete with real-time system data.

RowGen Platform Availability

RowGen is available for all Unix, Linux, and Windows platforms. The RA-RowGen GUI requires a Java Runtime Environment (JRE), at or above version 1.5.

RowGen Licensing and Support

RowGen can be licensed on a periodic lease or perpetual-use basis to meet your application and testing requirements. Fees start in US 4 figures and include the RA-RowGen GUI from RapidACE, LLC to read your data models and build RowGen scripts automatically to populate massive RDBs with referentially correct test data.

Solution Links:

1-800-333-SORT
1-321-777-8889

What Beats Real Data?

RowGen'd data may be better than real data. Why? Because real data may:

not yet exist. RowGen allows you to work on any subsequent phase of a project before an earlier phase is completed, since you do not have to rely on the real data from an earlier project phase.
be confidential. RowGen allows you to realistically simulate and outsource real data and/or file formats which may not be available for compliance, security, or other reasons.
not be robust enough for testing and development. RowGen's simulated data can produce any potential range of data values and volumes so that you can stress your application at the limits of what's possible in the future.

CoSort (IRI, Inc.)