User manual BUSINESS OBJECTS DATA INTEGRATOR 11.7.0.0 FOR WINDOWS RELEASE SUMMARY

[. . . ] 9 Performance Optimization Guide enhancements . 9 Trusted information--Data Quality XI integration . 10 Adapter interface installation enhancements . 10 Command line options to export to XML . [. . . ] You can store the lookup table in a persistent cache table which Data Integrator quickly pages into memory when each data flow executes. For more information, see Chapter 6, "Using Caches, " in the Data Integrator Performance Optimization Guide . Distributed data flows Data Integrator now provides the ability to distribute the workload across multiple CPUs in a grid. Data Integrator provides capabilities to distribute CPU-intensive and memoryintensive data processing work (such as join, grouping, table comparison and lookups) across multiple CPUs and computers. This work distribution provides the following potential benefits: · · Better memory management by taking advantage of more CPU resources and physical memory Better job performance and scalability by using concurrent sub data flow execution to take advantage of grid computing You can create sub data flows so that Data Integrator does not need to process the entire data flow in memory at one time. You can also distribute the sub data flows to different job servers within a server group to use additional memory and CPU resources. For more information, see Chapter 7, "Distributing Data Flow Execution, " in the Data Integrator Performance Optimization Guide . Load balancing enhancements Data Integrator now provides the ability to distribute the workload across multiple servers in a grid. 8 Data Integrator Release Summary Extreme scalability You can distribute the execution of a job or a part of a job across multiple Job Servers within a Server Group to better balance resource-intensive operations. You can specify the following distribution levels when you execute a job: · · · Job level--A job can execute on an available Job Server. Data flow level--Each data flow within a job can execute on an available Job Server. Sub data flow level--A resource-intensive operation (such as a sort, table comparison, or table lookup) within a data flow can execute on an available Job Server. For more information, see "Using grid computing to distribute data flows execution" on page 112 of the Data Integrator Performance Optimization Guide . Parallel join enhancements Data Integrator provides an additional parallel hash join that you can use to improve joins of large volumes of data. In addition, you can distribute the parallel join execution over multiple sub data flows. For more information, see "Degree of parallelism and joins" on page 86 of the Data Integrator Performance Optimization Guide . Performance Optimization Guide enhancements This version of Data Integrator provides a reorganized and enhanced Performance Optimization Guide to help you measure and tune performance of your ETL jobs. The new organization provides examples of tools to measure performance and determine performance bottlenecks, and it suggests tuning methods that subsequent chapters describe in detail. The enhancements include scenarios and examples that demonstrate usage of the new Extreme Scalability features in Data Integrator. For more information, see "Command line login to the Designer" on page 25 of the Data Integrator Advanced Development and Migration Guide. 10 Data Integrator Release Summary Maximum productivity Excel workbook as a source You can now import an Excel workbook directly without using ODBC. You can import the schema from a named range defined in the workbook, a custom range in a worksheet (for example A1:C10), or all fields. For details, see "Excel workbook format" on page 108 of the Data Integrator Reference Guide. Function enhancements This release includes several new functions to the list of available built-in functions in Data Integrator. These functions make it easier for Data Integrator developers to create more complex calculations and also adds to Data Integrator analytical capabilities. New functions include mathematical functions (sqrt, log, power), aggregation functions (count_distinct), string functions (asc, chr) and many more. A new category of functions will make it possible to compare values between different rows in a table, making it easy to calculate trends (Previous_Row_Value) and detect changes in values. Further, several functions were improved based on customer feedback, for example the week_in_year function can now return week numbers following the ISO standard which is dominantly used in Europe. For more information, see "Descriptions of built-in functions" on page 401 of the Data Integrator Reference Guide History preserving transform enhancements The History Preserving transform now contains an extra option to set the Valid to column of your old record. Now you can set the Valid to result for the same day or the previous day. [. . . ] You can view performance statistics in both graphical and tabular formats. When you execute jobs with the Collect statistics for monitoring option, you can view memory usage statistics. For more information, see "Reading the Performance Monitor for execution statistics" on page 29 of the Data Integrator Performance Optimization Guide . Self-tuning Data Integrator uses cache statistics to automatically determine the optimal cache type for subsequent job executions. For more information, see "Using statistics for cache self-tuning" on page 68 of the Data Integrator Performance Optimization Guide . Teradata UPSERT functionality The purpose of the Teradata UPSERT operation is to update a row, but if no row matches the update, the row is inserted. [. . . ]