Skip Headers
Oracle® Warehouse Builder User's Guide
11g Release 1 (11.1)

Part Number B31278-01
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

10 Introducing Oracle Warehouse Builder Transformations

One of the main functions of an Extract, Transformation, and Loading (ETL) tool is to transform data. Oracle Warehouse Builder provides several methods of transforming data. This chapter discusses transformations and describes how to create custom transformation using Warehouse Builder. It also describes how to import transformation definitions.

This chapter contains the following topics:

About Transforming Data Using Warehouse Builder

Warehouse Builder provides an intuitive user interface that enables you to define transformations required for your source data. Use one of the following methods to transform source data.

Benefits of Using Warehouse Builder for Transforming Data

Warehouse Builder enables you to reuse PL/SQL as well as to write your own PL/SQL transformations. To enable faster development of warehousing solutions, Warehouse Builder provides custom transformations written in PL/SQL. These custom transformations can be used in Warehouse Builder mappings.

Because SQL and PL/SQL are versatile and proven languages widely used by many information professionals, the time and expense of developing an alternative transformation language is eliminated by using Warehouse Builder. With Warehouse Builder, you can create solutions using existing knowledge and a proven, open, and standard technology.

All major relational database management systems support SQL and all programs written in SQL can be moved from one database to another with very little modification.This means that all the SQL knowledge in your organization is fully portable to Warehouse Builder. Warehouse Builder enables you to import and maintain any existing complex custom code.

About Transformations

Transformations are PL/SQL functions, procedures, packages, and types that enable you to transform data. You use transformations when designing mappings and process flows that define ETL processes.

Transformations are stored in the Warehouse Builder workspace and can be used in the project in which they are defined.

Transformation packages are deployed at the package level but executed at the transformation level.

Types of Transformations

Transformations, in Warehouse Builder, can be categorized as follows:

The following sections provide more details about these types of transformations.

Predefined Transformations

Warehouse Builder provides a set of predefined transformations that enable you to perform common transformation operations. These predefined transformations are part of the Oracle Library that consists of built-in and seeded functions and procedures. You can directly use these predefined transformations to transform your data. For more information on the Oracle Library, see "Types of Transformation Libraries".

Predefined transformations are organized into the following categories:

  • Administration

  • Character

  • Control Center

  • Conversion

  • Date

  • Numeric

  • OLAP

  • Others

  • SYS

  • Spatial

  • Streams

  • XML

For more information about the transformations that belong to each category, see "Transformations" in the Warehouse Builder Online Help.

Custom Transformations

A custom transformation is one this is created by the user. Custom transformations can use predefined transformations as part of their definition.

Custom transformations contains the following categories:

  • Functions: The Functions category contains standalone functions. This category is available under the Custom node of the Public Transformations node in the Global Explorer. It is also created automatically under the Transformations node of every Oracle module in the Project Explorer.

    Functions can be defined by the user or imported from a database. A function transformation takes 0-n input parameters and produces a result value.

  • Procedures: The Procedures category contains any standalone procedures used as transformations. This category is available under the Custom node of the Public Transformations node in the Global Explorer. It is also automatically created under the Transformations node of each Oracle module in the Global Explorer.

    Procedures can be defined by the user or imported from a database. A procedure transformation takes 0-n input parameters and produces 0-n output parameters.

  • Packages: The Packages category contains packages, which in turn contain functions, procedures, and PL/SQL types. This category is available under the Custom node of the Public Transformations node in the Global Explorer. It is also automatically created under the Transformations node of each Oracle module in the Global Explorer.

    PL/SQL packages can be created or imported in Warehouse Builder. The package body may be modified. The package header, which is the signature for the function or procedure, cannot be modified.

  • PL/SQL Types: The PL/SQL Types category contains any standalone PL/SQL types. This includes PL/SQL record types, REF cursor types, and nested table types. The PL/SQL Types category is automatically created in each package that you define using the Packages node in the Transformations node of the Project Explorer. It is also available under every package that you define in the following path of the Global Explorer: Public Transformations -> Custom -> Packages.

For more information about creating custom transformations, see "Defining Custom Transformations".

In addition to the above categories, you can also import PL/SQL packages. Although you can modify the package body of an imported package, you cannot modify the package header, which is the signature for the function or procedure. For more information on importing PL/SQL packages, see "Importing PL/SQL".

About Transformation Libraries

A transformation library consists of a set of reusable transformations. Each time you create a repository, Warehouse Builder creates a Transformation Library containing transformation operations for that project. This library contains the standard Oracle Library and an additional library for each Oracle module defined within the project.

Transformation libraries are available under the Public Transformations node of the Global Explorer in the Design Center.

Types of Transformation Libraries

Transformation libraries can be categorized as follows:

  • Oracle Library

    This is a collection of predefined functions from which you can define procedures for your Global Shared Library. The Oracle Library is contained in the Global Explorer. Expand the Pre-Defined node under the Public Transformations node. Each category of predefined transformations is represented by a separate node. Expand the node for a category to view the predefined transformations in that category. For example, expand the Character node to view the predefined character transformations contained in the Oracle Library.

  • Global Shared Library

    This is a collection of reusable transformations created by the user. These transformations are categorized as functions, procedures, and packages defined within your workspace.

    The transformations in the Global Shared Library are available under the Custom node of the Public Transformations node. Any transformation that you create under this node is available across all projects in the workspace. For information on creating transformations in the Global Shared Library, see "Defining Custom Transformations".

    When you deploy a transformation defined in the Global Shared Library, the transformation is deployed to the location that is associated with the default control center.

Accessing Transformation Libraries

Since transformations can be used at different points in the ETL process, Warehouse Builder enables you to access transformation libraries from different points in the Design Center.

You can access the transformation libraries using the following:

  • Expression Builder

    While creating mappings, you may need to create expressions to transform your source data. The Expression Builder interface enables you to create the expressions required to transform data. Since these expressions can include transformations, Warehouse Builder enables you to access transformation libraries from the Expression Builder.

    Transformation libraries are available under the Transformations tab of the Expression Builder. The Private node under TRANSFORMLIBS contains transformations that are available only in the current project. These transformations are created under the Transformations node of the Oracle module. The Public node contains the custom transformations from the Global Shared Library and the predefined transformations from the Oracle Library.

  • Add Transformation Operator Dialog Box

    The Transformation operator in the Mapping Editor enables you to add transformations, both from the Oracle library and the Global Shared Library, to a mapping. You can use this operator to transform data as part of the mapping.

  • Create Function Wizard, Create Procedure Wizard, Edit Function Dialog Box, or Edit Procedure Dialog Box

    The Implementation page on the these wizards or the Implementation tab of these editors enable you to specify the PL/SQL code that is part of the function or procedure body. You can use transformations in the PL/SQL code.

Defining Custom Transformations

Custom transformations include procedures, functions, and packages. Warehouse Builder provides wizards to create each type of custom transformation. Custom transformations can belong to the Global Shared Library or to a module in a project.

Custom Transformations in the Global Shared Library

Custom transformations that are part of the Global Shared Library can be used across all projects of the workspace in which they are defined. For example, you create a function called ADD_EMPL in the Global Shared Library of the workspace REP_OWNER. This procedure can be used across all the projects in REP_OWNER.

Use the Custom node of the Public Transformations node in the Global Explorer to define custom transformations that can be used across all projects in the workspace. Figure 10-1 displays the Global Explorer used to create such transformations.

To create a custom transformation in the Global Shared Library:

  1. From the Global Explorer, expand the Public Transformations node and then the Custom node.

    Warehouse Builder displays the type of transformations that you can create. This includes functions, procedures, and packages. Note that PL/SQL types can be created only as part of a package.

  2. Right-click the type of transformation you want to define and select New.

    For example, to create a function, right-click Functions and select New. To create PL/SQL types, expand the package in which you want to create the PL/SQL type, right-click PL/SQL Types and select New.

  3. For functions and procedures, Warehouse Builder displays the Welcome page of the Create Function Wizard or the Create Procedure wizard respectively. For PL/SQL types, Warehouse Builder displays the Welcome page of the Create PL/SQL Type Wizard.

    Click Next to proceed. See "Defining Functions and Procedures" for more information about the other pages in the wizard. For more information about creating PL/SQL types, see "Defining PL/SQL Types".

    For packages, Warehouse Builder displays the Create Transformation Library dialog box. Provide a name and an optional description for the package and click OK. The new package is added to the Packages node. You can subsequently create procedures, functions, or PL/SQL types that belong to this package.

Custom Transformations in a Project

Sometimes, you may need to define custom transformations that are required only in the current module or project. In this case, you can define custom transformations in an Oracle module of a project. Such custom transformations are accessible from all the projects in the current workspace. For example, consider the workspace owner called REP_OWNER that contains two projects PROJECT1 and PROJECT2. In the Oracle module called SALES of PROJECT1, you define a procedure called CALC_SAL. This procedure can be used in all modules belonging to PROJECT1, but is not accessible in PROJECT2.

Figure 10-1 displays the Project Explorer from which you can create custom transformations that are accessible within the project in which they are defined.

Figure 10-1 Creating Custom Transformations in an Oracle Module

Description of Figure 10-1 follows
Description of "Figure 10-1 Creating Custom Transformations in an Oracle Module"

To define a custom transformation in an Oracle module:

  1. From the Project Explorer, expand the Oracle warehouse module node and then the Transformations node.

  2. Right-click the type of transformation you want to create and select New.

    For example, to create a package, right-click Packages and select New. To create PL/SQL types, expand the package node under which you want to create the type, right-click PL/SQL Types and select New.

    For functions or procedures, Warehouse Builder displays the Welcome page of the Create Function Wizard or the Create Procedure Wizard respectively. For PL/SQL Types, the Welcome page of the Create PL/SQL Type Wizard is displayed. Click Next to proceed.

    See "Defining Functions and Procedures" for information about the remaining wizard pages. For more information about creating PL/SQL types, see "Defining PL/SQL Types".

    For packages, Warehouse Builder opens the Create Transformation Library dialog box. Provide a name and an optional description for the package and click OK. The package is added under the Packages node. You can subsequently create procedures, functions, or PL/SQL types that belong to this package.

Defining Functions and Procedures

Use the following pages of the Create Function Wizard or Create Procedure Wizard to define a function or procedure.

Name and Description Page

You use the Name and Description page to describe the custom transformation. Specify the following details on this page:

  • Name: Represents the name of the custom transformation. For more information about naming conventions, see "Naming Conventions for Data Objects".

  • Description: Represents the description of the custom transformation. This is an optional field.

  • Return Type: Represents the data type of the value returned by the function. You select a return type from the available options in the list. This field is applicable only for functions.

Parameters Page

Use the Parameters page to define the parameters, both input and output, of the transformation. Specify the following details for each parameter:

  • Name: Enter the name of the parameter.

  • Type: Select the data type of the parameter from the list.

  • I/O: Select the type of parameter. The options available are Input, Output, and Input/Output.

  • Required: Select Yes to indicate that a parameter is mandatory and No to indicate that it is not mandatory.

  • Default Value: Enter the default value for the parameter. The default value is used when you do not specify a value for the parameter at the time of executing the function or procedure.

Implementation Page

Use the Implementation page to specify the implementation details, such as the code, of the transformation. To specify the code used to implement the function or procedure, click Code Editor. Warehouse Builder displays the Code Editor window. This editor contains two panels. The upper panel displays the code and the lower panel displays the function signature and messages.

When you create a function, the following additional options are displayed:

  • Function is deterministic: This hint helps to avoid redundant function calls. If a stored function was called previously with the same arguments, the previous result can be used. The function result should not depend on the state of session variables or schema objects. Otherwise, results might vary across calls. Only DETERMINISTIC functions can be called from a function-based index or a materialized view that has query-rewrite enabled.

  • Enable function for parallel execution: This option declares that a stored function can be used safely in the child sessions of parallel DML evaluations. The state of a main (logon) session is never shared with child sessions. Each child session has its own state, which is initialized when the session begins. The function result should not depend on the state of session (static) variables. Otherwise, results might vary across sessions.

Summary Page

The Summary page provides a summary of the options that you chose on the previous wizard pages. Click Finish to complete defining the function or procedure. Warehouse Builder creates the function or procedure and displays it under the corresponding folder under the Public Transformations and Custom nodes in the Global Explorer.

Defining PL/SQL Types

Use the Create PL/SQL Type Wizard to create PL/SQL types. PL/SQL types must be defined within a package and they cannot exist independently.

About PL/SQL Types

PL/SQL types enable you to create collection types, record types, and REF cursor types in Warehouse Builder. You use PL/SQL types as parameters in subprograms or as return types for functions. Using PL/SQL types as parameters to subprograms enables you to process arbitrary number of elements. Use collection types to move data into and out of database tables using bulk SQL. For more information about PL/SQL types, see Oracle Database PL/SQL Language Reference.

Warehouse Builder enables you to create the following PL/SQL types:

  • PL/SQL Record types

    Record types enable you to define records in a package. A record is a composite data structure that contains multiple fields. Use records to hold related items and pass them to subprograms using a single parameter.

    For example, an EMPLOYEE record can contain details related to an employee such as ID, first name, last name, address, date of birth, date of joining, and salary. You can create a record type based on the EMPLOYEE record and use this record type to pass employee data between subprograms.

  • REF Cursor types

    REF cursor types enable you to define REF cursors within a package. REF cursors are not bound to a single query and can point to different result sets. Use REF cursors when you want to perform a query in one subprogram and process the results in another subprogram. REF cursors also enable you to pass query result sets between PL/SQL stored subprograms and various clients such as an OCI client or an Oracle Forms application.

    REF cursors are available to all PL/SQL clients. For example, you can declare a REF cursor in a PL/SQL host environment such as an OCI or Pro*C program, then pass it as an input host variable (bind variable) to PL/SQL. Application development tools such as Oracle Forms, which have a PL/SQL engine, can use cursor variables entirely on the client side. Or, you can pass cursor variables back and forth between a client and the database server through remote procedure calls.

  • Nested Table types

    Use nested table types to define nested tables within a package. A nested table is an unordered set of elements, all of the same data type. They are similar to one-dimensional arrays with no declared number of elements. Nested tables enable you to model multidimensional arrays by creating a nested table whose elements are also tables.

    For example, you can create a nested table type that can hold an arbitrary number of employee IDs. This nested table type can then be passed as a parameter to a subprogram that processes only the employee records contained in the nested table type.

Usage Scenario for PL/SQL Types

The SALES table stores the daily sales of an organization that has offices across the world. This table contains the sale ID, sale date, customer ID, product ID, amount sold, quantity sold, and currency in which the sale was made. Management wants to analyze global sales for a specified time period using a single currency, for example the US Dollar. Thus all sales values must be converted to US Dollar. Since the currency exchange rates can change every day, the sales amounts must be computed using the exchange rate of the sale currency on the sale date.

Solution Using PL/SQL Record Types 

Figure 10-2 displays the mapping that you use to obtain the sales amount in a specified currency using PL/SQL record types

Figure 10-2 PL/SQL Record Type in a Mapping

Description of Figure 10-2 follows
Description of "Figure 10-2 PL/SQL Record Type in a Mapping"

The mapping takes the individual sales data stored in different currencies, obtains the sales value in the specified currency, and loads this data into a target table. Use the following steps to create this mapping.

  1. In the Global Explorer, create a package. In this package, create a procedure called CURR_CONV_PROC.

    This procedure obtains the currency conversion values on each date in a specified time interval from a Web site. The input parameters of this procedure are the sales currency, the currency to which the sale value needs to be converted, and the time interval for which the currency conversion is required. This data is stored in a PL/SQL record type of type CURR_CONV_REC. This record type contains two attributes: date and conversion value.

    You create the PL/SQL record type as part of the package.

  2. Create a mapping that contains a Transformation operator. This operator is bound to the CURR_CONV_PROC procedure.

  3. Use a Mapping Input Parameter operator to provide values for the input parameters of the Transformation operator.

    The output group of the Transformation operator is a PL/SQL record type of type CURR_CONV_REC.

  4. Use an Expand Object operator to obtain the individual values stored in this record type and store these values in the table CURRENCY_TAB.

  5. Use an Aggregator operator to aggregate sales details for each order.

    The SALES table is a transactional table and stores data in normalized form. To obtain the aggregate sales for each order, use an Aggregator operator to aggregate sales data.

  6. Use a Joiner operator to join the aggregated sales details, which is the output of the Aggregator operator, with the data in the CURRENCY_TAB table. The sale date is used as the join condition.

  7. Use the Expression operator to multiply the sales amount with the currency exchange rate to get the total sales in the required currency. Load the converted sales data into the CONV_SALES_TAB table.

Creating PL/SQL Types

You can create PL/SQL types in the Project Explorer and Global Explorer of the Design Center. For more details about creating PL/SQL types, see "Defining PL/SQL Types".

Use the Create PL/SQL Types Wizard to create PL/SQL types. The wizard guides you through the following pages:

Name and Description Page

Use the Name and Description page to provide the name and an optional description for the PL/SQL type. Also use this page to select the type of PL/SQL type you want to create.

You can create any of the following PL/SQL types:

  • PL/SQL record type

  • REF cursor type

  • Nested table type

For more information about each PL/SQL type, see "About PL/SQL Types".

After specifying the name and selecting the type of PL/SQL type to create, click Next.

Attributes Page

Use the Attributes page to define the attributes of the PL/SQL record type. You specify attributes only for PL/SQL record types. A PL/SQL record must have at least one attribute.

For each attribute, define the following:

  • Name: The name of the attribute. The name should be unique within the record type.

  • Type: The data type of the attribute. Select the data type from the list.

  • Length: The length of the data type, for character data types.

  • Precision: The total number of digits allowed for the attribute, for numeric data types.

  • Scale: The total number of digits to the right of the decimal point, for numeric data types.

  • Seconds Precision: The number of digits in the fractional part of the datetime field. It can be a number between 0 and 9. The Seconds Precision is used only for TIMESTAMP data types.

Click Next to proceed to the next step.

Return Type Page

Use the Return Type page to select the return type of the PL/SQL type. You must specify a return type when you create REF cursors and nested tables.

To define REF cursors:

The return type for a REF cursor can only be a PL/SQL record type. If you know the name of the PL/SQL record type, you can search for it by typing the name in the Search For field and clicking Go.

The area below the Search For field displays the available PL/SQL types. These PL/SQL types are grouped under the two nodes: Public and Private. Expand the Public node to view the PL/SQL types that are part of the Oracle Shared Library. The types are grouped by package name. The Private node contains PL/SQL types that are created as part of a package in an Oracle module. Only PL/SQL types that belong to the current project are displayed. Each Oracle module is represented by a node. Within the module, the PL/SQL types are grouped by the package to which they belong.

To define nested tables:

For nested tables, the return type can be a scalar data type or a PL/SQL record type. Select one of the following options based on what the PL/SQL type returns:

  • Select a scalar type as return type

    This option enables you to create a PL/SQL type that returns a scalar type. Use the list to select the data type.

  • Select a PL/SQL record as return type

    This option enables you to create a PL/SQL type that returns a PL/SQL record type. If you know the name of the PL/SQL record type that is returned, type the name in the Search For field and click Go. The results of the search are displayed in the area below the option.

    You can also select the return type from the list of available types displayed. The area below this option contains two nodes: Public and Private. The Public node contains PL/SQL record types that are part of the Oracle Shared Library. The PL/SQL record types are grouped by the package to which they belong. The Private node contains the PL/SQL record types created as transformations in each Oracle module in the current project. These are grouped by module. Select the PL/SQL record type that the PL/SQL type returns.

Click Next to proceed with the creation of the PL/SQL type.

Summary Page

The Summary page displays the options that you have chosen on the wizard pages. Review the options. Click Back to modify any options. Click Finish to create the PL/SQL type.

Editing Custom Transformations

You can edit the definition of a custom transformation using the editors. Make sure you edit properties consistently. For example, if you change the name of a parameter, then you must also change its name in the implementation code.

Editing Function or Procedure Definitions

The Edit Function dialog box enables you to edit function definitions. To edit a procedure definition, use the Edit Procedure dialog box.

Use the following steps to edit functions, procedures, or packages:

  1. From the Project Explorer, expand the Oracle module in which the transformation is created. Then expand the Transformations node.

    To edit a transformation that is part of the Global Shared Library, from the Global Explorer, expand the Public Transformations node, and then the Custom node.

  2. Right-click the name of the function, procedure, or package you want to edit and select Open Editor.

    For functions or procedures, the Edit Function or Edit Procedure dialog box is displayed. Use the following tabs to edit the function or procedure definition:

    For packages, Warehouse Builder displays the Edit Transformation Library dialog box. You can only edit the name and description of the package. You can edit the functions and procedures contained within the package using the steps used to edit functions or packages.

Name Tab

Use the Name tab to edit the name and description of the function or procedure. For functions, you can also edit the return data type.

Parameters Tab

Use the Parameters tab to edit, add, or delete new parameters for a function or procedure. You can also edit and define the attributes of the parameters. The contents of the Parameters tab are the same as that of the Parameters page of the Create Transformation Wizard. For more information about the contents of this page, see "Parameters Page".

Implementation Tab

Use the Implementation tab to review the PL/SQL code for the function or procedure. Click Code Editor to edit the code. The contents of the Implementation tab are the same as that of the Implementation page of the Create Transformation Wizard. For more information on the contents of the Implementation page, see "Implementation Page".

Editing PL/SQL Types

The Edit PL/SQL Type dialog box enables you to edit the definition of a PL/SQL type. Use the following steps to edit a PL/SQL type:

  1. From the Project Explorer, expand the Oracle module that contains the PL/SQL type. Then expand the Transformations node.

    To edit a PL/SQL type stored in the Global Shared Library, expand the Public Transformations node in the Global Explorer, and then the Custom node.

  2. Expand the package that contains the PL/SQL type and then the PL/SQL Types node.

  3. Right-click the name of the PL/SQL type that you want to edit and select Open Editor.

    The Edit PL/SQL Type dialog box is displayed. Use the following tabs to edit the PL/SQL type:

Name Tab

The Name tab displays the name and the description of the PL/SQL type. Use this tab to edit the name or the description of the PL/SQL type.

To rename a PL/SQL type, select the name and enter the new name.

Attributes Tab

The Attributes tab displays details about the existing attributes of the PL/SQL record type. This tab is displayed for PL/SQL record types only. You can modify existing attributes, add new attributes, or delete attributes.

To add a new attribute, click the Name column of a blank row specify the details for the attribute. To delete an attribute, right-click the gray cell to the left the row that represents the attribute and select Delete.

Return Type Tab

Use the Return Type tab to modify the details of the return type of the PL/SQL type. For a REF cursor type, the return type must be a PL/SQL record. For a nested table, the return type can be a PL/SQL record type or a scalar data type.

Importing PL/SQL

Use the Import Metadata Wizard to import PL/SQL functions, procedures, and packages into a Warehouse Builder project. You can edit, save, and deploy the imported PL/SQL functions and procedures. You can also view and modify imported packages.

The following steps describe how to import PL/SQL packages from other sources into Warehouse Builder.

To import a PL/SQL function, procedure, or package:

  1. From the Project Explorer, expand the project node and then Databases node.

  2. Right-click an Oracle module node and select Import.

    Warehouse Builder displays the Welcome page of the Import Metadata Wizard.

  3. Click Next.

  4. Select PL/SQL Transformation in the Object Type field of the Filter Information page.

  5. Click Next.

    The Import Metadata Wizard displays the Object Selection page.

  6. Select a function, procedure, or package from the Available Objects list. Move the objects to the Selected Objects list by clicking the right arrow to move a single object or the Move All button to move multiple objects.

  7. Click Next.

    The Import Metadata Wizard displays the Summary and Import page.

  8. Verify the import information. Click Back to revise your selections.

  9. Click Finish to import the selected PL/SQL transformations.

    Warehouse Builder displays the Import Results page.

  10. Click OK proceed with the import. Click Undo to cancel the import process.

    The imported PL/SQL information appears under the Transformations node of the Oracle module into which you imported the data.

Restrictions on Using Imported PL/SQL

The following restrictions apply to the usage of imported PL/SQL:

  • You cannot edit imported PL/SQL packages.

  • Wrapped PL/SQL objects are not readable.

  • You can edit the imported package body but not the imported package specification.