Thursday, January 10, 2013

MAPPING PARAMETERS



Mapping parameters and variables represent values in mappings and mapplets.
When we use a mapping parameter or variable in a mapping, first we declare the mapping parameter or variable for use in each mapplet or mapping. Then, we define a value for the mapping parameter or variable before we run the session.

MAPPING PARAMETERS
  • A mapping parameter represents a constant value that we can define before running a session.
  • A mapping parameter retains the same value throughout the entire session.
Example: When we want to extract records of a particular month during ETL process, we will create a Mapping Parameter of data type and use it in query to compare it with the timestamp field in SQL override.
  • After we create a parameter, it appears in the Expression Editor.
  • We can then use the parameter in any expression in the mapplet or mapping.
  • We can also use parameters in a source qualifier filter, user-defined join, or extract override, and in the Expression Editor of reusable transformations.
MAPPING VARIABLES
  • Unlike mapping parameters, mapping variables are values that can change between sessions.
  • The Integration Service saves the latest value of a mapping variable to the repository at the end of each successful session.
  • We can override a saved value with the parameter file.
  • We can also clear all saved values for the session in the Workflow Manager.
We might use a mapping variable to perform an incremental read of the source. For example, we have a source table containing time stamped transactions and we want to evaluate the transactions on a daily basis. Instead of manually entering a session override to filter source data each time we run the session, we can create a mapping variable, $$IncludeDateTime. In the source qualifier, create a filter to read only rows whose transaction date equals $$IncludeDateTime, such as:
                    TIMESTAMP = $$IncludeDateTime
In the mapping, use a variable function to set the variable value to increment one day each time the session runs. If we set the initial value of $$IncludeDateTime to 8/1/2004, the first time the Integration Service runs the session, it reads only rows dated 8/1/2004. During the session, the Integration Service sets $$IncludeDateTime to 8/2/2004. It saves 8/2/2004 to the repository at the end of the session. The next time it runs the session, it reads only rows from August 2, 2004.
Used in following transformations:
  • Expression
  • Filter
  • Router
  • Update Strategy
Initial and Default Value:
When we declare a mapping parameter or variable in a mapping or a mapplet, we can enter an initial value. When the Integration Service needs an initial value, and we did not declare an initial value for the parameter or variable, the Integration Service uses a default value based on the data type of the parameter or variable.
Data ->Default Value
Numeric ->0
String ->Empty String
Date time ->1/1/1
Variable Values: Start value and current value of a mapping variable
Start Value:
The start value is the value of the variable at the start of the session. The Integration Service looks for the start value in the following order:
  1. Value in parameter file
  2. Value saved in the repository
  3. Initial value
  4. Default value
Current Value:
The current value is the value of the variable as the session progresses. When a session starts, the current value of a variable is the same as the start value. The final current value for a variable is saved to the repository at the end of a successful session. When a session fails to complete, the Integration Service does not update the value of the variable in the repository.
Note: If a variable function is not used to calculate the current value of a mapping variable, the start value of the variable is saved to the repository.
Variable Data type and Aggregation Type When we declare a mapping variable in a mapping, we need to configure the Data type and aggregation type for the variable. The IS uses the aggregate type of a Mapping variable to determine the final current value of the mapping variable.
Aggregation types are:
  •  Count: Integer and small integer data types are valid only.
  •  Max: All transformation data types except binary data type are valid.
  •  Min: All transformation data types except binary data type are valid.
Variable Functions
Variable functions determine how the Integration Service calculates the current value of a mapping variable in a pipeline.
SetMaxVariable: Sets the variable to the maximum value of a group of values. It ignores rows marked for update, delete, or reject. Aggregation type set to Max.
SetMinVariable: Sets the variable to the minimum value of a group of values. It ignores rows marked for update, delete, or reject. Aggregation type set to Min.
SetCountVariable: Increments the variable value by one. It adds one to the variable value when a row is marked for insertion, and subtracts one when the row is Marked for deletion. It ignores rows marked for update or reject. Aggregation type set to Count.
SetVariable: Sets the variable to the configured value. At the end of a session, it compares the final current value of the variable to the start value of the variable. Based on the aggregate type of the variable, it saves a final value to the repository.
Creating Mapping Parameters and Variables
  1. Open the folder where we want to create parameter or variable.
  2. In the Mapping Designer, click Mappings > Parameters and Variables. -or- In the Mapplet Designer, click Mapplet > Parameters and Variables.
  3. Click the add button.
  4. Enter name. Do not remove $$ from name.
  5. Select Type and Data type. Select Aggregation type for mapping variables.
  6. Give Initial Value. Click ok.
Example: Use of Mapping of Mapping Parameters and Variables
  • EMP will be source table.
  • Create a target table MP_MV_EXAMPLE having columns: EMPNO, ENAME, DEPTNO, TOTAL_SAL, MAX_VAR, MIN_VAR, COUNT_VAR and SET_VAR.
  • TOTAL_SAL = SAL+ COMM + $$BONUS (Bonus is mapping parameter that changes every month)
  • SET_VAR: We will be added one month to the HIREDATE of every employee.
  • Create shortcuts as necessary.
Creating Mapping
  1. Open folder where we want to create the mapping.
  2. Click Tools -> Mapping Designer.
  3. Click Mapping-> Create-> Give name. Ex: m_mp_mv_example
  4. Drag EMP and target table.
  5. Transformation -> Create -> Select Expression for list -> Create –>  Done.
  6. Drag EMPNO, ENAME, HIREDATE, SAL, COMM and DEPTNO to Expression.
  7. Create Parameter $$Bonus and Give initial value as 200.
  8. Create variable $$var_max of MAX aggregation type and initial value 1500.
  9. Create variable $$var_min of MIN aggregation type and initial value 1500.
  10. Create variable $$var_count of COUNT aggregation type and initial value 0. COUNT is visible when datatype is INT or SMALLINT.
  11. Create variable $$var_set of MAX aggregation type.

12. Create 5 output ports out_ TOTAL_SAL, out_MAX_VAR, out_MIN_VAR,
out_COUNT_VAR and out_SET_VAR.
13. Open expression editor for TOTAL_SAL. Do the same as we did earlier for SAL+ COMM. To add $$BONUS to it, select variable tab and select the parameter from mapping parameter. SAL + COMM + $$Bonus
14. Open Expression editor for out_max_var.
15. Select the variable function SETMAXVARIABLE from left side pane. Select
$$var_max from variable tab and SAL from ports tab as shown below.SETMAXVARIABLE($$var_max,SAL)

17. Open Expression editor for out_min_var and write the following expression:
SETMINVARIABLE($$var_min,SAL). Validate the expression.
18. Open Expression editor for out_count_var and write the following expression:
SETCOUNTVARIABLE($$var_count). Validate the expression.
19. Open Expression editor for out_set_var and write the following expression:
SETVARIABLE($$var_set,ADD_TO_DATE(HIREDATE,'MM',1)). Validate.
20. Click OK. Expression Transformation below:

21. Link all ports from expression to target and Validate Mapping and Save it.
22. See mapping picture on next page.


PARAMETER FILE
  • A parameter file is a list of parameters and associated values for a workflow, worklet, or session.
  • Parameter files provide flexibility to change these variables each time we run a workflow or session.
  • We can create multiple parameter files and change the file we use for a session or workflow. We can create a parameter file using a text editor such as WordPad or Notepad.
  • Enter the parameter file name and directory in the workflow or session properties.
A parameter file contains the following types of parameters and variables:
  • Workflow variable: References values and records information in a workflow.
  • Worklet variable: References values and records information in a worklet. Use predefined worklet variables in a parent workflow, but we cannot use workflow variables from the parent workflow in a worklet.
  •  Session parameter: Defines a value that can change from session to session, such as a database connection or file name.
  • Mapping parameter and Mapping variable
USING A PARAMETER FILE

Parameter files contain several sections preceded by a heading. The heading identifies the Integration Service, Integration Service process, workflow, worklet, or session to which we want to assign parameters or variables.
  • Make session and workflow.
  • Give connection information for source and target table.
  • Run workflow and see result.
Sample Parameter File for Our example:

In the parameter file, folder and session names are case sensitive.
Create a text file in notepad with name Para_File.txt
[Practice.ST:s_m_MP_MV_Example]
$$Bonus=1000
$$var_max=500
$$var_min=1200
$$var_count=0

CONFIGURING PARAMTER FILE :

We can specify the parameter file name and directory in the workflow or session properties.
To enter a parameter file in the workflow properties:
1. Open a Workflow in the Workflow Manager.
2. Click Workflows > Edit.
3. Click the Properties tab.
4. Enter the parameter directory and name in the Parameter Filename field.
5. Click OK.
To enter a parameter file in the session properties:
1. Open a session in the Workflow Manager.
2. Click the Properties tab and open the General Options settings.
3. Enter the parameter directory and name in the Parameter Filename field.
4. Example: D:\Files\Para_File.txt or $PMSourceFileDir\Para_File.txt
5. Click OK.


No comments:

Post a Comment

Thank you :
- kareem