Mapper

Mapper is a component to connect components and set the stream, variable and component's property.

Here is the instruction about the fields of the input side and output side in the mapping window and how the mapper that received record set works.

Input/Output fields and mapper function

Here is the instruction about the input field shown in the left of the mapping window, the output field in the right side and the mapper function configured in the middle mapping area.

Input

Stream

Field

Mapper maps value by using the input stream's record value. In the left, there are fields of the input stream shown under the "stream" in the left.

When the input stream is a container, the number of the input record is the total of the all streams in the container. For example, when the CSV files which have 3 records, 5 records, 7 records seperately have been made into a container, the number of the input record is 15.

When there are many input streams, each stream will be mapped seperately. The number of the record is the biggest one in the input streams. Once the mapper maps 1 record, each input record's pointer will move 1 record. When the records are the fewest of all the streams, after the records have been processed, the field value will be null.

Record No.

Return the input record's imdex based 1.

The number of the records

Return the total number of the records.

The running times that map value to the output stream is as same as the number of the records of the input stream. When the number of the records of the input stream is 0, and the "When there is no input data, error" property is set as "Yes", an error will happen. When the property is set as "No", the stream will be mapped only once. In this case, the stream's each field value will be null. When the "Begin loop" property is set as "Yes", the loop will begin from the Mapper Component, and the runnning times will be as same as the number of the records. In this case, the stream which has 1 record will be output, and the running time will be as same as the number of the input records.

Stream variable

When the stream variable is added to the stream, the value can be referred to by mapper.

Component

The input component's property can be checked by mapper. (But some properties can't be referred to by mapper.)

Running times

The component's running times can be checked with the all loop processings between the start and the end of the requested Flow. When there is a loop inside the loop, it's not necessary to reset the running times for the inner loop.

Error message

When an error happens in a component, after the error is ignored or the processing order moves from the Error Processing Flow to the next component, you can get the error message by putting a component after the error component.

Variable type

You can refer to the mapper variable, Flow variable, external variable set, system variable from the input side.

Output

Stream

The number of the records of the output stream is as same as the mapping times. But, when the stream which has only one record such as Binary or Text, the input data of the records will be merged.

If the "Output directly" property is set as "Yes", the input stream will be the output stream. In this case, the output stream won't be mapped.

Component

You can set the property of the component following to the mapper by mapper.(But, some properties can't be set by mapper.)

Value can be inserted to the property of the component of the output side only once regardless of the number of the records of the input stream. Namely, when the field of the input stream that has 10 records, the value will be the initial value. But, if the "Begin loop" property is set as "Yes", the stream will be mapped once the mapper processes.

Variable type

In the input side, setting value to the variable except mapper variable is as same as the property of the component. It can be done only once for running mapper once. The times of setting value to a mapper variable equals to the the number of the records. You can't set value for the system variable, the external variable set's constant and Flow variable that can't be changed. (It can't be shown in the output side.)

Status

The mapping content for the "Status" field can be recorded as the status string.

When you are running a request on which a lot of time will be spent, it's convenient for you to check how much it has been processed by checking the "Status" field. You can check the status string by displaying the the request's detail. The path is the Manage Console's "Status"-"Flow"-"Request list". Moreover, you can also check it from the external client such as flow-ctrl command. You can check it with the show request or show worker command of the flow-ctrl command.

Log

The mapping content for the "Log" field can be output to the execution log.

Under the initial status, the mapping content for the "Log" field is output to the FlowService.log. If the application log has been set, it will be output to the application log.

Mapper function

By configuring the function, the output data can be inserted to the field of the output side. The function is executed only once for 1 record's mapping. For example, when UUID function is mapped to the below 2 fields.

UUID → Field1
     → Field2

Please asssume that the input stream has 10 records here. In this case, the UUID function is executed only once for processing one record, so although the values of Field1 and Field2 are the same in each record, the values of the records are different with each other.

Mapper's action

Open XML in the input side

The number of the records of the XML is decided by the the number of the nodes that repeat in the field definition. There is an example for the XML instance as below.

<root>
	<Order form>
		<Order No.>1111</Order No.>
		<Products><Product name>Portable CD player</Product name><Price>1000</Price></Products>
		<Products><Product name>Boom box<Product name><Price>2000</Price></Products>
	</Order form>
	<Order form>
		<Order No.>2222</Order No.>
		<Products><Product name>Video camera</Product name><Price>3000</Price></Products>
		<Products><Product name>Digital camera</Product name><Price>4000</Price></Products>
		<Products><Product name>Scanner</Product name><Price>5000</Price></Products>
	</Order form>
</root>

Here the repeated nodes are "Order form" and "Products".

In this case, there are 2 "Order form" elements, and each has 2 and 3 "Products" elememts seperately. Namely, the number of the records is 2+3=5. If you spread the elememts that have values only, it will become as below.

1111, Portable CD player,1000
1111,Boom box,2000
2222,Video camera,3000
2222,Digital camera,4000
2222,Scanner,5000

The above record set is used in the mapper's input side. But, the field definition that isn't used for mapping is ignored in the mapper, so when the element for reference is only "Order number", and the elements under "Products" aren't for reference, the input instance is as below.

<root>
	<Order form>
		<Order number>1111</Order number>
	</Order form>
	<Order form>
		<Order number>2222</Order number>
	</Order form>
</root>

In this case, there will be 2 records in the input record set.

1111
2222

Conversely speaking, when you want to process 5 records by using order number only, it's necessary to map the elements under "Producrs" as dummy Flow variable.

Configure XML in the output side

There must be a document element's definition at least in the XML's field definition of the output side. For the processing result of the mapper, because the flat record set has been prepared, XML will be configured in the output side with this result set according to the rule as below.

The document element should has been created.
Don't create the node that wasn't mapped.
Don't create the node whose value is null, although it has been mapped.
When there is no node to repeat the above one, the mapped value can be overwritten.
When you map data under the node which can be repeated, you can search the node whose elements are as same as its own record from the created XML. If the node whose elements are the same exists, it will be considered as the same record, and can't be mapped (Namely, key is needed in the elements that will become record). When the node whose elememts that can't be repeated are the same exists, it can be considered as the record belongs to the node, the node that can be repeated will be created from there. When it doesn't exist, create a new node as a new record.

Please think about the below 2 examples of the field definitions that change the element to the attribute for the previous input instance.

A

Root                    Element
    Order form            Element (Repeat=True) 
        Order No.        Attribute
        Producrs            Element (Repeat=True) 
            Product name      Attribute
             Price       Attribute

B

Root                    Element
    Order form             Element (Repeat=True) 
        Order No.        Attribute
      Producrs             Element
            Product name       Attribute
             Price        Attribute

The difference between the field definitions of A and B is whether the "Products" element repeats or not. The results of A and B are shown as below.

A

<root>
	<Order form  Order No.="1111">
		<Products Product name="Portable CD player" Price="1000"/>
		<Products Product name="Boom box" Price="2000"/>
	</Order form >
	<Order form  Order No.="2222">
		<Products Product name="Video camera" Price="3000"/>
		<Products Product name="Digital camera" Price="4000"/>
		<Products Product name="Scanner" Price="5000"/>
	</Order form >
</root>

B

<root>
	<Order form Order No.="1111">
		<Products Product name="Portable CD player" Price="1000"/>
	</Order form>
	<Order form Order No.="1111">
		<Products Product name="Boom box" Price="2000"/>
	</Order form>
	<Order form Order No.="2222">
		<Products Product name="Video camera" Price="3000"/>
	</Order form>
	<Order form Order No.="2222">
		<Products Product name="Digital camera" Price="4000"/>
	</Order form>
	<Order form Order No.="2222">
		<Products Product name="Scanner" Price="5000"/>
	</Order form>
</root>

Whether A or B, when the first record is mapped, the result is same, so the instance will become as below.

<root>
	<Order form Order No.="1111">
		<Products Product name="Portable CD player" Price="1000"/>
	</Order form>
</root>

When the second record

1111,Boom box,2000

is inserted, the processing will be as below seperately.

Case A

Search the "Order form" element that can be repeated.
The node whose elements (@Order No.) that can't be repeated are the same exists, so the "Order form" element which is an object can be decided to it. (The new "Order form" element can't be created.)
Search the "Products" element that can be repeated under the object's "Order form" element.
The "Products" element whose elements (@product name, @price) that can't repeat are same exists, so create a new "Products" element.

Case B

Search the "Order form" element that can be repeated.
The node whose elements (@Order No., Products/@Product name, Product/@Price)that can't repeat are same doesn't exist, so create a new "Order form" element.
The "Products" element doesn't exist, so create a new "Products" element.

When the hierarchical structure of the node that can repeat becomes deep, or when complex XMLs that exist in parallel is created, you can deal with it by repeating the above processing.

Special Namespace

When there is an Attribute belonging to the "http://www.infoteria.com/asteria/mapper/void" Namespace in the field definition of the output XML, the Attribute won't be included in the instance of the output XML. Because the value will be mapped to the field, the node that the elements are the same can be output as another node by using it (as output content).

For example, map CSV that the same value is repeated for 5 lines in 1 field to XML.

Please assume that map this CSV to the below XML's field definition.

Root            Element
    Field1      Element (Repeat=True)   ←Map the CSV field here.

According to the previous instruction, the output result is shown as below.

<Root>
	<Field1>11111</Field1>
</Root>

The contents of Field1 keep same through 5 record processings, so they can be taken as 1 node.

Next, change the output field definition and mapping as below.

Root                Element
    Field1          Element (Repeat=True)  ※1
        recNo       Attribute ※2

※1 Map the CSV field here.
※2 Map the RecordNo here.

The result is as below.

<Root>
	<Field1 recNo="0">11111</Field1>
	<Field1 recNo="1">11111</Field1>
	<Field1 recNo="2">11111</Field1>
	<Field1 recNo="3">11111</Field1>
	<Field1 recNo="4">11111</Field1>
</Root>

Because the values of recNo attributes are different from each other according to the record processing, so the fields can't be got together.

At last, add "x=http://www.infoteria.com/asteria/mapper/void" as Namespace definition, and change the output field definition as below.

Root                Element
    Field1          Element (Repeat=True) ※1
    x:recNo         Attribute ※2

※1 Map CSV field here.
※2 Map RecordNo here.

The internal mapper action is as same as previous,
but the "x:recNo" attribute belonging to the "http://www.infoteria.com/asteria/mapper/void" can't be output, so the result is as below.

<Root>
	<Field1>11111</Field1>
	<Field1>11111</Field1>
	<Field1>11111</Field1>
	<Field1>11111</Field1>
	<Field1>11111</Field1>
</Root>

To the top of this page