Unable to load all the records into Target Cube

[aravind.cgns]

Hi Guys,

I have a TI Process which has datasource as "CSv File", the process is able to load most of the records into my target cube but it is unable to load few of the records.

It is diplaying below error:

"( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY","PKT - Premium Kits","1-VRT","ALL ITEMS","1","MFG","80","1",Data Source line (347) Error: Data procedure line (9): Invalid key: Dimension Name: "List Item", Element Name (Key): "( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY"
Error: Data procedure line (9): error repeats 6 times
"( 619) DSFCFAM - CHP SNFLR FAM - 8 Pk DISPLAY CA","PKT - Premium Kits","1-VRT","ALL ITEMS","1","MFG","80","1",Data Source line (348) Error: Data procedure line (9): Invalid key: Dimension Name: "List Item", Element Name (Key): "( 619) DSFCFAM - CHP SNFLR FAM - 8 Pk DISPLAY CA"
Error: Data procedure line (9): error repeats 6 times
"(3148) WMBB6X - WM 6-10oz BISTRO BLEND #3148","PVL - Private Label","1-VRT","ALL ITEMS","2","MFG","130","1",Data Source line (454) Error: Data procedure line (9): Invalid key: Dimension Name: "List Item", Element Name (Key): "(3148) WMBB6X - WM 6-10oz BISTRO BLEND #3148"


My data source is a "CSV File" with column seperator as | symbol.

My records in Csv looks Like this:

it_List_Item|Item_Group|Business|Root|Equiv_Factor|Item_Type|Uneq_Cases_Per_Pallet|visible

( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY|PKT - Premium Kits|1-VRT|ALL ITEMS|1|MFG|80|1
( 619) DSFCFAM - CHP SNFLR FAM - 8 Pk DISPLAY CA|PKT - Premium Kits|1-VRT|ALL ITEMS|1|MFG|80|1

(3148) WMBB6X - WM 6-10oz BISTRO BLEND #3148|PVL - Private Label|1-VRT|ALL ITEMS|2|MFG|130|1


the issue is due to the "Item_Group" attribute with extra spaces in the element, TIP is able load rest of the records but it is unable to load few records. Please let me know like where i am going wrong.

[Alan Kirk]

I have a TI Process which has datasource as "CSv File", the process is able to load most of the records into my target cube but it is unable to load few of the records.

It is diplaying below error:

"( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY","PKT - Premium Kits","1-VRT","ALL ITEMS","1","MFG","80","1",Data Source line (347) Error: Data procedure line (9): Invalid key: Dimension Name: "List Item", Element Name (Key): "( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY"
Error: Data procedure line (9): error repeats 6 times
...

My records in Csv looks Like this:

it_List_Item|Item_Group|Business|Root|Equiv_Factor|Item_Type|Uneq_Cases_Per_Pallet|visible

( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY|PKT - Premium Kits|1-VRT|ALL ITEMS|1|MFG|80|1
( 619) DSFCFAM - CHP SNFLR FAM - 8 Pk DISPLAY CA|PKT - Premium Kits|1-VRT|ALL ITEMS|1|MFG|80|1

(3148) WMBB6X - WM 6-10oz BISTRO BLEND #3148|PVL - Private Label|1-VRT|ALL ITEMS|2|MFG|130|1

the issue is due to the "Item_Group" attribute with extra spaces in the element,

Yeah. No it isn't.

First, TI generally ignores white space. Second, the log file is telling you clearly and unambiguously what the error is. It's that in the List Item dimension, the element ( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY does not exist. You need to create it before you run the process, or create it in the Metadata tab of the process so that by the time you reach the Data tab the element is there and ready to receive data.

[aravind.cgns]

Thanks Alan you are right, but the element "( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY" in my "List Item" dimension.

this is my data procedure :

If(it_List_Item@='');
it_List_Item='Default it_List_Item';
EndIf;

DimensionElementInsertDirect('List Item', '', it_List_Item, 'n');
CellPutS(Item_Group, 'a Item Attrs', it_List_Item, 'Item Group');
CellPutS(Business, 'a Item Attrs', it_List_Item, 'Business');
CellPutS(Root, 'a Item Attrs', it_List_Item, 'Root');
CellPutN(Equiv_Factor, 'a Item Attrs', it_List_Item, 'Equiv Factor');
CellPutS(Item_Type, 'a Item Attrs', it_List_Item, 'Item Type');
CellPutN(Uneq_Cases_Per_Pallet, 'a Item Attrs', it_List_Item, 'Uneq Cases Per Pallet');

[BariAbdul]

Please read Alan's advice carefully again.

eah. No it isn't.

First, TI generally ignores white space. Second, the log file is telling you clearly and unambiguously what the error is. It's that in the List Item dimension, the element ( 618) DASNFAM - CHP SES ASN FAM - 8 Pk DISPLAY does not exist. You need to create it before you run the process, or create it in the Metadata tab of the process so that by the time you reach the Data tab the element is there and ready to receive data.

[Wim Gielis]

If you are really sure that the element exists in the dimension (TM1 and I have my doubts though )
then consider debugging like shown here:
http://users.skynet.be/fa436118/wim/tm1 ... ers_EN.htm

How can you be sure that an element exists in a dimension ? Copy/paste the element to the Subset Editor and see if TM1 complains or not.

[Edward Stuart]

My first instinct was that the element was not added to the dimension in the metadata as per the posts above, however, checking the reference guide for DIMENSIONELEMENTINSERTDIRECT we have:

When the purpose of the TurboIntegrator process is to load large volumes of
data into a cube. In this case the process' Metadata procedure is deliberately
kept empty, and any element insertion needed to support data loading is
performed using direct calls in the Data procedure. When the Metadata
procedure is empty, the process skips an entire iteration over the external
datasource, which can result in faster data loads.

I've not tested this extensively myself but the issue still remains that TM1 cannot see the element, which means testing Wim's theory of special characters is where I would go next

[aravind.cgns]

Thanks Guys, u were right the problem was with the leading spaces in few of the "List Items" in the "CSV Files", and there are mismatch of the names in dimension elements and elements in the "CSV File.

Thanks,
Arvind