Adding ATTRPUTN

Post Reply
winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Adding ATTRPUTN

Post by winsonlee » Tue Dec 13, 2011 6:11 am

For newly added element, I would like to change the status from 0 to 1 which indicate that this element is newly added. Very simple process but somehow i coudlnt get it working.

Metadata

IF (DIMIX(vDim, MainNOrderJobNo) = 0);
AttrPutN(1, vDim, MainNOrderJobNo, 'NewlyAddedStatus');
ENDIF;
DimensionElementInsert(vDim, ' ', MainNOrderJobNo,'N');

rmackenzie
MVP
Posts: 733
Joined: Wed May 14, 2008 11:06 pm

Re: Adding ATTRPUTN

Post by rmackenzie » Tue Dec 13, 2011 6:22 am

You have to insert all your elements in the Metadata tab of the process before you try and update attribute data. The elements don't actually exist until the Metadata tab has been completed for every row in the data source. Then, and only then, can you write attributes against the new elements.
Robin Mackenzie

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: Adding ATTRPUTN

Post by winsonlee » Tue Dec 13, 2011 9:08 am

If that is the case how can i assign '1' attribute to newly added element ? I tried the code below it doesnt work either.

Metadata

status = 0;
IF (DIMIX(vDim, MainNOrderJobNo) = 0);
status = 1;
ENDIF;
DimensionElementInsert(vDim, ' ', MainNOrderJobNo,'N');

Data

IF(status = 1);
AttrPutN(1, vDim, MainNOrderJobNo, 'NewlyAddedStatus');
ENDIF;

Duncan P
MVP
Posts: 588
Joined: Wed Aug 17, 2011 1:19 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2
Excel Version: 2003 2007
Location: York, UK

Re: Adding ATTRPUTN

Post by Duncan P » Tue Dec 13, 2011 9:36 am

TI runs the metadata script for each source record, then it commits any dimension changes, then it runs the data script for each source record. This means that you need to keep a track of the status of all records. You could do it like this.

Prolog

Code: Select all

datarecord = 0;
status = '';
code1 = CODE('1',1);
Metadata

Code: Select all

IF (0 = DIMIX(vDim, MainNOrderJobNo));
	status = status | '1';
ELSE;
	status = status | '0';
ENDIF;
DimensionElementInsert(vDim, ' ', MainNOrderJobNo,'N');
Data

Code: Select all

datarecord = datarecord + 1;
IF ( CODE(status, datarecord) = code1 );
	AttrPutN(1, vDim, MainNOrderJobNo, 'NewlyAddedStatus');
ENDIF;
BTW - I had to paste the tab characters into the forum edit field. It made me think I was in the TI editor. :roll:

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: Adding ATTRPUTN

Post by winsonlee » Tue Dec 13, 2011 1:04 pm

Doesnt seems like there is any data in the text file when i try asciioutput status in "Data". when a variable is change in metadata, does the changes get pass onto data ?
Duncan P wrote:

Data

Code: Select all

datarecord = datarecord + 1;
ASCIIOUTPUT('c:\test.txt', status);
IF ( CODE(status, datarecord) = code1 );
	AttrPutN(1, vDim, MainNOrderJobNo, 'NewlyAddedStatus');
ENDIF;

Duncan P
MVP
Posts: 588
Joined: Wed Aug 17, 2011 1:19 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2
Excel Version: 2003 2007
Location: York, UK

Re: Adding ATTRPUTN

Post by Duncan P » Tue Dec 13, 2011 2:22 pm

Variables, once assigned are available in all sections of the process and the values persist across sections.

1. Is the Metadata section being executed? Try ASCIIOUTPUT of status in the Metadata section.

2. Have you put the status = ... lines in the Metadata section?

3. Have you initialised status in the prolog?

It works fine for me. If you can't find out why it doesn't for you then post the .PRO file. There may be something you've missed.

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: Adding ATTRPUTN

Post by winsonlee » Tue Dec 13, 2011 9:55 pm

The following are my prolog, metadata, data and epilog code. did i miss out anything ?

Code: Select all

#****Begin: Generated Statements***
#****End: Generated Statements****

count= 0;
status = '';
code1 = CODE('1',1);

vDim = 'p_Order';

AttrInsert(vDim, '', 'Name', 'A');
AttrInsert(vDim, '', 'Cust-JobTitle', 'A');
AttrInsert(vDim, '', 'JobTitle', 'S');
AttrInsert(vDim, '', 'Customer', 'S');

DimensionElementInsert(vDim, ' ', 'All Customers', 'C');

573,33

#****Begin: Generated Statements***
#****End: Generated Statements****

IF (0 = DIMIX(vDim, MainNOrderJobNo));
   status = status | '1';
ELSE;
   status = status | '0';
ENDIF;

ASCIIOUTPUT('c:\te.txt', MainNOrderJobNo, status);

if( CustomerNo   @<> '' & JobTitleNo @<> '');
vJobTitle = CustomerNo | '-' | JobTitleNo;
DimensionElementInsert(vDim, ' ', vJobTitle, 'C');
DimensionElementInsert(vDim, ' ', CustomerNo, 'C');
DimensionElementInsert(vDim, ' ', CustomerGroupNo, 'C');
DimensionElementInsert(vDim, ' ', MainNOrderJobNo,'N');


DimensionElementComponentAdd(vDim, vJobTitle , MainNOrderJobNo,1);
DimensionElementComponentAdd(vDim, CustomerNo, vJobTitle,1);
DimensionElementComponentAdd(vDim, CustomerGroupNo,  CustomerNo, 1);
DimensionElementComponentAdd(vDim, 'All Customers', CustomerGroupNo,1);

ENDIF;




574,35

#****Begin: Generated Statements***
#****End: Generated Statements****

vLength=LONG(CustomerNo);
vJobTitle = CustomerNo | '-' | JobTitleNo;

vCustomerNo = SCAN('.',CustomerNo);
vNewCustomerNo = SUBST(CustomerNo,1,vCustomerNo-1);


  AttrPutS(CustomerGroupName, vDim, CustomerGroupNo, 'Name');
  AttrPutS(CustomerName, vDim, CustomerNo, 'Name');
 AttrPutS(MainNOrderJobDescription, vDim, MainNOrderJobNo,'Name');

 AttrPutS(CustomerName | ' - ' | CustomerNo | ' | ' | JobTitleDescription, vDim, vJobTitle, 'Cust-JobTitle');

AttrPutS(CustomerNo, vDim, MainNOrderJobNo,'Customer');
AttrPutS(CustomerNo, vDim, vJobTitle,'Customer');


AttrPutS(vJobTitle , vDim, MainNOrderJobNo,'JobTitle');


count = count + 1;
ASCIIOUTPUT('c:\data.txt',MainNOrderJobNo,numbertostring(count),status);

IF ( CODE(status, count) = code1 );
   AttrPutN(1, vDim, MainNOrderJobNo, 'NewlyAddedStatus');
ENDIF;


575,9

#****Begin: Generated Statements***
#****End: Generated Statements****


576,CubeAction=1511DataAction=1503CubeLogChanges=0
638,1
804,0
1217,0
900,
901,
902,
903,

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: Adding ATTRPUTN

Post by winsonlee » Tue Dec 13, 2011 10:00 pm

Seems like I have included everything you have mention and when i try asciioutput there is data on the status in the metadata section.It is just that the variable content is not pass to data.
Duncan P wrote:Variables, once assigned are available in all sections of the process and the values persist across sections.

1. Is the Metadata section being executed? Try ASCIIOUTPUT of status in the Metadata section.

2. Have you put the status = ... lines in the Metadata section?

3. Have you initialised status in the prolog?

It works fine for me. If you can't find out why it doesn't for you then post the .PRO file. There may be something you've missed.

rmackenzie
MVP
Posts: 733
Joined: Wed May 14, 2008 11:06 pm

Re: Adding ATTRPUTN

Post by rmackenzie » Tue Dec 13, 2011 10:12 pm

winsonlee wrote:Doesnt seems like there is any data in the text file when i try asciioutput status in "Data". when a variable is change in metadata, does the changes get pass onto data ?
DuncanP wrote:Variables, once assigned are available in all sections of the process and the values persist across sections.
Yes, but the Data tab will have the last value assigned to 'status' in the Metadata tab. If you have 10 records and, in the Metadata tab, the 'status' is 1 for records 1-9 and 0 for record 10 then when the code in the Data tab executes the 'status' variable will have a value of 0.

Perhaps an alternative is if you create a 'Newly Created' consolidation point and add the new elements in there as well as adding them to other consolidations. Then you can look at the children of that consolidation later in your process(es) and take the relevant action.
Robin Mackenzie

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: Adding ATTRPUTN

Post by winsonlee » Tue Dec 13, 2011 11:27 pm

Using solution propose by DuncanP, after going through 10 record in metadata, the status will be = '1001100010' as each line of record will concatenate the previous status variable with 1 indicate element does not exist and 0 indicate element exist. and when it goes through the "Data", status should remain to have '1001100010' through out the execution of all records.

After looking through the ASCIIOUTPUT probably i might find out the reason why it is not working. Just to confirm is there any limit how many character a variable able to hold ? after 7987 record which means 'status' is holding 7987 character, the subsequent record, status becomes empty again.

will that be a reason why when i asciioutput 'status' in 'Data', there isnt any data in the variable ?

rmackenzie wrote:
winsonlee wrote:Doesnt seems like there is any data in the text file when i try asciioutput status in "Data". when a variable is change in metadata, does the changes get pass onto data ?
DuncanP wrote:Variables, once assigned are available in all sections of the process and the values persist across sections.
Yes, but the Data tab will have the last value assigned to 'status' in the Metadata tab. If you have 10 records and, in the Metadata tab, the 'status' is 1 for records 1-9 and 0 for record 10 then when the code in the Data tab executes the 'status' variable will have a value of 0.

Perhaps an alternative is if you create a 'Newly Created' consolidation point and add the new elements in there as well as adding them to other consolidations. Then you can look at the children of that consolidation later in your process(es) and take the relevant action.

User avatar
Alan Kirk
Site Admin
Posts: 5996
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2 Classic (PAW-free zone)
Excel Version: 2010 and 2016
Location: Sydney, Australia
Contact:

Re: Adding ATTRPUTN

Post by Alan Kirk » Tue Dec 13, 2011 11:42 pm

winsonlee wrote:Using solution propose by DuncanP, after going through 10 record in metadata, the status will be = '1001100010' as each line of record will concatenate the previous status variable with 1 indicate element does not exist and 0 indicate element exist. and when it goes through the "Data", status should remain to have '1001100010' through out the execution of all records.

After looking through the ASCIIOUTPUT probably i might find out the reason why it is not working. Just to confirm is there any limit how many character a variable able to hold ? after 7987 record which means 'status' is holding 7987 character, the subsequent record, status becomes empty again.
If I may quote the TI guide (and, at one time, I did):
TurboIntegrator is capable of handling string data in sizes of up to 8000 single-byte characters at a time. This limit applies when your TI process is performing actions such as assigning a value to a variable or importing individual records of data. Any value or record longer than 8000 singlebyte characters is truncated.
Of course, there's a difference between "truncated" and "obliterated", but that was from the 9.5.0 manual and I've noticed that the one thing that's been consistent about oversized string handling from version to version is... it's not consistent.

I haven't tested it in later incarnations of 9.5.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.

rmackenzie
MVP
Posts: 733
Joined: Wed May 14, 2008 11:06 pm

Re: Adding ATTRPUTN

Post by rmackenzie » Tue Dec 13, 2011 11:43 pm

Since the introduction of extended strings a few releases ago, the maximum length was set at 8000, up from 255. DuncanP's idea is nice but won't work for big input data sets.

Here's the link to the IBM website
Robin Mackenzie

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: Adding ATTRPUTN

Post by winsonlee » Wed Dec 14, 2011 12:01 am

It is a good to know info. Probably that is the reason why it is not working in my situation as my data set is more then 8000. Initially i thought it is just a simple solution of if element not found, assign 1 to the attribute.

But when comes to putting it in place, it is not as simple as i thought it is. Probably i will move towards the direction of adding element to 'Newly Created' consolidation and do the necessary checking from there.

Thanks for the help guys.
rmackenzie wrote:Since the introduction of extended strings a few releases ago, the maximum length was set at 8000, up from 255. DuncanP's idea is nice but won't work for big input data sets.

Here's the link to the IBM website

Duncan P
MVP
Posts: 588
Joined: Wed Aug 17, 2011 1:19 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2
Excel Version: 2003 2007
Location: York, UK

Re: Adding ATTRPUTN

Post by Duncan P » Wed Dec 14, 2011 10:20 am

OP doesn't say which flavour of 9.5 he is using but in 9.5.2 the 8000 character limit no longer applies to manipulation of string variables. The lines

Code: Select all

v = FILL('1',400000);
ASCIIOUTPUT( 'd:\temp\qq.txt', NumberToString(LONG(v)));
produce a file with the following line

Code: Select all

"400000"
so the variable is there and it has 400000 characters in it. If however I try to ASCIIOUTPUT the string itself I get nothing. There appears to be a limit of 64999 on the length of the overall line output in a single call to ACSIIOUTPUT.

So you would think that the concatenation would be OK still. However that is not the case. A simple test looping while concatenating a character to a variable and then outputting its length shows nothing after about 44399. However if I look in the server log it has the following error message :-

Code: Select all

4068   [2]   INFO   2011-12-14 09:26:16.562   TM1.Process   Process "qq" executed by user "Admin"
4068   [2]   ERROR   2011-12-14 09:26:22.968   TM1.Server.Memory   AllocateBlockFromOS: alloc (size = 44319) failed: (null)
4068   [2]   WARN   2011-12-14 09:26:22.968   TM1.Server.Memory   AllocateBigBlock() outOfMemory Exception <<< MEMORY_FATAL_LEVEL >>> -  apifunc# "196"
At the time of the failure the process (32bit) had just over a gigabyte allocated. What is probably happening is that of the available 2 Gb of address space there is not a single continuous block of the 44K required. This will be because each loop is allocating a block one byte larger than the largest one used so far and so TM1 is having to go to the operating system to give it another one. Furthermore TM1 is not giving back the previous blocks but is hanging on to them while the process is executing which means that the OS is allocating from an ever depleting pool. Using Address Space Monitor (http://hashpling.org/asm/) I got the following map of the address space just before it died.
tm1_asm.PNG
tm1_asm.PNG (6.15 KiB) Viewed 6393 times
So concatenation is not a goer, but in 9.5.2 not for the reasons stated in previous posts.

Besides rmackenzie's suggestion is much the better one in this instance. :)

rmackenzie
MVP
Posts: 733
Joined: Wed May 14, 2008 11:06 pm

Re: Adding ATTRPUTN

Post by rmackenzie » Wed Dec 14, 2011 10:28 pm

Duncan P wrote:in 9.5.2 the 8000 character limit no longer applies to manipulation of string variables.
Thanks for the update. Your idea is a good work-around to the absence of an array construct in TI.
Duncan P wrote:What is probably happening is that of the available 2 Gb of address space there is not a single continuous block of the 44K required. This will be because each loop is allocating a block one byte larger than the largest one used so far and so TM1 is having to go to the operating system to give it another one.
I guess the issue is string immutability. TM1 is often frustratingly awkward when it comes to string processing.
Robin Mackenzie

Post Reply