TM1 Forum

Posted: **Tue Jul 12, 2011 1:41 pm**

I have this cube with with a size of 200,000kb++

I have a view I used as a data source to input specific amounts to another version and tag it in another cube as finished. I just noticed as I keep looking into the TI process that this message occur even if the process is completed successfully.

For a cube that runs out of memory at some instance of opening the cube it I expect the transfer to take so long not a second or 2.

Is it possible that the TI process terminates it if runs out of memory during the process and still prompt a completed process?

Posted: **Tue Jul 12, 2011 4:10 pm**

As far as I'm aware, this message can be ignored as it is just a progress report as opposed to an error message. I have seen this on multiple occasions and have just ignored it.

If your message log tells you the process has completed successfully, I'd trust it over the screenshot you provided.

Posted: **Tue Jul 12, 2011 6:02 pm**

Typically this error message is caused by updating a dimension with a while loop on the prolog. All dimension updates on the prolog are comitted INSTANTANEOUSLY to memory (and to disk). It is just that the disk IO can't possibly keep up with TI. You don't get this on the meta data as updates are committed only once when reading the data source finishes. As Dan has said this can be ignorred, it is not really an error.

Posted: **Wed Jul 13, 2011 12:39 am**

lotsaram wrote:All dimension updates on the prolog are comitted INSTANTANEOUSLY to memory (and to disk).

This is news to me. I thought all metadata updates were done at the close of the metadata tab.

Is this a change, or has it always been this way?

Martin

Posted: **Wed Jul 13, 2011 1:22 am**

Martin Ryan wrote:
lotsaram wrote:All dimension updates on the prolog are comitted INSTANTANEOUSLY to memory (and to disk).
This is news to me. I thought all metadata updates were done at the close of the metadata tab.

Is this a change, or has it always been this way?

Yeah, due respect to Lotsa but I'm afraid I must disagree on this one. My understanding of dimension changes, based on work with the API which is I imagine not too different from how TI does it, is that you create a copy of the dim in memory, make the changes on the copy, then overwrite the original. This is, I believe, what happens at the end of the Metadata tab (and possibly the Prolog tab as well, but I haven't tested that). Instantaneous changes using that methodology would seem to hammer the system a bit hard.

Consider the following two processes:

Process 1 Prolog

Code: Select all

s_DimName = 'z_Test';

DimensionCreate(s_DimName);
DimensionElementInsert(s_DimName, '','Element 1','N');

ExecuteProcess( 'zTest2', 'Call', 'Prolog');

l = DimSiz ( s_DimName);

AsciiOutput ( 'C:\Temp\TestDimProc1.txt', NumberToString ( l ) );

Process 1 Epilog

Code: Select all

ExecuteProcess( 'zTest2', 'Call', 'Epilog');

Process 2 Prolog

Code: Select all

s_DimName = 'z_Test';

s_OutputFile = 'C:\Temp\TestDimProc' | Call | '.txt';

l = DimSiz ( s_DimName);

AsciiOutput (s_OutputFile, NumberToString ( l ) );

Now, what you get out of these is:
Prolog call to Process 2 (and its AsciiOutput) = 0
Call to Process 1's AsciiOutput = 1
Epilog call to Process 2 (and its AsciiOutput) = 1

This makes sense. What appears to be happening is that the copy of the dimension that Process 1 is (presumed, by me at least to be) working on does indeed have one element. Consequently the Prolog AsciiOut of that process shows 1. The copy is updated in real time... but that's not the dimension that the rest of the system knows about.

Consequently at the time that the Prolog call to process 2 is done, process 2 knows nothing of any such elements and returns 0. (NB: For some reason doing a DimSiz on a non-existent dim doesn't cause an error (in 9.5.2 at least), it just returns 0. My belief is that process 2 doesn't even know the new dim exists yet because it hasn't been registered. Certainly it clearly doesn't know anything about an element in that dim,even though the element was "created" before the call was made.)

However the Epilog call to Process 2 does know about the element, because, I believe, the dimension was registered either at the end of the Prolog tab or as the process slipped over the Metadata stage. (Neither process had a data source.)

Edit: Incidentally, an AsciiOutput in Process1's Prolog before the call to Process 2 will also yield "1".

Posted: **Wed Jul 13, 2011 5:55 am**

Martin Ryan wrote:
lotsaram wrote:All dimension updates on the prolog are comitted INSTANTANEOUSLY to memory (and to disk).
This is news to me. I thought all metadata updates were done at the close of the metadata tab.

Is this a change, or has it always been this way?

It's fortunate that I'm not a cat, because I'd be quite dead by now.

While waiting for some people to send me some data that I need, I got curious about the second arm of that statement; the question of whether it saves to disk immediately. The short version is that some disk writing can, but doesn't always, happen, so the reason for the problem can indeed be the one suggested.

I put a loop into the prolog after the dimension changes had been done to effectively pause the process for a couple of minutes (actually just loop for a couple of minutes), and watched what happened in the data directory.

When the process is used to create a new dimension, a .dim$ file (that is, the "half way house" file) is created straight away. When the loop (and therefore the Prolog ends, that file is replaced by the proper .dim file.

When the process is used to update a dimension, however, whether that's by only inserting new elements, or whether it's by inserting a new consolidation, inserting new elements and adding them to the consolidation) then the .dim$ file will not be created prior to the loop running. (It may well be created upon the save of the dimension at the end of the Prolog as an interim stage before the .dim is replaced, but there's no way to slow that down to see it.) In addition, the timestamp on the .dim file will not change while the loop is running, despite the addition of the new elements / hierarchy. That file's timestamp will only update when the process ends.

So the .dim file is not updated immediately... but in some circumstances an intermediate "$" file will be written to the disk, which could cause the error as Lotsaram described.

Posted: **Wed Jul 13, 2011 2:37 pm**

I just thought of asking the experts here if for some reason...this message during the process could probably roll back the process or something cause my TI to be completed but not work how its suppose to work. The TI is basic...use TM1 cube as a source. place some static text using certain dimensions in the view to another view and copy the data from one cube to another but for some reason the data isn't doing what its suppose to be doing.

TM1 Forum

...could not write dimension definition to disk storage...

...could not write dimension definition to disk storage...

Re: ...could not write dimension definition to disk storage.

Re: ...could not write dimension definition to disk storage.

Re: ...could not write dimension definition to disk storage.

Re: ...could not write dimension definition to disk storage.

Re: ...could not write dimension definition to disk storage.

Re: ...could not write dimension definition to disk storage.