TI Process - Save Action.

[santosh361]

Hi,

I observed that when ever we change the values in cube, .cub file is not updating with until will click save.
My question is if we run TI Process to load data, when/where this save action performs.

Also I have another question.

We are using SAN as data drive and we are migrated application to new high configuration server but data drive is in SAN. I understand from documentation that SAN is good kind of structure. But user raising issue that they are having latency and processes are taking long time to execute than prior.

Can you please suggest is SAN is good approach/how to calculate the performance/actions performed by TM1.

Thanks,
Santosh

[jim wood]

Hello there,

As TM1 is a memory resident application the data you load is stored in memory. If you have cube logging enabled then it also writes the changes to a log file. Only when you select save does it write the updated contents to disk. There is a command called SaveDataAll (look within the reference documentation for more details) within TI. I suggest you add this to the epilog of the last process executed,

Jim.

[santu469]

After the TI Process is executed, data will be available only in Physical memory. You need to have a SaveDataAll; process running after your data loads for the data to be saved on disk. If you have Data Write backs happening on your cubes, then it is a best practice to create a TI Process with SaveDataAll function and schedule it to run for every 5 or 10 min. This way u can mitigate the risk of loosing data during server crash.

[santosh361]

Here is my observation.

I have executed one TI Process to load a dimension after executing that TI Process and with out performing SaveDataAll ; operation. I am seeing that .dim file modified timestamp to current time. This is something I am confusing.

Why the timestamp got modified if it won't save on to the disk. I also tried to change the cube cell values and observed it did not updated with latest timestamp until I do save.

Is there any difference in Dimeniosn & Cubes.

Thanks,
Santosh

[tomok]

Changes to dimension and changes to cube data are two different things. Whenever you make a change to a dimension it is updated on disk to the .dim file. Changes to cube data are not reflected in disk (the .cub file) until it has been saved via a manual Save, a SaveDataAll function in a TI, or the service is taken down. As mentioned earlier, your model exists in RAM. You should think of the .cub file as only a hot backup and the only way to update that hot backup is to Save.

[mattgoff]

After the TI Process is executed, data will be available only in Physical memory. You need to have a SaveDataAll; process running after your data loads for the data to be saved on disk. If you have Data Write backs happening on your cubes, then it is a best practice to create a TI Process with SaveDataAll function and schedule it to run for every 5 or 10 min. This way u can mitigate the risk of loosing data during server crash.

Unless you're disabling logging, this is not necessary. If TM1 crashes, the transactions made since the last SaveDataAll will be replayed from the tx log. Furter, if you have cubes of any significant size, running SaveDataAll every 5 or 10 mins will make your server unusable since SaveDataAll places a global lock.

Matt

[lotsaram]

Unless you're disabling logging, this is not necessary. If TM1 crashes, the transactions made since the last SaveDataAll will be replayed from the tx log. Furter, if you have cubes of any significant size, running SaveDataAll every 5 or 10 mins will make your server unusable since SaveDataAll places a global lock.

I agree that I don't think anyone in their right mind woudl recommend a SaveDataAll every 5 - 10 minutes as a "best practice" in a system of any data volume, well any TM1 system at all. But at least in 10.1 and 9.5.2 and I think 9.5.1 as well that the lock placed by SaveDataAll is no longer global and no longer persists for the entire operation but is object (.cub file) based and only other users trying to access the cube being saved at the time are locked and each lock is released as each cube is saved. This at least is a huge improvement over 9.4, 9.1 and all before but still not any reason to by saving every few minutes given that transaction logging should be in place to handle any restore issues.

[santosh361]

Thanks for your response. It really justifies my argument.

Couple of other points I would like to raise.

1. We have TM1 cube as source for some of the processes : Now when we execute these TM1 processes it will collect the cube views from Memory only and not from Disk. Please confirm.
2. We are using SAN as datadrive and TM1 data folder is residing in this drive. Is this something causes the performance issue while running processes or browsing cubes.
Based on this thread it should not and it will completely depends on System Processors and Physical Memory challenges. What is your thoughts ?

What we need to consider while calculating the TM1 performance. (Our new server is having double capacity(Physical memorey) than old one. Other than Data Drive is residing in SAN. TM1 Business Administrators raising that they have huge performance issues after moving to new server).

Thanks,
Santosh

[lotsaram]

1. We have TM1 cube as source for some of the processes : Now when we execute these TM1 processes it will collect the cube views from Memory only and not from Disk. Please confirm.
2. We are using SAN as datadrive and TM1 data folder is residing in this drive. Is this something causes the performance issue while running processes or browsing cubes.
Based on this thread it should not and it will completely depends on System Processors and Physical Memory challenges. What is your thoughts ?

What we need to consider while calculating the TM1 performance. (Our new server is having double capacity(Physical memorey) than old one. Other than Data Drive is residing in SAN. TM1 Business Administrators raising that they have huge performance issues after moving to new server).

TM1 is totally in RAM. There is no reading or writing to disk during calculations or user queries, everything is in memory.

Performance issues with TM1 are usually either model related (poorly written rules, badly designed application) or network latency related. If the model has moved physical servers and not changed in any way then the most pertinent question is where is the new server relative to the old one. If the new server is somewhere else on the network that has higher latency to the clients then that's your reason. Other than that is the new server physical or virtual? If virtual are the additional memory and cores dedicated or shared and is TM1 really getting the resources it needs? There are quite a few things to watch out for when virtualizing TM1.

[santosh361]

We are using Physical servers other than Data Drive in SAN. Also now I just googled and found my new servers are 100 miles away than old once. Brand new environment and infrastructure claims that they are using advanced technology than old datacenter.
Our old boxes are windows 2003 and new once are windows 2008 r2. Not sure if this point needs to be consider and modify any settings to perform better.
If every thing runs in memory, only part left to mend is rules or TI code that executes. I am very curious to know if any other items we need to consider to weight the performance of model.

Thanks
Santosh

[qml]

my new servers are 100 miles away than old once

This is most likely the cause of your performance problems, just like lotsa says.

Compare the ping times and bandwidth between the client machines and the old and new servers. Perspectives and Architect use, how to put it mildly, sub-optimal protocols and so are very dependent on good network performance. Successful deployments over a WAN need to be carefully designed and implemented.

[santosh361]

@qml

I have verified the ping stats and it looks and those are almost similar and infact new server are little better(1 or 2 msecs fast enough).

I surprise how this impact over here, as processes will execute on TM1 server even though it executes from client - Am I wrong ?