TI process - Execution speed decrease

[Guillaume Galtier]

Hi everybody,

I'm facing a strange behaviour running a TI calculation process. After several working hours or several processes executions, the execution's speed of my calculation process is running low critically (from 10min initially to a couple of hours)

Looking at the memory state on the TM1 server (through Resource Monitor), I see that the "Free" memory is near 0 (or barely a few MB), but the "Standby" Memory is around 5GB.

Is it possible that this behaviour is due to the lack of Free memory ?

And more generally, can you explain me how TM1 and the TI processes are dealing with the memory ?

Many thanks in advance
Guillaume

[whitej_d]

I've seen TI processes cause paging if they get close to using all the memory, particularly if doing a lot of metadata manipulation. This really kills performance.

I don't know how close it needs to get before it starts paging though. Mine would start about 1 GB away from the limit. How does it look over time? On a freshly recycled server, does the TI process start off increasing memory, and then plateau? Can you test it on a system with loads of RAM?

[tomok]

I'm facing a strange behaviour running a TI calculation process.

Just what exactly is a TI Calculation process? Does this mean you have an algebraic formula on the data tab of the process and you are writing the result into a cube somewhere, or does it mean you are loading data into a cube, having a rule in the cube calculate something, and then extracting from the cube to load back into another cube? I ask because it can make a huge difference. Using a cube to calculate results and then using that as a data source to load another cube can be a huge memory hog as all the cells have to be calculated and they'll likely get cached (unless you've modified the default virtual memory settings on your system).

Looking at the memory state on the TM1 server (through Resource Monitor), I see that the "Free" memory is near 0 (or barely a few MB), but the "Standby" Memory is around 5GB.

Is it possible that this behaviour is due to the lack of Free memory ?

Lack of free memory ALWAYS means death to a TM1 system. You need to either add more RAM to your system or do an overhaul of the model.

[Guillaume Galtier]

First of all, thanks for your replies.

Tomok, the process i'm talking about, is a big process calling many sub processes to do a lot of mathematical calculation and to fill in a cube.
The sources of these processes are other cubes (including data calculated by rules) and the destination cube itself.

The aim is to cross the data from several cubes applying some mathematical calculations.

Lack of free memory ALWAYS means death to a TM1 system. You need to either add more RAM to your system or do an overhaul of the model.

So does it mean that TM1 doesn't use the Standby Memory which is a part of the Available Memory ?

[tomok]

So does it mean that TM1 doesn't use the Standby Memory which is a part of the Available Memory ?

Standby memory, if I'm not mistaken, consists of whatever RAM is provided to your system via the Paging File on disk. You never, ever, want TM1 to have to resort to using paging memory. Like I said earlier, you either need to add more RAM to your box or re-design how this process works.

[Duncan P]

If you are on 10.1 you might try putting all the sub-processes in a chore and setting the chore to commit after each process. This should have the effect of releasing back to TM1 the memory it is using to maintain a local copy of any data that the process has changed and local cache for any calculations that use those changed values.

Another option if you are using 9.5.2 and if no-one else needs to use the system at the time of the process is to use Bulk Load Mode.