TM1 RAM usage is more

[guru1988fortm1]

Hi All,

I am using TM1 9.5.2 in windows server R2 with 180 GB RAM and 32 bit cores

when the services are started, it takes almost 40 GB just for the steps before 'cube processing feeders'

Then it uses all of its 180 GB RAM and it takes one and half hours atleast, so the server starts up only at 180 GB

I have set the maximum cube load threads as 30 for faster server startup

The application is quite a huge around 10 GB size

persitent feeders =F and parallel interaction =T

can anyone suggest me that is this is something unusual or what could be the reason

Thanks
Regards
Guru Madhesh

[Gabor]

It sounds like (and/or) bad DB design + feeder issue.

MaximumCubeLoadThreads is not helpful during design phase, as it allows parallel feeding which may blow up the RAM.
The feeder target cube(s) must have an optimal dim order, otherwise feeder explosion is also possible.

Finally, it's always useful to start with less feeder rules to do quicker tests and optimization.
10 GB (if this is the size on harddisk) is not really huge. You should be able to get this loaded in about the same size of RAM, if no feeders there and dim order is ok.
That should be the starter.

[lotsaram]

when the services are started, it takes almost 40 GB just for the steps before 'cube processing feeders'

Then it uses all of its 180 GB RAM and it takes one and half hours atleast, so the server starts up only at 180 GB

I have set the maximum cube load threads as 30 for faster server startup

The application is quite a huge around 10 GB size

This doesn't make sense and/or is inconsistent. The "application is around 10 GB" yet the application "uses up all of its 180 GB RAM". Well which is it? Does the server not load at all because it runs out of RAM? Does the server load using 180 GB but of that 170 GB is garbage memory? Is the data directory 10 GB on disk? (This is next to meaningless by the way if that's what you mean).

Agree with Gabor that this sounds like an issue with feeders, in other words poor rule design. If this is the case then setting MaximumCubeLoadThreads to a very high value is the ABSOLUTELY LAST thing you want to do as this will consume lots of excess memory on startup and may actually (and counter-intuitively) result in much longer server load time than single threaded load.

I disagree with Gabor about MaximumCubeLoadThreads not being useful during development. Often during development the server needs to be taken down frequently and using the correct (correct != maximum) number of threads to load the server is useful to keep the downtime as short as possible. Using Persistent Feeders is also very helpful to minimize server load time (but NOT if the feeders need to be improved).

[jim wood]

I disagree with Gabor about MaximumCubeLoadThreads not being useful during development. Often during development the server needs to be taken down frequently and using the correct (correct != maximum) number of threads to load the server is useful to keep the downtime as short as possible.

I can see both sides of this argument. It can be useful as you say to reduce downtime to have it switched on, however it does not show you what the real memory usage is on start up which can be helpful especially when you're talking about this issue. As always with TM1 it depends on the circumstances,

Jim.

[lotsaram]

It can be useful as you say to reduce downtime to have it switched on, however it does not show you what the real memory usage is on start up which can be helpful especially when you're talking about this issue. As always with TM1 it depends on the circumstances

This is really a classic case of "it depends". I would argue that using PersistentFeeders=T combined with MaximumCubeLoadThreads>1 will give the "correct" memory usage on startup as garbage memory creation from multiple thread feeder evaluation is avoided and so therefore the model will use the right level of memory that it should (i.e. the same as single threaded load without persistent feeders.) That is of course assuming that the saved feeder files represent a correct state without over feeding which might not be the case in development where rules are being worked on and data might not be representative of what to expect in production.

[jim wood]

I'm not sure I would have persistent feeders enabled, especially when completing rule development, but hey we don't all do things the same way. Life would be very boring if we did,

Jim.