CPU Cores in use

[JamiseBondi]

Hi All,

We have three servers (Dev, QA and PRD) that have 16 CPU cores (lots of RAM and fast disc subsystem) but when we go to do a rule save or slice and dice data in views we sometimes have to wait many minutes for the calculations to take place and when I look at the CPU usage - out of the 16 cores that are available, we are using 1 or maybe 2 cores at 100%, the others sit at near 0%. How do we get TM1 to make use of more of the CPU cores? It feels like we have this big hardware engine that is simply idling along and I'm hoping there is a way to get more of it available for the TM1 engine to use.
I have used the "MaximumCubeLoadThreads=7" for example in the tm1s.cfg file to great effect, this actually does use the correct number of cores when starting up, but not while it's doing the ongoing calculations - it seems to use 1 or 2 cores for these calcs...

Many thanks.

[lotsaram]

As of 10.1

Loading of data at server startup is multi-threaded if using the MaximumCubeLoadThreads parameter.

If parallel interaction is enabled then then loading of data during a server session is also multi-threaded. (either via TI or user entry)

If parallel interaction is enabled then then view retrieval is multi-threaded.

However calculations are single-threaded. (provided sufficient caching is enabled and values are not invalidated by new data entry, subsequent retrieval of stored values can be multi-threaded).

Rule saving is certainly single-threaded.

Someone might want to elaborate or correct a simplification but I think that's it in a nutshell.

[Duncan P]

The key thing is that each client interaction or chore runs on a single thread, but that you can have multiple clients running simultaneously.

This means that any single TI runs on a single thread, but that you can have multiple TIs running at the same time, each of which can be reading or writing.

Certain actions can be done by multiple threads at a time (e.g. reading from a cube & performing calculations) while certain actions require everything else to stop while they do their work. A classic example of the latter is SaveDataAll. Other actions on specific objects will lock just those objects, e.g. dimension update will lock a dimension and all of its dependent cubes. While they are locked other threads will not be able to use them.

So if you have 100 users, each requesting data, then expect all CPUs to be pretty active. If you have an enormous data load through a single TI it will only use one thread. If you want to spread a load over multiple threads you will need to kick off multiple instances of the loading TI, each of which filters its input to a different section of the data.

As far as calculations are concerned each user thread does its own calculations (for cells that don't already have a cached value) and keeps hold of any that took longer than CalculationThresholdForStorage, but at the end of the operation it merges its private cached calculated values with the global cache. This merging can only be done one thread at a time on each cube.

[upali]

Recently at a conference, an IBM TM1 rep illustrated that the next version will have better multi-threaded capability. This addresses the exact issue you are having. So we'll have to wait and see if it materializes or not.

[stephen waters]

If you look at the sample agendas for the IoD conference in Las Vegas in November they include a couple of "TM1 10.2" sessions.

BPM-3728C: What's New in IBM Cognos TM1 V10.2 for Mobile, Scorecarding, and Microsoft Excel Clients
and
BPM-3726B: Scorecarding and Dashboarding with IBM Cognos TM1 V10.2 and IBM Cognos Insight

See http://www-01.ibm.com/software/data/2013-conference/ for info.

Obviously these are only provisional and there is nothing (yet) about specific performance improvements but hopefully we won't have to wait too long to find out