Server Load Performance

[mce]

Hi,

When starting TM1 service, it starts loading the data and calculations into memory and until it does server stays unavailable. I observe in Task Manager that tm1sd.exe is handling this.
However it is using only one CPU to do this. It does not utilize the rest of the CPUs.

Is this the case specific to our version of TM1 (9.0) or is it the same for all versions?
Is there a way to make it using more CPUs to finish it faster?

Thanks in advance for all replies.

Regards,

[kielmc]

Check out the MaximumCubeLoadThreads parameter in the TM1s.cfg file. You're better off testing to find how many processors you should use than following the suggested "Number of Processors - 1".

[kpk]

Check out the MaximumCubeLoadThreads parameter in the TM1s.cfg file. You're better off testing to find how many processors you should use than following the suggested "Number of Processors - 1".

It is better to check the release notes but as far as I remember the MaximumCubeLoadThreads parameter is only available in 9.1 and newer versions and not in 9.0.

[Alan Kirk]

Check out the MaximumCubeLoadThreads parameter in the TM1s.cfg file. You're better off testing to find how many processors you should use than following the suggested "Number of Processors - 1".

It is better to check the release notes but as far as I remember the MaximumCubeLoadThreads parameter is only available in 9.1 and newer versions and not in 9.0.

'Fraid not, it came in in 8.4.5. From the 8.4.5 release notes:

Multi-Threaded Feeder Processing
In previous versions of TM1, server loading (including the loading of cube data and the calculation of feeders) was a single-threaded process.

In TM1 8.4.5 the cube load and feeder calculation phases of server loading can be multi-threaded, so multiple processors can be used in parallel. This results in decreased server load times.

To enable multi-threaded cube loading and processing of feeders, you must add the MaximumCubeLoadThreads parameter to the Tm1s.cfg file.

However it does appear that it wasn't documented (outside of the release notes) until the 9.1 release; I can't find it referred to in the 9.0 .pdfs.

[kpk]

Hi Alan,

You have found the parameter in the RN of 8.4.5 and not in the RN of the 9.0.
One question: isn't it possible that the 8.4.5 was released later than the release of 9.0?

Of course it should be tested but this can be a logical reason if the parameter is in the version of 8.4.5 and missing not only from the RN of 9.0 but from that release at all.
I hope I am wrong.

Regards,
Peter

[Alan Kirk]

You have found the parameter in the RN of 8.4.5 and not in the RN of the 9.0.
One question: isn't it possible that the 8.4.5 was released later than the release of 9.0?

Of course it should be tested but this can be a logical reason if the parameter is in the version of 8.4.5 and missing not only from the RN of 9.0 but from that release at all.
I hope I am wrong.

This comes back to a recent post about about the release dates of each version. It would have actually been useful to keep track of that but it just didn't seem to be important at the time, and I notice that no-one else has had a reply for that post either.

The earliest install package of 9.0 that I have (which was originally going to be called 8.5 from memory) has a file date of 22-Dec-05. The 8.4.5 one is 18-May-07. The 9.0 SP3 one (not the U9 one) is 11-Jan-07. Unfortunately this isn't conclusive since I know that on occasions I went back and downloaded install packages of earlier versions just in case I ever needed them (a habit I picked up after the Great 8.2.10 and 8.2.11 Disasters) so these may bear no relation to the dates of release; 8.4.5 may have been released earlier than that. However given that Applix had jacked the proposed 8.5 into a complete new version number as 9.0, it would seem unusual that they'd go to the trouble of implementing the parameter in an 8.4 release (which was already shuffling its way to the boneyard at the time) and not implement it in the 9.0 flagship (at the time) edition. And of course it's not unknown for Applix to have {cough} "overlooked" doing documentation updates.

However this isn't academic for me either as I've recently moved from 8.2.12 servers to 9.0 ones and was interested in trying out the parameter, which could speed the load considerably. I've lodged a support request asking whether it was implemented in 9.0 and will post the answer back here. (SR 04004,999,616 so that I can find this thread again when I get an answer.)

[kpk]

Based on the posts (mining:) from the previous TM1 forum this parameter should be in 9.0 SP3 but probably not in previous 9.0 versions.

Wednesday, March 21, 2007 11:35 AM: Multiprocessor Performance...
Applix have introduced the multi-thread load option for 8.4.5 and for 9 SP3. This should help utilise your cpus. (Does not work if you have variable feeders.)

Friday, January 05, 2007 3:06 PM: 9.0 SP3 is now available for download
TM1 9.0 SP3 is now available for download from the Applix web site. TM1 9.0 SP3 contains all Hot Fixes developed to date for TM1 9.0 SP2.
Details of the fixes can be found in the attached release notes (also available for download).
Please note that all other manuals and documentation have not been changed, and are available from the SP2 download location.

Wednesday, November 08, 2006 5:41 PM: TM1 8.4.5 released...
2. Multi-Threaded Feeder Processing during Server Loading
Using a configuration file setting, the cube load and feeder calculation phases of server loading can leverage multiple processors in parallel, resulting in faster server load times.
Note: This option is not applicable for TM1 models using 'conditional feeders'.

Wednesday, June 28, 2006 11:21 PM: TM1 9 SP2 is now available...

Wednesday, December 21, 2005 11:23 PM: Applix announces the general availability of TM1 9.0...
... TM1 9.0 is built on the same core engine as TM1 8.4.3 with some additional fixes. ...

[David Usherwood]

I'd have to recheck but I'm pretty sure that multithreaded load was _removed_ in 9.0 SP anything.
It came back in 9.1+ but didn't always help startup time, and if you gave it too many cores memory could baloon. I've tried it on 9.5.1 and it seemed fine there.
Note also that if you write your conditional feeders sensibly, controlling them from a set of flag cubes, conditional feeders can work very well with multithreaded load.

[Alan Kirk]

I'd have to recheck but I'm pretty sure that multithreaded load was _removed_ in 9.0 SP anything.

As per IBM support this morning:

MaximumCubeLoadThreads is supported in 9.0 SP3

I'm looking forward to trying it out, actually...

[David Usherwood]

Just added the MCLT parameter to the config of a good hefty client server (set to 4) and run it up on an 8 core box under 9.0 SP3 U9 (the version the app was built on 6 months ago). There are _no_ 9.1+ style threading feeder messages and _no_ use of > 1 core.
(Alzheimer's holding off for a few weeks more then.... )

[Alan Kirk]

Just added the MCLT parameter to the config of a good hefty client server (set to 4) and run it up on an 8 core box under 9.0 SP3 U9 (the version the app was built on 6 months ago). There are _no_ 9.1+ style threading feeder messages and _no_ use of > 1 core.
(Alzheimer's holding off for a few weeks more then.... )

Not at IBM Support if that's correct.

I'm still going to take a whack at it on mine and see what happens, just for curiosity's sake if nothing else.

[mce]

I tried using this MCLT parameter in 9.0 SP1, and it did not let tm1sd.exe use more than one CPUs.

[kpk]

I tried using this MCLT parameter in 9.0 SP1, and it did not let tm1sd.exe use more than one CPUs.

Hi,

Only sp3 of 9.0 was released later than the 8.4.5 (first official release of this parameter). Therefore you should try sp3.

[David Usherwood]

I did. (It's the main one most of our larger clients use.) It's not supported.

[mce]

Strangely I received the responde from IBM Support as "the MaximumCubeLoadThreads parameter is supported in TM1 9.0 SP1".

[David Usherwood]

I'm afraid I'm not prepared to go back and dig out 9.0 SP1 to disprove this statement - it wasn't a very good version as I recall. Anybody who has a testbed to hand is welcome to do this and post the results.

[mce]

I already asked for a re-confirmation from IBM for 9.0 SP1. I will post the response here when received.

[Alan Kirk]

I already asked for a re-confirmation from IBM for 9.0 SP1. I will post the response here when received.

If it's anything like the last one it won't be worth the electrons it's printed on. I just re-opened my own SR with a request for "clarification":

OK, so the parameter is supposedly available in 9.0 SP3. So, how is it that David Usherwood found that it isn't (http://forums.olapforums.com/viewtopic.php?f=3&t=3141), and I found that when I loaded a feeder-rich (and we're talking, feeders EVERYWHERE) server without the parameter it took 8 minutes 5 seconds to load, and when I implemented the parameter and set it to 3 then (a) I saw no additional threads launch and (b) the server load took 8 minutes and 6 seconds, close enough to make no difference?

Curious minds want to know, and all that.

[Alan Kirk]

I already asked for a re-confirmation from IBM for 9.0 SP1. I will post the response here when received.

If it's anything like the last one it won't be worth the electrons it's printed on. I just re-opened my own SR with a request for "clarification":

OK, so the parameter is supposedly available in 9.0 SP3. So, how is it that David Usherwood found that it isn't (http://forums.olapforums.com/viewtopic.php?f=3&t=3141), and I found that when I loaded a feeder-rich (and we're talking, feeders EVERYWHERE) server without the parameter it took 8 minutes 5 seconds to load, and when I implemented the parameter and set it to 3 then (a) I saw no additional threads launch and (b) the server load took 8 minutes and 6 seconds, close enough to make no difference?

Curious minds want to know, and all that.

I think they've kinda-sorta conceded that it isn't implemented in version 9.

I'm not 100% clear on whether they have or not, but for all intents and purposes I'm taking the last reply as saying that. Effectively. And in the face of empirical evidence, that's the way I'm going to read it.

[Michel Zijlema]

This maybe a bit off-topic regarding the discussion above, but I think still worthwile to mention.
I remember testing the multithread load parameter on a 8.4.5U2 (IA64) TM1 server that at that moment took at least 4 hours to start. Using this version the parameter indeed enabled the TM1 server to use multiple cores on loading and could have been of big help... if it didn't have the side effect of almost doubling RAM usage. With a TM1 server using 24 Gb of RAM on a machine with 32 Gb of RAM using the parameter forced the TM1 server into virtual memory, rendering this 'optimization' unusable. According to Applix Support at that time, the doubling of RAM usage was 'expected behaviour'.

Michel