TM1 Forum

Hi,
I would like to measure the impack of multithreaded support in Cognos 10.2 to performance.

I have read article http://ykud.com/blog/cognos/tm1-cognos/ ... ance-today There are tips to improve performance. The biggest benefit is stated as migrating from Cognos 10.1.+ to Cognos 10.2.+ There is info: "With MTQ they’ll finally be calculated on multiple CPUs at the same time, giving you near linear increase in performance. That’s right, with 4 cores the same view will be rendered almost 4 times faster"


I did the test.


I. Original Machine
a) Intel 4 CPU core machine (I can see four CPU columns in Windows Task Manager)
b) Windows 2008 R2 64-bit
c) Cognos BI 10.2.1 64-bit fixpack 6 (reports are here)
d) Cognos TM1 10.1.1
e) Cognos 10.1.1 64-bit fixpack 2


II. New Cognos 10.2.2 machine
I created virtual machine clone of above I. machine, so a) to d) are exactly the same.
e) Cognos 10.2.2 64-bit fixpack 2
In tm1s.cfg file I have added the new line at the bottom of file:
MTQ=4

Before test both virtual machines Windows restarted to be sure there is all started from zero and having no benefit of already in memory data.


Now tests:
In both machines (I. and II.) I have run Cognos BI one single report at the time to measure up an execute time with stopwatch and observe the Task Manager Process tab.

The result reports where executed in almost the same time and in both cases I have seen tm1sd.exe occupying 25% of CPU which is exactly 1 of 4 CPUs.
The vast majority of time I see tm1sd.exe in Task Manager in both I. and II. machine, so I think performance problem is at TM1 and not at Cognos BI site.

It looks to me that multithreading is not working in my case on Cognos 10.2.2 or at least it appears so - 25% of CPU consumption and the same reports execution times.

On II. computer I checked the tm1server.log file:

5572 [] INFO 2015-05-15 08:43:09.872 TM1.Server The server will use Parallel Interaction.
5572 [] INFO 2015-05-15 08:43:09.872 TM1.Server Server threading mode is thread per connection.
5156 [] INFO 2015-05-15 08:43:10.247 TM1.Server Dynamic Client Message Port: 52425
5572 [] INFO 2015-05-15 08:43:10.247 TM1.Server MTQ is enabled with 3 threads


From blog post: http://blog.tm1tutorials.com/2013/09/11 ... #more-1052 I see: "“Server threading mode is thread per connection” – weird enough since MTQ was enabled when this line appeared.

If I understand correctly multithreading is on, but for some reason is not accepted in my case.

Do I need to set something else to enable multithreading in TM1?

Is Cognos BI able to get benefit of TM1 multithreading?

Thanks

In tm1s.cfg file I have added the new line at the bottom of file:
MTQ=4

I guess you have to add it as
MTQ=T

Where as Tm1s.cfg file has the parameter IntegratedSecurityMode set to 4 or 5 for security.

BariAbdul wrote:

In tm1s.cfg file I have added the new line at the bottom of file:
MTQ=4

I guess you have to add it as
MTQ=T

Where as Tm1s.cfg file has the parameter IntegratedSecurityMode set to 4 or 5 for security.

No.

MTQ=4 would be correct as you have to specify the number of threads it is to use.

You would be better off monitoring this in the operations console as a query is executed and you would see something along the lines of "> Work Unit for 2258" etc multiple times.

Note that anything saying you would see a "linear performance increase" is very dubious. There are a number of posts on here detailing the circumstances but a number of factors will reduce the increase; for example the fact that the initial query request and final compilation of data can only be single threaded - it will only be that actual retrieval of values that can be performed in multiple threads; if you have rule calculated values you may find that a number calculated in Thread 1 is also required to calculate a number derived in Thread 4 but Thread 4 cannot communicate with Thread 1 and as such the number would be calculated twice etc etc etc.

To really see the performance increase I would enable stargate debug level of logging and see how long the view takes to calculate via the logs.

declanr wrote:You would be better off monitoring this in the operations console as a query is executed and you would see something along the lines of "> Work Unit for 2258" etc multiple times.

I currently don't have TM1 Operations Console installed. I don't use TM1 Web server, so I didn't even install TM1 Operation Console. If I am not mistaken for TM1 Operation Console it also has to be some java server like Apache Tomcat installed. Isn't it?

declanr wrote:Note that anything saying you would see a "linear performance increase" is very dubious.

I was sceptical about this, but at least if there is multiple CPU threads are used it should be at least some performance increase. Now I have been monitoring with Process Explorer (freeware tool), to see if multiple threads were used during report execute, but it never was. So I suspect that this multithread support does not work in my case (see print-screen cpu_threads.jpg bellow).

declanr wrote:To really see the performance increase I would enable stargate debug level of logging and see how long the view takes to calculate via the logs.

Can you please point me to some documentation how to do this? What do you mean by "how log the view takes to calculate"? I don't use any views in TM1 - see views.jpg - I was never able to establish Cognos BI to use TM1 views, so I don't use them. I don't know if this is not supported or I just was unable to perform this task.

Stanislav2 wrote:Can you please point me to some documentation how to do this?

This is the best post on the forum about logging stargates.

Stanislav2 wrote: 5572 [] INFO 2015-05-15 08:43:10.247 TM1.Server MTQ is enabled with 3 threads

MTQ is in fact enabled with 3 threads.

Stanislav2 wrote:5572 [] INFO 2015-05-15 08:43:09.872 TM1.Server Server threading mode is thread per connection.

This is always printed and has been for a while. Doesn't seem to have anything to do with MTQ (or anything else). I'd assume it's another one of those log messages added as part of a new feature that IBM never finished/shipped/told us about.


Looks like MTQ is enabled but not getting triggered for your particular query.

Here's IBM's documentation http://www-01.ibm.com/support/knowledge ... ption.html

Duncan has a better suggestion for stargate logging (setting will reduce noise from the logs) but the IBM link mentions some other MTQ specific loggers.

It is important to note that MTQ will not be triggered by all queries. Many of the operations that BI kicks off are not multi threaded on TM1 while some are. The same goes for things like a some TI process operations. The MTQ engine tries to determine if a query is a candidate for multi threading, if not it will stay single threaded.

The best way to ensure MTQ is enabled in your environment is to make a large view in the cube view and monitor operations console/tm1top. If you see the "work unit" items then you have it enabled.

The activity that gets split into multiple threads is stargate view creation; things like rule calculations etc will not be split out. Remember the view has to be sufficiently large to trigger branching off. By default worker threads appear after 10000 cells are processed, but that can be lowered by using the MTQ.OperationProgressCheckSkipLoopSize parameter in tm1s.cfg (e.g. setting it to 1).