9.5.2 - New Features and Issues

[moby91]

Where did you get the information that the new Data Reservation feature is tied to the new Parallel Interaction feature ?

According to the chapter on Data Reservation in the manual TM1 9.5.2 Developer Guide it can be used no matter if Parallel Interaction is enabled or disabled.


See the Data Reservation Overview in the manual TM1 9.5.2 Developer Guide

http://publib.boulder.ibm.com/infocente ... rvw_N32049
TM1 Developer Guide 9.5.2
Using Data Reservations >
Data Reservation Overview

How to use Data Reservations

To use Data Reservation, you must first enable and configure the feature for individual cubes and user groups and then use TI or API functions to programmatically apply and manage DRs.

To enable Data Reservation, use the following TM1 tools:
* }CubeProperties control cube - Enables and configures Data Reservation for individual cubes. For details, see Enabling Data Reservation for cubes.
* Capabilities Assignments - Determines if the members of a user group can manage (acquire and release) Data Reservations for themselves and other users.

[David Usherwood]

Thanks for that. Clearly the ADE (Applix Documentation Elf) is still on the payroll - remember PersistingofFeeders?
And, depending on your font, lowercase el followed by uppercase eye is not good - thank heavens case doesn't matter (I believe).
Rereading the docs, firstly, I see nothing in the section on Data Reservation saying that it does, or doesn't, link with Parallel Interaction aka Cube Versioning. However it seems to me that they are both aiming, in different ways, to address the concurrency issue.
My post focused on whether the calculation cache was retained when a data reservation was made and the evidence so far it that it is not.

[paulsimon]

Alan

I thought it might be worth mentioning another thing about .EXD files. One of our users had Excel 2010 with TM1 9.5.1. As it didn't work, IT uninstalled 2010 and re-installed 2007. Even then the Client would not work. I had to delete the EXD file, and repeat the process of uninstaling and reinstalling a few times before I could finally get it to work, and even then it was only happy to work if I started Excel 2007 with a blank workbook first. The moral of the story is don't try 2010 with anything before 9.5.2 as you cannot even back it out reliably. I am guessing that some links get stuck in the registry, but cleaning the registry is something that I leave to IT. I could do it, but I don't want to be held responsible.

I am just negotiating to get the go ahead to test 9.5.2 now. They are rather cautious. However, they also have a CRM system which required 2010, so that may provide the necessary push.

Regards

Paul Simon

[marco]

In the 9.5.2 NewFeatureGuide I have found the following:

"New functions for Rules and TurboIntegrator
The following new functions are available in TM1, version 9.5.2 for use with rules and TurboIntegrator:
● ConsolidatedMin
● ConsolidatedMax
● ConsolidatedAvg
● ConsolidatedCount
● ConsolidatedCountUnique
Note: These functions were not documented in the standard TM1 9.5.2 documentation. For complete
details, search for these functions in the latest release documentation and updates on the IBM®
Cognos® TM1® Info center."

Has anyone found further documentation of these new functions from some source?

There is a technote with an example:
https://www-304.ibm.com/support/docview ... wg21472749

ConsolidatedMax; ConsolidatedMin; ConsolidatedAvg; ConsolidatedCount
These functions calculate the maximum, minimum, average or count across all the elements in a consolidation and returns a single value. These functions can be used in TurboIntegrator processes or Rules.

Syntax

ConsolidatedMax(flag-value, cube-name, element_1, element_2,
… );
ConsolidatedMin(flag-value, cube-name, element_1, element_2,
… );
ConsolidatedAvg(flag-value, cube-name, element_1, element_2,
… );
ConsolidatedCount(flag-value, cube-name, element_1, element_2,
… );

he flag value is the sum of the following values:
1 - do not use consolidation weighting when computing the value. If this is turned on the raw value of the consolidated element will be used.
2 - ignore zero values. If this is set, zero values will not be used as part of computing an average.
Example
['Apr']=ConsolidatedMax( 0, '', !actvsbud, '1 Quarter' );

[Steve Rowe]

Interesting I wonder what happens to performance when you set the calc to ignore zeros, looks like a lot of care would be needed when that was used.

[George Regateiro]

Just saw this come across the RSS from IBM

https://www-304.ibm.com/support/docview ... s=swgimgmt


A fairly good run down on the Parallel Interaction feature

[mattgoff]

A fairly good run down on the Parallel Interaction feature

Reader vs writer locking is the last major performance issue I face. Through cube (re)design I've managed to get rid of most of the other blocks. 9.1 helped but was a bit of a disappointment-- fingers crossed it's for real this time. Now if I can only get them to fix replication....

Matt

[Steve Rowe]

It's good to see this level of detail in the docs, in the Applix days the detail of how the engine work seemed to more of mystic lore passed from mouth to ear.
The new functionality looks good as long as it doesn't slow readers down to much when it's off.
Just the calculation engine to get multi-threaded I guess?
Cheers

[David Usherwood]

Just running up a 952 server
a ParallelInteraction is _not_ documented in the Ops Guide
b (Printable) suggestions on the meaning of the last two lines in the log are invited. I'd look in the docs, but hey....

3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server The server will use Parallel Interaction.
3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server Server threading mode is thread per connection.
3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server Client Message Port does not accept Tm1Top connections.

[Alan Kirk]

My attention has been drawn to a March technote about parallel interaction which may be of interest to some, and which as far as I can see hasn't been mentioned yet:

https://www-304.ibm.com/support/docview ... wg21460097

Problem(Abstract)
While running in Parallel Interaction mode, the IBM® Cognos® TM1® server uses only a single thread to process requests in the Job Queue.

Resolving the problem
The JobQueueThreadPoolSize parameter is a new server configuration parameter that enables TM1 to use multiple threads to process the TM1 Job Queue, providing greater thru-put and processing of sandbox requests when the TM1 server is configured to use Parallel Interaction (PI).

NOTE: The Parallel Interaction feature was initially documented and referred to as Cube Versioning.

When the server is not running in PI mode, this parameter has no effect, and only a single thread is used to process requests in the Job Queue.

This parameter is static. If you change this parameter value, you must restart the TM1 server to apply the new value.

Configure this parameter in the Tm1s.cfg file using the following format:

JobQueueThreadPoolSize=x

where x represents the number of threads you want to use for processing TM1 Job Queue requests.

For example:

JobQueueThreadPoolSize=3

By default, if this parameter is not set, then TM1 uses a value of 1.

That last point may be causing some problems for those who want to use this feature.

[comma]

That last point may be causing some problems for those who want to use this feature.

Which point do you mean exactly?

[Martin Ryan]

a ParallelInteraction is _not_ documented in the Ops Guide

Ah. Glad I wasn't the only one who couldn't find it.

b (Printable) suggestions on the meaning of the last two lines in the log are invited. I'd look in the docs, but hey....

3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server The server will use Parallel Interaction.
3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server Server threading mode is thread per connection.
3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server Client Message Port does not accept Tm1Top connections.

No idea on number two unless it relates to Alan's post.

For number three I just noticed when installing TM1 that you need to specify a client message port, and that by default it is blank. Perhaps you left it blank? Or if you did set it to something then maybe a firewall setting needs to be changed.

It's been a while since I've done a clean install but I don't remember the Client Message Port being required before. Is this new? Or am I being forgetful?

[Alan Kirk]

a ParallelInteraction is _not_ documented in the Ops Guide

Ah. Glad I wasn't the only one who couldn't find it.

b (Printable) suggestions on the meaning of the last two lines in the log are invited. I'd look in the docs, but hey....

3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server The server will use Parallel Interaction.
3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server Server threading mode is thread per connection.
3904 [] INFO 2011-04-10 19:29:00.907 TM1.Server Client Message Port does not accept Tm1Top connections.

No idea on number two unless it relates to Alan's post.

For number three I just noticed when installing TM1 that you need to specify a client message port, and that by default it is blank. Perhaps you left it blank? Or if you did set it to something then maybe a firewall setting needs to be changed.

It's been a while since I've done a clean install but I don't remember the Client Message Port being required before. Is this new? Or am I being forgetful?

You may not have come across it in past versions; as far as I can tell it first came in in 9.4.

My reading of the operations guide is that it isn't in fact required and will be assigned if not specified. Reading more between the lines than the lines themselves, it sounds more like David got himself a conflict with the port number that the server had assigned. I don't imagine that the message port number has a valid context when it comes to TM1 Top, since that isn't a real "client".

Identifies a secondary port used to accept client messages concerning the progress and ultimate cancellation of a lengthy operation without tying up thread reserves.

This additional port ensures that other server requests can continue to process while waiting for a cancellation from the user.

By default, this port number is automatically and dynamically assigned when the TM1 server starts. You do not have to set ClientMessagePortNumber to a specific number unless firewalls or other network issues require the listener port to be a well‐known number.

NOTE: Be sure to assign unique port numbers for the server and client message ports. If you have two servers running on the same machine with the same port number, the message activity may cause a system failure.

See also PortNumber and ProgressMessage.

[Steve Rowe]

Problem(Abstract)
While running in Parallel Interaction mode, the IBM® Cognos® TM1® server uses only a single thread to process requests in the Job Queue.

Resolving the problem
The JobQueueThreadPoolSize parameter is a new server configuration parameter that enables TM1 to use multiple threads to process the TM1 Job Queue, providing greater thru-put and processing of sandbox requests when the TM1 server is configured to use Parallel Interaction (PI).

Alan quoted the above re a tech note and the reference to sandboxing is confusing to me.

Does the parameter JobQueueThreadPoolSize just apply to user sand boxes?
or
Does the whole Parallel Interaction mode and JobQueueThreadPoolSize cfg just apply to sand boxes?
or
Is there some kind of system sand box being created behind the scences which is how Parallel Interaction is being delivered and this is where the reference to sandboxing comes from in the tech note?
or Something else??

The message seems a little confused or am I misssing something?

Cheers,

[Martin Ryan]

As I understand it parallel interaction is separate from sandboxes. It allows simultaneous writes and reads to occur without slowing each other down. It uses multiple calculation trees to deliver the read request at the time it is requested, allowing a separate calculation tree to deal with subsequent writes. Once that read is complete the out of date calculation tree is cleared out.

However one type of writes that will be occurring during parallel interaction is sandboxes being submitted into the main (i.e. non-sandbox) version. These submissions are maintained by the job queue. If multiple users are submitting multiple sandboxes to the main version then by default only one thread will be used, which can create a bottleneck. This parameter will allow multiple threads to write back to the main version.

I'm not sure how that actually works in terms of the calculation trees that PI is creating, but I suspect while it will generally faster, it may lead to a high RAM overhead if there are multiple reads going on at the same time as the writes are happening.

[dfrench77]

mikko wrote:
kpk wrote:
In the 9.5.2 NewFeatureGuide I have found the following:

"New functions for Rules and TurboIntegrator
The following new functions are available in TM1, version 9.5.2 for use with rules and TurboIntegrator:
● ConsolidatedMin
● ConsolidatedMax
● ConsolidatedAvg
● ConsolidatedCount
● ConsolidatedCountUnique
Note: These functions were not documented in the standard TM1 9.5.2 documentation. For complete
details, search for these functions in the latest release documentation and updates on the IBM®
Cognos® TM1® Info center."

Has anyone found further documentation of these new functions from some source?

I've got a service request in on it.

Did anyone get any information on the above new functions? Did the IBM service request provide any information on the documentation for these functions?

[Alan Kirk]

mikko wrote:
kpk wrote:
In the 9.5.2 NewFeatureGuide I have found the following:

"New functions for Rules and TurboIntegrator
The following new functions are available in TM1, version 9.5.2 for use with rules and TurboIntegrator:
● ConsolidatedMin
● ConsolidatedMax
● ConsolidatedAvg
● ConsolidatedCount
● ConsolidatedCountUnique
Note: These functions were not documented in the standard TM1 9.5.2 documentation. For complete
details, search for these functions in the latest release documentation and updates on the IBM®
Cognos® TM1® Info center."

Has anyone found further documentation of these new functions from some source?

I've got a service request in on it.

Did anyone get any information on the above new functions? Did the IBM service request provide any information on the documentation for these functions?

Beware of the invalid attribution... it was me who wrote that a service request had been submitted, not any of the people named above.

In response the following Technote was created, which I didn't post the details of because Marco had already seen it before I got into work that day. He had already posted a link to it in a post further up the page:
https://www-304.ibm.com/support/docview ... 72749&wv=1

[dfrench77]

mikko wrote:

Beware of the invalid attribution... it was me who wrote that a service request had been submitted, not any of the people named above.

In response the following Technote was created, which I didn't post the details of because Marco had already seen it before I got into work that day. He had already posted a link to it in a post further up the page:
https://www-304.ibm.com/support/docview ... 72749&wv=1

My apologies for the invalide attribute...thanks for pointing out that the link is available for the documentation.

[Alan Kirk]

Just came across an interesting new "Proven Practice" that Iboglix has posted in relation to creating a summary cube to track data reservations. It's not something I've played around with myself, nor have I tested the method proposed, but it may be of interest to some 9.5.2 users:

http://www.ibm.com/developerworks/data/ ... ge577.html

[Andy Key]

I've just had a look at that document and thought I would give it a go as I haven't touched DR yet.

Apart from the fact that the code doesn't save due to a missing '1' on the RHS of the Control Cube check line, and the fact that the resulting DRsummary cube will only show the first DR for a }Cube/}Client combination not the complete list as the description of 'all the data reservations on all available cubes in a TM1 server' suggests...