PI and Cube Dependancies on Server start up.

[Steve Rowe]

Hi,

We've recently rewritten one of large data processing jobs to take advantage of PI and the 32 cores on our box and now have a job that was taking an hour to run taking about 5 minutes, happy days in deed.

The one space we have not cracked is getting the PI to work on first run when the server launches.
1. Server launches.
2. We've a TI that runs that sets the cube dependencies, this is run first separately from the main job.
3. The main job runs but in serial mode.
4. No new cube dependencies are created in the log file while 3 is running.
5. Subsequent runs of the main job execute in parallel, so whatever is breaking on the first run is being fixed during the first run.

The main job does include some dimension maintenance but this happens in the initial serial part of the job where the preparation for the parallel chore launch happens and in any case the job runs in parallel every time after the first run so there is nothing in the job itself that breaks PI.

Has anyone encountered this before or have any clues? Any suggestions of log file settings that might help track what is locking the PI?

(I've not really been a fan of the recent trend for many cored "slow" processor boxes but use of PI really makes full use of all this spare capacity, you do need a lot of cores to take advantage of it though, we are using 20 at once in our new job and it has been a revelation, to me at least.)

Cheers,

[TableManagerOne]

Hi Steve, if the serialization is due to conflicting lock modes (IXC in top) you might start with the TM1.Lock.Exception logger at DEBUG level. It'll produce something similar to the following:

Code: Select all

1232   [2]   DEBUG   2013-08-09 23:24:33.919   TM1.Lock.Exception   Lock conflict acquiring 'IX' lock. Thread '9192' holds 'IX' lock. Object 'PriceCube' , type 'Cube'.

This logger was introduced somewhere in the 9.5.2 stream... Maybe FP2?

If you don't have it, you can fall back on some of the more verbose loggers (like setting TM1.Lock=DEBUG.)

[Steve Rowe]

Hi, Thanks for the tip I'll try that out, much appreciated.

[David Usherwood]

Wouldn't it be luverly if IBM saw fit to document the logging options?

[Duncan P]

Hi Steve,

What version are you running on, what mechanism are you using to run this job on server start-up and what are you using to multi-thread the main job (perhaps TM1RUNTI.EXE)?

Cheers,
Duncan.

[Steve Rowe]

Hi,

We are on 952 SP2 (I think, pretty hard for me to confirm here so take that with a pinch of salt, the SP bit anyway).

We are only running the jobs manually on start-up, next step would be some form of automation around running the cube dependency job on start-up. Maybe since it appears to be a zero cost job we can just run it every time we run the serial part of the main job, anyone know if there is a downside to this?

In terms of launching the parallel jobs we have a cube dimensioned by thread that contains the parameters of the work we want to do (lets call this thread.properties), this is populated by the serial part of the job that runs every 2-3 minutes which looks in a schedule cube to see if there is any work to do in a given 15 minute slot.

Alongside this we have 20 chores that all run the same TI but have a different parameter so that they take their main call parameters from the thread.properties. These run every 30 seconds but only do work when thread.properties is correctly populated.

Not had chance to test out the logging parameter yet.

Cheers,

Steve