TM1.Lock Contention happens on some dimension objects due to read operations in concurrent TI processes

[mce]

Hello,

We run the same TI process in parallel with different parameters to update different portions of a cube. However, we observe some locking contentions on some dimension objects causing the processes to roll back and re-execute. There is no dimension operations in this process that is running in parallel and there is no public MDX subsets in those dimensions. There are many dimensions in the model but this contention occurs only in few dimensions which has no different usage than other dimensions in the process.

In summary, Lock contentions occur where there is no reason for it to occur in TM1. This is causing lots of trouble and our concurrent processes that are updating data in parallel are all failing to commit and keeps rolling back and hence finish in very long time at the end since they rolled back.

We believe this is a defect in TM1's locking behaviour as per our observations.
Any comments would be helpful to guide us how we can resolve or work around this issue?
Thanks in advance.

Regards

[qml]

Do you have any functions in this TI which would operate on these locking dimensions, even as read-only? DIMIX/DIMNM/ATTRS...?

Are any of the locking dimensions ever updated by any of the *Direct functions - not necessarily in this process, but in general?

[mce]

Do you have any functions in this TI which would operate on these locking dimensions, even as read-only? DIMIX/DIMNM/ATTRS...?

Yes, we use DIMIX a lot. In an occasion, I replaced DIMIX by SubsetElementGetIndex, and it was still locking on that dimension.
We also use CellGetS for a cube that use the dimension that is being locked with contention.

Are any of the locking dimensions ever updated by any of the *Direct functions - not necessarily in this process, but in general?

The process that is running in parallel, does not make any dimension updates. It only updates data and parallel threads all hit different slices of the same cube.
In some occasions, we use direct function when updating dimensions, but this happens at a very different time than data update. After updating dimensions, we also make save to disk before we start updating cubes. Hence this should not impact. Are there any known issues with Direct functions?

[ykud]

We saw something similar in one of the models and unfortunately, there wasn't any real solution, we just had to 'commit' locks in different parts of the process earlier by running each portion as it's own RunTi with ExecuteCommand(...,1).

Can you try running this process in the TIDebugger and see what Locks it gets? If you enable Lock=Debug logging, you'd see what object causes contention before rollback and you'd be able to see where this object get's locked by the TI?

[qml]

I still don't trust the *Direct metadata functions very much. The fact you need to remember to run DimensionUpdateDirect after any such updates in order for the new elements to be fully functional tells you a bit about the downsides of overusing these functions. But I digress.

When I had issues with similar contention locking in the past, I discovered that the first DIMIX after a (regular, not necessarily Direct) dimension update would require an IX/W lock rather than the regular R lock. I worked around it by adding a dummy DIMIX in the Epilog of the processes that were updating the dimensions in question and that solved the issue. My hypothesis is that the first DIMIX call triggers some internal index caching or something similar.

[mce]

We saw something similar in one of the models and unfortunately, there wasn't any real solution, we just had to 'commit' locks in different parts of the process earlier by running each portion as it's own RunTi with ExecuteCommand(...,1).

We are already running the threads in different sessions using RunTI.

If you enable Lock=Debug logging, you'd see what object causes contention before rollback and you'd be able to see where this object get's locked by the TI?

It is already enabled, and we see the name of the dimension that is causing contention. But this dimension is never updated at all during the process. It is only ready using DIMIX or ELPAR or SubsetElementGetIndex. The same operations are done in all other dimensions, but it is causing contention only in this dimension. The stranger point about it, it does not happen in all executions, it happen only in some executions. I also observe that contention is more likely to happen when DIMIX function returns 0 in a lot more records.

[mce]

I still don't trust the *Direct metadata functions very much. The fact you need to remember to run DimensionUpdateDirect after any such updates in order for the new elements to be fully functional tells you a bit about the downsides of overusing these functions. But I digress.

When I had issues with similar contention locking in the past, I discovered that the first DIMIX after a (regular, not necessarily Direct) dimension update would require an IX/W lock rather than the regular R lock. I worked around it by adding a dummy DIMIX in the Epilog of the processes that were updating the dimensions in question and that solved the issue. My hypothesis is that the first DIMIX call triggers some internal index caching or something similar.

Thanks for the note. I will check this.

[PlanningDev]

Couldn't quite tell from the post but do you turn logging on/off in your TI's?

If so it turns out that CubeGetLogChanges/CubeSetLogChanges is locking. You would need to use CellPutS to the }CubeProperties cube for Logging instead. It is definetly causing locking for us especially when Chores start to run at the same time.

https://code.cubewise.com/blog/10-tips- ... pplication

[EvgenyT]

On the subject of CubeGetLogChanges/CubeSetLogChange
Had an issue with new }bedrock processes where it would cause a lock when executed in parallel via RushTI or RunTI.
There is a new parameter for logging in those processes (I can't remember exactly which }bedrock TIs were they). In the end, I had to hash out that code in }berock TIs to avoid lock contention.