Pause between Meta and Data Parts of TI Process

[pdaser]

Hello everyone!

Maybe someone experienced the same phenomenon I currently got on my hands and can give me a hint:
I have a TI process (in 9.4.1) which reads data from SAP BW and writes it into a TM1 cube. In its Meta part the data coming in from SAP BW gets reformatted to match the elements in TM1's dimensions. After reformatting, test are run to see if the matching elements exist in TM1. If not, the process will break off and write some error message to a log file. That's everything this part of the process does. And it takes more than 5 minutes for roughly 2.000 sets of data!
I inserted timestamp outputs as the first and last statements in the Meta part. According to these timestamps working through the 2.000 sets takes 2 seconds.
I inserted a timestamp output as the first statement in the Data part. It says that the Data part starts 5 minutes after the last timestamp written by the Meta part.

What happens between the conclusion of the observable work in the Meta part and its effective end? (The log file of the Meta part won't be released for the 5 minutes it takes until the Data part starts.)

I'm thankful for any help!

Peter

[jim wood]

I've never really looked in to this in detail. I guess it depends on what you are doing in the meta tab and what selections you have in place. For example if you are updating a dimension in the meta tab and you have selected automatic hierarchy formatting it may be applying said formatting? I know the actual selection of this is applied in the prolog, but if you updated the dimension it may wait until the update is complete before applying the formatting?

This is only an educated guess on my part. I would use this as place to start rather than the answer,

Jim.

[qml]

Generally TM1 commits metadata changes in bulk after all iterations of the Metadata tab are finished. So depending on what exactly is happening in your code, this could be the cause of the delay.

[pdaser]

Guys,

thanks for your fast answers!

I changed my code somewhat, i.e. I removed a conditional metadata change from the Meta tab's code and another conditional metadata change from the Prologue tab's code. (Which in sum is all the metadata changes.) It shouldn't have mattered in my opinion because both conditions are not met in my test runs. Actually, there is a change: 1.5 of the 5 minutes delay happen somewhere during the Meta part now (in bulk), the other 3.5 minutes still happen at its end!?

Have a nice weekend everyone!

Peter

[lotsaram]

BW is not such a conventional data source as ODBC for example. Although it is true that meta data will be saved (i.e. dimension changes committed and saved to disk)at the conclusion of the meta data section, I think it is much more likely that the reason for the pause in execution is that TM1 is waiting at this point for BW to prepare the data query. Try optimizing the query or breaking it up into sub-sections and you might get a different result.

[pdaser]

Good morning, lotsaram!

Thank you for your insight.
But I don't understand it at the moment. The data from BW has already been queried for the Meta part, there is no different data to be read for the Data part!? (And there is no pause worth mentioning between Prologue and Meta part.)

Peter