TI import data for 100 lines take 15 min.

[harrytm1]

Hi all,

I'm using 9.5.2 version. Previously, TI is able to import flat files with tens of thousands of rows and it's ligtning fast.

Now, when I try to import a file with 100 rows into a Profit Loss cube, it takes 15 min and freezes up my Excel. Is this unique to 9.5.2 due to changes? if so, what changes and what can be done? Is it also due to TM1 updating all dependent calculations once a value is written a data point which is fed to those calculations? Hence, recalculation is done at every line.

Appreciate your kind advice.

Harry

[jim wood]

Have you tried running the process using architect rather than the excel add-on?

[lotsaram]

Have you tried running the process using architect rather than the excel add-on?

How would that make a difference? Once a process is running it is running on the server.

[jim wood]

I'm only trying to discount any possible excel issues. Even though the process is running through the server it still reports progress back through the add-in interface. It's not a solution only a question. I'm sorry if you couldn't see it's value.

[Alan Kirk]

I'm only trying to discount any possible excel issues. Even though the process is running through the server it still reports progress back through the add-in interface.

I have to admit I was wondering about that too; the potential issue with the dialog didn't occur to me. It's worth checking out. On a related point it's probably worth checking that there isn't a conflict between the server's port number and the ClientMessagePortNumber, if the latter has been set in the tm1s.cfg file. (Not likely, but no harm looking.)

Another way of avoiding that issue would be to schedule the chore rather than running it manually so that there's no dialog anywhere, and seeing whether that makes a difference.

I'm using 9.5.2 version. Previously, TI is able to import flat files with tens of thousands of rows and it's ligtning fast.

Now, when I try to import a file with 100 rows into a Profit Loss cube, it takes 15 min and freezes up my Excel. Is this unique to 9.5.2 due to changes?

I've noticed a couple of PMRs in the IBM Notifications e-mail which talk about customers encountering performance issues with 9.5.2 after upgrading from 9.5.1 though TI isn't mentioned in the synopsis. While I'm not suggesting that there is a chronic performance issue with 9.5.2 on such a limited sample, if you have a dev machine it might be an idea to set it up with 9.5.1, do the exact same import, repeat it on 9.5.2 and if you're getting the same slow-down, raise it as a PMR as soon as possible.

[Gregor Koch]

Hi

What do you mean by "Previously"?

Last week, no changes made to anything?
Before you changed the rules on the cube you are loading to?
Before you upgraded to 9.5.2, from which version?

Have you tried unloading the rules of the cube your are loading to?


Cheers

[harrytm1]

"Previously" means I used 9.4.1 and 9.5.1 to do similar import.

I did use ClientMessagePortNumber parameter in the cfg file.

I ran the TI through Server Explorer. Have not tried Architect though. Will give it a go.

Judging from the posts, seems like everyone else is not experiencing this problem?

[Steve Vincent]

By the wording of your question am i right in assuming your models have a lot of rules? If so check the server config file for "AllRuleCalcStargateOptimization". I've have finally found what was murdering my models after upgrading from 9.0 to 9.5.2 nearly 3 weeks ago. No matter what you set that parameter to (T/F) in some cases my models crash. #'d out and they are fine. If you use that config parameter get rid of it and try again (server restart required). I'm about to update IBM with my findings and hopefully get a fix for it.

[harrytm1]

hi Steve,

thanks for the heads-up on the cfg parameter. I tried it and it did not seem to improve the performance. My colleague did the following test by removing the rules on cubes that require calculations from the cube where the flat file is importing the data into (i.e. JIB PL and BS). here are the timings:

Enabled rules Time
enable rules in Only JIB PL & BS (cubes that import data) 0.23s
enable rules in Balance Sheet 12.54 s
enable rules in Profit Loss 432.85s
All rules enabled 1365.93 s

so it seems that 9.5.2 had introduced some feeders or rules handling that recalculate an imported data and the related calculation before moving onto the next row in the flat file.

Hope someone has some ideas how this whole thing works in TM1 now... appreciate your help in advance!

[Steve Vincent]

My models are very light on rules but i'd look in to all the various cfg parameters that may help or hinder your processes. Might even be worth posting the cfg settings for others to have a look at, they may spot something

[Martin Ryan]

Sounds like something worth raising with IBM. To keep things working in the meantime though you could delete the rule in the prolog of the TI process, do the import, then reimport the rule from a text file in the epilog.

Obviously less than ideal but if it saves you 14 minutes every time you load it might be worth trying until you can find something more permanent.

Martin