Rules effect on a Data load

Post Reply
bubidibabadi
Posts: 10
Joined: Tue Jul 23, 2019 7:19 pm
OLAP Product: TM1
Version: 10.2.2, Planning Analytics wit
Excel Version: 2010

Rules effect on a Data load

Post by bubidibabadi »

Hello All,

We have a cube 'A' which has a lot of rules and feeders in it. When i try to load the data via a TI (around 84 millions records!), the speed is really slow. However, if i load the same data into a cube that doesn't have any rules, its way faster.

Does this mean that the rules affect data loads? I am just curious on why this would happen. Are there any other ways of improving the performance apart from removing and re applying the rules after the load?

Thank you in advance for your reply.
Regards,
bubidi.
burnstripe
Regular Participant
Posts: 165
Joined: Wed May 06, 2020 2:58 pm
OLAP Product: Planning Analytics
Version: 2.0.9
Excel Version: 2016

Re: Rules effect on a Data load

Post by burnstripe »

I suspect you have the parameter forcereevaluationoffeedersforfedcellsondatachange in the tm1s.cfg (service config for the model) set to T (True), when this is in place each time data is changed the feeder is reevaluate, which is causing the slow performance.

If this parameter were to be switched to F (false) the load would run a lot faster because it wouldn't be re-evaluating the feeders constantly.

If you have conditional feeders in the model then you'll need to control when to refire the feeders manually, via process or a service restart (if using persistent feeders set to F also in tm1s.cfg)
User avatar
Elessar
Community Contributor
Posts: 296
Joined: Mon Nov 21, 2011 12:33 pm
OLAP Product: PA 2
Version: 2.0.9
Excel Version: 2016
Location: Russia
Contact:

Re: Rules effect on a Data load

Post by Elessar »

Hi,

As said before, it's definitely feeders. You need to switch them off before loading data

Create a blank rule file (or a rule file with rules, but without feeders): let it be "dummy"
In Prolog: RuleLoadFromFile('Sales', 'dummy');
In Epilog: RuleLoadFromFile('Sales', '');
Best regards, Alexander Dvoynev

TM1 and Data Science blog: 6th article - PAfE + VBA: Commit each cell without pressing “Commit” button.
Emixam
Posts: 135
Joined: Tue May 21, 2019 3:33 pm
OLAP Product: TM1
Version: PA 2.0.x
Excel Version: 2016
Location: The Internet

Re: Rules effect on a Data load

Post by Emixam »

Hello,

I had the same problem and I used the bedrock process to unload/load rule in my cube before I load my data

https://github.com/cubewise-code/bedroc ... manage.pro

And then make sure to reprocess feeders at the end

https://github.com/cubewise-code/bedroc ... eeders.pro
lotsaram
MVP
Posts: 3590
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Rules effect on a Data load

Post by lotsaram »

Emixam wrote: Fri Oct 01, 2021 1:26 pm Hello,

I had the same problem and I used the bedrock process to unload/load rule in my cube before I load my data

https://github.com/cubewise-code/bedroc ... manage.pro

And then make sure to reprocess feeders at the end

https://github.com/cubewise-code/bedroc ... eeders.pro
We do the same but man you really wonder why TI just doesn't have a native function like DisableFeederProcessing('CubeName') or DisableRules('CubeName') which could just be called on the Prolog and then reset on the Epilog. Would make things so much easier. (Hint, hint Stuart King, Hubert Heijkers. Or just give us a version of TM1 where there is no performance impact loading data to a slice which feeds dependent calculations).
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Wim Gielis
MVP
Posts: 2957
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.14
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Rules effect on a Data load

Post by Wim Gielis »

lotsaram wrote: Fri Oct 01, 2021 3:07 pm We do the same but man you really wonder why TI just doesn't have a native function like DisableFeederProcessing('CubeName') or DisableRules('CubeName') which could just be called on the Prolog and then reset on the Epilog. Would make things so much easier. (Hint, hint Stuart King, Hubert Heijkers. Or just give us a version of TM1 where there is no performance impact loading data to a slice which feeds dependent calculations).
Never used before, but there is a setting called SaveFeedersOnRuleAttach.
https://www.ibm.com/docs/en/planning-an ... ruleattach
It's a dynamic parameter.
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly https://github.com/wimgielis
lotsaram
MVP
Posts: 3590
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Rules effect on a Data load

Post by lotsaram »

SaveFeedersOnRuleAttach !
Interesting, another tm1s.cfg parameter I never heard of or used before! :lol:
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Post Reply