Page 1 of 1

Rules effect on a Data load

Posted: Tue Sep 28, 2021 10:10 pm
by bubidibabadi
Hello All,

We have a cube 'A' which has a lot of rules and feeders in it. When i try to load the data via a TI (around 84 millions records!), the speed is really slow. However, if i load the same data into a cube that doesn't have any rules, its way faster.

Does this mean that the rules affect data loads? I am just curious on why this would happen. Are there any other ways of improving the performance apart from removing and re applying the rules after the load?

Thank you in advance for your reply.

Re: Rules effect on a Data load

Posted: Tue Sep 28, 2021 11:09 pm
by burnstripe
I suspect you have the parameter forcereevaluationoffeedersforfedcellsondatachange in the tm1s.cfg (service config for the model) set to T (True), when this is in place each time data is changed the feeder is reevaluate, which is causing the slow performance.

If this parameter were to be switched to F (false) the load would run a lot faster because it wouldn't be re-evaluating the feeders constantly.

If you have conditional feeders in the model then you'll need to control when to refire the feeders manually, via process or a service restart (if using persistent feeders set to F also in tm1s.cfg)

Re: Rules effect on a Data load

Posted: Fri Oct 01, 2021 8:30 am
by Elessar
Hi,

As said before, it's definitely feeders. You need to switch them off before loading data

Create a blank rule file (or a rule file with rules, but without feeders): let it be "dummy"
In Prolog: RuleLoadFromFile('Sales', 'dummy');
In Epilog: RuleLoadFromFile('Sales', '');

Re: Rules effect on a Data load

Posted: Fri Oct 01, 2021 1:26 pm
by Emixam
Hello,

I had the same problem and I used the bedrock process to unload/load rule in my cube before I load my data

https://github.com/cubewise-code/bedroc ... manage.pro

And then make sure to reprocess feeders at the end

https://github.com/cubewise-code/bedroc ... eeders.pro

Re: Rules effect on a Data load

Posted: Fri Oct 01, 2021 3:07 pm
by lotsaram
Emixam wrote: Fri Oct 01, 2021 1:26 pm Hello,

I had the same problem and I used the bedrock process to unload/load rule in my cube before I load my data

https://github.com/cubewise-code/bedroc ... manage.pro

And then make sure to reprocess feeders at the end

https://github.com/cubewise-code/bedroc ... eeders.pro
We do the same but man you really wonder why TI just doesn't have a native function like DisableFeederProcessing('CubeName') or DisableRules('CubeName') which could just be called on the Prolog and then reset on the Epilog. Would make things so much easier. (Hint, hint Stuart King, Hubert Heijkers. Or just give us a version of TM1 where there is no performance impact loading data to a slice which feeds dependent calculations).

Re: Rules effect on a Data load

Posted: Mon Oct 04, 2021 7:19 am
by Wim Gielis
lotsaram wrote: Fri Oct 01, 2021 3:07 pm We do the same but man you really wonder why TI just doesn't have a native function like DisableFeederProcessing('CubeName') or DisableRules('CubeName') which could just be called on the Prolog and then reset on the Epilog. Would make things so much easier. (Hint, hint Stuart King, Hubert Heijkers. Or just give us a version of TM1 where there is no performance impact loading data to a slice which feeds dependent calculations).
Never used before, but there is a setting called SaveFeedersOnRuleAttach.
https://www.ibm.com/docs/en/planning-an ... ruleattach
It's a dynamic parameter.

Re: Rules effect on a Data load

Posted: Mon Oct 04, 2021 7:51 am
by lotsaram
SaveFeedersOnRuleAttach !
Interesting, another tm1s.cfg parameter I never heard of or used before! :lol: