TM1 Forum

Hi all got something to ask about feeding.

Within same cube,
We got,
Active version: we use rule DB, stet rule for the non-active version
and Non-active version: we use TI to DB

We then use conditional feeder to feed only active version
Note: condition of version whether active or non-active, only to be changed approx. once a month.

we can accept the slowness once a month when condition change, if it will be quicker during when the conditions stay same.

Question:
1) Is this combination the best for the performance?

2) Rule vs TI : regards feeding same data
What if, we use rule for version, rule without version specification.
With number of non-active version never gonna change, will TI or rule be indifferent for the non-active version?
What make feeders slower than rules except the change of number in cells?

3) Conditional vs. non-conditional feeders
Assume we use TI for non-active version, does conditional feeders enhance performance?
(apart from the slowness of refeeding when change condition, i mean will conditional feeder help while condition not changed?)
In case the format of conditional feeders must be considered too
our conditional feeders currently are like:
[]=> DB(
IF(version=active, 'Source cube', '')
, dim1, dim2, dim3);


Thank you so much

Rather than using a conditional feeder my suggestion would be to use a consolidation of "active versions" and feed that as opposed to having a conditional feeder. (feeding a consolidation is shorthand for feeding all leaf descendants of the consolidated node.) Then to manage the active version all that needs to be done is edit the hierarchy, I think this would be easier and easily accommodates expanding or contracting the number of active versions. Note that this approach still has the same limitations as your current approach of having to reprocess feeders or resave rules when there is such a configuration change in order to ensure that all calculations are correctly fed and calculating as changing the hierarchy won't trigger retrospective feeding.

For performance it is a good idea to convert "non-active" versions to static data entry and fill via TI, but there is always a trade-off both in terms of dynamism of the model and the once-off processing time and complexity of coding to ensure the switch is 100% right with no loss or corruption of data.

Cside wrote:What make feeders slower than rules except the change of number in cells?

OK. Here you are getting confused. A feeder doesn't calculate anything, a feeder is not "slower than a rule". N level rules require feeders in order to consolidate (or calculate) correctly - you can think of a feeder as a "flag" to indicate to the consolidation engine that "this cell requires evaluation despite the fact it contains no input data". Without correct feeders N level rules will display incorrect results when queried at consolidated levels. Rules and feeders are 2 different halves of getting a calculation right.

"Slowness" comes from overfeeding. That is if the feeder flags are not only in cells where a rule is calculating something but also in cells that evaluate to zero, or worse cells where no rule is attached at all. This makes a cube more "dense" than it should be and therefore the consolidation engine has to stop off and evaluate a whole lot of cells that have no value. This is a wast of time (and processing power) and this is why feeders can make a model "slow" (when not done right). You get the same effect if feeders are not used at all (no SkipCheck statement) as this turns the sparse consolidation algorithm off entirely and effectively adds up and evaluates each and every cell in the cube as if the cube were 100% dense.

Cside wrote:Assume we use TI for non-active version, does conditional feeders enhance performance?

Yes if you get it right! The purpose of conditional feeders is to avoid feeding where you don't need to (that is where the target cell has no rule or has a rule but would evaluate to zero). Less feeder flags means faster consolidation/calculation as less cells need to be evaluated. You pay for this during server start-up and rule saves where the conditional feeders require additional evaluation to determine whether an input value should trigger population or not to the target cell(s).

Thanks a lot, appreciate your explanation. Now we know conditional feeder do help.

a little furthermore, this mean we must not use the persistent feeder, to let it reflag when conditional change?
(i assume persistent feeder mean not letting feeders reflag)

In addition, would like your recommendation.
As we use conditional feeder, active version condition change dynamicly to data of another cube, so there will be no rule saving after condition (active or non-active version) changed.
so i think it will auto re-evaluate the feeders when we open the cube cuz we did experience openning cube become slower.
Will it be a good idea to put the Cubereprocessfeeder in the TI after so as to let it reflag and so when open cube it does not re-feed?

What make feeders slower than rules except the change of number in cells?

This i'm very very sorry, my mistake. I intend to ask what make rule+feeders slower than TI except reflagging (which occur when condition change (condition which conditional feeder based on) )

As I'm comparing at those cells of non-active version between:
1.having rule+feeder attached to it or
2.having no rule, use TI

In other word,
what feeders do that add slowness (in case of not overfeeding)
- refeeding (or reflag), and
- evaluating as flagged? --> if this cause significant slowness then it make TI superior than rule+feeders

Hi
I was going to wait until lotsa comes back with the answers, which no doubt he will be able to provide if I get anything wrong here or misunderstand some of your questions/comments.
But I just can't contain myself...

Cside wrote: a little furthermore, this mean we must not use the persistent feeder, to let it reflag when conditional change?
(i assume persistent feeder mean not letting feeders reflag)

Generally Persistent Feeders won't prevent feeders, conditional or not, from re firing at run time. Persistent Feeders are basically a snapshot of what the feeders are at a certain point in time (when you do a SaveDataAll or a rule save on that cube). This snapshot is stored to disk (.feeders files) and is used at server start up time. If you use this setting feeders won't be set 'from scratch' or evaluated at start up time, they are only loaded from these files.
This will only help you speed up the start time of you server, not more not less.

What makes a feeder fire/reflag is discussed in many threads on this forum and I'll let you read those first. All I say is: changes in the LHS

Cside wrote: In addition, would like your recommendation.
As we use conditional feeder, active version condition change dynamicly to data of another cube, so there will be no rule saving after condition (active or non-active version) changed.

Going with your example

[]=> DB(IF(version=active, 'Source cube', ''), dim1, dim2, dim3);

If you change what is in active (say in a lookup cube), this won't re fire feeders unless you do a rule save or a CubeProcessFeeders.
It also won't re fire if you have set ReevaluateConditionalFeeders=T in your .cfg file.

Apart from this I would hardly ever use a [] on the left side of the feeder statement because this will run through every single stored number or fed calculated cell in your cube and then check whether it should feed something else - that can't be good, can it?

Cside wrote: so i think it will auto re-evaluate the feeders when we open the cube cuz we did experience openning cube become slower.

God no, what makes you think that. Opening a cube view has absolutely nothing (not directly anyways) to do with the re-evaluation of feeders. It's the re calculation of that view. What you might have experienced is the difference in reading cached and un-cached data. Please do a search on 'Stargate'.

Cside wrote: Will it be a good idea to put the Cubereprocessfeeder in the TI after so as to let it reflag and so when open cube it does not re-feed?

On the first part, as mentioned above it would be one way of making sure your feeders re fire if you can't re write them. On the second part "open cube = re feed", see above (shudders).

Cside wrote: This i'm very very sorry, my mistake. I intend to ask what make rule+feeders slower than TI except reflagging (which occur when condition change (condition which conditional feeder based on) )

As I'm comparing at those cells of non-active version between:
1.having rule+feeder attached to it or
2.having no rule, use TI

In other word,
what feeders do that add slowness (in case of not overfeeding)
- refeeding (or reflag), and
- evaluating as flagged? --> if this cause significant slowness then it make TI superior than rule+feeders

Here I am struggling a bit to understand you.
Generally speaking, consolidating stored numbers will be quicker than performing a calculation on the fly and then consolidating the results. But if the calculations are not complex and you are not overfeeding the difference won't be all that unbearable. So it really depends on the nature of and how "live" the numbers need to be to make a call what is 'superior'.
Maybe I don't quite understand your whole active version situation and what it is supposed to do.
You mentioned that the active version changes once a month, but I take it the data within that version might then change subsequently within that month?
If yes, and with the given information you could keep the TI approach for inactive versions as they don't change any more. But why not change your feeder to

[Active Version]=> something with no condition;

Change the active version in the rule, re save.
Yes, you would have to change it every month, but it is very easy and keeps you from running TIs as soon as the numbers change.
It's not ideal in terms of maintenance and others probably would frown upon it but hey...it would perform.

This is only if your active version really needs to change. Because better even, have one version (or more but they are predefined) that is always your active version and store it of to static versions with TI when appropriate.


Cheers

Thanks a lot Gregor, really appreciate for clearing the cloud.

I forgot to mention that the cell to which TI run data to, is a stet-rule cell.
I guess you knew because i think the only time we use conditional feeder is that when the rule must be stet for one or another condition.
Here is my doubt,
which argument is correct, and are supporting ideas relevant?

Argument A: The conditional feeders is useful because on stet-rule cell, the flags (RHS of feeder statement) still do slower consolidation by means of checking or occupying RAM?

Argument B: The flags do not affect speed of consolidation. If this is the case, conditional feeder will not different from non-conditional feeders (feeding every condition).

Support idea for argument A
If flags on stet-rule cells do not affect speed of consolidation,
what for and when conditional feeder is useful anyway?
(cuz my understanding is: rule that use conditional feeders is always stet rule, and feeding on stet-rule cell and not feeding it are the same)

Support idea for argument B
I suspect that the flagged might not get checked because
1) when stet-rule cell are not fed, it did not get suppressed when suppress zero (I tried restarting the model after commenting feed already)
It is like the tool perceives the stet-rule cell as not having rule.
If the act of deciding whether the cell is rule applied-- and if finding no rule (stet rule) does eliminate the step of checking flags in the Consolidation calculation
then this means the flags will not slow down the consolidation.

Cside, I think you should really read again about what feeders are and how they work. I'm sure it'll start making sense after going through a couple of threads on the forum. The way you've worded your post makes me wonder if you got the concept wrong or if you're just not using the right words to explain what you mean.

There is plenty of material on feeders (including conditional feeders) on this forum, but one of the simplest explanations can be found in this old thread. Hope it helps.

This link might also help.
http://bedrocktm1.org/resources/files/W ... _Rules.pdf

Thanks a lot guys, I'll try my best