How do feeders actually work?

Post Reply
User avatar
Martin Ryan
Site Admin
Posts: 1988
Joined: Sat May 10, 2008 9:08 am
OLAP Product: TM1
Version: 10.1
Excel Version: 2010
Location: Wellington, New Zealand
Contact:

How do feeders actually work?

Post by Martin Ryan »

I started an Olapedia article on TM1 Feeders, explaining them as I understand them and promptly got corrected by another user.

I said
"SkipCheck;" indicates that the rules should only calculate cells with a feeder flag.
He said
Not quite correct
"Skipcheck" turns the Sparse Cosnolidation Algorythm back on which by default gets turned of once there is a rule on a cube. The rule will always calculate but the result will not necessarily (unless fed) consolidated. Skipcheck doesn't mean 'don't calculate unless fed'.
I was aware of the term sparse consolidation algorithm, but ignored it for simplicity's sake. I was however surprised by the assertion that rules always calculate. To my mind, the calculation is what actually takes the time, so if all the zero cells are still being calculated and then ignored at consolidation, it will still take a long time.

Surely an unfed cell will not be calculated AND it will also be ignored by the consolidation?

Martin
Please do not send technical questions via private message or email. Post them in the forum where you'll probably get a faster reply, and everyone can benefit from the answers.
Jodi Ryan Family Lawyer
User avatar
Steve Rowe
Site Admin
Posts: 2410
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: How do feeders actually work?

Post by Steve Rowe »

It's a pretty fine distinction.

An _N_ level rule based cell will always get calculated if you ask for the result in an un-zero suppressed view, this is ture irrespective of skipcheck and feeder status. This holds even if the cell is dependant on other unfed _N_ level values, but not if it is dependant on C level values that are ruled/skipchecked and unfed.

The feeder flags indicate to the system that a cell might have a value if you are using zero-suppression at the N level, or if the system is calculating a consoldated value.

An unfed cell will not be displayed (or calculated?) if looking at the cube zero suppressed, it will always be ignored by the consolidation.

I think the above is true, whish is mostly the same as what you said.

It does get confusing though as even in a zero suppressed view you can get un fed values displaying.

Say you have a cube viewer laid like with 9 N level values like this and zero suppression on.

A B C
D E F
G H I

Say value E is unfed, it will still display the correct result in the cube viewer since you are forcing the system to return the value as it is required to complete the layout of the cube viewer. If you were to reorientate the cube viewer so all the values appeared as a column then value E will not display since it is not fed and the cell is not required to complete the display.

This is probably confused the issue even more :lol:
Technical Director
www.infocat.co.uk
User avatar
Martin Ryan
Site Admin
Posts: 1988
Joined: Sat May 10, 2008 9:08 am
OLAP Product: TM1
Version: 10.1
Excel Version: 2010
Location: Wellington, New Zealand
Contact:

Re: How do feeders actually work?

Post by Martin Ryan »

Steve Rowe wrote: An unfed cell will not be displayed (or calculated?) if looking at the cube zero suppressed, it will always be ignored by the consolidation.
Your question in the parenthesis is precisely what I'm getting at. If an unfed cell is not displayed, is it actually calculated? The more I think about it, the more I think it makes no sense for it to be calculated. If it was calculated and all that skipcheck did was prevent it being consolidated, that would mean only a minimal time saving as consolidations are pretty straight forward once the number is there, whereas calculations to arrive at the number (that is subsequently being ignored by the consolidation) can be all over the place.
It does get confusing though as even in a zero suppressed view you can get un fed values displaying.
Yes, I think the fact that sometimes you can see unfed numbers being calculated does confuse things a lot.

Last time feeders came up we had a veerrry long conversation. And there's a note in that thread that the original Yahoo group also had a really long discussion about them. Funny how it's really difficult to completely understand them!

Martin
Please do not send technical questions via private message or email. Post them in the forum where you'll probably get a faster reply, and everyone can benefit from the answers.
Jodi Ryan Family Lawyer
User avatar
John Hobson
Site Admin
Posts: 330
Joined: Sun May 11, 2008 4:58 pm
OLAP Product: Any
Version: 1.0
Excel Version: 2020
Location: Lytham UK
Contact:

Re: How do feeders actually work?

Post by John Hobson »

Scary isn't it that some of the most experienced and sharpest TM1 brains around with decades of experience between us are sill not QUITE sure how feeders work.

Dealing in relatively dense and high level models as I generally do in planning, and where the rules can get quite complex, I generally avoid feeders wherever I can in the interest of maintainability. (I know this is probably heretical, but I am sure I am not really alone). In my experience over 13 years now, only seasoned admin have anything approaching a decent understanding of feeders and even then the potential for error is still vast.

When I tested the difference in our OTB model, we were seeing a model with skipcheck and feeders calculating top level consolidations in about 5 second instead of 15. With most of the planning taking place at low level and 1000 lines of rules the benefits of maintaining feeders for the performance increase just don't stack up to me. The only thing that makes me wonder about it is the abysmal performance over a WAN in Excel, where any performance increase would be welcome.

I do realise that with large sparse reporting model with a few simple rules the performance gain is well worth the candle.

What still worries me is the fact that when I last checked ASCII export was still unreliable for calculated variables in cubes without skipcheck and feeders. I haven't checked in 9.1.4 and all my enquiries have now been resolves - or should that be "disappeared" so I have no idae if they thik they fixed it.
John Hobson
The Planning Factory
User avatar
Martin Ryan
Site Admin
Posts: 1988
Joined: Sat May 10, 2008 9:08 am
OLAP Product: TM1
Version: 10.1
Excel Version: 2010
Location: Wellington, New Zealand
Contact:

Re: How do feeders actually work?

Post by Martin Ryan »

John Hobson wrote: I do realise that with large sparse reporting model with a few simple rules the performance gain is well worth the candle.
Most definitely. I recently fine tuned some feeders that cut down a high level view from 90 seconds to 30 seconds. I hate to think what it'd be without the feeders at all.

I agree though, for small or dense cubes they're often not worth the extra work.

Martin
Please do not send technical questions via private message or email. Post them in the forum where you'll probably get a faster reply, and everyone can benefit from the answers.
Jodi Ryan Family Lawyer
User avatar
Steve Rowe
Site Admin
Posts: 2410
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: How do feeders actually work?

Post by Steve Rowe »

I think you can infer that the unfed cells do not get calculated in a suppressed view from the response time you get.
Also logically since the cell is unfed the calculation engine does not know that there might be a value there and so doesn't calculate it. The trouble is you cannot actually prove this since looking to see if the value is there forces it to calculate. Quantum Physics in action! Sort of...

Actually you might be able to prove it by looking in the performance monitor at the calculated cell counts?

I think the problem with both the statements in the OP is that they don't make a distinction on zero-suppression. Your statement is true if looking at zero-suppresed views, the other true if not.

On the subject of deep technical understanding, I did do a document when I was working at InfoCat that I thought did a pretty good job of explaining what was going on under the hood, not sure if they would be prepared to post it...
Technical Director
www.infocat.co.uk
ScottW
Regular Participant
Posts: 152
Joined: Fri May 23, 2008 12:08 am
OLAP Product: TM1 CX
Version: 9.5 9.4.1 9.1.4 9.0 8.4
Excel Version: 2003 2007
Location: Melbourne, Australia
Contact:

Re: How do feeders actually work?

Post by ScottW »

The rule will always calculate
The distinction that is missing from this quote is " ... if queried directly" for example in an non zero supressed view or directly in Excel via DBRW. The result of the calculation however won't necessarily be correct if the rule is not fed or intermediate calculations that it relies on are not fed.

If one rule calculation flowed into another and the first was not fed and the downstream cell was then queried the first cell would NOT calculate if it were not fed* due to the sparse consolidation algorithm (provided SkipCheck was turned on of course.)

* unless it were a C level rule and leaf cells below it contained real input data

Complicated ain't it.
Cheers,
Scott W
Cubewise
www.cubewise.com
Gregor Koch
MVP
Posts: 263
Joined: Fri Jun 27, 2008 12:15 am
OLAP Product: Cognos TM1, CX
Version: 9.0 and up
Excel Version: 2007 and up

Re: How do feeders actually work?

Post by Gregor Koch »

Hi

Starting to read the thread I was going to add Scott's bolt statement to clear things up.
As far as I understand it this applies to both calculations and consolidations: they are only calculated if queried. But imo it would be incorrect to state that a rule is not calculated if not fed.
The other way around: a rule calculated cell is skipped (not caclulated) when consolidating if it is not fed.

Cheers
glaurens
Posts: 41
Joined: Thu Jan 08, 2009 3:42 pm

Re: How do feeders actually work?

Post by glaurens »

Gregor Koch wrote:Hi

Starting to read the thread I was going to add Scott's bolt statement to clear things up.
As far as I understand it this applies to both calculations and consolidations: they are only calculated if queried. But imo it would be incorrect to state that a rule is not calculated if not fed.
The other way around: a rule calculated cell is skipped (not caclulated) when consolidating if it is not fed.

Cheers
or, phrased differently: If a tree in a forest falls, with nobody around to hear it, does it make a sound?

G
User avatar
John Hobson
Site Admin
Posts: 330
Joined: Sun May 11, 2008 4:58 pm
OLAP Product: Any
Version: 1.0
Excel Version: 2020
Location: Lytham UK
Contact:

Re: How do feeders actually work?

Post by John Hobson »

or, phrased differently: If a tree in a forest falls, with nobody around to hear it, does it make a sound?
Schrödinger's cube?
John Hobson
The Planning Factory
David Usherwood
Site Admin
Posts: 1453
Joined: Wed May 28, 2008 9:09 am

Re: How do feeders actually work?

Post by David Usherwood »

There's got to be a way of explaining this simply. Feeders are a notorious barrier for TM1 users climbing the learning curve - Stephen Waters suggests the climbing term 'overhang' as a term for a major barrier which puts people off. I suspect many of us are over the overhang, but we mustn't let it block other users - especially the EP community who will (should) be getting involved.
User avatar
Martin Ryan
Site Admin
Posts: 1988
Joined: Sat May 10, 2008 9:08 am
OLAP Product: TM1
Version: 10.1
Excel Version: 2010
Location: Wellington, New Zealand
Contact:

Re: How do feeders actually work?

Post by Martin Ryan »

David Usherwood wrote:There's got to be a way of explaining this simply.
I think there is a simple way of explaining it, but the simple way isn't necessarily perfectly correct. I wrote a simple explanation on Olapedia and was corrected because some parts were an oversimplification. So a nice simple explanation for a new user is fine, but then a more advanced user comes along and things need to be clarified further.

Intracube feeders are actually pretty simple in their essence I think, and when you explain it with a simple x=y*z formula thus y=>x people can see immediately what's going on. The trick is when you get into intercube feeders and feeding subsequent periods using attributes and what not.

Hopefully between us all we can manage to fine tune the feeder article into something that is simple, as well as correct.

Martin
Please do not send technical questions via private message or email. Post them in the forum where you'll probably get a faster reply, and everyone can benefit from the answers.
Jodi Ryan Family Lawyer
User avatar
John Hobson
Site Admin
Posts: 330
Joined: Sun May 11, 2008 4:58 pm
OLAP Product: Any
Version: 1.0
Excel Version: 2020
Location: Lytham UK
Contact:

Re: How do feeders actually work?

Post by John Hobson »

Feeder syntax question:

I just came across a feeder written in the form

['a'] => ['b'],['c];

Now I would normally have written 2 feeders and the manual doesn't say that you can do it like this, so I thought I'd check if the panel sees this as valid.

B and C are both elements in the same dimension as A.
John Hobson
The Planning Factory
User avatar
Martin Ryan
Site Admin
Posts: 1988
Joined: Sat May 10, 2008 9:08 am
OLAP Product: TM1
Version: 10.1
Excel Version: 2010
Location: Wellington, New Zealand
Contact:

Re: How do feeders actually work?

Post by Martin Ryan »

Yep, another short cut. Works well.

Might have been omitted from the manual to save confusion.

Martin
Please do not send technical questions via private message or email. Post them in the forum where you'll probably get a faster reply, and everyone can benefit from the answers.
Jodi Ryan Family Lawyer
User avatar
John Hobson
Site Admin
Posts: 330
Joined: Sun May 11, 2008 4:58 pm
OLAP Product: Any
Version: 1.0
Excel Version: 2020
Location: Lytham UK
Contact:

Re: How do feeders actually work?

Post by John Hobson »

Thanks Martin - the things you learn after 13 years eh :shock:
Might have been omitted from the manual to save confusion.
Like everything else on feeders is clearas crystal huh ;)
John Hobson
The Planning Factory
User avatar
Steve Vincent
Site Admin
Posts: 1054
Joined: Mon May 12, 2008 8:33 am
OLAP Product: TM1
Version: 10.2.2 FP1
Excel Version: 2010
Location: UK

Re: How do feeders actually work?

Post by Steve Vincent »

John Hobson wrote:Feeder syntax question:

I just came across a feeder written in the form

['a'] => ['b'],['c];

Now I would normally have written 2 feeders and the manual doesn't say that you can do it like this, so I thought I'd check if the panel sees this as valid.

B and C are both elements in the same dimension as A.
One of the Applix consultants wrote some of our rules that way. Its not always the clearest method, but sometimes it saves a lot repetition too.
If this were a dictatorship, it would be a heck of a lot easier, just so long as I'm the dictator.
Production: Planning Analytics 64 bit 2.0.5, Windows 2016 Server. Excel 2016, IE11 for t'internet
sarahmilius
Posts: 14
Joined: Tue Nov 18, 2008 5:48 pm
OLAP Product: TM1 EP
Version: 9.4x 9.5x
Excel Version: 2007 2010
Location: US-Northeast

Re: How do feeders actually work?

Post by sarahmilius »

John Hobson wrote:Feeder syntax question:

I just came across a feeder written in the form

['a'] => ['b'],['c];
John,
this syntax has appeared in 2 versions of class manuals (May, Sept 2008) for the developer classes taught by IbmCog.
That said, the remaining class coverage on Feeders will do little to help new folks over the overhang. The coverage is rather sparse ;)

Feeders seem to be one of those topics where you realize a nuance from every discussion. Today, for me, it was, ...won't display if the cell depends on an unfed calculation elsewhere....

- Sarah
ParisHilton
Posts: 73
Joined: Fri Apr 23, 2010 11:35 am
OLAP Product: Tm1
Version: 9.5
Excel Version: 2007 2010

Re: How do feeders actually work?

Post by ParisHilton »

John Hobson wrote:Feeder syntax question:
['a'] => ['b'],['c];
This syntax is still being taught ( June 2010).

The course does 'teach' skipcheck and feeders. Probably enough to get you through the developer exam but way short of what is being discussed here.

I am trying to set up a trivial example to show at a simplistic level what's going on, but at the moment it seems to ignore my skipcheck, most likely this is my novice status shining through, or it could be the cube is so tiny, it doesn't bother with skipcheck. My gut feeling is I am examining the intermediate results and forcing a calc.

Will post back a nice TI process if I get it sorted!
“The way I see it, you should live everyday like its your birthday”


TM1 9.5 Cognos BI 8.4 Excel 2007. 128GB Ram. E7450@2.40Ghz -24 core. Fluffy white dog.
Herman Moller
Posts: 70
Joined: Thu May 22, 2008 3:38 pm

Re: How do feeders actually work?

Post by Herman Moller »

To feed or not to feed.....

If you don't feed a system it starts up time will be lighting fast but when doing calculations it will take a long time to cache. If you have feeders in a system the start up time is slower but the response time for the calculations is much faster.

“Some small, simple rules applications can work in TM1's default mode (without sparse consolidations), but applications of larger size begin to run very slowly. To solve this problem, TM1 lets you restore sparse consolidation and instruct the consolidation engine where to look for rule-derived values. During consolidations, TM1 uses a sparse consolidation algorithm to skip over cells that contain zero or are empty. This algorithm speeds up consolidation calculations in cubes that are highly sparse. A sparse cube is a cube in which the number of populated cells as a percentage of total cells is low

To do this, you use a set of instructions in the rule for the cube:

In the Rules Editor, insert a SKIPCHECK declaration, which forces TM1 to use the sparse consolidation algorithm in all cases.

SKIPCHECK;


Create feeder statements, which cause placeholder values to be stored in rules-calculated cells, so that the cells are not skipped during consolidations.

Feeder area => reference to rules-calculatedvalue;

The feeder area identifies a component of a rule-derived value. When this component contains a non-zero value, a placeholder is fed to cells containing the rules-calculated value identified on the right-hand side of the statement. This placeholder alerts the TM1 consolidation engine to include the rule-derived value when performing consolidations.”

In the end it depends how much is going on in the system
iansdigby
Community Contributor
Posts: 109
Joined: Thu Feb 26, 2009 8:44 am
OLAP Product: TM1
Version: 9 + 10 + Plan An
Excel Version: All
Location: Isle of Wight, UK

Re: How do feeders actually work?

Post by iansdigby »

It helps me to remember that memory is just a great (organised) pile of bits which are either on or off. Program instructions point to selected bits to see if they are on or off and act accordingly. What follows is my (quite probably wrong) understanding of the process:

Here is an imaginary TM1 cell, that has a value, in memory (I invented all the addresses):
  • Addresses 1-100 contain dimension element names that define the area.
    Addresses 102-200 contain the value.
    Address 201 - contains the feeder flag (maybe set to 1).
When you write ['a']=>['b'];
and save the rule, the program looks for every cell whose area coordinates include ['a'] and, if there is a value in it, turns on the feeder bit for the corresponding'['b'] cell.

Before a user or process requests a view, the only data in a cube is:
  • area co-ordinates
    raw numbers
    feeder flags
. To meet the view request, TM1:

1. Calculates everything in the view according to the rules.
2. Consolidates the view along the dimension hierarchies. If the cell has a real (not calculated) value, it gets included. If the cell has a feeder flag it gets included. If it has neither, it doesn't get included.

Most of the 'cube' is still just sitting in memory, being values and feeder flags. The view is probably a tiny part of the whole cube.

I think there must be two passes by the program, one to calculate the requested cells and another to consolidate. TM1 doesn't decide not to calculate an un-fed cell, because you can see the calculated values in un-fed view cells at the N level. And it wouldn't gain much by doing so (unless the rule were badly written) - millions of zeros skipped probably far outweigh the number of calculated cells.

TM1 is ipso facto efficient because it only stores raw values and feeder flags, as well as calculating only when a view is requested.
"the earth is but one country, and mankind its citizens" - Baha'u'llah
Post Reply