Huge amount of memory used for input data

[its_me]

Hi all.

I'm after some help regarding memory use. A cube I have has the below values from stats by cube after restarting the instance. What I'm struggling to work out is how the memory used for input data is so huge in relation to the number of populated numeric cells.
The cube has 8 dimensions and none of them have a silly number of elements in either.

Is there something obvious I am missing here or is it just that the numbers displayed to me in this cube could be inaccurate?
Any help appreciated



Memory Used for Views 0
Number of Stored Views 0
Number of Stored Calculated Cells 0
Number of Populated String Cells 0
Number of Populated Numeric Cells 236167
Number of Fed Cells 54290624
Memory Used for Calculations 0
Memory Used for Feeders 16777216
Memory Used for Input Data 1811936256
Total Memory Used 1828713472

Thanks in advance

[Edward Stuart]

A very recent post looked at a very similar issue - http://www.tm1forum.com/viewtopic.php?f=3&t=12279

[its_me]

Thanks Edward. I think that thread was slightly different though, If I'm reading it correctly that post was more to do with memory used by input data increasing even if the same data was reloaded to the cube. My issue is just around the amount of memory used per input data value, straight after a restart.

[BrianL]

I was going to suggest you look at reordering the dimensions in the cube since that can sometimes result in a decrease in memory consumption for the cube. However, If a sub-optimal dimension ordering is used I'd expect to see the populated numeric cells to memory used for input data ratio to be very close to the ratio between number of fed cells and memory used for feeders. Might still be worth looking into...

[qml]

There have been issues with the reliability of these stats with some versions of TM1. Which version are you on?

To me it looks like in your case 'Memory Used for Input Data' and 'Memory Used for Feeders' are both wrong. Your feeder-to-populated-cell ratio is 230. This makes me think that your 'Memory Used for Input Data' includes some wrongly attributed memory that's really used by feeders. In a well-ordered 8-dimensional cube I would expect the average feeder flag to take up about 30 bytes and the average populated cell to take up about 50 bytes (both are ballpark figures of course!). The numbers calculated on the basis of the stats you provided are 3 and 7672, respectively - both orders of magnitude outside the believable range. So in your case I would trust the 'Total Memory Used', but not the split reported.

The true issue I see a strong potential for is the feeder-to-populated-cell ratio. While it's not impossible to have a cube where each populated cell legitimately feeds 230 other calculated cells, it is a very rare occurrence. You likely have some massive overfeeding and if you solve that then your total memory use will drop drastically (right now it's very high for the amount of data you are storing).

[Duncan P]

To me it looks like in your case 'Memory Used for Input Data' and 'Memory Used for Feeders' are both wrong

this is highly likely particularly given that

Memory Used for Feeders 16777216

is an exact power of 2.