TM1 Forum

Hi,

Has anyone seen a situation where the CUB file size has dramatically increased, without any noticeable increase in data ?

I have a cube that has been running at circa 330-380Mb for some time. Its a Balance Sheet cube that contains a dimension "Version", containing around 40 elements. Overnight I have run a simple process to create a new Version element and copy all data from one version to the new one. This will increase the size to some extent, but the process took 10 times the length expected and the CUB file has grown to 700Mb, and completed successfully, with entirely normal results (other than the size).


I have removed all rules and feeders and restarted the server with a backup of the cube at 350Mb, and the current 700Mb, so I can compare them side by side.

Reviewing in Server Explorer gives no additional data on-screen, running Performance Monitor and comparing in Stats by Cube gives me almost identical numbers of numeric cells and memory size. If I dump out the contents of the cubes to CSV files, I get very similar record counts (just different by the data in the copied version). Other than the one new element, all the dimensions and elements are the same and are shared by both cubes of course. I have also checked for temporary files but there are no cub$ files floating around so I am assuming it's not temporary data that didn't rollback for some reason.

It's almost as if there is some orphaned data held in the CUB file somehow, but oddly if I purge both cubes with a Clear All process, the both go back to 572Kb basic size.


I am about to try exporting, purging, and reimporting the data to see if this reverts the cube back to normal size, but in the meantime does anyone know why/how this might happen and whether there are any other checks I could make to try and understand why the filesize has grown so much?

I'm using 10.2.2 FP3

Thanks

Has there been a recent dimension re-order? That can pretty dramatically affect cube size.

conflax wrote:I have a cube that has been running at circa 330-380Mb for some time. Its a Balance Sheet cube that contains a dimension "Version", containing around 40 elements. Overnight I have run a simple process to create a new Version element and copy all data from one version to the new one. This will increase the size to some extent, but the process took 10 times the length expected and the CUB file has grown to 700Mb, and completed successfully, with entirely normal results (other than the size).

Just how "simple" was this process? Did you copy only leaf level and also skip rule calculated values during the copy? If either of those statements is false then i can see how it might take a long time and take up more space in the new .cub file. What about rules? Do all the same rules apply to both versions or did you range-value some of the stuff?

Hi,

Thanks for the quick responses

BrianL, good point but no there hasn't been a dimension re-order

Tomok the process initially builds subsets based on MDX filtering to lowest level - written in the TI at runtime not taken from a dynamic subset - apart from the one Dimension that's the copy which contains just the destination element. It does select Rule driven values as it is also used for range-value copies elsewhere, so this could explain why it takes a long time to run, but the CELLPUT is wrapped in a CELLISUPDATEABLE so I am assuming that whilst it is reading these values it wouldn't be doing anything with them in the Data tab.

However, in terms of the CUB file size I was under the impression that this was purely related to saved lowest-level data and not affected by Rules or Feeders?

conflax wrote:However, in terms of the CUB file size I was under the impression that this was purely related to saved lowest-level data and not affected by Rules or Feeders?

It is, which is why I asked the question about range-valuing. You gave precious little detail about the process so I had to assume the worst case scenario. Since you don't know what the problem is, you haven't re-ordered the cube, and I am positive that cube size is a factor of stored leaf level data I suggest you export the cube, skipping consols and rule-calculated intersections to a file and load it into Excel to compare the values in the two versions. My guess is that you've got a lot more data in the "copied" version than you think you have.

Hi,

Apologies yes I probably should have posted the TI code but I suppose I was asking a more general question about whether this had been seen before. I have now exported both cubes and haven't looked at the detail yet as too big for Excel/Notepad/Access - but the text file sizes are almost identical, I also then purged the 700Mb cube and reimported the same text file (from the 700Mb cube source), and having saved data it's now back to 350Mb! Immediate problem solved but I am going to continue to try and understand why it happened in the first place and what caused the increase.

Do you have or did you have UNDEFVALS in the rules file for the cube?

No the rules file doesn't have UNDEFVALS set.

Also FYI I have since realised my data export was only for N level cells, so am running another export to include consolidations where cell type is string, just in case there are some rogue string entries in a consol that wouldn't have been picked up before.