.blb and .rux files - maintenance

Post Reply
iahobson
Posts: 3
Joined: Fri Mar 18, 2016 8:27 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

.blb and .rux files - maintenance

Post by iahobson »

Hi

This is my first post on this forum. I have read posts multiple times and find that it is more useful than other sources when troubleshooting! I realize that this topic has been discussed multiple times but I can't find an agreed way of maintaining rules in performance modeler.

We introduced TM1 to our business last year (10.2.2) and we are now at the stage where we are trying to improve performance. We upgraded to Fix pack 5 last week too. Our model is using approximately 30GB of memory and the regeneration of feeders is taking anything up to 5 hours. We use performance modeler unless it is absolutely necessary to use architect.

To address the feeders issues, we are looking to remove rules and replace with TI processes wherever we can. My question is what is the safest process to remove/modify rules? We have run some tests but we're not seeing the performance improvements we hoped. Some extra info that may help to understand our efforts so far are as follows:

- Some forum articles suggest to 'disable' the rules within the rule editor (right click the rule and select disable). This changes the RUX file for that cube but not the BLB file. How does the disabling of the rules get flowed through to the BLB file - assuming that the BLB file is being used over the RUX?

- Changing a rule within the cube calculation window (right clicking within the cube and selecting 'Open Cube Calculation') creates/modifies a number of BLB files (prefixed 'CompileCubeCalc...', 'CubeCalculation...', 'CompiledLinkDefn...') as well as the RUX file. In theory this should mean that this approach gives consistency between RUX and BLB files, but I think I am right in saying that the model uses the BLB files if they exist.

- Despite disabling/deleting countless rules, the .FEEDERS files don't appear to be getting any smaller. The RAM usage appears to be dropping, but I would have expected the FEEDERS files to shrink given that there are now less feeders. We have stopped the service, deleted the .feeders files, and restarted a number of times now.

- We tried deleting BLB files and restarting but this prevented us from using the rule editor in performance modeler, so we rolled back that change.

Thanks in advance for your help. I apologise in advance if I've missed anything obvious!

Ian
BrianL
MVP
Posts: 264
Joined: Mon Nov 03, 2014 8:23 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2 PA2
Excel Version: 2016

Re: .blb and .rux files - maintenance

Post by BrianL »

The TM1 server exclusively uses the .RUX files as the definitive source of rule statements. Any .BLB files are artifacts created by the client application (in this case performance modeller). Typically they store things like additional formatting details, etc...

You will need to shutdown the server, delete the .feeder files, then restart to see any decrease in size for the .feeder files. You may find it worthwhile to enable the performance monitor tracking and check the }StatsByCube cube to see which cubes are consuming the most memory for feeders to better help you target the highest value areas.
User avatar
garry cook
Community Contributor
Posts: 209
Joined: Thu May 22, 2008 7:45 am
OLAP Product: TM1
Version: Various
Excel Version: Various

Re: .blb and .rux files - maintenance

Post by garry cook »

The TM1 server exclusively uses the .RUX files as the definitive source of rule statements. Any .BLB files are artifacts created by the client application (in this case performance modeller). Typically they store things like additional formatting details, etc...
It is worth pointing out though incase someone stumbles on this in a future search that if you have old .blb files for the rules, what is in the blb is what will be displayed when you open the rule editor even though it doesn't tie to the .RUX. This can cause significant confusion and creates a risk of an old version of the rules being accidentally saved over the live version - an annoyance that has been about for years and still exists in 10.2.2 FP4 :cry:
iahobson
Posts: 3
Joined: Fri Mar 18, 2016 8:27 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: .blb and .rux files - maintenance

Post by iahobson »

Thanks Garry & Brian for your responses.

My understanding is that BLB's are used for more than just formatting in performance modeler, maybe architect is different? Garry mentioned the difference between rules in the RUX and the BLB, and I guess this is why I am asking for further clarification. The FEEDERS statements in the RUX files should update when a rule is changed, regardless of whether the rule is changed in the rule editor or the performance modeler cube calculation, right?

So ultimately, I need to know that when I delete a rule in performance modeler, the feeders files are recalculated using the changed RUX file. That doesn't appear to be happening at the moment as the FEEDERS file for the cube is identical after making the changes. As per my previous post, we have shutdown the server, deleted FEEDER files and restarted on a number of occasions, but the size of the feeders files remains unchanged.

I guess the other question is given the multiple ways to edit rules, what is the most effective way to do this without generating discrepancies between the RUX and BLB's - it's not always practical to use the cube calculation window, so I need to know how we can prevent rules getting overwritten by BLB files.
User avatar
gtonkin
MVP
Posts: 1192
Joined: Thu May 06, 2010 3:03 pm
OLAP Product: TM1
Version: Latest and greatest
Excel Version: Office 365 64-bit
Location: JHB, South Africa
Contact:

Re: .blb and .rux files - maintenance

Post by gtonkin »

Ignoring the .RUX and .BLB files, I would not trust PM to create the "right" feeders for me. You mention the model is consuming 30Gb of memory. It may be useful to look at the stats cubes to identify which cube(s) are using the memory and then try and address those first.
Also, if you are flushing your persistent feeders and recreating them each time you restart, you will incur overhead on the disk writes and you may need to test if you are better off with or without them. Remember too that shutting down needs to flush from memory to disk which can also take time re-writing the .feeders files.
iahobson
Posts: 3
Joined: Fri Mar 18, 2016 8:27 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: .blb and .rux files - maintenance

Post by iahobson »

Hi Gtonkin

One issue we have with performance modeler is the auto generated feeders, and we do have issues with cells not being fed and there have been instances where we have had to supplement the auto feeders with manual ones (as you can't edit the auto generated ones in PM). Our memory usage is primarily contained to ~5 cubes, and we are focusing on removing as many rules as possible from these cubes, replacing them with processes. There are some rules that we simply cannot remove though, so we need to understand more about feeders and the best practices when setting them up/maintaining them.

In our live environment, we have enabled 'Generate Feeders Automatically' to 'Only for auto generated rules'. This kicks off feeder regeneration whenever rule/dimension is changed and creates quite a lot of disruption, so we generally promote out of hours from our DEV environment. This tends to reduce the necessity for restarts/rebuilding of feeders.
User avatar
gtonkin
MVP
Posts: 1192
Joined: Thu May 06, 2010 3:03 pm
OLAP Product: TM1
Version: Latest and greatest
Excel Version: Office 365 64-bit
Location: JHB, South Africa
Contact:

Re: .blb and .rux files - maintenance

Post by gtonkin »

Try do some review in your }StatsByCube after starting performance monitor - there are plenty of articles througout the forum to give you some guidance on interpreting the results and where to once you identify the "offending" cube(s). The main things to try and focus on would be: Number of Fed Cells, Memory Used for Calculations, Memory Used for Feeders, Total Memory Used.
tomok
MVP
Posts: 2831
Joined: Tue Feb 16, 2010 2:39 pm
OLAP Product: TM1, Palo
Version: Beginning of time thru 10.2
Excel Version: 2003-2007-2010-2013
Location: Atlanta, GA
Contact:

Re: .blb and .rux files - maintenance

Post by tomok »

iahobson wrote:There are some rules that we simply cannot remove though, so we need to understand more about feeders and the best practices when setting them up/maintaining them.
Step #1 should be to extricate yourselves from "Performance" Modeler. No TM1 developer worth his/her weight in salt would ever use that tool for a production environment that included anything other than some simple rudimentary cubes.
Tom O'Kelley - Manager Finance Systems
American Tower
http://www.onlinecourtreservations.com/
babytiger
Posts: 78
Joined: Wed Jul 31, 2013 4:32 am
OLAP Product: Cognos TM1, EP, Analyst
Version: 10.2.2
Excel Version: 2013
Location: Sydney AU

Re: .blb and .rux files - maintenance

Post by babytiger »

tomok wrote:
iahobson wrote:There are some rules that we simply cannot remove though, so we need to understand more about feeders and the best practices when setting them up/maintaining them.
Step #1 should be to extricate yourselves from "Performance" Modeler. No TM1 developer worth his/her weight in salt would ever use that tool for a production environment that included anything other than some simple rudimentary cubes.
+1 agree. That would be my recommendation too, as the first step to adopt best practice for TM1 development.
MK
deepakjain2020
Regular Participant
Posts: 152
Joined: Sat May 25, 2013 10:32 am
OLAP Product: TM1
Version: 9.5.2; 10.2.2
Excel Version: 2007

Re: .blb and .rux files - maintenance

Post by deepakjain2020 »

Hi Ian,

You can check TM1S.CFG file, and see if Persistent Feeders is turned on?
If it is so, please stop service, then turn off and start your instance, it could give you some info.

Regards,
Deepak Singhvi
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: .blb and .rux files - maintenance

Post by TrevorGoss »

babytiger wrote:
tomok wrote:
iahobson wrote:There are some rules that we simply cannot remove though, so we need to understand more about feeders and the best practices when setting them up/maintaining them.
Step #1 should be to extricate yourselves from "Performance" Modeler. No TM1 developer worth his/her weight in salt would ever use that tool for a production environment that included anything other than some simple rudimentary cubes.
+1 agree. That would be my recommendation too, as the first step to adopt best practice for TM1 development.
Quite right.
We use performance modeler unless it is absolutely necessary to use architect.
A lot of people, I think, would reverse that statement when describing their use of TM1.

It may well be the case that you have overfed some of your cubes. This post explains how to develop a tool to help you detect overfeeding http://www.tm1forum.com/viewtopic.php?t=4002

Trevor.
ByronB
Posts: 17
Joined: Tue Mar 01, 2016 5:55 am
OLAP Product: TM1 + BI
Version: TM1 10.2
Excel Version: Excel 2010

Re: .blb and .rux files - maintenance

Post by ByronB »

Hi iahobson,

I think you're using PersistentFeeders ON,

With persistent feeders, there are 2 important things that you need to know

1 - when you restart your TM1 instance, if the feeders file exists, TM1 will not recreate the feeders file.

2 - In the rule file, if you change a rule, TM1 will not reprocess the feeders, TM1 will only relcalculate the area between Skipcheck and Feeders.
TM1 will reprocess the feeders only if you change a feeder in the rule (only if you make a change in the area below Feeders)

It is best practice, to stop your service, delete all feeders files and restart your service every week, so yo umake sure that every week your feeders are uptodate

By the way, you should get rid of Performance modeler, and build TM1 application with Architect or perspective

Cheers,

Vincent
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: .blb and .rux files - maintenance

Post by TrevorGoss »

ByronB wrote:
By the way, you should get rid of Performance modeler, and build TM1 application with Architect or perspective
Performance modeler can still be useful in some circumstances. For example, attaching a TI to an application, so the customers can fire a load TI from Applications Web. You can also run reports that contain information about your models, such as details regarding your feeders (which is relevant to the original post), how many cells you have and more.

So in its defence, it can be useful, but for development I will steer clear until it is drastically improved.

Trevor.
Post Reply