memory allocation of cubes

Post Reply
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

memory allocation of cubes

Post by mincharug.shulft »

Hello,

Current functionality of one of application is
Loading all the user cubes(there are four types of cubes has individual user) into memory whenever user logged into the application and also whenever user’s log off from the app the user cubes won’t free up the memory until TM1 restart happens SO
Here I am thinking of to write TI script whenever users log off from app release the cubes from memory using the Load on demand.

Is it right approach or can one suggest me here?
Thank in advance.
David Usherwood
Site Admin
Posts: 1458
Joined: Wed May 28, 2008 9:09 am

Re: memory allocation of cubes

Post by David Usherwood »

TM1 never hands back memory to the OS, so unloading the cube just creates more 'garbage' memory.
What are you trying to achieve?
tomok
MVP
Posts: 2836
Joined: Tue Feb 16, 2010 2:39 pm
OLAP Product: TM1, Palo
Version: Beginning of time thru 10.2
Excel Version: 2003-2007-2010-2013
Location: Atlanta, GA
Contact:

Re: memory allocation of cubes

Post by tomok »

mincharug.shulft wrote:Loading all the user cubes(there are four types of cubes has individual user) into memory whenever user logged into the application
On-demand individual user cubes??? Sounds like a really, really bad OLAP design. Just curious, but what would be the rationale behind this?
Tom O'Kelley - Manager Finance Systems
American Tower
http://www.onlinecourtreservations.com/
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: memory allocation of cubes

Post by mincharug.shulft »

Actually we are currently
With windows server 2003 enterprise x64 service pack2
Applix 32-bit and with 4GB RAM.
So when user are accessing the application all most all 3.9GB of memory is using by TM1 server and
Bringing down the TM1 server.
So here my understanding from the data model is the following
As soon as users are accessing the application and all the users cubes (there are four types of cubes has individual user) are getting loaded into the memory and it never free up until tm1 server restart happens even then users are logged off from the application.
So here I am thinking of to write TI script whenever users log off from the application I want to release the cubes from memory using the Load on demand option.
Please help me if you’re still getting you satisfied.

Thanks advance.
tomok
MVP
Posts: 2836
Joined: Tue Feb 16, 2010 2:39 pm
OLAP Product: TM1, Palo
Version: Beginning of time thru 10.2
Excel Version: 2003-2007-2010-2013
Location: Atlanta, GA
Contact:

Re: memory allocation of cubes

Post by tomok »

Take a look at the CubeUnload TI function. I doubt it will solve your problem however. Your're running the 32-bit version of TM1 on a 64-bit OS and you only have 4GB installed on the machine? This not a good combination. You would have more memory for TM1, assuming you are hell bent on running the 32-bit version, if you ran a 32-bit OS and added the /3GB option to the OS.

The real answer is to run the 64-bit version of TM1 on a 64-bit OS and add more RAM to the box. I'm guessing you don't own a license to the 64-bit version and that's why you are trying to squeeze it into 32-bit. Good luck.
Tom O'Kelley - Manager Finance Systems
American Tower
http://www.onlinecourtreservations.com/
User avatar
Steve Rowe
Site Admin
Posts: 2464
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: memory allocation of cubes

Post by Steve Rowe »

So the theory is that you unload the cubes when people log off and have them set to load on demand when a user queries them, this should release the RAM into and out of the garbage box. You'll still have the 4GB limit though it might take longer to get there. Another thing to watch out for is if you have a set of cubes that are set to load on demand and they are rules and feeders involved then you'll need to load the cubes in the right order or the rules and feeders won't work. Plus your other users will take a performance hit when the cube is loaded and feeders fire.

Its possible this may work but how are you planning on detecting when someone logs off? I guess you'll just scan client properties at a certain frequency and record who is not on since the last run.

Lke others though I'm curious as to what you are using TM1 for that has a set of cubes per user?

Cheers,
Technical Director
www.infocat.co.uk
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: memory allocation of cubes

Post by mincharug.shulft »

HI Steve Rowe,and all,

We don’t have rules here.
And I am planning to detect the following way:
There is a control cube called “notification” which can usually monitoring as whoever logged in/logged off from the application so I am taking that as logic and
Also using the CubeUnload function to unloading the cube from memory.
Please let help if I am wrongly going?

Thanks in advance.
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: memory allocation of cubes

Post by mincharug.shulft »

Hi all,

i have written TI script using "CubeUnload" function and i want to see the unloads a specified cube from memory.
where i can see this? please help me?

Thanks in advance.
David Usherwood
Site Admin
Posts: 1458
Joined: Wed May 28, 2008 9:09 am

Re: memory allocation of cubes

Post by David Usherwood »

Suggest you refresh your memory concerning Performance Monitor....
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: memory allocation of cubes

Post by mincharug.shulft »

Actually we donot enable this performance monitor option
so i think i should enable this?
i am correct?

Really thanks for your reply.
David Usherwood
Site Admin
Posts: 1458
Joined: Wed May 28, 2008 9:09 am

Re: memory allocation of cubes

Post by David Usherwood »

If you want to use it then turning it on is a useful first step :)
IMHO PM is a pretty essential diagnostic tool for any serious TM1 app.
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: memory allocation of cubes

Post by mincharug.shulft »

Hi Gentle mans,

please help with below explanation with below TI coding.

here where "vendorname" is coming as parameter.
and here i am trying to do is the following:
if the cube name is equal to e_cube then unload the ''}ElementAttributes_' | vendor_cube;" like this doing for other cubes.

when i trying to excute the script its throwing an error line 26 "statement of out place" .
could you guys help whether the logic that i have written is correct or not.and also errors

vendor_cube = 'app_' |vendorname | '_cost';
CubeUnload(vendor_cube);

e_cube = '}ElementAttributes_' | vendor_cube;
e_m_cube = '}ElementAttributes_memo_' | vendor_cube;
m_cube = 'memo_' | vendor_cube;

max_Cubes = DIMSIZ( '}Cubes' );
i = 1;
While ( i <= max_Cubes );
CubeName = DIMNM( '}Cubes', i);
if(CubeName@=e_cube);
CubeUnload(e_cube);

elseif(CubeName@=e_m_cube);
CubeUnload(e_m_cube);

elseif(CubeName@=m_cube);
CubeUnload(m_cube);

endif;
endif;
end;
i=i+1;
end;
Alan Kirk
Site Admin
Posts: 6667
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: memory allocation of cubes

Post by Alan Kirk »

mincharug.shulft wrote:Hi Gentle mans,

please help with below explanation with below TI coding.

here where "vendorname" is coming as parameter.
and here i am trying to do is the following:
if the cube name is equal to e_cube then unload the ''}ElementAttributes_' | vendor_cube;" like this doing for other cubes.

when i trying to excute the script its throwing an error line 26 "statement of out place" .
could you guys help whether the logic that i have written is correct or not.and also errors
I'll leave others to comment on the correctness or otherwise of using separate cubes for each vendor. It is perhaps not an approach that I would use as I think that having all these separate cubes will cause more overhead than having a single (well designed) cube.

As for your code, when you post can you please make sure you put it in a CODE block; you'll find the button for this at the top of the posting window. In that way it's possible to see all of the indentations, which in this case shows where your syntax errors are pretty easily.

Code: Select all

vendor_cube = 'app_' |vendorname | '_cost';
CubeUnload(vendor_cube);

e_cube = '}ElementAttributes_' | vendor_cube;
e_m_cube = '}ElementAttributes_memo_' | vendor_cube;
m_cube = 'memo_' | vendor_cube;

max_Cubes = DIMSIZ( '}Cubes' );

i = 1;

While ( i <= max_Cubes );

    CubeName = DIMNM( '}Cubes', i);

    if(CubeName@=e_cube);
         CubeUnload(e_cube);

    elseif(CubeName@=e_m_cube);
        CubeUnload(e_m_cube);

    elseif(CubeName@=m_cube);
          CubeUnload(m_cube);

    endif;

# Two Endifs for one If. Delete the one below.
    endif;

# Two Ends for one While, and this one would give you an infinite loop.
# Delete this.
end;

    i=i+1;

end;

Incidentally, rather than doing such a long, laborious loop, wouldn't it be easier just to use CubeExists() to check whether the three attribute cubes are there and if they are, unload them? (Or try to; whether unloading attribute cubes works at all I'm not sure of, but you could try it and see.)
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
lotsaram
MVP
Posts: 3706
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: memory allocation of cubes

Post by lotsaram »

mincharug.shulft wrote:...if the cube name is equal to e_cube then unload the ''}ElementAttributes_' | vendor_cube;" like this doing for other cubes.
Design of your model aside, provided each vendor cube is independent of all the others then there should be no issue with unloading the cubes apart from the performance hit you are inflicting on users when they load the cubes for the first time. I would however advise against unloading any system cubes like }ElementAttributes. It might be OK but these are system objects and so who knows what the TM1 server might or might not be doing in the background that might require access.

Regardless of the design with separate cubes for each user/vendor, although I would question it on face value, assuming you have done the requirements gathering and analysis correctly then if this makes data loading or security management easier then so be it, it's a valid design. BUT I do question why you are wanting to unload cubes. RAM is a very cheap and plentiful resource compared to TM1 consulting time, it would seem a much better investment to properly spec the memory capacity of the server to accommodate all cubes as opposed to the time you are spending (wasting) implementing logic to track which vendors are logged in and unload cubes of those who aren't. 1/ this is needless development, 2/ it inflicts a performance impact on users due to needing to load cubes into memory 3/ there is a risk of server crash should too many users log on at once.
Alan Kirk
Site Admin
Posts: 6667
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: memory allocation of cubes

Post by Alan Kirk »

lotsaram wrote: Regardless of the design with separate cubes for each user/vendor, although I would question it on face value, assuming you have done the requirements gathering and analysis correctly then if this makes data loading or security management easier then so be it, it's a valid design. BUT I do question why you are wanting to unload cubes. RAM is a very cheap and plentiful resource compared to TM1 consulting time,
While I'm not disagreeing with the general thrust of your argument, RAM may be cheap and plentiful.

On the other hand upgrades from 32 to to 64 bit TM1 for those on a 32 bit only licence (and there are some such animals, though I don't know for sure that it applies here however I can't imagine too many other reasons for still fluffing about on 32 bit these days) are merely plentiful, as is the concomitant year on year increases in maintenance.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: memory allocation of cubes

Post by mincharug.shulft »

Hi ,
its workning now and thanks a lot for your valuable help.

and i plan to see the unload cube from memory by ON the "performance monitor on" BUT i could not able to see the deffernce in the }StatsByCube

Am i seeing worng or is there any other way can we see?

Thanks a lot agin
asutcliffe
Regular Participant
Posts: 164
Joined: Tue May 04, 2010 10:49 am
OLAP Product: Cognos TM1
Version: 9.4.1 - 10.1
Excel Version: 2003 and 2007

Re: memory allocation of cubes

Post by asutcliffe »

mincharug.shulft wrote:and i plan to see the unload cube from memory by ON the "performance monitor on" BUT i could not able to see the deffernce in the }StatsByCube

Am i seeing worng or is there any other way can we see?
So you have the performance monitor switched on and don't see any change to 'Total Memory Used' in }StatsByCube when you unload a cube?

Note, you need to wait a minute or so for changes to be reflected in the stats cubes. Also, if I recall correctly, even when unloaded, a cube still consumes some memory.
Alan Kirk
Site Admin
Posts: 6667
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: memory allocation of cubes

Post by Alan Kirk »

mincharug.shulft wrote:Hi ,
its workning now and thanks a lot for your valuable help.

and i plan to see the unload cube from memory by ON the "performance monitor on" BUT i could not able to see the deffernce in the }StatsByCube

Am i seeing worng or is there any other way can we see?

Thanks a lot agin
ASutcliffe is correct in that you have to wait for the next minute to tick over and that "Total Memory Used" won't hit zero. (It still needs to be registered with the server so that it'll show up in the cubes list for a start.) However if the memory still doesn't drop substantially after that minute (and it does look like element attribute cubes will unload the way normal ones will), I'd be looking at whether there are any rules or feeders which might be holding the cubes in memory despite your attempt to unload them.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
Post Reply