TM1 Garbage Memory

[mce]

Hi,

What might be the reason to have big proportion of the allocated memory to TM1 is in Garbage and not used by any of the cubes even just after restarting the TM1 service?

Any comments would really be appreciated.

Regards

[David Usherwood]

Using multithreaded feeder loading (MaximumCubeLoadThreads>1).
Bear in mind that the calculation cache (and probably the stargate caches) will make use of the garbage. NB the TM1 instance never hands memory back to Windows.

[mce]

Thanks David. This explains then. MaximumCubeLoadThreads=22

[David Usherwood]

22!
How many physical cores do you actually have?

[mce]

We have 24 cores.

[wissew]

SWEET!!!

[David Usherwood]

Shame TM1 makes so little use of modern multi core technology. You can buy lots of servers, of course - that works well - but then the server to server integration is hardly there (replication - hah!).

[mce]

Data updates and report queries can obviously utilize multi core as far as locking issues are handled properly. Obviously there a lot to be done on the locking area.

[David Usherwood]

Calculation is still single threaded - and when data is changed, the calculation cache is cleared for the cube in question and all dependent cubes. Multi user locking is better than it used to be but for modelling apps with little input and lots of calculation, the architecture could be a lot better.

[wissew]

What's the ram on the server?

[mce]

Calculation is still single threaded - and when data is changed, the calculation cache is cleared for the cube in question and all dependent cubes. Multi user locking is better than it used to be but for modelling apps with little input and lots of calculation, the architecture could be a lot better.

You are right. I meant to say that read queries from multiple users and write operations in batch update mode by multiple processes can utilize multiple cores. It would certainly be much better if it was able to utilize multiple cores for the same query but I think that is a very hard requirement to achieve and not sure if it would get any priority in near feature considering the existence of much more important requirements and more svere issues with TM1.

[ioscat]

Hello, community!
achieved same trouble with garbage memory from MaximumCubeLoadThreads 4,5 GB for cubes and 5 GB for garbage. Now I've disables this option but garbage is still not 0. Now it's about 2,5 GB. Is this OK? How much garbage should just restarted model have?
Also found that consolidatechildren may cause large memory use but we don't use this function in model.

[jim wood]

There'll be all kinds of stuff like feeders or security going on in there. 2.5GB does seem very high but without knowing exactly what options you have set in your CFG it's hard to say what is causing it.

[ioscat]

GroupsCreationLimit=10

[TM1S]
PerformanceMonitorOn=T
AllowSeparateNandCRules=T
ReevaluateConditionalFeeders=T
ServerLogging=F
SecurityPackageName=Kerberos
IntegratedSecurityMode=5
UseSSL=F
ServerName=<***>
DataBaseDirectory=<***>
LoggingDirectory=<***>
AdminHost=<***>
PortNumber=<***>
Language=ENG
SaveTime=
DownTime=
ProgressMessage=True
AuditLogOn=F
AuditLogMaxFileSize= 200 MB
AuditLogUpdateInterval=60
PersistentFeeders=F
ParallelInteraction=T
JobQueueThreadPoolSIze = 12
MaximumUserSandboxSize=1024
CalculationThresholdForStorage=100000000000000000000000000000000000000000000000000

ServerCAMURI=<***>
ClientCAMURI=<***>
#ServerCAMURI=<***>
#ClientCAMURI=<***>

ClientPingCAMPassport=110
CAMPortalVariableFile = portal\variables_TM1.xml



#IdleConnectionTimeOutSeconds=7200
DefaultMeasureDimension=T
#MaximumCubeLoadThreads=2
ViewConsolidationOptimizationMethod=TREE

#Optional CAM parameters
#CAMSSLCertificate=
#CAMSSLCertRevList=
#SkipSSLCAMHostCheck=TRUE

[Duncan P]

Off topic but I would put your CalculationThresholdForStorage down to a more sensible value. Most programs use the C function atoi to interpret strings as integers and the documentation for it says this

If the converted value would be out of the range of representable values by an int, it causes undefined behavior.

[jim wood]

I'd also switch performance monitor off. I'm not sure if that would account for some of the garbage.

[ioscat]

just deleted all files from data folder to test garbage - and get non zero garbage memory value. is it normal? shouldn't it be zero?

Безымянный.png (5.74 KiB) Viewed 15413 times

[lotsaram]

just deleted all files from data folder to test garbage - and get non zero garbage memory value. is it normal? shouldn't it be zero?

Безымянный.png

Oh dear, oh dear. I thought we had gotten a bit further than this. What exactly would you think that any FILES would have to do with anything? Remember the TM1 server is completely in memory, the data directory files only represent a static copy of the database at a point in time. Also be very careful with deleting any files in the data directory while the server is in session as this can cause instability (i.e. crashes).

[ioscat]

wow, lotsaram, of course I stopped service, then deleted files and started server again. So, as I suppose, I got "Game over" TM1 server. As you can see, there is only about 9 MB of memory used.

[lotsaram]

Well I think if you meant that you were starting a server from an empty data directory there were maybe clearer or better ways of phrasing it.

Some garbage memory is normal. It just means that TM1 has requested more memory from the OS that what it currently requires. I don't know why you would be concerned about it.