Slow Login in Architect

User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Slow Login in Architect

Post by macsir »

Tm1 10.2.2 fp7
I noticed that for around two weeks, it will sometimes become very slow to login from Architect or Perspectives or Web. The Architect I was testing just sits on the TM1 server itself. Thus there shouldn't be any network issue. After a reboot, the duration of login went back to normal (less than 10 seconds). I am aware that the login duration is pretty much depend on the number of TM1 objects residing in the server.
However, after a period of time (days or weeks), it goes up to around 20 seconds, or 40 seconds or even 60 seconds randomly.
I checked the logs in every place and seems to be normal. No special findings. The only thing I can tell is when it happened, the server has many (~90-100) sessions concurrently although most of them are idle.
Could anyone share any idea on it? Thanks.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
Edward Stuart
Community Contributor
Posts: 247
Joined: Tue Nov 01, 2011 10:31 am
OLAP Product: TM1
Version: All
Excel Version: All
Location: Manchester
Contact:

Re: Slow Login in Architect

Post by Edward Stuart »

Which IntegratedSecurityMode are you using and are you using single sign on etc..?
User avatar
gtonkin
MVP
Posts: 1192
Joined: Thu May 06, 2010 3:03 pm
OLAP Product: TM1
Version: Latest and greatest
Excel Version: Office 365 64-bit
Location: JHB, South Africa
Contact:

Re: Slow Login in Architect

Post by gtonkin »

Is there a difference when you use the server name vs the IP address? Have had numerous issues when the DNS is fritzed - it takes forever to connect the dots.
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Edward Stuart wrote: Fri Jun 07, 2019 1:31 pm Which IntegratedSecurityMode are you using and are you using single sign on etc..?
Cam security mode 5 with sso. The cognos is installed on same server where TM1 is. Thanks.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

gtonkin wrote: Fri Jun 07, 2019 4:18 pm Is there a difference when you use the server name vs the IP address? Have had numerous issues when the DNS is fritzed - it takes forever to connect the dots.
I tried using IP in architect. No difference and still slow. Thanks
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
Edward Stuart
Community Contributor
Posts: 247
Joined: Tue Nov 01, 2011 10:31 am
OLAP Product: TM1
Version: All
Excel Version: All
Location: Manchester
Contact:

Re: Slow Login in Architect

Post by Edward Stuart »

Once you are logged in are there further issues or is it only login that sees the delays?

Do you have access to Cognos BI Security to see if anything odd is occuring at that end? My guess would be that the 'normal' identifiers for TM1 are clear so it could be a BI related issue?
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Edward Stuart wrote: Mon Jun 10, 2019 12:11 pm Once you are logged in are there further issues or is it only login that sees the delays?

Do you have access to Cognos BI Security to see if anything odd is occuring at that end? My guess would be that the 'normal' identifiers for TM1 are clear so it could be a BI related issue?
Thanks, no further slowness issue after logged in as I observed.
I checked BI side and no special finding. What I can see from TM1 top is, after double clicking the server, it took around 5 seconds for showing "??? - SystemServerConnectWithCAM" and then after around 4 seconds, my logging name is shown correctly. After that, TM1 server took 1 or 2 seconds to show all objects. Therefore, the total duration is around 10 seconds now after a fresh reboot several days ago. Is it quite normal if someone can test it as well? Thanks. I don't think it is slow. When the issue really happened last time, the server took ages to show all objects after authenticating my user name with CAM.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

By the way, TM1 web was also slow when I had the issue. Now it takes around 15 seconds to show applications and views after open the URL in Chrome.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
gtonkin
MVP
Posts: 1192
Joined: Thu May 06, 2010 3:03 pm
OLAP Product: TM1
Version: Latest and greatest
Excel Version: Office 365 64-bit
Location: JHB, South Africa
Contact:

Re: Slow Login in Architect

Post by gtonkin »

Anything suspicious in your tm1admsrv log files?

May be worthwhile adding some additional loggers in your tm1s-log.properties to see if there is anything untoward going on e.g.

Code: Select all

log4j.logger.TM1.Server.Network=DEBUG, STATS
log4j.appender.STATS=org.apache.log4j.SharedMemoryAppender
log4j.appender.STATS.MemorySize=5 MB
log4j.appender.STATS.File=tm1STATS.log
log4j.appender.STATS.MaxFileSize=5 MB
log4j.appender.STATS.MaxBackupIndex=5
log4j.appender.STATS.Timezone=Local
Have a look here for more loggers.
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

gtonkin wrote: Tue Jun 11, 2019 11:34 am Anything suspicious in your tm1admsrv log files?

May be worthwhile adding some additional loggers in your tm1s-log.properties to see if there is anything untoward going on e.g.

Code: Select all

log4j.logger.TM1.Server.Network=DEBUG, STATS
log4j.appender.STATS=org.apache.log4j.SharedMemoryAppender
log4j.appender.STATS.MemorySize=5 MB
log4j.appender.STATS.File=tm1STATS.log
log4j.appender.STATS.MaxFileSize=5 MB
log4j.appender.STATS.MaxBackupIndex=5
log4j.appender.STATS.Timezone=Local
Have a look here for more loggers.
Thanks, that's a good point. I will take action on them and report back.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Just want to share,
The login duration was slowed down to around 20 seconds and I checked the DEBUG server log. There is no ERROR or WARNING at all.
Compared to a login from fresh reboot, the log is showing more

Code: Select all

105416   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
85292   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
28796   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForSingleObject: Timed out on event 0x0000000000000144.
33076   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
28796   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForSingleObject: Waiting for event 0x0000000000000144.
71340   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
28796   []   DEBUG   2019-06-19 05:59:55.400   TM1.Event   mt_WaitForSingleObject: Timed out on event 0x0000000000000144.
28796   []   DEBUG   2019-06-19 05:59:55.400   TM1.Event   mt_WaitForSingleObject: Waiting for event 0x0000000000000144.
28796   []   DEBUG   2019-06-19 05:59:55.416   TM1.Event   mt_WaitForSingleObject: Timed out on event 0x0000000000000144.
Not sure what that mean but seems those timeout events were consuming considerable time. Maybe I am wrong as it is out of my technical scope. But for now, the most practical workaround would be rebooting server weekly rather than monthly. :?
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
Elessar
Community Contributor
Posts: 331
Joined: Mon Nov 21, 2011 12:33 pm
OLAP Product: PA 2
Version: 2.0.9
Excel Version: 2016
Contact:

Re: Slow Login in Architect

Post by Elessar »

Hi,

When you log in to TM1 with ISM=5, TM1 checks CAM client*group mapping (and adds new client if any). This can be reason

How many groups do you have in TM1?
How many groups does the "slow" user belong to?
Do you have any rules in }Clientgroups cube? Any feeders from it?
Does the problem affect all users?
Best regards, Alexander Dvoynev

TM1 and Data Science blog: 6th article - PAfE + VBA: Commit each cell without pressing “Commit” button.
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Elessar wrote: Wed Jun 19, 2019 6:33 am Hi,

When you log in to TM1 with ISM=5, TM1 checks CAM client*group mapping (and adds new client if any). This can be reason

How many groups do you have in TM1?
How many groups does the "slow" user belong to?
Do you have any rules in }Clientgroups cube? Any feeders from it?
Does the problem affect all users?
1. Around 70 groups but not all of them have members
2. when it slows, all groups I tested are slow
3. No
4. all users I know including myself, admin.

Thanks.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
Elessar
Community Contributor
Posts: 331
Joined: Mon Nov 21, 2011 12:33 pm
OLAP Product: PA 2
Version: 2.0.9
Excel Version: 2016
Contact:

Re: Slow Login in Architect

Post by Elessar »

There should be "CAM_LPSvr.exe" processes. Do they consume CPU during slow user connections?
It can consume the whole core (for example, 25% in 4-core processor). If so, try to kill it in Task manager. Cognos will recreate new, if it needs.

And the most obvious (sorry if I didn't find this above ;) ): did you try to restart Cognos service?
Best regards, Alexander Dvoynev

TM1 and Data Science blog: 6th article - PAfE + VBA: Commit each cell without pressing “Commit” button.
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Elessar wrote: Thu Jun 20, 2019 6:38 am There should be "CAM_LPSvr.exe" processes. Do they consume CPU during slow user connections?
It can consume the whole core (for example, 25% in 4-core processor). If so, try to kill it in Task manager. Cognos will recreate new, if it needs.

And the most obvious (sorry if I didn't find this above ;) ): did you try to restart Cognos service?
I did search "CAM_LPSvr.exe" and nothing special on the consumption.
I will try to restart Cognos service next time to see if it was improving.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
Bakkone
Posts: 119
Joined: Mon Oct 27, 2014 10:50 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: Slow Login in Architect

Post by Bakkone »

Do you use SSL? If so, between which components?

Is BI and TM1 installed on the same server? Is TM1 installed on one or multiple servers?

Do you have any anti-virus on the servers?

Would it be possible for you to test with native login once to see if you still if its still slow?
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Bakkone wrote: Mon Jun 24, 2019 5:31 pm Do you use SSL? If so, between which components?

Is BI and TM1 installed on the same server? Is TM1 installed on one or multiple servers?

Do you have any anti-virus on the servers?

Would it be possible for you to test with native login once to see if you still if its still slow?
1.Yes, UseSSL=T but CAM is not using SSL, means it is HTTP rather than HTTPS
2.Yes
3.Yes but seems not to be relevant
4.Good point, I will try next time.
Thanks.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
Bakkone
Posts: 119
Joined: Mon Oct 27, 2014 10:50 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: Slow Login in Architect

Post by Bakkone »

macsir wrote: Mon Jun 24, 2019 10:49 pm 1.Yes, UseSSL=T but CAM is not using SSL, means it is HTTP rather than HTTPS
2.Yes
3.Yes but seems not to be relevant
4.Good point, I will try next time.
Thanks.
1, Are you using custom certificates?

2a, Are you using IIS as a webserver for your CAM-installation?
2b, Is the database server for the BI-installation on the same machine?

3, AV can be the death of everything. Would it be possible to turn it off when things are slow to see if things are faster?

Whats the server OS?
What version of Cognos BI?
What Database are you using for your content store?
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

1, Are you using custom certificates?
Yes, tm1ca_v2.pem
2a, Are you using IIS as a webserver for your CAM-installation?
Yes
2b, Is the database server for the BI-installation on the same machine?
DB Server is another server

3, AV can be the death of everything. Would it be possible to turn it off when things are slow to see if things are faster?
Has tried disabled but no improvement

Whats the server OS?
Windows 2012
What version of Cognos BI?
10.2.2
What Database are you using for your content store?
SQL Server
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: Slow Login in Architect

Post by macsir »

Just want to update, looks like the login duration is stable and fast after cleaning lots of obsolete Performance Modeler additions out. Can't say it is 100% the reason but the login period is indeed improved. The cleaning is still in progress.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
Post Reply