Page 1 of 1
Registering of Private Subsets at Startup time is X7 in one week
Posted: Mon Jul 08, 2024 5:47 pm
by John Hammond
We encountered a problem where the time to load private subsets into memory went from 1hr 5min to 8 hrs. This occurs as pretty much the last operation of start up and there was a week between the two startups.
We have a lot of users but we don't see what is changed to make this jump since we likely only added a couple of users in the week.
It's mode 5 users and Unix so plenty of squiggly characters in the subset names, but the log only provides a single line of info not where it gets stuck.
If anyone has experienced this before I would be most grateful for your thoughts.
Thank you
John
Re: Registering of Private Subsets at Startup time is X7 in one week
Posted: Mon Jul 08, 2024 6:51 pm
by ascheevel
Set LoadPrivateSubsetsOnStartup=F in the config and see who complains about super long login time?
Re: Registering of Private Subsets at Startup time is X7 in one week
Posted: Mon Jul 08, 2024 9:55 pm
by ascheevel
Sorry for my spitball idea above, it does nothing today or even this week to help fix your slow restart problem and if we're dependent on enduser complaints, our sample size will forever be too large. I tried to replicate your issue of a slower restart by creating 5,00 and 15,000 private subsets for a couple test users and confirming LoadPrivateSubsetsOnStartup=T before restarting but couldn't detect any noticeable difference in server load time.
I then wondered about using the rest-api and python to track impersonation timings for clients after setting LoadPrivateSubsetsOnStartup=F and a restart. The 5k and 15k subsets users do take a while to login, but that could simply be reflecting the count and not necessarily size of subset or complexity of mdx. Still, impersonation and time tracking might be worth a try, especially if you can get your hands on an archived data folder of the tm1 instance from before the slowdown happened. You could then do a before/after comparison by user to see if anyone stands out. Below is some python code I used to test login impersonation timing of impersonation after a restart when LoadPrivateSubsetsOnStartup=F.
Code: Select all
from TM1py.Services import TM1Service
import pandas as pd
import time
with TM1Service(address='...', port='...', user='...', password='...', ssl=True) as tm1:
mdx = 'Filter(TM1SubsetAll([}Clients]), [}ClientGroups].([}Groups].[Admin]) = "")'
users = [user[0]['Name'] for user in tm1.elements.execute_set_mdx(mdx=mdx)]
login_times = {'user': [], 'login_time': []}
for user in users:
start = time.time()
with TM1Service(address='...', port='...', user='...', password='...', ssl=True, impersonate=user) as tm2:
end = time.time()
login_times['user'].append(user)
login_times['login_time'].append(round(end-start, 3))
df1 = pd.DataFrame.from_dict(login_times)
Re: Registering of Private Subsets at Startup time is X7 in one week
Posted: Tue Jul 09, 2024 8:46 am
by lotsaram
Holy hell you weren't concerned already that loading subsets was taking over 1 hour?!
Re: Registering of Private Subsets at Startup time is X7 in one week
Posted: Tue Jul 09, 2024 9:32 am
by Wim Gielis
Curious to know how a TM1 model can get to tens of thousands of private objects.
Is it subsets ? Views ? Application entries ?
Are they created automatically with some sort of automation ?
If you are to migrate to a different namespace/environment, this can mean a lot of work to get the objects in the correct folders (folders named by namespace, usernames that can change for the }TM1_DefaultDisplayValue alias) etc.
BTW, nice code Andrew.
Re: Registering of Private Subsets at Startup time is X7 in one week
Posted: Tue Jul 23, 2024 3:20 pm
by John Hammond
ascheevel wrote: ↑Mon Jul 08, 2024 6:51 pm
Set LoadPrivateSubsetsOnStartup=F in the config and see who complains about super long login time?
Exactly my thoughts, but it turns out this default setting (which worked fine in windows) had to be changed to =T on IBM's advice for Unix when we started getting long logins.
This worked fine for four years until the current problem cropped up. So, nice try - Thanks!
Re: Registering of Private Subsets at Startup time is X7 in one week
Posted: Tue Jul 23, 2024 3:23 pm
by John Hammond
ascheevel wrote: ↑Mon Jul 08, 2024 9:55 pm
Sorry for my spitball idea above, it does nothing today or even this week to help fix your slow restart problem and if we're dependent on enduser complaints, our sample size will forever be too large. I tried to replicate your issue of a slower restart by creating 5,00 and 15,000 private subsets for a couple test users and confirming LoadPrivateSubsetsOnStartup=T before restarting but couldn't detect any noticeable difference in server load time.
I then wondered about using the rest-api and python to track impersonation timings for clients after setting LoadPrivateSubsetsOnStartup=F and a restart. The 5k and 15k subsets users do take a while to login, but that could simply be reflecting the count and not necessarily size of subset or complexity of mdx. Still, impersonation and time tracking might be worth a try, especially if you can get your hands on an archived data folder of the tm1 instance from before the slowdown happened. You could then do a before/after comparison by user to see if anyone stands out. Below is some python code I used to test login impersonation timing of impersonation after a restart when LoadPrivateSubsetsOnStartup=F.
Code: Select all
from TM1py.Services import TM1Service
import pandas as pd
import time
with TM1Service(address='...', port='...', user='...', password='...', ssl=True) as tm1:
mdx = 'Filter(TM1SubsetAll([}Clients]), [}ClientGroups].([}Groups].[Admin]) = "")'
users = [user[0]['Name'] for user in tm1.elements.execute_set_mdx(mdx=mdx)]
login_times = {'user': [], 'login_time': []}
for user in users:
start = time.time()
with TM1Service(address='...', port='...', user='...', password='...', ssl=True, impersonate=user) as tm2:
end = time.time()
login_times['user'].append(user)
login_times['login_time'].append(round(end-start, 3))
df1 = pd.DataFrame.from_dict(login_times)
Like your approach with Python. We used to go Loadrunner to simulate timings but with Python this is so much more universal...
Thank you