Registering of Private Subsets at Startup time is X7 in one week

[John Hammond]

We encountered a problem where the time to load private subsets into memory went from 1hr 5min to 8 hrs. This occurs as pretty much the last operation of start up and there was a week between the two startups.

We have a lot of users but we don't see what is changed to make this jump since we likely only added a couple of users in the week.

It's mode 5 users and Unix so plenty of squiggly characters in the subset names, but the log only provides a single line of info not where it gets stuck.

If anyone has experienced this before I would be most grateful for your thoughts.

Thank you

John

[ascheevel]

Set LoadPrivateSubsetsOnStartup=F in the config and see who complains about super long login time?

[ascheevel]

Sorry for my spitball idea above, it does nothing today or even this week to help fix your slow restart problem and if we're dependent on enduser complaints, our sample size will forever be too large. I tried to replicate your issue of a slower restart by creating 5,00 and 15,000 private subsets for a couple test users and confirming LoadPrivateSubsetsOnStartup=T before restarting but couldn't detect any noticeable difference in server load time.

I then wondered about using the rest-api and python to track impersonation timings for clients after setting LoadPrivateSubsetsOnStartup=F and a restart. The 5k and 15k subsets users do take a while to login, but that could simply be reflecting the count and not necessarily size of subset or complexity of mdx. Still, impersonation and time tracking might be worth a try, especially if you can get your hands on an archived data folder of the tm1 instance from before the slowdown happened. You could then do a before/after comparison by user to see if anyone stands out. Below is some python code I used to test login impersonation timing of impersonation after a restart when LoadPrivateSubsetsOnStartup=F.

Code: Select all

from TM1py.Services import TM1Service
import pandas as pd
import time

with TM1Service(address='...', port='...', user='...', password='...', ssl=True) as tm1:
    mdx = 'Filter(TM1SubsetAll([}Clients]), [}ClientGroups].([}Groups].[Admin]) = "")'
    users = [user[0]['Name'] for user in tm1.elements.execute_set_mdx(mdx=mdx)]

    login_times = {'user': [], 'login_time': []}
    for user in users:
        start = time.time()
        with TM1Service(address='...', port='...', user='...', password='...', ssl=True, impersonate=user) as tm2:
            end = time.time()
            login_times['user'].append(user)
            login_times['login_time'].append(round(end-start, 3))

df1 = pd.DataFrame.from_dict(login_times)

[lotsaram]

Holy hell you weren't concerned already that loading subsets was taking over 1 hour?!

[Wim Gielis]

Curious to know how a TM1 model can get to tens of thousands of private objects.
Is it subsets ? Views ? Application entries ?
Are they created automatically with some sort of automation ?

If you are to migrate to a different namespace/environment, this can mean a lot of work to get the objects in the correct folders (folders named by namespace, usernames that can change for the }TM1_DefaultDisplayValue alias) etc.

BTW, nice code Andrew.

[John Hammond]

Set LoadPrivateSubsetsOnStartup=F in the config and see who complains about super long login time?

Exactly my thoughts, but it turns out this default setting (which worked fine in windows) had to be changed to =T on IBM's advice for Unix when we started getting long logins.

This worked fine for four years until the current problem cropped up. So, nice try - Thanks!

[John Hammond]

Sorry for my spitball idea above, it does nothing today or even this week to help fix your slow restart problem and if we're dependent on enduser complaints, our sample size will forever be too large. I tried to replicate your issue of a slower restart by creating 5,00 and 15,000 private subsets for a couple test users and confirming LoadPrivateSubsetsOnStartup=T before restarting but couldn't detect any noticeable difference in server load time.

I then wondered about using the rest-api and python to track impersonation timings for clients after setting LoadPrivateSubsetsOnStartup=F and a restart. The 5k and 15k subsets users do take a while to login, but that could simply be reflecting the count and not necessarily size of subset or complexity of mdx. Still, impersonation and time tracking might be worth a try, especially if you can get your hands on an archived data folder of the tm1 instance from before the slowdown happened. You could then do a before/after comparison by user to see if anyone stands out. Below is some python code I used to test login impersonation timing of impersonation after a restart when LoadPrivateSubsetsOnStartup=F.

Code: Select all

from TM1py.Services import TM1Service
import pandas as pd
import time

with TM1Service(address='...', port='...', user='...', password='...', ssl=True) as tm1:
    mdx = 'Filter(TM1SubsetAll([}Clients]), [}ClientGroups].([}Groups].[Admin]) = "")'
    users = [user[0]['Name'] for user in tm1.elements.execute_set_mdx(mdx=mdx)]

    login_times = {'user': [], 'login_time': []}
    for user in users:
        start = time.time()
        with TM1Service(address='...', port='...', user='...', password='...', ssl=True, impersonate=user) as tm2:
            end = time.time()
            login_times['user'].append(user)
            login_times['login_time'].append(round(end-start, 3))

df1 = pd.DataFrame.from_dict(login_times)

Like your approach with Python. We used to go Loadrunner to simulate timings but with Python this is so much more universal...

Thank you