Page 1 of 1
Order of loading files in memory
Posted: Wed Nov 27, 2019 9:50 am
by Wim Gielis
Hello,
Just a question that popped up with me.
I (re)start a TM1 model, let's say 20 GB and 5 minutes of loading time.
Does anybody know in what order, more or less, files are read into memory ?
The question is, after starting a model, can I still change the contents of a PRO file or copy/paste a PRO file in the TM1 data directory such that it is still picked during loading ? IF processes are "loaded" at the end of the transaction, this would avoid starting all over again with the load.
My real user case is a TM1 model of 115 GB where I needed to import a couple of TI processes but I forgot... until 80 GB was loaded and half an hour was gone
Thanks
Re: Order of loading files in memory
Posted: Wed Nov 27, 2019 11:22 am
by Napshot
Hey
I did some small test on this and from what i can see processes are some of the last that are loaded in. I could copy/delete past *.pro in to the data library at around 90% (might be abit higher than that) and see the effects at startup.
So in a order like this, based on the log file:
Cube headers > Dimension > Rules > Cube body > feeders >processes > chores
Morten
Re: Order of loading files in memory
Posted: Wed Nov 27, 2019 11:47 am
by gtonkin
Wim Gielis wrote: ↑Wed Nov 27, 2019 9:50 am
...
My real user case is a TM1 model of 115 GB where I needed to import a couple of TI processes but I forgot... until 80 GB was loaded and half an hour was gone...
Would be useful to be able to use AutoHotKey to effectively retype the necessary details from your non-prod to prod environment then save/compile.
A nice project for those long weekends!
Re: Order of loading files in memory
Posted: Wed Nov 27, 2019 11:49 am
by Wim Gielis
gtonkin wrote: ↑Wed Nov 27, 2019 11:47 am
Wim Gielis wrote: ↑Wed Nov 27, 2019 9:50 am
...
My real user case is a TM1 model of 115 GB where I needed to import a couple of TI processes but I forgot... until 80 GB was loaded and half an hour was gone...
Would be useful to be able to use AutoHotKey to effectively retype the necessary details from your non-prod to prod environment then save/compile.
A nice project for those long weekends!
Indeed, use the TM1 REST API within tools like AutoHotKey

Hmmm, I already do that

As in: you have a big daily zipped file of a TM1 directory and you specify a cube name. Then the AHK script will grab all necessary files (*.cub, dim, feeder, rux, ...). It will see what are the dimensions of the cube (through REST API), and retrieve those dim and }ElementAttributes and }ElementSecurity files. The output is a folder of objects

Selecting multiple cubes is possible too.
This works fine for TM1 authentication mode = 1. If someone wants to cooperate with me to get it working for CAM security, let me know. I would be interested in getting the REST API calls from AHK or PowerShell or ... working in other security modes.
The benefit is of course gaining time. It takes a lot of time to find cub and dim and other files in a (big) zip file. With this method, we don't need to unzip the file first. In bigger TM1 models I support, unzipping such a file of about 100 GB takes about 2 hours.
Re: Order of loading files in memory
Posted: Wed Nov 27, 2019 11:51 am
by Wim Gielis
Napshot wrote: ↑Wed Nov 27, 2019 11:22 am
Hey
I did some small test on this and from what i can see processes are some of the last that are loaded in. I could copy/delete past *.pro in to the data library at around 90% (might be abit higher than that) and see the effects at startup.
So in a order like this, based on the log file:
Cube headers > Dimension > Rules > Cube body > feeders >processes > chores
Morten
Many thanks Morten. Next time I restart a model that is not loaded very quickly, I will test it too.
Re: Order of loading files in memory
Posted: Thu Dec 12, 2019 6:15 pm
by Wim Gielis
Wim Gielis wrote: ↑Wed Nov 27, 2019 11:49 amThis works fine for TM1 authentication mode = 1. If someone wants to cooperate with me to get it working for CAM security, let me know. I would be interested in getting the REST API calls from AHK or PowerShell or ... working in other security modes.
It works by exploiting the log on information specified in this thread:
https://www.tm1forum.com/viewtopic.php?f=3&t=15063