Migrating & Maintaining Diminsions in Parallel Services

[RJ!]

Hi All,

Does anyone know if a Dimension's .dim file can be easily used by 2 TM1 Services?

The reason I ask is that we are looking to create a new TM1 Service that will be an upgrade to the existing service and we'll need to have these running in parallel (i.e. Something like PROD & NewPROD).
One of the biggest issues we have is that our Account, Cost Centre & Product Dimensions have a lot of structure changes over the course of the month and obviously we don't want to be duplicating our effort inserting these changes between the 2 services.

Has anyone else been in this situation? Is there a better way to keep all of these dimensions in sync without copying in the .Dim file & restarting the 2nd service?

[RJ!]

Just confirming that we'll have the Services in Parallel for atleast 3mths...

[Alan Kirk]

The short answer to your specific question is no. A server must have its own (and only one) data directory. By definition a file can't be in two places at once. There is another option, though... and I hesitate to suggest this since I'm sure it's on the Defence Against The Dark Arts syllabus. It's...

{Whispers...} Replication.

There, I said it.

Specifically, you can replicate the dimension's Element Attributes cube which will take the dimension with it. I did this for a while with dims that were shared between development and production but then stopped because I just don't trust it. At times it would stop replicating a specific dimension for no particular reason that I could discern other than "don't wanna", and you won't get any warning about it unless you add reading the replication logs to your daily "to do" list. But for a few dimensions, for three months... you can probably get away with it.

Just don't expect subsets to replicate reliably, even if you check the relevant option.

[tomok]

The short answer to your specific question is no. A server must have its own (and only one) data directory.

Actually you can reference multiple data directories in a single TM1 service. Just separate them with a semi-colon in the tm1s.cfg file. I discovered this trick many years ago. Although I almost never use it, I have had occasion to give it a whirl every now and then. In the OP's case he could separate the "shared" dimensions in a separate data directory and then reference that folder in the tm1s.cfg file of each service and voila!, shared dimensions. On the downside, you have to be careful on dimension maintenance. You need to pick only one of the services to do the maintenance and then the only way to get the updates to the other service will be to restart it. Not ideal, but it can be done.

[Alan Kirk]

The short answer to your specific question is no. A server must have its own (and only one) data directory.

Actually you can reference multiple data directories in a single TM1 service. Just separate them with a semi-colon in the tm1s.cfg file. I discovered this trick many years ago.

Interesting; I of course knew that you could specify different directories on the client side like that but had never looked at doing it on the server side.

Although I almost never use it, I have had occasion to give it a whirl every now and then. In the OP's case he could separate the "shared" dimensions in a separate data directory and then reference that folder in the tm1s.cfg file of each service and voila!, shared dimensions. On the downside, you have to be careful on dimension maintenance. You need to pick only one of the services to do the maintenance and then the only way to get the updates to the other service will be to restart it.

Yes, that was my first thought when I read your comment above. The Dim file is only read on startup. Similarly I'd imagine that it would be necessary to manually separate the attributes dim and cube into that folder as well. Still, it's an interesting option.

[RJ!]

Thanks guys! Might try the replication path first, then if that doesn't work, then try Tomok's suggestion!

Atleast if we try Tomok's suggestion we can finally give the automatic Service Restart chore to work ( http://www.tm1forum.com/viewtopic.php?f=21&t=7266 )

[Martin Ryan]

Another option is to create TI processes to export and import the dimensions from one server to the other via csv files. I did this a while back when I had multiple instances (for valid reasons) that shared common dimensions. There was a master instance that spat out the dimension and all the attributes in two separate files, then the other instances would pick up those files and rebuild the dimensions.

Only trick is that you need to be careful on import to destroy the hierarchy (i.e. delete parent/child relationships), don't delete any elements otherwise you can lose data if anything goes wrong. Also, this method won't get subsets across.

[RJ!]

Hi Martin,

That was one option that we considered though were not too sure how this would go, we've copied over the Bedrock processes that can do it, just not 100% confident that we can unwind & then rewind all of the hierachies. So you have successfully been able to do this?

[lotsaram]

Hi Martin,

That was one option that we considered though were not too sure how this would go, we've copied over the Bedrock processes that can do it, just not 100% confident that we can unwind & then rewind all of the hierachies. So you have successfully been able to do this?

How large are the dimensions, how numerous, and for what purpose do you want to do the sync? It would be helpful to know.

Having contributed to the bedrock 1.0 code this was one area I would have done different as I am pretty sure the file format only allows for a certain number of alternate hierarchies and aliases. This might be fixed in 2.0, it was certainly an area that needed tweaking. Having said that the bedrock processes will definitely work (assuming max number of hierarchies aren't exceeded) but if the dimensions are quite large then rather than using an unwind process it is going to be much more time efficient to simply delete all C elements and rebuild rather than unwind. (Just bear in mind that if C elements are deleted as opposed to unwound then all attribute values will need to be reset as well as any static subsets using any C elements. Depending on the extra overhead of this the additional processing time of unwinding may be the "lesser evil").

But as opposed to a purely TECHNICAL solution could you provide some more detail on the WHY aspect with your "prod and new prod" scenario? It might (probably I suspect) be a bit overblown to set up a full real-time or near real-time syncing of dimensions while both servers are up. If you are developing a new model not yet in production which will replace the production model (eventually) but in the meantime you want to ensure that meta data stays in sync and only needs to be maintained once then the solution could be a lot simpler with just a periodic copy of the dim files and element attribute cub files and a server bounce ...

[Martin Ryan]

Yup, something along the lines of (not tested, just typed out in a couple of minutes, so may have some errors):

Code: Select all

sDim='MyDimension';
numElem=dimsiz(sDim);
while(numElem>0);
  sParent=dimnm(sDim, numElem);
  nNumChildren=elcompn(sDim, sParent);
  while(nNumChildren>0);
    sChild=elcomp(sDim, sParent, nNumChildren);
    DimensionElementComponentDelete(sDim, sParent, sChild);
    nNumChildren=nNumChildren-1;
  end;
  numElem=numElem-1;
endif;

Note it's important to count from the number of children down to zero, rather than from zero up to the number of children, because of course as you delete children the number of children decreases. The alternative is to keep deleting the first child until elcompn=0.

HTH

[RJ!]

Hi Lotsaram,

You are right, we are building a new service that will eventually replace our current Prod Service (oddly called COB1) with a new one (PROD). Our Dimensions don't need to be updated between the services in real time, its just that we'd be looking to keep them sync'd daily as we want our Users to start using the new environment before we do a hard cut over. I think that sharing the files in the TM1s.cfg file & bouncing the server in the morning once our Prod Environment is updated might be the better solution if it'll work (the services are likely to be on 2 separate servers - not sure if that affects anything). It seems we are destined to be bouncing the server unless we want to do the replication method which no one seems to have much confidence in.

BTW:
Account Dimension has about 90,000 elements under 5 separate structures
Product Dimension has about 15,000 elements under 3 separate structures
Cost Centre Dimension has about 25,000 elements under 6 separate structures
All have atleast 10 columns of attributes/metadata attached to them...

These are the 3 main dimensions that would definately need to be shared between the services, I'm sure there are maybe 2-3 more that would need to be also added.

[tomok]

(the services are likely to be on 2 separate servers - not sure if that affects anything)

What that means is that you can't just list the path to the shared dimensions on the different server. You'll have to create a share for the folder that has the shared dimensions and make sure the second server has access to that share, meaning the account that TM1 is running under must have access to the share. Then you'll need to make sure the reference in the tm1s.cfg file has the full UNC path.

[Gabor]

If you are going to use more than 1 server folder in .cfg, please keep in mind there are several bugs with recent TM1 releases (9.5.2 & 10.1.1).
For example ASCIIOUTPUT creates the file in first folder (if no path specified), but ASCIIDELETE seems to look in last folder, as files remain untouched in first folder.
Similar issues are existing for other TI-functions like DeleteAllPersistentFeeders, but also for Audit Logs (TM1.Audit.PostProcessor Error xx) etc. -> APAR's existing.