More info on TM1Project.json file

Post Reply
luizg2019
Posts: 41
Joined: Thu Sep 12, 2019 11:02 pm
OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
Version: PA 2.0.9 - PAW 2.0.73
Excel Version: office 2016

More info on TM1Project.json file

Post by luizg2019 »

Hi!

I'm trying to use GIT on our servers but IBM's documentation is not very detailed on this subject, they say you "SHOULD" create a tm1project.json file for each model but don't say where to save it, I tried creating it on my data folder but it made no difference when I sent a GitPush command.

Does anyone know where should I create this file and how should I proceed to use this?

Thank you.
User avatar
Mike Cowie
Site Admin
Posts: 482
Joined: Sun May 11, 2008 7:07 pm
OLAP Product: IBM TM1/PA, SSAS, and more
Version: Anything thru 11.x
Excel Version: 2003 - Office 365
Location: Alabama, USA
Contact:

Re: More info on TM1Project.json file

Post by Mike Cowie »

Hello:

The only way to work with the project file that I'm aware of is via the REST API-- I don't believe it can be managed directly with a file in the data directory structure.

To create/update that project file create json which defines your tm1project spec, for instance:

Code: Select all

{
  "Version": "1.0",
  "Name": "My Model",
  "Settings": {
    "Performance": {
      "MTQ": {
        "MTFeedersAtStartup": false
      }
    }
  },
  "Tasks": {
		"Refresh Security":
    {
      "Process": "Processes('Refresh Security')",
      "Parameters": []
    }
	},
  "Objects": {},
  "Files": [
		"\\admin\\*\\*"
	],
  "Ignore": [
		"Processes('Ignore Me')"
  ],
  "Deployment": {},
  "PrePull": [],
  "PostPull": [],
  "PrePush": [],
  "PostPush": []
}
And POST or PUT that json to this special TM1 REST API url used to access the tm1project definition:
.../api/v1/!tm1project

You can review what's currently in there with a GET-- by default I believe it's blank until you POST/PUT something there.

Hope that helps! (and, yes, it's not that clearly documented right now)
Mike Cowie
QueBIT Consulting, LLC

Are you lost without Print Reports in Planning Analytics for Excel (PAfE)? Get it back today, for free, with Print Reports for IBM Planning Analytics for Excel!
Edward Stuart
Community Contributor
Posts: 247
Joined: Tue Nov 01, 2011 10:31 am
OLAP Product: TM1
Version: All
Excel Version: All
Location: Manchester
Contact:

Re: More info on TM1Project.json file

Post by Edward Stuart »

Excellent work Mike, I'm looking forward to testing this further.

For those also looking into this Yury has posted (yet) another great blog on Git Integration

https://ykud.com/blog/cognos/tm1-cognos ... tm1-part-1
luizg2019
Posts: 41
Joined: Thu Sep 12, 2019 11:02 pm
OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
Version: PA 2.0.9 - PAW 2.0.73
Excel Version: office 2016

Re: More info on TM1Project.json file

Post by luizg2019 »

Mike Cowie wrote: Thu Dec 05, 2019 11:00 pm And POST or PUT that json to this special TM1 REST API url used to access the tm1project definition:
.../api/v1/!tm1project
Wow. I would've never guessed based on the documentation :lol: :lol: :lol:
Thank you very much for your help Mike!
Edward Stuart wrote: Fri Dec 06, 2019 8:54 am For those also looking into this Yury has posted (yet) another great blog on Git Integration
https://ykud.com/blog/cognos/tm1-cognos ... tm1-part-1
Thank you for sharing this link Edward, it's definitely more informative than the official documentation. :D
luizg2019
Posts: 41
Joined: Thu Sep 12, 2019 11:02 pm
OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
Version: PA 2.0.9 - PAW 2.0.73
Excel Version: office 2016

Re: More info on TM1Project.json file

Post by luizg2019 »

Mike Cowie wrote: Thu Dec 05, 2019 11:00 pm
To create/update that project file create json which defines your tm1project spec, for instance:

Code: Select all

{
  "Version": "1.0",
  "Name": "My Model",
  "Settings": {
    "Performance": {
      "MTQ": {
        "MTFeedersAtStartup": false
      }
    }
  },
  "Tasks": {
		"Refresh Security":
    {
      "Process": "Processes('Refresh Security')",
      "Parameters": []
    }
	},
  "Objects": {},
  "Files": [
		"\\admin\\*\\*"
	],
  "Ignore": [
		"Processes('Ignore Me')"
  ],
  "Deployment": {},
  "PrePull": [],
  "PostPull": [],
  "PrePush": [],
  "PostPush": []
}
And POST or PUT that json to this special TM1 REST API url used to access the tm1project definition:
.../api/v1/!tm1project

You can review what's currently in there with a GET-- by default I believe it's blank until you POST/PUT something there.

Hope that helps! (and, yes, it's not that clearly documented right now)
Hi mike! Do you know how can I specify for control cubes to be included in the git context?
I tried using the "(not) ignore" section in tm1project but the control cubes were not added to git, and it made the gitpush took a lot more of time to complete.

Code: Select all

{
  "Version": "1.0",
  "Name": "Test",
  "Ignore": [
		"!Processes('}*')",
		"!Dimensions('}*')",
		"!Cubes('}*')"
  ]
}
Thank you.


EDIT:
Nevermind.

Found the answer. Turns out the wildcard is NOT supported for control objects :oops:
So you have to manually list one by one if you want them included in the git as well, which is very annoying because when you're moving new dimensions between servers, its attributes will not be transferred unless you specify each "}ElementAttributes_Dimension" cubes to be transferred as well.
Last edited by luizg2019 on Mon Dec 09, 2019 7:28 pm, edited 1 time in total.
User avatar
ykud
MVP
Posts: 148
Joined: Sat Jan 10, 2009 10:52 am
Contact:

Re: More info on TM1Project.json file

Post by ykud »

luizg2019 wrote: Mon Dec 09, 2019 2:07 pm So you have to manually list one by one if you want them included in the git as well, which is very annoying because when you're moving new dimensions between servers, its attributes will not be transferred unless you specify each "}ElementAttributes_Dimension" cubes to be transferred as well.
Hi there,

How did you manage to include the control cubes in the end?
Via !Ignore, like this?

Code: Select all

  "Ignore": ["!Cubes('}ElementAttributes_DimensionA')"]
Or you're 'including' them explicitly in {objects}? Can you please share your tm1project.json?
I'm trying to add them in something like all_cubes.json file to be able to use Link: all_cubes.json, but failing so far. Quite annoying :(

Also, maybe you or Mike have played with it and know the answer:
Say I want to promote only 3 changed objects from one server (A) to another (B), but I have 10 more objects changed on A that I don't want to promote yet. I thought this would be something like listing the 3 objects in the "deployment" as objects so that it overrides the primary definition, but so far I am struggling to get any Pull request to ignore the other 10 objects. Any pointers?

Thanks,
Yuri
luizg2019
Posts: 41
Joined: Thu Sep 12, 2019 11:02 pm
OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
Version: PA 2.0.9 - PAW 2.0.73
Excel Version: office 2016

Re: More info on TM1Project.json file

Post by luizg2019 »

ykud wrote: Mon Jan 06, 2020 5:01 am Hi there,

How did you manage to include the control cubes in the end?
Via !Ignore, like this?

Code: Select all

  "Ignore": ["!Cubes('}ElementAttributes_DimensionA')"]
Hello!
Yes, using the (not) !ignore feature. But you have to save the tm1project settings before doing a git pull.
I didn't get to play a lot with the tm1project setting because I didn't have time and the bad documentation also discourages me.

But my tm1project.json was something like:

Code: Select all

{
  "Version": "1.0",
  "Name": "Test",
  "Ignore": [
		"!Dimensions('}Clients')",
		"!Dimensions('}ClientProperties')",
		"!Cubes('}ClientProperties')"
  ]
}
ykud wrote: Mon Jan 06, 2020 5:01 am Say I want to promote only 3 changed objects from one server (A) to another (B), but I have 10 more objects changed on A that I don't want to promote yet. I thought this would be something like listing the 3 objects in the "deployment" as objects so that it overrides the primary definition, but so far I am struggling to get any Pull request to ignore the other 10 objects. Any pointers?
I also ran into that issue. What I did was manually ignoring the objects on the tm1project.json from server A before doing a commit/gitpush so when you do a gitpull on server B it won't overwrite the objects that you ignored.
User avatar
ykud
MVP
Posts: 148
Joined: Sat Jan 10, 2009 10:52 am
Contact:

Re: More info on TM1Project.json file

Post by ykud »

luizg2019 wrote: Tue Jan 07, 2020 1:04 pm Hello!
Yes, using the (not) !ignore feature. But you have to save the tm1project settings before doing a git pull.
I didn't get to play a lot with the tm1project setting because I didn't have time and the bad documentation also discourages me.
Thanks heaps! That document is indeed very confusing and vague, I'm quite annoyed about having to 'guess & try' so much around it.

luizg2019 wrote: Tue Jan 07, 2020 1:04 pm
ykud wrote: Mon Jan 06, 2020 5:01 am Say I want to promote only 3 changed objects from one server (A) to another (B), but I have 10 more objects changed on A that I don't want to promote yet. I thought this would be something like listing the 3 objects in the "deployment" as objects so that it overrides the primary definition, but so far I am struggling to get any Pull request to ignore the other 10 objects. Any pointers?
I also ran into that issue. What I did was manually ignoring the objects on the tm1project.json from server A before doing a commit/gitpush so when you do a gitpull on server B it won't overwrite the objects that you ignored.
Yep, I know that if you don't have any other objects in the repo (don't push or delete from git), they will no be pulled. Just hoping this is not the only way as that's kinda the end of trying to have a single git repo for your tm1 server. I'll try and see if I can get some pointers from IBM re this, things can't be that bad.

Thanks again,
Yuri
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: More info on TM1Project.json file

Post by macsir »

Hi, is there a way to add control objects into Git now?
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
scrumthing
Posts: 81
Joined: Tue Jan 26, 2016 4:18 pm
OLAP Product: TM1
Version: 11.x
Excel Version: MS365

Re: More info on TM1Project.json file

Post by scrumthing »

macsir wrote: Thu May 14, 2020 7:41 am Hi, is there a way to add control objects into Git now?
Still not as far as I know but I think IBM is by now aware of the issue.
It does not stop there. Even if you add all control objects manually you still do not have the data within those cubes (aka attribute values) which I find kind of really annoying especially if it comes down to stuff like picklists. I don't want to write a rule or process for every single picklist in my model.
ykud wrote: Tue Jan 07, 2020 9:33 pm Yep, I know that if you don't have any other objects in the repo (don't push or delete from git), they will no be pulled. Just hoping this is not the only way as that's kinda the end of trying to have a single git repo for your tm1 server. I'll try and see if I can get some pointers from IBM re this, things can't be that bad.
I tried it with only one repo as well and so far failed to do it properly. I think their idea is to have a separate branch for every merge you want to do and constantly experience "merge hell" between development and testing and production. :-(
I would love to set up a centralized repo with all standard processes for multiple instances and deploy them from there. But I am not sure that that is possible without merging and forking within GIT.
There is no OLAP database besides TM1!
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: More info on TM1Project.json file

Post by lotsaram »

scrumthing wrote: Thu May 14, 2020 9:20 am I tried it with only one repo as well and so far failed to do it properly. I think their idea is to have a separate branch for every merge you want to do and constantly experience "merge hell" between development and testing and production. :-(
I would love to set up a centralized repo with all standard processes for multiple instances and deploy them from there. But I am not sure that that is possible without merging and forking within GIT.
This is why I think ultimately this is doomed to fail, not becasue of missing or misleading documentation or unfinished development and feature support, but becasue fundamentally this is not how TM1 development works.

The way IBM have conceptualized Git integration working is that each developer works on a separate branch, and each branch is essentially an individual copy of the entire TM1 server. The thing is when you are developing in TM1 it's not just code ("code" in TM1 sense being .pro and .rux files) but a model consisting of hierarchies and cube data. You can't develop the code in isolation of the model. Certainly not for more complex models, and definately not when it comes to most unit testing, UAT and troubleshooting. And well TM1 models can get pretty big, and having multiple developers each working on their own separate model can add up to a lot of resources and a lot of interdependencies.

Most of the time having a shared model being worked on by multiple developers makes a lot more sense. With developers reserving & checking-in / checking-out discrete packages of objects. And simple mechanisms to select a collection of objects to migrate between instances. But this is very difficult to manage (or is currently unmannegable) with how IBM have done the Git integration.

I'm still hopeful this might evolve into a decent tool, but I do admit to not being overly optimistic about it.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: More info on TM1Project.json file

Post by macsir »

The way IBM have conceptualized Git integration working is that each developer works on a separate branch, and each branch is essentially an individual copy of the entire TM1 server.
Actually, I think the working mode should be working. Each branch doesn't have to be the copy of entire TM1 server. They could have several objects you working on and later want to deploy to a target TM1 server. The central repo could have its own history and each branch could have history too. At any time, they could be merged back to central history. This is how Git works and I have tested. They are working fine. In Git, you can track history of code files, update and hot promote.
The only block for me is to figure out how to write proper tm1project file (I have learnt Ignore section but need more other sections) and how to deal with control objects (not individually). Hopefully IBM can provide more later.
For the data files, I am not concerning too much as it is not relevant to Git anyway. Git shouldn't maintain any history of any data file.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
User avatar
macsir
MVP
Posts: 782
Joined: Wed May 30, 2012 6:50 am
OLAP Product: TM1
Version: PAL 2.0.9
Excel Version: Office 365
Contact:

Re: More info on TM1Project.json file

Post by macsir »

Further on control objects for tm1project files, it is not a big deal anymore. I can easily build the json file with extractions for all control objects from }Cubes, }Processes and }Dimensions dimensions. And they can be initialized into Git completely.
In TM1,the answer is always yes though sometimes with a but....
http://tm1sir.blogspot.com.au/
MSM1323
Posts: 1
Joined: Tue Nov 03, 2020 8:54 am
OLAP Product: TM1
Version: 11.7.00002.1
Excel Version: Office 2016

Re: More info on TM1Project.json file

Post by MSM1323 »

Hi,
1) is there a way to add attribute values into Git now?
2) can anyone here add in this thread some example for "Link" and "Files" in the tm1project file?
3) how to transfer files from Application folder through Git? How to write it in the tm1project file in"Ignore"? All my attempts ended with error 290 and smth like "Failed to parse ignore string \"Applications('ForProcesses')/DocumentReferences('res.csv')\". Cannot find property \"Applications\" in entity type \"EntityContainer\"."

As stated earlier, the documentation is mostly incomprehensible and I have some difficulty using it.
Anyway, this thread has already helped me a lot, thanks everyone :D
bobgillvan
Posts: 3
Joined: Tue May 24, 2022 9:14 pm
OLAP Product: TM1
Version: PA 2.0.9
Excel Version: 365

Re: More info on TM1Project.json file

Post by bobgillvan »

I finally figured out how the TM1Project.json file and PA Cloud all play nicely together. I did a video to share with the community too.

https://www.youtube.com/watch?v=9XT9ALcKf4k
User avatar
gtonkin
MVP
Posts: 1192
Joined: Thu May 06, 2010 3:03 pm
OLAP Product: TM1
Version: Latest and greatest
Excel Version: Office 365 64-bit
Location: JHB, South Africa
Contact:

Re: More info on TM1Project.json file

Post by gtonkin »

Thanks for sharing that insightful video - have saved for future use too.
Wim Gielis
MVP
Posts: 3105
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: More info on TM1Project.json file

Post by Wim Gielis »

That's very useful, thanks for that.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
bobgillvan
Posts: 3
Joined: Tue May 24, 2022 9:14 pm
OLAP Product: TM1
Version: PA 2.0.9
Excel Version: 365

Re: More info on TM1Project.json file

Post by bobgillvan »

bobgillvan wrote: Tue May 24, 2022 9:19 pm I finally figured out how the TM1Project.json file and PA Cloud all play nicely together. I did a video to share with the community too.

https://www.youtube.com/watch?v=9XT9ALcKf4k
With more experimenting, I realized some of what I understood was wrong. Here are two followup videos to correct and expand on my approach.

Postman / TM1 - Variables - https://youtu.be/xfMJOFhnCsE

Postman / TM1 - GitHub Integration and Usage - https://youtu.be/BqvqoYb1QU0
Post Reply