More info on TM1Project.json file
-
- Posts: 41
- Joined: Thu Sep 12, 2019 11:02 pm
- OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
- Version: PA 2.0.9 - PAW 2.0.73
- Excel Version: office 2016
More info on TM1Project.json file
Hi!
I'm trying to use GIT on our servers but IBM's documentation is not very detailed on this subject, they say you "SHOULD" create a tm1project.json file for each model but don't say where to save it, I tried creating it on my data folder but it made no difference when I sent a GitPush command.
Does anyone know where should I create this file and how should I proceed to use this?
Thank you.
I'm trying to use GIT on our servers but IBM's documentation is not very detailed on this subject, they say you "SHOULD" create a tm1project.json file for each model but don't say where to save it, I tried creating it on my data folder but it made no difference when I sent a GitPush command.
Does anyone know where should I create this file and how should I proceed to use this?
Thank you.
- Mike Cowie
- Site Admin
- Posts: 482
- Joined: Sun May 11, 2008 7:07 pm
- OLAP Product: IBM TM1/PA, SSAS, and more
- Version: Anything thru 11.x
- Excel Version: 2003 - Office 365
- Location: Alabama, USA
- Contact:
Re: More info on TM1Project.json file
Hello:
The only way to work with the project file that I'm aware of is via the REST API-- I don't believe it can be managed directly with a file in the data directory structure.
To create/update that project file create json which defines your tm1project spec, for instance:
And POST or PUT that json to this special TM1 REST API url used to access the tm1project definition:
.../api/v1/!tm1project
You can review what's currently in there with a GET-- by default I believe it's blank until you POST/PUT something there.
Hope that helps! (and, yes, it's not that clearly documented right now)
The only way to work with the project file that I'm aware of is via the REST API-- I don't believe it can be managed directly with a file in the data directory structure.
To create/update that project file create json which defines your tm1project spec, for instance:
Code: Select all
{
"Version": "1.0",
"Name": "My Model",
"Settings": {
"Performance": {
"MTQ": {
"MTFeedersAtStartup": false
}
}
},
"Tasks": {
"Refresh Security":
{
"Process": "Processes('Refresh Security')",
"Parameters": []
}
},
"Objects": {},
"Files": [
"\\admin\\*\\*"
],
"Ignore": [
"Processes('Ignore Me')"
],
"Deployment": {},
"PrePull": [],
"PostPull": [],
"PrePush": [],
"PostPush": []
}
.../api/v1/!tm1project
You can review what's currently in there with a GET-- by default I believe it's blank until you POST/PUT something there.
Hope that helps! (and, yes, it's not that clearly documented right now)
Mike Cowie
QueBIT Consulting, LLC
Are you lost without Print Reports in Planning Analytics for Excel (PAfE)? Get it back today, for free, with Print Reports for IBM Planning Analytics for Excel!
QueBIT Consulting, LLC
Are you lost without Print Reports in Planning Analytics for Excel (PAfE)? Get it back today, for free, with Print Reports for IBM Planning Analytics for Excel!
-
- Community Contributor
- Posts: 248
- Joined: Tue Nov 01, 2011 10:31 am
- OLAP Product: TM1
- Version: All
- Excel Version: All
- Location: Manchester
- Contact:
Re: More info on TM1Project.json file
Excellent work Mike, I'm looking forward to testing this further.
For those also looking into this Yury has posted (yet) another great blog on Git Integration
https://ykud.com/blog/cognos/tm1-cognos ... tm1-part-1
For those also looking into this Yury has posted (yet) another great blog on Git Integration
https://ykud.com/blog/cognos/tm1-cognos ... tm1-part-1
-
- Posts: 41
- Joined: Thu Sep 12, 2019 11:02 pm
- OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
- Version: PA 2.0.9 - PAW 2.0.73
- Excel Version: office 2016
Re: More info on TM1Project.json file
Wow. I would've never guessed based on the documentationMike Cowie wrote: ↑Thu Dec 05, 2019 11:00 pm And POST or PUT that json to this special TM1 REST API url used to access the tm1project definition:
.../api/v1/!tm1project
Thank you very much for your help Mike!
Thank you for sharing this link Edward, it's definitely more informative than the official documentation.Edward Stuart wrote: ↑Fri Dec 06, 2019 8:54 am For those also looking into this Yury has posted (yet) another great blog on Git Integration
https://ykud.com/blog/cognos/tm1-cognos ... tm1-part-1
-
- Posts: 41
- Joined: Thu Sep 12, 2019 11:02 pm
- OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
- Version: PA 2.0.9 - PAW 2.0.73
- Excel Version: office 2016
Re: More info on TM1Project.json file
Hi mike! Do you know how can I specify for control cubes to be included in the git context?Mike Cowie wrote: ↑Thu Dec 05, 2019 11:00 pm
To create/update that project file create json which defines your tm1project spec, for instance:And POST or PUT that json to this special TM1 REST API url used to access the tm1project definition:Code: Select all
{ "Version": "1.0", "Name": "My Model", "Settings": { "Performance": { "MTQ": { "MTFeedersAtStartup": false } } }, "Tasks": { "Refresh Security": { "Process": "Processes('Refresh Security')", "Parameters": [] } }, "Objects": {}, "Files": [ "\\admin\\*\\*" ], "Ignore": [ "Processes('Ignore Me')" ], "Deployment": {}, "PrePull": [], "PostPull": [], "PrePush": [], "PostPush": [] }
.../api/v1/!tm1project
You can review what's currently in there with a GET-- by default I believe it's blank until you POST/PUT something there.
Hope that helps! (and, yes, it's not that clearly documented right now)
I tried using the "(not) ignore" section in tm1project but the control cubes were not added to git, and it made the gitpush took a lot more of time to complete.
Code: Select all
{
"Version": "1.0",
"Name": "Test",
"Ignore": [
"!Processes('}*')",
"!Dimensions('}*')",
"!Cubes('}*')"
]
}
EDIT:
Nevermind.
Found the answer. Turns out the wildcard is NOT supported for control objects
So you have to manually list one by one if you want them included in the git as well, which is very annoying because when you're moving new dimensions between servers, its attributes will not be transferred unless you specify each "}ElementAttributes_Dimension" cubes to be transferred as well.
Last edited by luizg2019 on Mon Dec 09, 2019 7:28 pm, edited 1 time in total.
Re: More info on TM1Project.json file
Hi there,luizg2019 wrote: ↑Mon Dec 09, 2019 2:07 pm So you have to manually list one by one if you want them included in the git as well, which is very annoying because when you're moving new dimensions between servers, its attributes will not be transferred unless you specify each "}ElementAttributes_Dimension" cubes to be transferred as well.
How did you manage to include the control cubes in the end?
Via !Ignore, like this?
Code: Select all
"Ignore": ["!Cubes('}ElementAttributes_DimensionA')"]
I'm trying to add them in something like all_cubes.json file to be able to use Link: all_cubes.json, but failing so far. Quite annoying
Also, maybe you or Mike have played with it and know the answer:
Say I want to promote only 3 changed objects from one server (A) to another (B), but I have 10 more objects changed on A that I don't want to promote yet. I thought this would be something like listing the 3 objects in the "deployment" as objects so that it overrides the primary definition, but so far I am struggling to get any Pull request to ignore the other 10 objects. Any pointers?
Thanks,
Yuri
-
- Posts: 41
- Joined: Thu Sep 12, 2019 11:02 pm
- OLAP Product: TM1 - PAX-PAW-Perspectiv-Arc
- Version: PA 2.0.9 - PAW 2.0.73
- Excel Version: office 2016
Re: More info on TM1Project.json file
Hello!ykud wrote: ↑Mon Jan 06, 2020 5:01 am Hi there,
How did you manage to include the control cubes in the end?
Via !Ignore, like this?Code: Select all
"Ignore": ["!Cubes('}ElementAttributes_DimensionA')"]
Yes, using the (not) !ignore feature. But you have to save the tm1project settings before doing a git pull.
I didn't get to play a lot with the tm1project setting because I didn't have time and the bad documentation also discourages me.
But my tm1project.json was something like:
Code: Select all
{
"Version": "1.0",
"Name": "Test",
"Ignore": [
"!Dimensions('}Clients')",
"!Dimensions('}ClientProperties')",
"!Cubes('}ClientProperties')"
]
}
I also ran into that issue. What I did was manually ignoring the objects on the tm1project.json from server A before doing a commit/gitpush so when you do a gitpull on server B it won't overwrite the objects that you ignored.ykud wrote: ↑Mon Jan 06, 2020 5:01 am Say I want to promote only 3 changed objects from one server (A) to another (B), but I have 10 more objects changed on A that I don't want to promote yet. I thought this would be something like listing the 3 objects in the "deployment" as objects so that it overrides the primary definition, but so far I am struggling to get any Pull request to ignore the other 10 objects. Any pointers?
Re: More info on TM1Project.json file
Thanks heaps! That document is indeed very confusing and vague, I'm quite annoyed about having to 'guess & try' so much around it.
Yep, I know that if you don't have any other objects in the repo (don't push or delete from git), they will no be pulled. Just hoping this is not the only way as that's kinda the end of trying to have a single git repo for your tm1 server. I'll try and see if I can get some pointers from IBM re this, things can't be that bad.luizg2019 wrote: ↑Tue Jan 07, 2020 1:04 pmI also ran into that issue. What I did was manually ignoring the objects on the tm1project.json from server A before doing a commit/gitpush so when you do a gitpull on server B it won't overwrite the objects that you ignored.ykud wrote: ↑Mon Jan 06, 2020 5:01 am Say I want to promote only 3 changed objects from one server (A) to another (B), but I have 10 more objects changed on A that I don't want to promote yet. I thought this would be something like listing the 3 objects in the "deployment" as objects so that it overrides the primary definition, but so far I am struggling to get any Pull request to ignore the other 10 objects. Any pointers?
Thanks again,
Yuri
- macsir
- MVP
- Posts: 785
- Joined: Wed May 30, 2012 6:50 am
- OLAP Product: TM1
- Version: PAL 2.0.9
- Excel Version: Office 365
- Contact:
Re: More info on TM1Project.json file
Hi, is there a way to add control objects into Git now?
- scrumthing
- Posts: 81
- Joined: Tue Jan 26, 2016 4:18 pm
- OLAP Product: TM1
- Version: 11.x
- Excel Version: MS365
Re: More info on TM1Project.json file
Still not as far as I know but I think IBM is by now aware of the issue.
It does not stop there. Even if you add all control objects manually you still do not have the data within those cubes (aka attribute values) which I find kind of really annoying especially if it comes down to stuff like picklists. I don't want to write a rule or process for every single picklist in my model.
I tried it with only one repo as well and so far failed to do it properly. I think their idea is to have a separate branch for every merge you want to do and constantly experience "merge hell" between development and testing and production.ykud wrote: ↑Tue Jan 07, 2020 9:33 pm Yep, I know that if you don't have any other objects in the repo (don't push or delete from git), they will no be pulled. Just hoping this is not the only way as that's kinda the end of trying to have a single git repo for your tm1 server. I'll try and see if I can get some pointers from IBM re this, things can't be that bad.
I would love to set up a centralized repo with all standard processes for multiple instances and deploy them from there. But I am not sure that that is possible without merging and forking within GIT.
There is no OLAP database besides TM1!
-
- MVP
- Posts: 3687
- Joined: Fri Mar 13, 2009 11:14 am
- OLAP Product: TableManager1
- Version: PA 2.0.x
- Excel Version: Office 365
- Location: Switzerland
Re: More info on TM1Project.json file
This is why I think ultimately this is doomed to fail, not becasue of missing or misleading documentation or unfinished development and feature support, but becasue fundamentally this is not how TM1 development works.scrumthing wrote: ↑Thu May 14, 2020 9:20 am I tried it with only one repo as well and so far failed to do it properly. I think their idea is to have a separate branch for every merge you want to do and constantly experience "merge hell" between development and testing and production.
I would love to set up a centralized repo with all standard processes for multiple instances and deploy them from there. But I am not sure that that is possible without merging and forking within GIT.
The way IBM have conceptualized Git integration working is that each developer works on a separate branch, and each branch is essentially an individual copy of the entire TM1 server. The thing is when you are developing in TM1 it's not just code ("code" in TM1 sense being .pro and .rux files) but a model consisting of hierarchies and cube data. You can't develop the code in isolation of the model. Certainly not for more complex models, and definately not when it comes to most unit testing, UAT and troubleshooting. And well TM1 models can get pretty big, and having multiple developers each working on their own separate model can add up to a lot of resources and a lot of interdependencies.
Most of the time having a shared model being worked on by multiple developers makes a lot more sense. With developers reserving & checking-in / checking-out discrete packages of objects. And simple mechanisms to select a collection of objects to migrate between instances. But this is very difficult to manage (or is currently unmannegable) with how IBM have done the Git integration.
I'm still hopeful this might evolve into a decent tool, but I do admit to not being overly optimistic about it.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
- macsir
- MVP
- Posts: 785
- Joined: Wed May 30, 2012 6:50 am
- OLAP Product: TM1
- Version: PAL 2.0.9
- Excel Version: Office 365
- Contact:
Re: More info on TM1Project.json file
Actually, I think the working mode should be working. Each branch doesn't have to be the copy of entire TM1 server. They could have several objects you working on and later want to deploy to a target TM1 server. The central repo could have its own history and each branch could have history too. At any time, they could be merged back to central history. This is how Git works and I have tested. They are working fine. In Git, you can track history of code files, update and hot promote.The way IBM have conceptualized Git integration working is that each developer works on a separate branch, and each branch is essentially an individual copy of the entire TM1 server.
The only block for me is to figure out how to write proper tm1project file (I have learnt Ignore section but need more other sections) and how to deal with control objects (not individually). Hopefully IBM can provide more later.
For the data files, I am not concerning too much as it is not relevant to Git anyway. Git shouldn't maintain any history of any data file.
- macsir
- MVP
- Posts: 785
- Joined: Wed May 30, 2012 6:50 am
- OLAP Product: TM1
- Version: PAL 2.0.9
- Excel Version: Office 365
- Contact:
Re: More info on TM1Project.json file
Further on control objects for tm1project files, it is not a big deal anymore. I can easily build the json file with extractions for all control objects from }Cubes, }Processes and }Dimensions dimensions. And they can be initialized into Git completely.
-
- Posts: 1
- Joined: Tue Nov 03, 2020 8:54 am
- OLAP Product: TM1
- Version: 11.7.00002.1
- Excel Version: Office 2016
Re: More info on TM1Project.json file
Hi,
1) is there a way to add attribute values into Git now?
2) can anyone here add in this thread some example for "Link" and "Files" in the tm1project file?
3) how to transfer files from Application folder through Git? How to write it in the tm1project file in"Ignore"? All my attempts ended with error 290 and smth like "Failed to parse ignore string \"Applications('ForProcesses')/DocumentReferences('res.csv')\". Cannot find property \"Applications\" in entity type \"EntityContainer\"."
As stated earlier, the documentation is mostly incomprehensible and I have some difficulty using it.
Anyway, this thread has already helped me a lot, thanks everyone
1) is there a way to add attribute values into Git now?
2) can anyone here add in this thread some example for "Link" and "Files" in the tm1project file?
3) how to transfer files from Application folder through Git? How to write it in the tm1project file in"Ignore"? All my attempts ended with error 290 and smth like "Failed to parse ignore string \"Applications('ForProcesses')/DocumentReferences('res.csv')\". Cannot find property \"Applications\" in entity type \"EntityContainer\"."
As stated earlier, the documentation is mostly incomprehensible and I have some difficulty using it.
Anyway, this thread has already helped me a lot, thanks everyone
-
- Posts: 3
- Joined: Tue May 24, 2022 9:14 pm
- OLAP Product: TM1
- Version: PA 2.0.9
- Excel Version: 365
Re: More info on TM1Project.json file
I finally figured out how the TM1Project.json file and PA Cloud all play nicely together. I did a video to share with the community too.
https://www.youtube.com/watch?v=9XT9ALcKf4k
https://www.youtube.com/watch?v=9XT9ALcKf4k
- gtonkin
- MVP
- Posts: 1235
- Joined: Thu May 06, 2010 3:03 pm
- OLAP Product: TM1
- Version: Latest and greatest
- Excel Version: Office 365 64-bit
- Location: JHB, South Africa
- Contact:
Re: More info on TM1Project.json file
Thanks for sharing that insightful video - have saved for future use too.
-
- MVP
- Posts: 3194
- Joined: Mon Dec 29, 2008 6:26 pm
- OLAP Product: TM1, Jedox
- Version: PAL 2.0.9.18
- Excel Version: Microsoft 365
- Location: Brussels, Belgium
- Contact:
Re: More info on TM1Project.json file
That's very useful, thanks for that.
Best regards,
Wim Gielis
IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
Wim Gielis
IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
-
- Posts: 3
- Joined: Tue May 24, 2022 9:14 pm
- OLAP Product: TM1
- Version: PA 2.0.9
- Excel Version: 365
Re: More info on TM1Project.json file
With more experimenting, I realized some of what I understood was wrong. Here are two followup videos to correct and expand on my approach.bobgillvan wrote: ↑Tue May 24, 2022 9:19 pm I finally figured out how the TM1Project.json file and PA Cloud all play nicely together. I did a video to share with the community too.
https://www.youtube.com/watch?v=9XT9ALcKf4k
Postman / TM1 - Variables - https://youtu.be/xfMJOFhnCsE
Postman / TM1 - GitHub Integration and Usage - https://youtu.be/BqvqoYb1QU0