REST API Dimension Clone

[dan.kelleher]

Hello,

We have a simple Powershell script to hot promote TI processes, and am looking to implement something similar for dimension metadata.

I'm looking to replicate a dimension between 2 services using the REST API as opposed to TI. It should be an exact copy, including attributes. Does anyone know if this is possible without deleting/ recreating consolidated elements and re-processing the edges? i.e. is there a simple shortcut I'm not aware of?

http://${"host"}:${"port"}/api/v1/Dimensions('from')/Hierarchies('from')/Elements
http://${"host"}:${"port"}/api/v1/Dimensions('from')/Hierarchies('from')/Edges

I am aware of how this can be done by exporting/ importing using file-based approach, and am also aware of git integration as a potential solution, although I'd prefer to be able to trigger this from the source TM1 service in isolation:

https://www.ibm.com/docs/en/planning-an ... ntegration.

Many thanks for looking,

Dan

[Wim Gielis]

AFAIK, indeed you need the elements and the edges.
No direct endpoint to do this although that would be useful.

[lotsaram]

There's no single endpoint to duplicate a dimension. You can get the elements and structure, but you then need separate calls to create the attributes and then copy the data.

[tm123]

Hello,

We have a simple Powershell script to hot promote TI processes, and am looking to implement something similar for dimension metadata.

I'm looking to replicate a dimension between 2 services using the REST API as opposed to TI. It should be an exact copy, including attributes. Does anyone know if this is possible without deleting/ recreating consolidated elements and re-processing the edges? i.e. is there a simple shortcut I'm not aware of?

http://${"host"}:${"port"}/api/v1/Dimensions('from')/Hierarchies('from')/Elements
http://${"host"}:${"port"}/api/v1/Dimensions('from')/Hierarchies('from')/Edges

I am aware of how this can be done by exporting/ importing using file-based approach, and am also aware of git integration as a potential solution, although I'd prefer to be able to trigger this from the source TM1 service in isolation:

https://www.ibm.com/docs/en/planning-an ... ntegration.

Many thanks for looking,

Dan

Can you provide some details about your Powershell Script you use to hot promote TI Processes?
I am trying to do the same thing but I cant find any help online

[Wim Gielis]

Can you provide some details about your Powershell Script you use to hot promote TI Processes?
I am trying to do the same thing but I cant find any help online

Where are you in your TM1 REST api journey ?
Do you use a tool like Postman to execute requests and capture the response ?
Do you use Python scripting and tm1py to do the heavy lifting ?
What programming language or tool do you use to launch the requests ?

To hot promote TIs, you would typically do a GET request towards Processes(‘nameoftheprocess’) (source TM1 server). Empty body in the request.
Then a POST or PATCH request towards Processes to create or update a process (destination TM1 server). Body contains the JSON representation of the process contents (reworked from the GET output above)

If you show what you now have, people might be able to provide more help.

[tm123]

Can you provide some details about your Powershell Script you use to hot promote TI Processes?
I am trying to do the same thing but I cant find any help online

Where are you in your TM1 REST api journey ?
Do you use a tool like Postman to execute requests and capture the response ?
Do you use Python scripting and tm1py to do the heavy lifting ?
What programming language or tool do you use to launch the requests ?

To hot promote TIs, you would typically do a GET request towards Processes(‘nameoftheprocess’) (source TM1 server). Empty body in the request.
Then a POST or PATCH request towards Processes to create or update a process (destination TM1 server). Body contains the JSON representation of the process contents (reworked from the GET output above)

If you show what you now have, people might be able to provide more help.

I have used Postman but I was hoping I can use powershell script to do the following

Lets say I have all the processes I want to promote in a folder. Loop through all the processes and execute a GET Request for each of them in the Source Server, and the execute a POST/Patch request for that process in Target Server.

I clearly need some help, so if someone can share a powershell script that does something similar, i would really appreciate it

[Wim Gielis]

I clearly need some help, so if someone can share a powershell script that does something similar, i would really appreciate it

Hello, I do hope that someone can post code here.
Until then, I would advise to start yourself because if no one does, you still have nothing.
Build it gradually, hardcoded in the beginning, then later on more advanced and dynamic.
In general Google or Chat GPT can be good staring points or even more.

[WilliamSmith]

I clearly need some help, so if someone can share a powershell script that does something similar, i would really appreciate it

I'm willing to dig into this to see what programmatic solution is possible. Can you give me a definition of "hot promote" in this context, that's one that's not in my vocabulary

[tm123]

I clearly need some help, so if someone can share a powershell script that does something similar, i would really appreciate it

I'm willing to dig into this to see what programmatic solution is possible. Can you give me a definition of "hot promote" in this context, that's one that's not in my vocabulary

Thank you William,

What I am trying to achieve is migrate a Process from DEV to QA for example, without having to restart the QA Instance. I was hoping I can do that with Rest API / Powershell

[gtonkin]

This series may help you get familiar with some of the commands and then those concepts could be translated to Powershell.

YKud’s article may be very helpful too…

[Edward Stuart]

I've not toyed with Powershell for some time but a historical script I have used looked like this:

Code: Select all

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", <Authentication Method - TM1/ CAM/ PA On Cloud/ AWS?>)

$csvfolder = <MyExportLocation>
$csvfile = $csvfolder + '<MyExportFileName>.csv'

$response = Invoke-RestMethod 'http://<ServerName>:<HttpPortNumber>/api/v1/Dimensions(''<DimName>'')/Hierarchies(''<DimName>'')?$select=Name&$expand=Elements' -Method 'GET' -Headers $headers
$response | ConvertTo-Json
$response.Elements | Export-Csv -Delimiter "," -Encoding "UTF8" -path $csvfile -NoTypeInformation

Write-Host $response

I assume it still works! But should get you started

There are a few guides around on Authentication but George's code covers some of this but if you are hitting issues let us know

[dan.kelleher]

We use something like this

Code: Select all

#.\hot_promote_process.ps1 my_process_name DEV UAT PATCH

$processName  =$args[0]
$sourceEnv =$args[1]
$targetEnv = $args[2]
$method = $args[3]

function Set-Header {
    param (
        $EncodedText
    )
    
    $header = @{
        "Content-Type"="application/json; odata.metadata=none; odata.streaming=true; charset=utf-8"
        "Authorization"="Basic ${EncodedText}"
    }
    
    return $header
}

$sourceServer = switch ( $sourceEnv )
{
    "DEV" { "ip address/ host name"}
    "UAT" { "ip address/ host name"}
}

$targetServer = switch ( $targetEnv )
{
    "DEV" { "ip address/ host name"}
    "UAT" { "ip address/ host name"}
}

$sourcePort = switch ( $sourceEnv )
{
    "DEV" { "1234"}
    "UAT" { "5678"}
}

$targetPort = switch ( $targetEnv )
{
    "DEV" { "1234"}
    "UAT" { "5678"}
}

$Bytes = [System.Text.Encoding]::UTF8.GetBytes("user:pass")
$EncodedText =[Convert]::ToBase64String($Bytes)

$url = $url = "http://${sourceServer}:${sourcePort}/api/v1/Processes('$processName')"
$header = Set-Header $EncodedText
$response = Invoke-WebRequest -Uri $url -Method Get -Headers $header

$x = $response.content | ConvertFrom-Json
$x = $x | Select-Object * -ExcludeProperty Attributes
$body = $x | ConvertTo-Json

$url = $url = "http://${targetServer}:${TargetPort}/api/v1/Processes('$processName')"
$header = Set-Header $targetEncodedText
$response = Invoke-WebRequest -Uri $url -Method $method -Headers $header -Body $body

[Wim Gielis]

Thanks Dan.Kelleher.

After some modifications, this works fine for me:

Code: Select all

#.\hot_promote_process.ps1 my_process_name DEV PRD PATCH

$processName = $args[0]
$sourceEnv = $args[1]
$targetEnv = $args[2]
$method = $args[3]

function Set-Header {
    param (
        $EncodedText
    )

    $header = @{
        "Content-Type"="application/json; odata.metadata=none; odata.streaming=true; charset=utf-8"
        "Authorization"="Basic ${EncodedText}"
    }

    return $header
}

$sourceServer = switch ( $sourceEnv )
{
    "DEV" { "20.123.178.99" }
    "PRD" { "ip address/host name" }
    "UAT" { "ip address/host name" }
}

$targetServer = switch ( $targetEnv )
{
    "DEV" { "ip address/host name" }
    "PRD" { "20.123.178.99" }
    "UAT" { "ip address/host name" }
}

$sourcePort = switch ( $sourceEnv )
{
    "DEV" { "8003" }
    "PRD" { "1234" }
    "UAT" { "1234" }
}

$targetPort = switch ( $targetEnv )
{
    "DEV" { "5678" }
    "PRD" { "6002" }
    "UAT" { "5678" }
}

$sourceUser = switch ( $sourceEnv )
{
    "DEV" { "admin" }
    "PRD" { "wim" }
    "UAT" { "wim" }
}

$targetUser = switch ( $targetEnv )
{
    "DEV" { "wim" }
    "PRD" { "admin" }
    "UAT" { "wim" }
}

$sourcePassword = switch ( $sourceEnv )
{
    "DEV" { "apple" }
    "PRD" { "apple" }
    "UAT" { "apple" }
}

$targetPassword = switch ( $targetEnv )
{
    "DEV" { "apple" }
    "PRD" { "apple" }
    "UAT" { "apple" }
}

$sourceUseSSL = switch ( $sourceEnv )
{
    "DEV" { 0 }
    "PRD" { 0 }
    "UAT" { 0 }
}

$targetUseSSL = switch ( $targetEnv )
{
    "DEV" { 1 }
    "PRD" { 1 }
    "UAT" { 1 }
}

$sourceProtocol = if ("${sourceUseSSL}" -eq 1) { "s" } else { "" }
$targetProtocol = if ("${targetUseSSL}" -eq 1) { "s" } else { "" }

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}


# The source side

$sourceBytes = [System.Text.Encoding]::UTF8.GetBytes("${sourceUser}:${sourcePassword}")
# $sourceBytes = [System.Text.Encoding]::UTF8.GetBytes("${sourceUser}:${sourceUser}:${sourceNamespace}")
$sourceEncodedText = [Convert]::ToBase64String($sourceBytes)

$sourceURL = "http${sourceProtocol}://${sourceServer}:${sourcePort}/api/v1/Processes('$processName')"
$header = Set-Header $sourceEncodedText
$sourceResponse = Invoke-WebRequest -Uri $sourceURL -Method Get -Headers $header


# The target side

$x = $sourceResponse.content | ConvertFrom-Json
$x = $x | Select-Object * -ExcludeProperty Attributes
$body = $x | ConvertTo-Json

$targetBytes = [System.Text.Encoding]::UTF8.GetBytes("${targetUser}:${targetPassword}")
# $targetBytes = [System.Text.Encoding]::UTF8.GetBytes("${targetUser}:${targetUser}:${targetNamespace}")
$targetEncodedText = [Convert]::ToBase64String($targetBytes)


$header = Set-Header $targetEncodedText

if ("${method}" -eq "POST") { 
$targetURL = "http${targetProtocol}://${targetServer}:${targetPort}/api/v1/Processes"
} else { 
$targetURL = "http${targetProtocol}://${targetServer}:${targetPort}/api/v1/Processes('$processName')"
}

$targetResponse = Invoke-WebRequest -Uri $targetURL -Method $method -Headers $header -Body $body

[ardi]

Here is the version of the code I use to promote TIs from one TM1 Instance to another.
I have a config file where I save all the connection details and I created a function to get all the information i need from the config file
Also, I check if the Process exists in Target Instance. If it exists, I use PATCH method, otherwise, I use POST method

Code: Select all

$InputFolder = $args[0]
$srcEnv = $args[1]
$tgtEnv = $args[2]


$Directory = $InputFolder + "\pro\"

function Get-ConfigIni 
{  
    param(  
        [parameter(Mandatory = $true)] [string] $filePath  
    )  
    
    $anonymous = "NoSection"
  
    $ini = @{}  
    switch -regex -file $filePath  
    {  
        "^\[(.+)\]$" # Get Sections from Ini File 
        {  
            $section = $matches[1]  
            $ini[$section] = @{}  
            $CommentCount = 0  
        }  

        "^(;.*)$" # Comment  
        {  
            if (!($section))  
            {  
                $section = $anonymous  
                $ini[$section] = @{}  
            }  
            $value = $matches[1]  
            $CommentCount = $CommentCount + 1  
            $name = "Comment" + $CommentCount  
            $ini[$section][$name] = $value  
        }   

        "(.+?)\s*=\s*(.*)" # Key  
        {  
            if (!($section))  
            {  
                $section = $anonymous  
                $ini[$section] = @{}  
            }  
            $name,$value = $matches[1..2]  
            $ini[$section][$name] = $value  
        }  
    }  

    return $ini  
}  

$configIni = Get-ConfigIni .\config.ini

$srcServer = $configIni.$srcEnv.admin_host
$srcPort = $configIni.$srcEnv.port_number
$srcUser = $configIni.$srcEnv.user
$srcPassword = $configIni.$srcEnv.password
$srcUseSSL = $configIni.$srcEnv.useSSL

$tgtServer = $configIni.$tgtEnv.admin_host
$tgtPort = $configIni.$tgtEnv.port_number
$tgtUser = $configIni.$tgtEnv.user
$tgtPassword = $configIni.$tgtEnv.password
$tgtUseSSL = $configIni.$tgtEnv.useSSL

function Set-Header {
    param (
        $EncodedText
    )

    $header = @{
        "Content-Type"="application/json; odata.metadata=none; odata.streaming=true; charset=utf-8"
        "Authorization"="Basic ${EncodedText}"
    }

    return $header
}


$srcHttp = if ("${srcUseSSL}" -eq 1) { "https" } else { "http" }
$tgtHttp = if ("${tgtUseSSL}" -eq 1) { "https" } else { "http" }

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}


# The source side

$srcBytes = [System.Text.Encoding]::UTF8.GetBytes("${srcUser}:${srcPassword}")
# $srcBytes = [System.Text.Encoding]::UTF8.GetBytes("${srcUser}:${srcUser}:${srcNamespace}")
$srcEncodedText = [Convert]::ToBase64String($srcBytes)

foreach ($FileName in Get-ChildItem $Directory) {
	$FileType = $FileName.Extension
	if ("${FileType}" -eq ".pro") { 
		$processName = $FileName.Basename
		##
		$srcURL = "${srcHttp}://${srcServer}:${srcPort}/api/v1/Processes('$processName')"
		$header = Set-Header $srcEncodedText

		#$srcResponse = Invoke-WebRequest -Uri $srcURL -Method Get -Headers $header

		$srcErrorCode = 0
		try {
			$srcResponse = Invoke-WebRequest -Uri $srcURL -Method Get -Headers $header
		} catch {
			$StatusCode = [int]$_.Exception.Response.StatusCode
			$srcErrorCode = $StatusCode

		}
		if ("${srcErrorCode}" -eq 0) 
		{ 
			$srcErrorCode = $(Invoke-WebRequest -Uri $srcURL -Method Get -Headers $header).statuscode
		}

		# The target side
		if ("${srcErrorCode}" -eq 200) 
		{
			$x = $srcResponse.content | ConvertFrom-Json
			$x = $x | Select-Object * -ExcludeProperty Attributes
			$body = $x | ConvertTo-Json

			$tgtBytes = [System.Text.Encoding]::UTF8.GetBytes("${tgtUser}:${tgtPassword}")
			# $tgtBytes = [System.Text.Encoding]::UTF8.GetBytes("${tgtUser}:${tgtUser}:${tgtNamespace}")
			$tgtEncodedText = [Convert]::ToBase64String($tgtBytes)


			$header = Set-Header $tgtEncodedText

			#== Check if Process Exists in Target Server. If it Exists, use PATCH, otherwise, use POST

			$tgtErrorCode = 0
			$tgtURLCheck = "${tgtHttp}://${tgtServer}:${tgtPort}/api/v1/Processes('$processName')"
			try {
				$tgtResponse = Invoke-WebRequest -Uri $tgtURLCheck -Method Get -Headers $header
			} catch {
				$StatusCode = [int]$_.Exception.Response.StatusCode
				$tgtErrorCode = $StatusCode

			}

			if ("${tgtErrorCode}" -eq 0) 
			{ 
				$tgtErrorCode = $(Invoke-WebRequest -Uri $tgtURLCheck -Method Get -Headers $header).statuscode
			}	
			if ("${tgtErrorCode}" -eq 200 ) 
			{ 
				$method = "PATCH"
				Write-Output "Process $processName exists, therefore, using $method method"
			}
			else
			{
				$method = "POST"
				Write-Output "Process $processName does not exist, therefore, using $method method"
			}

			if ("${method}" -eq "POST") { 
				$tgtURL = "${tgtHttp}://${tgtServer}:${tgtPort}/api/v1/Processes"
			} else { 
				$tgtURL = "${tgtHttp}://${tgtServer}:${tgtPort}/api/v1/Processes('$processName')"
			}
			if ("${method}" -eq "POST") { 
				$tgtResponse = Invoke-WebRequest -Uri $tgtURL -Method $method -Headers $header -Body $body
			} else { 
				$tgtResponse = Invoke-WebRequest -Uri $tgtURL -Method $method -Headers $header -Body $body
			}
		}
		else
		{
			Write-Output "Unable to Hot promote process $processName"
		}
##
		
	}
    
}

[Wim Gielis]

Thank you Ardian. Good additions.
Maybe useful to post a sample file for 'config.ini' (though probably standard and not too difficult to do it ourselves) ?
Of course without sensitive information but just the structure and some examples of entries.

I would also skip the If test for the PRO extension:

Code: Select all

foreach ($FileName in Get-ChildItem $Directory -Filter "*.pro") {
	$processName = $FileName.Basename

[ardi]

Thank you Ardian. Good additions.
Maybe useful to post a sample file for 'config.ini' (though probably standard and not too difficult to do it ourselves) ?
Of course without sensitive information but just the structure and some examples of entries.

I would also skip the If test for the PRO extension:

Code: Select all

foreach ($FileName in Get-ChildItem $Directory -Filter "*.pro") {
	$processName = $FileName.Basename

Thanks Wim,

Here is the sample config.ini file

Code: Select all

[DEV]
admin_host=localhost
port_number=8881
user=admin
password=apple
useSSL=0

[UAT]
admin_host=localhost
port_number=8882
user=admin
password=apple
useSSL=0

Good suggestion regarding the if statement, the folder though is supposed to have only pro files and in my code, i write to an audit file the names of files that should not be on that folder.

[Wim Gielis]

Thank you. Happy new year to all