Help on Data Validation using TM1PY

[ravi.tm1]

Hi All,

I am new to TM1PY. We have requirement to do data validation using TM1PY and send email notification if there are differences in data.

I am extracting data in csv file from source TM1 server and loading the data in target TM1 server.

As we extracting data in files there is possibility of any modification in files who has access to these files. So, we want to do data validations using TM1PY.

Can we do data validations using TM1PY if yes can anyone guide me steps to do data validations between TM1 servers and send email notification.

Regards,
Ravi

[Wim Gielis]

What kind of data validations do you intend to do ?

[lotsaram]

So let me get this straight.
* data from Cube A on server 1 is extracted as CSV
* this CSV is used as a data source to load data to cube B on server 2
* because of concerns that someone with access to the CSV may have changed the data before loading to cube B you want to check that CSV values are still the same as cube A

This doesn't sound ridiculous to you? It sure does to me. There are several better ways to solve the problem, for example:
* keep file based data exchange but chain export and import together in one workflow
* export the files to a location that users don't have access to
* encrypt the CSVs after export and decrypt prior to import
* have tm1py connect to both servers and do the extraction and load in the same routine (no need for any files, no opportunity for anyone to change any data)

[Emixam]

That's what we are doing. TM1py is connected to both server and we do something like this:

Code: Select all

mdx_server1 = "my_mdx1"
mdx_server2 = "my_mdx2"
df_server1 = tm1_server1.cells.execute_mdx_dataframe(mdx_server1)
df_server2 = tm1_server2.cells.execute_mdx_dataframe(mdx_server2)
if df_server1.equals(df_server2):
	# they are equals

Depending on your use case, you might have to do some transformations on your dataframes. I also like using MDXpy to create my mdx

If possible, i would do the extract, transformation and load in TM1py. No need to export to csv.