TM1 10.2: Get Filesize via TI

[Mark2007]

Hi there,

I'm driving a daily import of ASCII-Files. Sometime, the sourcesystem delivers files without any data, so the filesize =0. Unfortunately, if my import starts, I start with a zeroout, so all data in the cube are gone and as the sourcefile does not contain any data, the cube keeps empty until a correct file is delivered the next day.

Is there any possibility to get the filesize in the TI, so that I can processbreak, if filesize = 0?

Thanks in advance!

Mark

[Alan Kirk]

I'm driving a daily import of ASCII-Files. Sometime, the sourcesystem delivers files without any data, so the filesize =0. Unfortunately, if my import starts, I start with a zeroout, so all data in the cube are gone and as the sourcefile does not contain any data, the cube keeps empty until a correct file is delivered the next day.

Is there any possibility to get the filesize in the TI, so that I can processbreak, if filesize = 0?

There is no direct way of doing that that (I haven't read through the documentation of the latest version yet, but I doubt it). However I used to have a similar problem with an ODBC read from our then-PeopleSoft ledger. Sometimes the database connection would be down in some way but it wouldn't error, it would just return zero records. What I did was to put a "check" process in front of the one where I did the ViewZeroOut; it would open the connection, start to read the records and if it found any then a line in the Data tab would write to my control cube, and the rest of the chore would proceed. (It wouldn't read the whole data set obviously; it quit after (I think it was) 5 or 6 records.) If the flag had not been set, then the main process knew that there was no data. Instead of calling the ViewZeroOut and the main load, the chore would abort.

In your shoes I would probably not just abort but run a batch file to archive the file out to another folder.

[upali]

As a practice I usually use the MetaData tab to scan the file and find the intersections in the source file to determine what sections should be used for ZeroOut. However, you would need to relay on some assumptions and a contract between the Source System sending the file because it is not very efficient to ZeroOut every single combination of data. For example, the file will provide ALL departments but just one or more days of data in the file. The main assumption here is ALL departments will be present for a given day, not some in one day and some in another day.

I would also suggest a generation of an Exception report if "invalid or missing" items are present in the file and send an email if the TI completed with an exception report. This was you'll be on-top of the issue before people start to complain of missing data. A combination of these steps will give you an avenue to identify files that do not have "enough" data.

[gtonkin]

You could also create and execute a batch file to test for the size of the file - if the file size is non-zero, copy it to another file which you will then use as the datasource for your process e.g.

Code: Select all

@ECHO OFF
SET size=%~z1
if %size% GTR 0 (
	rem File is greater than zero
	rem add commands to copy file to new name for import etc.
) ELSE (
	rem File is zero size - do nothing
)

You could then use TI to check if the file you copy to, to be used as your source exists. If it does, process it.

The %~z1 gets the size of the file referred to by the first parameter.

[BariAbdul]

Thanks , gtonkin.Quite useful.