ASCIIAPPEND

Suggest and discuss enhancements for TM1
Post Reply
User avatar
Steve Vincent
Site Admin
Posts: 1054
Joined: Mon May 12, 2008 8:33 am
OLAP Product: TM1
Version: 10.2.2 FP1
Excel Version: 2010
Location: UK

ASCIIAPPEND

Post by Steve Vincent »

Something that i'd like to do is append data to an existing ascii file from a TI, but current options do not allow this. Even writing to the same file from the same TI from different tabs will overwrite the existing file.

My solution would be to add another command called ASCIIAPPEND, to have the same criteria as ASCIIOUTPUT but adds to the bottom of the file rather than overwriting it. If the file doesn't exist, it should create it.

If others believe that to be a worthwhile addition i'm quite happy to raise a request to Cognos. I can't see it being too difficult, so it'll probably take as long as undo spread has... :lol:
If this were a dictatorship, it would be a heck of a lot easier, just so long as I'm the dictator.
Production: Planning Analytics 64 bit 2.0.5, Windows 2016 Server. Excel 2016, IE11 for t'internet
User avatar
Mike Cowie
Site Admin
Posts: 482
Joined: Sun May 11, 2008 7:07 pm
OLAP Product: IBM TM1/PA, SSAS, and more
Version: Anything thru 11.x
Excel Version: 2003 - Office 365
Location: Alabama, USA
Contact:

Re: ASCIIAPPEND

Post by Mike Cowie »

Hi Steve,

It would be nice to see some better log file management - maybe even a constant/variable (like ASCIIOutputAppend=1;) that let you switch on append for ASCIIOUTPUT written files for the duration of a process rather than a separate function...

To tide you over in the interim, you can write out each tab's log to a separate file and combine them with a DOS command in a separate process or (if you don't care about adding in an Epilog log portion) in the Epilog. From an old Applix forum thread (http://applixforum.olapforums.com/viewP ... eadID=6929:

Code: Select all

ExecuteCommand('CMD /C Copy Prolog.cma+Metadata.cma+Data.cma CompleteLog.cma',1);


Hopefully you get the general idea... not perfect, but it can get you to a single custom log file in the end.

Regards,
Mike
Mike Cowie
QueBIT Consulting, LLC

Are you lost without Print Reports in Planning Analytics for Excel (PAfE)? Get it back today, for free, with Print Reports for IBM Planning Analytics for Excel!
User avatar
Steve Vincent
Site Admin
Posts: 1054
Joined: Mon May 12, 2008 8:33 am
OLAP Product: TM1
Version: 10.2.2 FP1
Excel Version: 2010
Location: UK

Re: ASCIIAPPEND

Post by Steve Vincent »

Thanks Mike, in my current case i need to combine files that will be created daily but i can deal with that using the same code in a different way. Its useful to have that option of combining output from tab in a single TI tho, thanks :)
If this were a dictatorship, it would be a heck of a lot easier, just so long as I'm the dictator.
Production: Planning Analytics 64 bit 2.0.5, Windows 2016 Server. Excel 2016, IE11 for t'internet
User avatar
Alan Kirk
Site Admin
Posts: 6606
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk »

Steve Vincent wrote:Something that i'd like to do is append data to an existing ascii file from a TI, but current options do not allow this. Even writing to the same file from the same TI from different tabs will overwrite the existing file.

My solution would be to add another command called ASCIIAPPEND, to have the same criteria as ASCIIOUTPUT but adds to the bottom of the file rather than overwriting it. If the file doesn't exist, it should create it.

If others believe that to be a worthwhile addition i'm quite happy to raise a request to Cognos. I can't see it being too difficult, so it'll probably take as long as undo spread has... :lol:
Steve, I came across a situation where this would have been handy a day or two back. Have you raised it?

The only down side is what would happen if another process has a write lock on the file. My bet is that this is why we have the current situation where each tab creates its own file; the advantage is that since the tab creates the file itself, there's no possibility that the file could be locked.

My feeling is that the workaround for this chould be an argument in the AsciiAppend function which lets us tell the server what to do if it encounters such a write lock on the file that it's trying to append to:
0 = Ignore and continue processing (default)
1 = Fail the process with an error
2 = Create a new file with a timestamped name and write to that instead of the intended file.

Of course this would require a break away from the Applix mindset of "an error crashes the process even if we don't need or want it to". (See also, creating 256+ character strings in 9.0 which crash the process as opposed to just being ignored as they were in 8.2.12.)

The process should, of course, open the file with its own write lock.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: ASCIIAPPEND

Post by lotsaram »

I hadn't noticed this thread before but an ASCIIAppend function would certainly have my vote. The current workaround of declaring a separate test file for each tab of a process where using custom logging then having a batch file concatenate the files together is a long way from optimal.
User avatar
Alan Kirk
Site Admin
Posts: 6606
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk »

Alan Kirk wrote:
Steve Vincent wrote:Something that i'd like to do is append data to an existing ascii file from a TI, but current options do not allow this. Even writing to the same file from the same TI from different tabs will overwrite the existing file.

My solution would be to add another command called ASCIIAPPEND, to have the same criteria as ASCIIOUTPUT but adds to the bottom of the file rather than overwriting it. If the file doesn't exist, it should create it.

If others believe that to be a worthwhile addition i'm quite happy to raise a request to Cognos. I can't see it being too difficult, so it'll probably take as long as undo spread has... :lol:
Steve, I came across a situation where this would have been handy a day or two back. Have you raised it?
{Knock-knock} Steve? Anyone home? {Knock-knock, peers through gap in the curtains} Steve? Y'there?
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
User avatar
mattgoff
MVP
Posts: 516
Joined: Fri May 16, 2008 1:37 pm
OLAP Product: TM1
Version: 10.2.2.6
Excel Version: O365
Location: Florida, USA

Re: ASCIIAPPEND

Post by mattgoff »

I like it. I jump through a lot of hoops w/o this capability. Technically I would prefer a flag added to the ASCIIOutput function a la Perl e.g.

read-only: open FILEHANDLE, "filename.txt"
read-write: open FILEHANDLE, ">filename.txt"
append: open FILEHANDLE, ">>filename.txt"
Please read and follow the Request for Assistance Guidelines. It helps us answer your question and saves everyone a lot of time.
mincharug.shulft
Regular Participant
Posts: 155
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: ASCIIAPPEND

Post by mincharug.shulft »

Steve ,
i have only one ASCIIOUTPUT under the prolog tab
and as soon as TI excutes asciioutout is overwriting the file.
BUT what i want it is Once TI excutes i have to add the new data into the existed txt file.is it possible?
and also i have seen is asciiappend=1 parameter so where i have to add this parameter either in cfg or..?
and otherhand
by walking through all gentle man posts and help guide....
tried using excutecommand but i cannot rechable.

actually my ascii as it was thereb in the prolog tab.
asciioutput('para.log',str_date | ' ' |user_cube);
asciioutput('para.log',str_date | ' '|cube1);
asciioutput('para.log',str_date |' '|cube2);
and soon TI runs its ganarating the para.log into database directory.
and when next time TI runs its overwriting the file and content of data as well.
but i want it is when next time TI runs -the new data and old data sould be there in the para.log.
please give your aid?
winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: ASCIIAPPEND

Post by winsonlee »

You can use dos to append the new logfile to an existing log file.

ExecuteCommand('cmd /c type para.log >> paraTemp.log',0);
User avatar
Harvey
Community Contributor
Posts: 236
Joined: Mon Aug 04, 2008 4:43 am
OLAP Product: PA, TM1, CX, Palo
Version: TM1 8.3 onwards
Excel Version: 2003 onwards
Contact:

Re: ASCIIAPPEND

Post by Harvey »

Logging is an area that the new product I'm working on, Flow Responder, would fit nicely.

I think we've all had pain points in this area, but if you guys have any specific needs and requirements, please let me know and I'll create a logging plugin for the Flow Responder. This is a good opportunity to get in on the ground level and make sure I cover all the functionality on your wish list.

Feel free to get me privately if you prefer, at team@flowolap.com.
Take your TM1 experience to the next level - TM1Innovators.net
User avatar
Alan Kirk
Site Admin
Posts: 6606
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk »

I've raised this in the DeveloperWorks RFE group as request 63589. You need to be logged into DeveloperWorks to see it but if you have an account and want this, please do log in and vote for it if you want it too.

Description
A problem occurs when you want to write an output file which is either:
- Sourced from multiple tabs of a TI process;
- Sourced from multiple TI processes; or
- Both.

Each TI tab (procedure) will create and write any file that it is told to when using AsciiOutput / TextOutput. That means that if you write a file in the Prolog of process 1, then want to write to the same file in the Metadata tab then the Metadata tab output will overwrite the Prolog tab's output.

The same is true if you try to output to the same file in the Data tab, the Epilog tab, or a subsequent process.

In many cases we want only a single file of output. Traditionally the only approach to this has been to output to separate files with different filenames and then use (for example) the command line Copy command to merge the files into one. This is awkward and unwieldy especially if there will be an unknown number of output files, depending on the conditions that exist at runtime.

This could be overcome if in addition to AsciiOutput and TextOutput we had AsciiAppend and TextAppend functions which would allow the new output to be appended to the end of an existing file.

As there would be the danger of the file being locked by another process I would suggest that there would need to be a function parameter to specify what to do in such a case:

0 = Ignore and continue processing (default)
1 = Fail the process with an error
2 = Create a new file with a timestamped name and write to that instead of the intended file.
User Case
There is a range of situations that users have come across where this is needed. One is where you want to create a single audit log for a range of processes. Another is where you want to create a single output file from multiple data sources (cubes in some cases, views in another).

Given the limitations on output sort order from views, for example, in some cases the only way to get the output into particular orders is to create multiple views and export them separately. As things stand this requires outputting to multiple files and combining them.

The specifics will vary from site to site but the need is wide ranging.
Business justification
Without this function we need to rely on unreliable and cumbersome workarounds to be able to unite output into a single file. It unnecessarily consumes disk space by having many separate files on the disk system, and introduces unnecessary complexity by having the combination process reliant on code which exists outside of the TM1 server. This could result in incomplete files which may have an impact on the recipients of those files.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
User avatar
Alan Kirk
Site Admin
Posts: 6606
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: PA2.0.9.18 Classic NO PAW!
Excel Version: 2013 and Office 365
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk »

Joyous news, boys and girls, I just noticed that this request was last updated on the 7th of this month and now has a status of "Planned for Future Release".

How far in the future is unknown of course, but when it finally does get here (OK, I'll be Optimistic Ollie for a moment), I'll be the first to be patting Iboglix on the back for it.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.
VanBrink
Posts: 6
Joined: Wed Jul 18, 2012 11:50 am
OLAP Product: TM1, PA, PAW, PAX
Version: PA 2.0.x
Excel Version: MS Office 365
Location: Belgium,

Re: ASCIIAPPEND

Post by VanBrink »

Was this feature ever delivered in any of the later releases? I am not aware of any function like this? I have also looked at the available functions on Knowledge Centre and have looked whether an idea like this has been submitted to the IBM Data and AI ideas site.
Wim Gielis
MVP
Posts: 3103
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: ASCIIAPPEND

Post by Wim Gielis »

No, it wasn't.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
Post Reply