RE: ETL for Flexcab Sample Requests failed with file size error

[Techow, Ric]

Hi Paul,

Wondering if you could help me out.  Should gzip be able to handle files 
produced by PKZIP in ZIP64 format as referred to below?

Thanks for any help you give.

Ric Techow
Performance Analyst
Cost Value Engineering Group
Telstra Account | Communications, Media and Entertainment Group

HP Enterprise Services
Mob 1: +61 415 637 097
Mob 2: +61 402 815 984
E-mail:address@hidden
4, Wesley Court
East Burwood VIC 3151

From: Jakupi, Sezar
Sent: Friday, 27 July 2012 11:11 AM
To: Pontifex, Mark; Techow, Ric
Cc: AS AU RADTI.NET; Watson, Andrew
Subject: RE: ETL for Flexcab Sample Requests failed with file size error
Importance: High

Mark

I created ITAM  Incident INC000009519423. Priority: High. Description: PKZIP 
compression errors FLEXCAB application and assigned to IBM Software Support 
group who install and maintain PKZIP on z/OS platform.

This is the latest reply from the IBM Support group.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

A compressed file format of 'ZIP64' is apparently used by PKZIP to compress 
files > 4gb in size.

Really need to have details of jobs that were run at z/OS end (ie. specific 
output for a job whose filer produced the issue at the Solaris end and one that 
did not) so we can see if PKZIP behaviour was different.

Also need to have SOLARIS support ensure their 'GZIP'  understands the ZIP64 
file format.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Mark - can you please review and action with SOLARIS support ?

Regards,
Sezar Jakupi

Capacity Planning, Performance Tuning, Cost Value Engineering Group
Communications, Media and Entertainment Group
HP Enterprise Services
Mobile +61 450 266 799
Email address@hidden<mailto:address@hidden>
Level 1, 4 Wesley Place East Burwood, Victoria 3151
5th Floor, 484 St.Kilda Road Melbourne, Vic 3004.

From: Pontifex, Mark
Sent: Thursday, 19 July 2012 9:40 PM
To: Techow, Ric
Cc: AS AU RADTI.NET; Watson, Andrew; Jakupi, Sezar
Subject: RE: ETL for Flexcab Sample Requests failed with file size error

Hi Ric

Has anybody been able to look into this further with Flexcab ?

Based on Nishant being able to un-compress and then re-compress Flexcab files 
on our side using PKZIP and then ETL them using our normal (gunzip) to 
un-compress the files seems to indicate a PKZIP issue on the Mainframe side, 
but please correct me if my assumption is incorrect or unfounded ?

In the end of the day we're all HP and trying to avoid dockets and although 
Anton is quite patient his patience only extends so far.

Thank you

regards

Mark Pontifex
Application Manager - Mercury & RADTI (Telstra)
HP Enterprise Services
Telephone +61 2 9012 5519
Email address@hidden<mailto:address@hidden>
iTIL v3 Foundations + Service Operations
Level 4, 36 - 46 George Street Burwood NSW 2134
[Description: cid:image001.png@01CA798F.FD7CCFE0]

From: Pontifex, Mark
Sent: Tuesday, 17 July 2012 7:13 PM
To: Techow, Ric
Cc: AS AU RADTI.NET; Watson, Andrew
Subject: RE: ETL for Flexcab Sample Requests failed with file size error

Hi Ric

Whilst this is a fresh issue would you or possibly Andrew be able to follow up 
with Flexcab ?

Given Nishant un-compressed and then recompressed the Flexcab file using PKZIP 
on the RADTI side then went through our normal ETL process without issue looks 
like an issue with PKZIP used on the mainframe ?

regards

Mark Pontifex
Application Manager - Mercury & RADTI (Telstra)
HP Enterprise Services
Telephone +61 2 9012 5519
Email address@hidden<mailto:address@hidden>
iTIL v3 Foundations + Service Operations
Level 4, 36 - 46 George Street Burwood NSW 2134
[Description: cid:image001.png@01CA798F.FD7CCFE0]

From: Pontifex, Mark
Sent: Tuesday, 17 July 2012 2:49 PM
To: Techow, Ric; Vaishnaw, Nishant
Cc: Thiruvengadam, Kumaraguru; AS AU RADTI.NET
Subject: RE: ETL for Flexcab Sample Requests failed with file size error

Hi Ric

No we would have to direct that to Flexcab and their Mainframe jobs.

regards

Mark Pontifex
Application Manager - Mercury & RADTI (Telstra)
HP Enterprise Services
Telephone +61 2 9012 5519
Email address@hidden<mailto:address@hidden>
iTIL v3 Foundations + Service Operations
Level 4, 36 - 46 George Street Burwood NSW 2134
[Description: cid:image001.png@01CA798F.FD7CCFE0]

From: Techow, Ric
Sent: Tuesday, 17 July 2012 2:40 PM
To: Vaishnaw, Nishant
Cc: Thiruvengadam, Kumaraguru; AS AU RADTI.NET
Subject: RE: ETL for Flexcab Sample Requests failed with file size error

That looks entirely correct analysis.

Have we got the jobs which zip on the mainframe side??

Ric Techow
Performance Analyst
Cost Value Engineering Group
Telstra Account | Communications, Media and Entertainment Group

HP Enterprise Services
Mob 1: +61 415 637 097
Mob 2: +61 402 815 984
E-mail:address@hidden
4, Wesley Court
East Burwood VIC 3151

From: Vaishnaw, Nishant
Sent: Tuesday, 17 July 2012 2:37 PM
To: Techow, Ric
Cc: Thiruvengadam, Kumaraguru; AS AU RADTI.NET
Subject: RE: ETL for Flexcab Sample Requests failed with file size error

HI Ric,

As per the analysis the below two script are involve in the Flexcab file zip 
issue:
Job data_prepare.pm\job_af.pl


if ($suffix eq "zip" or $suffix eq "gz" or $suffix eq "ZIP") {
                                &log($logLevel, "dataPrepare", "gzip -df 
required for suffix: $suffix\n");

                                $command_line="gzip -df 
$FilePath/$FinalBaseFile.gz";   ------------------------> gzip command causing 
problem for +2 GB file
                                $status=system($command_line);
                                if ($status != 0)
                                {
                                                &log($logLevel, 
"dataPreparation: - Unable to gzip -df from file: $FinalBaseFile.gz\n");  
---------->  error message which we got after Job failure
                                                return 1;
                                }

                }



Please correct\suggest  me if iam wrong





Thanks & Regards
Nishant Vaishnaw
Telstra - Mediation and Rating
Level 7, Wing 3, Cluster C, EON, Kharadi, Pune - 411014 |
Ext: 5237111
Mobile +91-9011040128
e-mail - address@hidden<mailto:address@hidden>




From: Vaishnaw, Nishant
Sent: Wednesday, July 11, 2012 1:21 PM
To: Techow, Ric
Cc: Thiruvengadam, Kumaraguru; AS AU RADTI.NET; Pontifex, Mark
Subject: RE: ETL for Flexcab Sample Requests failed with file size error

Hi Ric,

As part of analysis ,
1)We have Unzip using PKZIP the file "FLEXETLFAL100001070612650712BIT.gz"
2)Then we have zip again the file

After triggering ETL process we have noticed that The ETL process "316498" 
completed successfully without any length error .






Thanks & Regards
Nishant Vaishnaw
Telstra - Mediation and Rating
Level 7, Wing 3, Cluster C, EON, Kharadi, Pune - 411014 |
Ext: 5237111
Mobile +91-9011040128
e-mail - address@hidden<mailto:address@hidden>



From: Pontifex, Mark
Sent: Tuesday, July 10, 2012 10:20 AM
To: Chakraborty, Arunuday; Techow, Ric; AS AU TC Invoicing
Cc: AS AU RADTI.NET
Subject: FW: ETL for Flexcab Sample Requests failed with file size error

Hi Arun / Ric

We've seen a repeat of the issue with compressed Flexcab files, could you check 
the same file on your side please.

Ric anything you would like to try on RADTI side ?

From: Patil, Ajay
Sent: Tuesday, 10 July 2012 2:44 PM
To: AS AU RADTI.NET
Subject: ETL for Flexcab Sample Requests failed with file size error


Hi Kumar , Nishant,

Today AF jobs for Flexcab Sample Requests ETLFAL1, ETLFAIL5, ETLFAIL7  have 
failed for following files
with gz: invalid compressed data--length error

-rwxrwxr-x   1 radti    radti        80M Jul  9 13:08 
FLEXETLFAL100001070612650712BIT.gz
-rwxrwxr-x   1 radti    radti       154M Jul  9 14:03 
FLEXETLFAIL50001070612050712BIT.gz
-rwxrwxr-x   1 radti    radti       106M Jul  9 15:18 
FLEXETLFAIL70001070612050712BIT.gz


Thanks,
Ajay Patil
Telstra - Rating and Mediation
SEZ EON Kharadi | Cluster C | Level 7 | Pune | 411014 | India
Desk -+91 2030540112  CUG- #55237112 Mobile +91 9011038537
Email :address@hidden<mailto:address@hidden>