Portal | IBM Manuals | Downloads | Products | Refer | Info | Programs | JCLs | Forum Rules*| Site Map | Mainframe CD 
IBMMAINFRAMES.com - IBM Mainframe Support Forums Index
 
Register
 
IBMMAINFRAMES.com - IBM Mainframe Support Forums Index FAQ Search Memberlist Usergroups Profile Log in to check your private messages Log in
 
How to reduce the file size (No. of records)

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
Author Message
leo_sangha

Active User


Joined: 11 Aug 2005
Posts: 51
Location: England
PostPosted: Tue Aug 19, 2008 1:48 pm    Post subject: How to reduce the file size (No. of records)
 Reply with quote
Input File VB, having 200,000 records

Field Name Position length
Record-Code 31 CHAR 4
(possible values 100,200,300, etc.)


Every record-code is associated with Record-Type field

Field Name Position length
Record-Type 10 BI(2)
(possible values 116,118,138, etc.)

Every record-code can have more than 1000 records with same record-type.

Requirement :
Create an output file in which every record-code should not have more than 20 records of same record-type.

Can this be done using DFSORT ? Any help will be appreciated.
Back to top
View user's profile Send private message
References
leo_sangha

Active User


Joined: 11 Aug 2005
Posts: 51
Location: England
PostPosted: Tue Aug 19, 2008 2:07 pm    Post subject: Reply to: How to reduce the file size (No. of records)
 Reply with quote
Just to give you an idea of how the records look like:

Customer Record-code Record-Type
000001 100 116
000002 200 116
000003 100 116
000004 200 116
000005 300 116
000006 100 116
000007 300 116
-
-
-
100001 100 118
100002 200 118
100003 100 118
100004 200 118
100005 300 118
100006 100 118
100007 300 118

Let me know if the req is still not clear
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 3502
Location: Brussels once more ...
PostPosted: Tue Aug 19, 2008 2:35 pm    Post subject:
Reply with quote
Use tags to make your post more easily readable
Back to top
View user's profile Send private message
arcvns

Senior Member


Joined: 17 Oct 2006
Posts: 706
Location: Chennai, India
PostPosted: Tue Aug 19, 2008 3:26 pm    Post subject:
Reply with quote
Are your input records sorted on Customer field?

Thanks,
Arun
Back to top
View user's profile Send private message
leo_sangha

Active User


Joined: 11 Aug 2005
Posts: 51
Location: England
PostPosted: Tue Aug 19, 2008 4:02 pm    Post subject: Reply to: How to reduce the file size (No. of records)
 Reply with quote
Yes the records are sorted on the customer number
Back to top
View user's profile Send private message
arcvns

Senior Member


Joined: 17 Oct 2006
Posts: 706
Location: Chennai, India
PostPosted: Tue Aug 19, 2008 5:36 pm    Post subject:
Reply with quote
leo_sangha,

Try out the below ICETOOL job.

I have assumed a VB file, LRECL=80 as input and the position of "Customer" as 1-6. You may change the field positions as per your file attributes.

Code:
//STEP00   EXEC PGM=ICETOOL                                       
//TOOLMSG  DD SYSOUT=*                                             
//DFSMSG   DD SYSOUT=*                                             
//TEMP1    DD DSN=&&TEMP1,DISP=(,PASS),SPACE=(CYL,(5,5)),UNIT=SYSDA
//IN       DD DISP=SHR,DSN=your.input.file                         
//OUT      DD DISP=SHR,DSN=your.output.file                       
//TOOLIN   DD *                                                   
SORT FROM(IN)    TO(TEMP1) USING(CTL1)                             
SORT FROM(TEMP1) TO(OUT)   USING(CTL2)                             
//*                                                               
//CTL1CNTL DD *                                                   
  INREC OVERLAY=(81:14,2,83:35,4)                                 
  SORT FIELDS=(81,6,CH,A)                                         
  OUTREC IFTHEN=(WHEN=INIT,OVERLAY=(87:SEQNUM,8,ZD,RESTART=(81,6)))
  OUTFIL INCLUDE=(87,8,ZD,LE,20),BUILD=(1,80)                     
//*                                                               
//CTL2CNTL DD *                                                   
  SORT FIELDS=(5,6,CH,A)   


Thanks,
Arun
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 4574
Location: San Jose, CA
PostPosted: Tue Aug 19, 2008 9:27 pm    Post subject:
Reply with quote
Arun,

This is a VB input file, but you've padded the output records as if it were an FB file. If the input records started out as 38 bytes, the output records will be 80 bytes instead of 38 bytes. Not good!

leo_sangha,

Here's a DFSORT/ICETOOL job that uses the new SELECT FIRST(n) function available with z/OS V1R5 PTF UK90013 (July, 2008) to do what you asked for and preserve the record lengths:

Code:

//S1   EXEC  PGM=ICETOOL
//TOOLMSG   DD  SYSOUT=*
//DFSMSG    DD  SYSOUT=*
//IN DD DSN=...  input file (VB)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(MOD,PASS)
//OUT DD DSN=...  output file (VB)
//TOOLIN DD *
SELECT FROM(IN) TO(T1) ON(14,2,CH) ON(35,4,CH) FIRST(20)
SORT FROM(T1) TO(OUT) USING(CTL1)
/*
//CTL1CNTL DD *
  OPTION EQUALS
  SORT FIELDS=(5,6,CH,A)
/*


If you don't have the July, 2008 PTF and can't get your System Programmer to install it, you can use this DFSORT/ICETOOL job without SELECT FIRST(n) which will also preserve the record lengths:

Code:

//S2   EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//IN DD DSN=...  input file (VB)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=...  output file (VB)
//TOOLIN   DD *
SORT FROM(IN) USING(CTL1)
SORT FROM(T1) TO(OUT) USING(CTL2)
/*
//CTL1CNTL DD *
  INREC BUILD=(1,4,5:14,2,7:35,4,11:8X,19:5)
  SORT FIELDS=(5,6,CH,A)
  OUTREC IFTHEN=(WHEN=INIT,
     OVERLAY=(11:SEQNUM,8,ZD,RESTART=(5,6)))
  OUTFIL FNAMES=T1,INCLUDE=(11,8,ZD,LE,20),BUILD=(1,4,5:19)
/*
//CTL2CNTL DD *
  SORT FIELDS=(5,6,CH,A)
/*
Back to top
View user's profile Send private message
arcvns

Senior Member


Joined: 17 Oct 2006
Posts: 706
Location: Chennai, India
PostPosted: Wed Aug 20, 2008 8:07 am    Post subject:
Reply with quote
Frank,

I had assumed both input and output files as VB, LRECL=80 and the results were looking just fine. I have a few questions here.

Will padding all the records at 81 st position make the VB file look like an FB file of fixed record length?

How do we verify whether record lengths are preserved for a VB file?

Thanks,
Arun
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 4574
Location: San Jose, CA
PostPosted: Wed Aug 20, 2008 8:55 pm    Post subject:
Reply with quote
Arun,

Quote:
I had assumed both input and output files as VB, LRECL=80 and the results were looking just fine.


You are confusing the LRECL of a VB file with the lengths of the records. The LRECL gives the maximum length. The records can be shorter than the maximum length (that's the point of a variable-length record).

The results would only look fine if every input record was 80 bytes long which, though possible, is not usual for VB files. And it's certainly not something you should ever assume about a VB file.

Quote:
Will padding all the records at 81 st position make the VB file look like an FB file of fixed record length?


You question really doesn't make any sense. A VB file has an RDW in positions 1-4 with the length of the record in the first two bytes (in binary) - the data starts in position 5. An FB file does not have an RDW - the data starts in position 1. VB records can have different lengths up to the LRECL. FB records are all the LRECL length. Padding VB records unnecessarily just wastes space - the point of a VB file is to have different length records as needed.

Quote:
How do we verify whether record lengths are preserved for a VB file?


You can display the record length and data before and after. You can use a DFSORT job like the following to display VB records for verification:

Code:

//SHOWV EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN  DD DSN=...  input file (VB)
//SORTOUT DD SYSOUT=*
//SYSIN    DD    *
  OPTION COPY
  OUTREC BUILD=(1,4,1,2,BI,C'|',5)
/*


SORTOUT might show:

Code:

   14 |AAA FIELD1       
   20 |BBB FIELD1FIELD2


When I used this for your output, it showed the record length as 80 for every output record even though my input file had shorter records. The job I showed preserves the record lengths from input to output.
Back to top
View user's profile Send private message
arcvns

Senior Member


Joined: 17 Oct 2006
Posts: 706
Location: Chennai, India
PostPosted: Wed Aug 20, 2008 10:06 pm    Post subject:
Reply with quote
Quote:
Will padding all the records at 81 st position make the VB file look like an FB file of fixed record length?


Frank,

I should have put this in a different way. What I meant to ask was, whether padding makes the RDW field of all the records the same value thereby causing wastage of space.

Thanks for making things clear.

Thanks,
Arun
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 4574
Location: San Jose, CA
PostPosted: Wed Aug 20, 2008 11:35 pm    Post subject:
Reply with quote
Yes, if you pad the records to 80 bytes, the RDW length will be 80 (4 byte RDW + 76 bytes of data).
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1