Performance Issue While SORTing

ksk · Posted: Wed Aug 06, 2008 2:47 pm Post subject: Performance Issue While SORTing

Hi,

One JOb in our shop is concatenating 5 files and sorting on the following creteria.

gcicchet · Senior Member Joined: 28 Jul 2006 Posts: 696

Hi,

can you provide the SYSOUT messages from the job and job log ?

Gerry

ksk · Posted: Wed Aug 06, 2008 3:27 pm Post subject:

Following is SYSOUT.

gcicchet · Senior Member Joined: 28 Jul 2006 Posts: 696

Hi,

you are sorting some large files, one way to speed this up and I'm sure there will other suggestions would be to have 5 jobs sorting individual files followed by a merge job.

Gerry

gcicchet · Senior Member Joined: 28 Jul 2006 Posts: 696

also see http://www-304.ibm.com/systems/support/storage/software/sort/mvs/tuning/

Gerry

Dave Betten · New User Joined: 24 Jan 2006 Posts: 13

You are sorting 29GB of data and using 2GB of Hiperspace. So that means the remaining 27GB has to be written/read from DASD sortwork. So 3 hours is not entirely unreasonable but it's tough to say since I don't know what kind of I/O response times you are getting. This job is going to be heavily dependent on I/O response times for SORTIN, SORTWORK and SORTOUT.

Reducing the key field from 9,262 to 9,37 will help reduce CPU time but I don't think it will have a huge difference in elapsed.

I'd suggest you add //SORTDIAG DD DUMMY to the jcl for this sort and then send the sysout to dfsort@us.ibm.com. That's the email address for our DFSORT hotline and I can take a look and see if there's any tuning we can do for this sort. Be sure to send the entire sysout, not just the DFSORT messages.

ksk · Posted: Thu Aug 07, 2008 4:05 pm Post subject:

Hi Dave/Gerry,

Thanks for your immediate responses. I have increased work files and space and executed in test region with Production files, now it's taking 45 mins.

We will incorporate the same in production and if we face any problem I will let you know.

KSK

ksk · Posted: Thu Aug 07, 2008 4:09 pm Post subject:

Dave,

I forgot to ask one question. can you let me know what is the use of the following statement.

Dave Betten · New User Joined: 24 Jan 2006 Posts: 13

The SORTDIAG causes additional diagnostic messages to be printed and these help us when performing indepth analysis. It does not do anything to improve performance so it has nothing to do with your decreased run time. How much Hiperspace did the sort use when you ran it in your test environment? It could be that there was less resource contention in the test environment and that's why it ran so much faster. Increasing work space reduces the risk of failure but shouldn't have that drastic an impact on elapsed time. Why don't you send me the entire sysout from that run and I can take a look.

ksk · Posted: Thu Aug 07, 2008 5:53 pm Post subject:

Here is the SYSOUT of changed JOB.

Dave Betten · New User Joined: 24 Jan 2006 Posts: 13

It would be better if you could send the entire sysout, not just the dfsort messages, as a text file to dfsort@us.ibm.com. Also if you can run the original sort with only the SORTDIAG change (not the sort control card and work space changes) and send that entire sysout as well? Then I can compare both and provide some feedback.

ksk · Posted: Fri Aug 08, 2008 11:09 am Post subject:

Dave,

Just I want to know an information from you on Hiperspace. Is hyperspace allocated by default by DFSORT or it would depend on other parameters like CLASS parameter, number of Work files, space allocated for work files and output files etc...

I would like to know because when I didn't use any Sort Work files it used 2GB hyperspace. When added Workfiles and more space, used 3GB. Today we changed CLASS parameter also apart from workfiles and space, it used Hyperspace 4GB.

With CLASS change, Work files and more space, above mentioned 2 steps are executed in 70 Mins in production.

I will send the entire Sysout of 2 cases in a day or 2.

KSK

Dave Betten · New User Joined: 24 Jan 2006 Posts: 13

The amount of Hiperspace used depends on the amount of storage available to DFSORT at the time it executes. So it's quite possible there was just more available on the system at the time you ran the other sorts. Adding workspace does not effect how much Hiperspace we use.

Guru Bob · New User Joined: 31 Jan 2008 Posts: 21 Location: Malaysia

My estimate is 20 SORTWK files at 2450 cylinders each. I assumed 3390 device going by the BLKSizes in the SORT output.

2GB for memory is good if you can get it all the time.

Even your output file requires 21,248 cylinders on a 3390.

Is the input on DISK or CART?