Tuesday, March 01, 2005

Splitting Input into Multiple Jobs

Splitting Input into Multiple Jobs
Having multiple concurrent jobs each perform part of an application’s work on a system can drive up CPU usage and increase application throughput. Once you’ve determined the number of jobs you want to run, distribute the input data evenly among them and start them running.
This may be easier said than done. The methodology you choose to distribute input among multiple jobs may depend on your application’s design and require close study of the application to see what steps are necessary.
If you have a single input file of transactions to be applied to your database, and you have one job to process the data, start multiple instances of the job (k of them) and spread the processing among them. The first job would process from record 1 to record (1 x nbr_rcds)/k, and the second job would process from record (((1 x nbr_rcds)/k) + 1) to (2 x nbr_rcds)/k (and so on), where nbr_rcds is the number of active records in the physical file. In an example with a 300-record file and three jobs (k = 3), the records processed by each job would be:
Job 1: From record 1 to record (1 x 300)/3
(From record 1 to record 100)
Job 2: From record (((1 x 300)/3) + 1) to record (2 x 300)/3
(From record 101 to record 200)
Job 3: From record (((2 x 300)/3) + 1) to record (3 x 300)/3
(From record (201 to record 300)

No comments:

Post a Comment