The general principle behind batch processing is automating repetitive tasks. Single tasks are known as jobs, while a set of jobs is known as a batch. This distinction is mostly academic, since the terms job and batch job are now mostly synonymous, but here we'll use the terms separately.
There are three basic steps in a batch or job-oriented workflow:
- 1.Copy input data from archival storage to scratch space
- 2.Run computational tasks over the input data
- 3.Copy output to archival storage
Depending on the scale and characteristics of a particular job, different jobs may require different combinations of computational resources. Garnering these resources is a combination of:
Last modified 4yr ago