Parse the infile for HPC params, jobs, and batches
Parse the file looking for the following conditions
lines ending in `\` wait nextnode
Batch commands in groups of $self->cpus_per_task, or smaller as wait and nextnode indicate
Check all jobs to make sure they have commands
Iterate through all lines in the job file 1. Sanity check - can't use nohup or push commands to background 2. Check for HPC meta - #HPC 3. Check for Note meta
Append to the command
We check for a few cases
1. A line that is terminated by the usual newline character
2. A multiline command in the usual bash sense
echo "goodbye!" && \ echo "not again!"
3. The command is wait. Submit jobs we already have to the scheduler, and any jobs after 'wait', depend upon jobs before 'wait' finishing.
4. Deprecated! The command is 'newnode' on a line by itself. Submit all the previous jobs, but no dependenciies. Instead please use '#HPC commands_per_node' within your job file.
#HPC jobname=job01 #HPC commands_per_node=1 #HPC cpus_per_task=1 gzip VERY_LARGE_FILE gzip OTHER_VERY_LARGE_FILE
Do some sanity checks. So far we only check for nohup, because nohup confuses schedulers.
#TODO Add check for when line ends with &. This also confuses schedulers
Check for lines starting with #TASK - used to pass per process task_tags
allow for changing parameters mid through the script
#Job1 echo "this is job one" && \ bin/dostuff bblahblahblah
echo "This is my new job with new HPC params!"
Make sure our hpc variables are current for filling in the template #HPC cpus_per_task=1 to #SBATCH --cpus-per-task=1