Submit Jobs with CLI Commands

This chapter provides examples of submitting jobs using CLI (command line interface) commands.

Note: CLI commands are case insensitive. For example, timetolerance and timeTolerance represent the same command.

nc run

Note: nc modify -res now support binary unit conversion for all memory based resources as a convenience from Petabytes (PB), TerabyteS (TB), or GigabyteS (GB) to Megabytes (MB), which is still used internally and reported by all commands. The input conversion will accept either decimal or integer form and are all case-insensitive, so for example both nc run -r SWAP/1GB — sleep 0, and nc run -r RAM/0.1Tb — sleep 0 are supported.

The currently supported parameter names for which this conversion is supported are RAM/, RAM#, RAMFREE#, RAMFREE/, RAMTOTAL#, RAMFREE/, SWAP/, SWAP#, SWAPFREE#, SWAPFREE/, SWAPTOTAL#, SWAPTOTAL/ and TMP# or TMP/. By default the unit is MB (Megabytes), where 1MB is 1<<20 bytes.

vnc: Usage Message
      Run one or more jobs. The jobs are added to the system, and
      will remain in the system until you use 'forget' to forget them
      or they are automatically forgotten by the system.
      If taskers and resources are available, the jobs are dispatched
      immediately, else they are queued.
      % nc run [OPTIONS] command ...
      -h               -- Print this message.
      -v  <level>      -- Verbose level from 0 (silent) to 9 (very verbose)
      --               -- Null option. In case of ambiguity, use
      this to separate the options from the command.
      In addition, the value of the environment variable NC_RUN_ARGS is
      prepended to the argument list for this command, while the value of
      NC_RUN_ARGS_AFTER is appended.
      -autokill <time> -- Kill job if it runs longer than specified time.
                          Set it to zero to disable autokill (the default).
      -clearcase       -- This is a job to be run in a ClearCase view
                          (see docs).
      -C <class>       -- The job belongs to the given class.
                          If argument is empty, the option is ignored.
                          Option can be repeated.  The jobclass of the job
                          will be the last one specified.
      -e <env>         -- Set the environment. Default is current env, as
                          defined by the variable VOV_ENV.
                          Setting this to the null string "" or to
                          "SNAPSHOT", forces the use of an environment
      -ep              -- Capture environment in a SNAPSHOT property. Uses
                          SNAPPROP environment.
      -first           -- Schedule job first in its bucket.
      -forceterm       -- In the case of interactive jobs where the output is
                          piped, the job's TERM environment variable is set
                          to 'network'.  This option disables that behavior.
      -g <group>       -- Specify the fairshare group.
                          The .user:subgroup suffix will be added.
                          You need permission to run in the specified group.
      -G <group.user>  -- Specify the fairshare group complete with the .user:
                          nothing will be added or changed. The group must be
                          well formed and include a .<user> component.
                          You need permission to run in the specified group.
      -I, -Ir          -- Run interactive job. TTY signals like <ctrl>C are
                          propagated to the job.  If the environment variable
                          VOV_INTERACTIVE_PING is set, its value (TIMESPEC
                          format. minimum is 1m) will be used to keep the
                          connection with the interactive job alive by pinging
                          the job at the specified interval.
      -Il              -- Run interactive job. TTY signals like <ctrl>C are
                          kept local, not propagated to the job. Appropriate
                          for piping stdout to a file or command.  See above
                          for usage of VOV_INTERACTIVE_PING.
      -Ix              -- Run X Window based interactive job, no tty, no wait.
                          Adds env D(DISPLAY=...) so job displays on submission
                          display.  See above for usage of
      -jobproj <name>  -- The job project is set to <name>. The default is
                          determined by the environment variables VOV_JOBPROJ,
                          LM_PROJECT and RLM_PROJECT.
      -jpp  <JPP>      -- Specify a job-placement-policy. Legal values for JPP
                          are a comma separated list of one or more words from
                          the following list:
                          At most one of these:
                             fastest   -- Dispatch to the fastest tasker
                             slowest   -- Dispatch to the slowest tasker.
                             first     -- Dispatch to the first tasker found.
                             largest   -- Dispatch to tasker with largest value
                                          of '(MB of unused RAM) +
                                          16000*(Number of unused cores)'
                             smallest  -- Dispatch to tasker with smallest value
                                          of '(MB of unused RAM) +
                                          16000*(Number of unused cores)'
                             smallram  -- Dispatch to tasker with smallest RAM.
                          At most one of these (Linux-only):
                             pack      -- NUMA control: assign the job to a
                                          NUMA node with the least number of
                                          available resources that will fit the
                                          job. If none of the NUMA nodes have
                                          sufficient job slots and RAM, the job
                                          will be allowed to run on as many
                                          NUMA nodes as needed to satisfy its
                                          resource requirement.
                             spread    -- NUMA control: assign the job to a
                                          NUMA node that has the largest number
                                          of available resources.
                             none      -- NUMA control: allow Linux to place
                                          jobs. The Linux CPUs Allowed affinity
                                          list will be all the CPUs on the
                                          system (default).
                          -jpp slowest
                          -jpp spread
                          -jpp smallest,pack
                          -jpp first,spread
                          Note: To place jobs on the same machines,
                                use first or smallest.
      -N <jobname>     -- Same as -J <jobname>.
                          (compatible with FDL 'N' procedure)
      -J <jobname>     -- Set the job name (same as -N <jobname>)
      -keep            -- Keep job after completion, disabling auto-forget.
      -keepfor <time>  -- Keep job after completion for specified time.
                          Disables auto-forget. Requires the NC administrator
                          to copy $VOVDIR/etc/liveness/live_keepfor_jobs.tcl
                          into the $SWD/tasks directory.
      -limit <spec>    -- Add limit to jobs submitted with this option. <spec>
                          could be just a number, or a name followed by a
                          number. Throttles the running jobs of a user
                          submitted with the same -limit option to the
                          specified number.
      -L <exitstatus>  -- Legal exit status list (default is 0). You can also
                          use commas to separate the valid statuses.
                          -L 0,2,10        -L 0,200-208
      -maxresched <N>  -- Maximum number of times the job can be rescheduled.
                          Must be >= 1 and <= 10 (default 10). Implemented
                          via the MAX_RESCHEDULE property on the job.
      -p <priorities>  -- Set priorities for scheduling and execution of job.
                          Format is <schedulingPriority>[.executionPriority]
                          Priority is either a number from 1 (low) to 15 (top)
                          or a symbolic value 'low' 'normal' 'high' 'top' or
                          any abbreviation thereof.
                          Examples:  -p n   -p 4.high   -p high.low
      -pre <SCRIPT>    -- Execute <SCRIPT> as precondition. The JOBID will be
      -precmd <SCRIPT>    appended to the arguments of the script. If the
                          script exits with non-zero status, the job is not
                          run. See examples in $VOVDIR/etc/pre.
      -post <SCRIPT>   -- Execute <SCRIPT> as postcondition.  The JOBID and
      -postcmd <SCRIPT>   EXITSTATUS will be appended to the arguments of the
                          script.  The script is executed irrespective of the
                          success of the job.  The exit status of the script
                          becomes the exit status of the job.
                          See examples in $VOVDIR/etc/post.
      -preemptable <N> -- Set preemptable mode:
                          N=0    not preemptable
                          N=1    preemption allowed (default)
      -profile            Activate job profiling, to track the history of RAM,
                          CPU, License usage over time.  Without this option,
                          only the current usage is computed.
      -r <r1> [r2..rN] -- Set requested resources of the job.  Accepts
                          multiple resource arguments and may be repeated.
                          If -r is the last option, use '--' to separate
                          the last resource from the command line.
      -r+ <resource>   -- Append one resource to resource list.
                          No -- termination necessary.
      -r- <resource>   -- Remove one resource from the resource list.
                          It is an error to remove a resource
                          that does not exist.
                          No termination necessary.
      -rf                 Add Filer:<FILER_NAME> resource
                          (computed from run dir)
      -rundir <dir>    -- Specify a different run directory (default ".")
                          If the <dir> specification is quoted by single
                          quotes, the directory is taken exactly as given,
                          instead of being canonicalized. When using -rundir
                          with the SNAPSHOT environment, the -ep argument
                          must also be passed. Implies -D.
      -set <setName>   -- Assign the job(s) to the given set.
      -sg <subgroup>   -- Specify a subgroup for fairshare for the current user
      -splitstderr     -- Write the stderr output of the interactive job to
                          stderr. Default is to write the job's stderr output
                          to stdout. Note that using this option will probably
                          result in garbled terminal output due to interleaving
                          of stdout and stderr outputs.
      -fstokens <N>    -- Multiply weight of this job in FairShare by N.
                          Default 1, range is [0..50000]
      -tool <toolName> -- Specify a "toolName" different from the tail of the
                          first command line argument.  The argument must be
                          less than 100 characters long and contain only
                          alphanumeric chars.
      -x | -xdur <xdur> -- Set the expected duration of the job.
      -after <time>    -- Fire job after specified time.
      -array <n>       -- Submit a jobarray of 'n' repeated commands
                          Some fields may contain the strings @INDEX@, @JOBID@,
                          and @ARRAYID@, which are substituted when the array
                          is created.
                          These fields are: command, env, wd, toolname, jobname
                          The output files are also subject to the same
                          Three comma-separated formats for <n> are supported:
      -at <date>       -- Specify earliest date to fire job
                          The date is parsed by the Tcl command
                          [clock scan $date]
      -dp N            -- Run a Distributed Parallel (DP) job requiring N
      -dpactive <n>    -- The n-th component is the one that becomes active
      (default 1).
      -dpres RESLIST   -- Specification of the resources required by a parallel
                          job.  Example: -dpres "RAM/200 CORES/2"
                          See vovcreatepartialjobs for more info.
      -dpres+ <rsrc>   -- Append one resource to the distributed processing
                          resource list.
                          No -- termination necessary.
      -dpres1 RESLIST
      -dpres2 RESLIST
      -dpres<n> RESLIST -- Specify resources for a component <n> of a DP job.
                           The number <n> is in the range from 1 to <N>
                           (option -dp)
                           Example:  -dpres1 "RAM/200"
                                     -dpres2 "CORES/4 RAM/10"
      -dpwait TIMESPEC -- The time the components wait to rendezvous
                          (default 30s, minimum 3s). The wait is increased with
                          each attempt.  The maximum wait is controlled by the
                          property DP_WAIT_MAX
      -dpnocohortwait  -- Partial jobs may exit without waiting for master.
      -dpinitialport N -- Specifiy starting port for partial jobs to 
                          communicate on
      -D               -- Do not check the validity of the directory.
      -f <file>        -- Get a list of commands from a file, one per line.
                          Jobs are created and then scheduled in blocks
                          of 200 jobs (unless otherwise specified by -fb).
      -fb <n>          -- Change the size of blocks of jobs scheduled with -f
                          (default 200).
      -fw  <S>         -- Specify delay between blocks of jobs, in seconds.
                          Value must be >= 0, default is 0.  Use with -f.
      -dribble         -- Short hand for -fb 1 -fw 0.1
      -F               -- Force running of job even if it is already valid.
                          This is useful only if you are also using option -l
                          to set the name of the log file, otherwise this
                          option has no effect.
      -atomic          -- Create jobarray using a single RPC between 
                          client and server.
      -dep <Id|Name>   -- Specify a dependency on the list of jobs.
      -d   <Id|Name>      The argument can be a list of job Ids or job names.
                          In the case of job names, the dependency is looked
                          for in the set of jobs belonging to the submitting
                          The current job will not start until the
                          specified jobs have completed successfully.
                          May be repeated.
                          Performance note: dependencies on job names are much
                          slower than dependencies on job ids.
      -depset <Name>   -- Specify a dependency on all jobs in the named set at
                          the time of submission. If other jobs are added to
                          the set later, they will not be added to the
                          dependencies. May be repeated.
      -forcelog        -- Force the declared output log to be the output of
      -force              this job. If another job was declaring the same
                          output, it will become black (SLEEPING).
      -i <in_file>     -- Specify an input dependency.
      -l <logfile>     -- Specify name of logfile.
                          As with -rundir, if the <logfile> is quoted with
                          either "  or ', then the name is taken literally
                          and not canonicalized.
                          Quoted or not, variable substitution on the file name
                          is performed for the following variables
                              @JOBID@    -> Id of job.
                              @ARRAYID@  -> Id of job array (if applicable).
                              @DATE@     -> ISO_TIMESTAMP
                              @UNIQUE@   -> %Y%m%d_%H%M%S.SUBMISSION_PID
                              @JOBCLASS@ -> job class (the alphanumeric part)
                              @JOBNAME@  -> job name  (the alphanumeric part)
                          You may need to use -forcelog together with -l.
                          Timestamp in format '%Y%m%d_%H%M%S.SUBMISSION_PID'
                          will be added to the logfile name for array jobs
                          when '@UNIQUE@' is not present in the logfile name.
      -n               -- Use no wrapper (default: use 'vw').
      -nolog           -- Do not keep a log.
      -o <out_file>    -- Specify an output dependency.
      -P <NAME=VALUE>  -- Add the given property to the jobs (may be repeated).
      -s               -- Declare that the logfile is SHARED (see docs).
                          You rarely need this option. If misused, this option
                          causes extra buckets to be created in the scheduler.
                          Probably you need '-forcelog -F' instead.
      -uniqueid        -- Force NC to use a unique new
                          VovId for each job submission,
                          even when the same job is submitted multiple times.
      -wrapper <W>     -- Use specified wrapper '<W>' (default: use 'vw').
      -m               -- Send me mail upon job completion.
      -M <mail rule>   -- Send mail according to the given rule (see docs).
      -w               -- Wait for the job(s) to finish: do not show any log.
                          For the meaning of the exit status,
                          please check nc wait.
      -wl              -- Wait for the job(s) to finish: show the log of
                          the last job.
                          For the meaning of the exit status,
                          please check nc wait.
      -nodb            -- The job is not stored in the jobs log or in the
      -nopolicy        -- For ADMIN only.  Disable the policy layer.
      % nc run sleep 10
      % nc run -autokill 30m   sleep 10000000
      % nc run -array 10 sleep 1          # submit 10 sleep jobs via
      % nc run -array 10,200,10 sleep 1   # submit sleep jobs with index
      % nc run -g /teams/chipA  -sg session12  sleep 1
      % nc run -G /teams/chipA.any   sleep 1
      % nc run -C longjobs sleep 10000
      % nc run -C longjobs -r+ RAM/200   sleep 10000
      % nc run -r unix -- sleep 10
      % nc run -p high sleep 10
      % nc run -e BASE -p h sleep 10
      % nc run -e SNAPSHOT+SIM -p h sleep 10
      % nc run -m sleep 10;               # email when job finishes.
      % nc run -M ":ERROR" sleep 10;    # email only if
      % nc run -dp 3 -dpres sun7,linux vovparallel clone sleep 10
      % nc run -at 6pm sleep 10
      % nc run -at "tomorrow 6pm" sleep 10
      % nc run -after 10m sleep 10
      % nc run -forcelog -F -l mylog.txt ./myjob

Default Output of nc run

The default output from nc run includes the following information:
  • The resource list assigned to the job, which can be controlled with the option -r.
  • The environment used for the job, which can be controlled with the option -e.
  • The command line.
  • The log file used to store both stderr and stdout of the command, which can be controlled with the option -l
  • The JobId assigned by Accelerator to this job. JobIDs are used as handles with many of the Accelerator commands.