LSF Emulation

This document is intended for those who have been using the Platform LSF batch system and are now moving to using Accelerator Plus on top of the LSF batch system.

The following information describes the available resources, specific scripts, and guidelines of using those resources.

The Altair Accelerator installation includes scripts that are designed to minimize the impact of transitioning workloads from using LSF directly and instead using the Accelerator Plus hierarchical scheduler. Ideally, the use of these scripts, will enable the redirection of workloads from using an LSF cluster directly to one using an Accelerator Plus scheduler in front of the same LSF cluster. While the scripts cover the common usage of LSF, higher performance and a richer set of scheduling directives will be achieved with using the native Accelerator Plus command set.

The emulated commands include the following:
  • bhist
  • bhosts
  • bjobs
  • bkill
  • bmgroup
  • bpeek
  • bqueues
  • bstat
  • bsub
  • lshosts
  • lsid

The scripts that emulate the above commands are available in the directory $VOVDIR/scripts/lsfemulation, which is not in the PATH in the default setup. This setup avoids collision with the platform scripts and commands.

The emulated commands are added to the path by adding the Altair Accelerator named environment LSFEMUL. The LSFEMUL environment setup is installed in $VOVDIR/etc/environments/LSFEMUL.*.
% ves +LSFEMUL
% bsub sleep 10
% bjobs -a

Configure Resource Mapping

The resources used by LSF, expressed by the -R option in bsub, are significantly different from the resource maps in Accelerator.

To map resources from one system to the other, customize the file $VOVDIR/local/lsfemulation/config.bsub.tcl

This file contains a set of assignments to the Tcl arrays MAP_LSF2NC() and MAP_RUSAGE().
Note: This file is used only by the Altair Accelerator bsub emulator.

An example for config.bsub.tcl can be found in the usual location for configuration files, a subdirectory of $VOVDIR/etc/config:

File: $VOVDIR/etc/config/lsfemulation/config.bsub.tcl

#
# Sample configuration of the bsub emulation.
#
# This file must be placed in $VOVDIR/local/lsfemulation/config.bsub.tcl
#

# If you want support for exclusive access to machines (option -x)
# you need to:
# 1. Uncomment the line below 'set bsubopt(percent) 1'
# 2. Make sure all taskers offer the resource PERCENT/100
# 3. Make sure all jobs request at least PERCENT/1 (see vnc_policy.tcl)
set bsubctrl(percent) 1

# Set this to 1 to cause bsub to always send email
set bsubctrl(alwaysmail) 1

# Truncate emailed log files after this many bytes
# negative values (e.g. -1) mean mail the whole file (BEWARE)
# zero means accept the default (65536 bytes)
set bsubctrl(logmax) 0

# Emulation transforms -m hostnames into an OR expression,
# which can slow down the NC scheduler if too complex.  
# Hosts after this count are silently dropped to avoid slow scheduling
set bsubctrl(moptmax) 6

# Map LSF 'select' resources into NC resources.
# select[rhel4]  -> "linux"
set MAP_LSF2NC(rhel3) "linux"
set MAP_LSF2NC(rhel4) "linux"
set MAP_LSF2NC(rhel5) "linux"
set MAP_LSF2NC(RH4_64) "linux x86_64"

# Map LSF 'rusage' resources into NC resources,
# typically resources of type License:
# Example: 
#  rusage[dc=1]   -> "License:Design-Compiler"

set MAP_RUSAGE(dc)   "License:Design-Compiler"
set MAP_RUSAGE(pt)   "License:PrimeTime"
set MAP_RUSAGE(drc)  "License:lic_drc"
set MAP_RUSAGE(lvs)  "License:lic_lvs"
set MAP_RUSAGE(erc)  "License:lic_erc"

Emulate the LSF Report in the Output Log

Some legacy scripts expect some LSF specific lines in the log file of a job. This can be achieved with a post-command that adds those lines to the log. An example of such command is post_job_report.csh.

#!/bin/csh -f
# -*- Tcl -*- \
   exec vovsh -f $0 $*:q

set usage "
Description:
  post_job_report.csh    

  Used for some jobs submitted with the bsub emulator:

Example:
% ves +LSFEMUL
% bsub -Ep $VOVDIR/etc/post/post_job_report.csh -J test.lsf_with_jobreport cal 2015
"

if { $argv == {} } {
    VovPrintUsage $usage
}

source $env(VOVDIR)/tcl/vtcl/vovlsfemulib.tcl

set jobId       [lindex $argv 0]
set logFileName [lsfEmuGetJobLogFileName $jobId]
set report      [lsfEmuFmtJobReport $jobId]

if { $logFileName ne "" } {
    VovMessage "Adding job report to $logFileName"
    set fp [open $logFileName "a"]
    puts $fp [lsfEmuFmtJobReport $jobId]
    close $fp
} else {
    VovMessage "No log file found for job $jobId\n$report"

    set whyOld ""
    set whyNew "Cannot find a log file for this job $jobId"
    if { [catch {set whyOld [vtk_prop_get $jobId WHY]}] } {
        set why $whyNew
    } else {
        set why "$whyOld\n$whyNew\n$report"
    }
    catch {vtk_prop_set $jobId WHY $why}
}

exit 0
The post command can be specified with the option -Ep of the bsub emulator. For example:
% bsub -Ep $VOVDIR/etc/post/post_job_report.csh   [OTHER OPTIONS]
... command

Debug the LSF Emulation Layer Usage

To debug as well as test and verify an LSF emulation script, it can be helpful to view the issued commands and the used options and values.

If the environment is set with variable VOV_LOG_LSFEMUL to the name of a file, all emulation commands will be logged in that file. For example:
% setenv VOV_LOG_LSFEMUL ~/lsfemul.log
% bsub sleep 11
% lsid
% cat $VOV_LOG_LSFEMUL