Overbooking in Allocator

The Overbooking function (Controller) is vtk_flexlm_overbook for vovresourced and LA::AddResource in Allocator. These procedures have a number of tuning parameters for overbooking:

-thresh real
The threshold at which overbooking becomes active. For example if the value is set to 0.8 it means that when the job count reaches 80% of the allowed total, overbooking starts increasing the number of submitted jobs. The default value for the parameter is 0.9.
-factor real
This number is used to scale the Measured Error into the additional number of jobs to submit. Common values are 0.8. The default value for this parameter is 1.0.
-maxfactor real
This number represents the maximum overbooking multiplicative growth factor on top of the existing licenses that is desired. For example if 10 licenses are available and we want to use overbooking to grow them to 100, the maxfactor should be set to 9.0 (10 + 9.0x10 = 100) The default value for this parameter is 2.0.
-headroom int
When the license in use counts is greater than the maximum number of licenses less the headroom, the overbooking quantity is throttled. Negative headroom values are often used. The default value of headroom is 0.
-queued int
When Allocator sees this number of licenses queued (ie in vendor-queuing) then the overbooking quantity is throttled. Generally this should be a small number. The default value is 1.
-lowpass int
To smooth the effect of overbooking, the actual correction is low-pass filtered. This option controls the delay of the filter. The larger the number, the longer the delay and the smoother the correction. The default value is 8.
-enable BOOL
Simple way to enable/disable an overbooking rule
-verbose BOOL
Increase verbosity
-v
Same as -verbose 1
To activate overbooking, use either the browser interface for a resource or call:
vtk_mq_set_resource_field License:abcd  overbook 1

Please note that in Allocator, activating overbooking implies disabling handle/job matching.

Similarly, to disable overbooking, you can use:
vtk_mq_set_resource_field License:abcd  overbook 0
vtk_mq_set_resource_field License:abcd  match    1

To configure the overbooking parameters (see License Overbooking for details) you also need to use the same vtk interface.

Example:
vtk_mq_set_resource_field License:abcd  overbook.thresh      0.9
vtk_mq_set_resource_field License:abcd  overbook.boostfactor 1.0
vtk_mq_set_resource_field License:abcd  overbook.maxfactor   2.0
vtk_mq_set_resource_field License:abcd  overbook.headroom    0
vtk_mq_set_resource_field License:abcd  overbook.vqthresh    1
vtk_mq_set_resource_field License:abcd  overbook.lpf.n       8; # Length of low-pass filter.

In Allocator, these settings have immediate effect. You can store them in your vovlad/config.tcl file.

Tuning Overbooking

he optimal values for the overbooking parameters are workload specific. While the default values work in many cases, it is worthwhile to review the overbooking operation. Use the -verbose 1 option to monitor the behavior of the overbooking routines in the resource daemon log file. The goal should be to increase the license utilization to near 100% but without pushing too many jobs into vendor queueing. Vendor queuing implies that a hardware slot is taken but the job is stalled waiting for that license; normally a few slots being idle for a few seconds is a reasonable trade-off for high license utilization. However administrators should also be aware that having too many jobs in vendor queueing can cause the license daemons to stall, dramatically reducing overall throughput.

If your workload ends up pushing too many jobs into vendor queuing before backing off, even with a small value of -queued then you should consider reducing factor from 1.0 to say 0.9 (or lower) and increasing headroom from 0 to perhaps 2-5% of the maximum license quantity.

If your workload struggles to get any vendor queuing, it may be because the threshold is not reached or maintained - particularly when the new total (maximum number of jobs) has been increased. Reducing the threshold to 0.8 or 0.7 will enable overbooking to continue to be active. For a large number of short duration jobs, there can be many jobs "in-flight": jobs for which license matching by Accelerator or Monitor has not yet occurred. In these cases it may make sense to increase -factor N to say 1.2 and have a negative value of headroom equal to say 2-5% of the total license count.

While overbooking exhibits the self compensation characteristics expected from a control loop, tuning the parameters is often worthwhile for optimal usage.

Default values were used in the example above, which can be set to any other value using these commands.