Batch Process to Define and Run a Flow

In production mode, you will have scripts that create flows and run them. The console GUI will be one interface you will use to monitor the flows that get run this way. The other interface is through the browser.

In these tutorials, you are using the simple command cp to emulate more complicated programs, which might take much longer to run that a file copy does. This tutorial does not show you long running programs, as you will see in production. This means that during this exercise you do not get to view long lasting states of an intermediate job. The intermediate jobs run too quickly to notice their state changes. For longer lasting flows, the GUI and browser interface provide useful ways to watch the details of what is happening.

  1. The control over what is supposed to happen is in scripts that define the flows and get them running. To demonstrate this, reset FlowTracer as you did earlier, to get rid of the current flow. It is not needed any more. Remove the dependent files and the primary input file too.
    % vovforget -elements System:nodes
    message: Forgotten 10 nodes
    % rm aa bb cc dd1 dd2
    The console should reflect all this and show an empty Set View panel.
  2. Edit the file in the simple_test directory to look like this:
    % cat 
    rm -f aa bb cc dd1 dd2
    vovforget -elements System:nodes
    sleep 5
    vovbuild -f Flow.tcl
    sleep 5
    sleep 10
    touch aa
    sleep 10
    This shell script is going to run the commands you have been entering interactively, with sleep commands in the sequence to emulate the time between entering the commands. This is intended to give you a chance to notice the state change in the console's Set Viewer when the script is run.
  3. Run the shell script while watching the GUI console.
    % sh 
    The script starts with removing the primary and dependent files so it can be run repeatedly.
  4. Run the shell script again, and then again, to get familiar with having flows defined starting from an empty canvas, seeing the flow dependency graph before any jobs of the flow are run, seeing the jobs scheduled to run but not succeed, and then seeing a successful run of all the jobs.
The flow is defined by FDL language statements in a flow description file. The flow is created and run by commands in a batch shell script. You monitor the progress of the entire system by looking at the GUI console.