Configure Google Cloud Platform Cloud Bursting

Create the necessary components in GCP that are required for cloud bursting and then create a Control cloud bursting scenario.

Create a Project

Create a GCP Project so that you can create the necessary cloud components for cloud bursting.

GCP documentation can be found at Creating and Managing Projects.

Google Cloud Platform projects form the basis for creating, enabling, and using all Cloud Platform services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for Cloud Platform resources.

  1. Login to the GCP console.
  2. Click located in the upper left-hand corner of the web page.
  3. Click Home.
  4. Click Create.
  5. For Project Name, enter a name for the project.
    The name can be anything meaningful to your organization, e.g., pc_cloudproject.
  6. Click Create.
    It may take a few moments to create the project.
  7. Using a browser, navigate to the following URL: https://console.developers.google.com/apis/library/compute.googleapis.com?project=PROJECTNAME
    where PROJECTNAME is the name of the project.
  8. Click Enable.

Create a Service Account

Create a Google Cloud Platform service account so that you can later create a cloud account in Control.

You must already have signed up for an GCP account.

GCP documentation can be found at Understanding Service Accounts and Compute Engine IAM Roles .

A service account is a special Google account that belongs to an application or a virtual machine, instead of to an individual end user. Create a service account and give this account administrative permissions for all Compute Engine resources. Once this task is complete, you should have downloaded a JSON file containing the following:
  • Project ID
  • Client ID
  • Client email
  • Private key ID
  • Private key
This information is necessary to create a cloud account in Control.
  1. Login to the GCP console.
  2. Click located in the upper left-hand corner of the web page.
  3. Click IAM & admin > Service accounts.
  4. Click CREATE SERVICE ACCOUNT.
  5. Enter the following to create a service account:
    1. For Service account name, enter a name for the service account.
      The name can be anything meaningful to your organization, e.g., pc-service-account.
    2. Click CREATE.
    3. For the Project role, click Select a role > Compute Engine > Compute Admin.
      This role gives full control of all Compute Engine resources.
    4. Click CONTINUE.
    5. Under Create key (optional), click CREATE KEY.
    6. For Key type, enable JSON.
    7. Click CREATE.
    8. Save the JSON file in a secure location.
      The information in this file is used later to add a cloud account to Control.
    9. Click CLOSE.
    10. Click DONE.
      The service account is created.

Add a GCP Cloud Account

Add an existing GCP service account to Control so that you can create a cloud bursting scenario or an appliance.

You must have created a GCP service account, given it Compute Admin permissions and downloaded a JSON file containing information about the service account. The information in the JSON file is necessary to create a cloud account in Control. If you do not have this information see Create a Service Account.
  1. Login to Control.
  2. Click the Cloud tab.
  3. Click Cloud from the Infrastructure menu located on the left-hand side of the web page.
  4. Click .
  5. Enter the following to add a cloud account:
    1. For Account name, enter any name for the cloud account.
      The name can be anything meaningful to your organization, e.g., gcp_cloudaccount.
    2. For Project ID, copy and paste the project_id value from the JSON file.
    3. For Client ID, copy and paste the client_id value from the JSON file.
    4. For Client Mail, copy and paste the client_email value from the JSON file.
    5. For Private Key ID, copy and paste the private_key_id value from the JSON file.
    6. For Private Key, copy and paste the private_key value from the JSON file.
      Important: Do not copy the \n characters at the end of the private_key value.
  6. Click Create account.

Create a Virtual Private Cloud Network

Create a Virtual Private Network (VPC) in GCP so that you can later create a virtual machine for cloud bursting.

GCP documentation can be found at Virtual Private Cloud Documentation and Using VPC Networks .

  1. Login to the GCP console.
  2. Click located in the upper left-hand corner of the web page.
  3. Click VPC network > VPC networks.
  4. Click CREATE VPC NETWORK.
  5. Enter the following to create a VPC:
    1. For Name, enter a name for the VPC.
      The name can be anything meaningful to your organization, e.g., bursting-vpc.
    2. In the Subnets section, click the Custom tab under Subnet creation mode.
    3. For Name, enter a name for the subnet.
      The name can be anything meaningful to your organization, e.g., bursting-subnet.
    4. For Region, select a Region based on the geographical location of users.
    5. For IP address range, enter an IP address range using CIDR notation
    6. For Private Google access, enable Off.
    7. Click Done.
    8. For Dynamic routing mode, enable Regional.
  6. Click Create.
    Creating the VPC network may take some time. Do not proceed until the VPC is created.
  7. Select the VPC by clicking on its name.
  8. Click the Firewall rules tab.
  9. Click CREATE FIREWALL RULE.
  10. Enter the following to create a firewall rule:
    1. For Name, enter a name for the firewall rule.
      The name can be anything meaningful to your organization, e.g., ssh-all.
    2. For Direction of Traffic, enable Ingress.
    3. For Action on match, enable Allow.
    4. For Targets, select All instances in the network.
    5. For Source filter, select IP ranges.
    6. For Source IP ranges, enter 0.0.0.0/0.
    7. For Protocols and ports, enable Specified protocols and ports.
    8. Enable tcp.
    9. Enter 22.
  11. Click Create.

Creating a Virtual Machine

Create a CentOS 7.3 virtual Machine in GCP.

You must have already created a virtual network with a subnet and added a firewall rule.

GCP documentation can be found at Virtual Machine Instances and Creating and Starting a VM Instance.

  1. Login to the GCP console.
  2. Click located in the upper left-hand corner of the web page.
  3. Click Compute Engine > VM instances.
  4. Click CREATE INSTANCE.
  5. Enter the following to create a virtual machine:
    1. For Name, enter a name for the virtual machine.
      The name can be anything meaningful to your organization, e.g., bursting-vm.
    2. For Zone, select a zone that is in the same Region as the subnet of the previously created VPC.
    3. In the Machine type, click the Customize link.

      Customize the Machine Type
      Figure 1. Customize the Machine Type
    4. Specify the CPUs, GPUs and RAM.
      Consider the nature of the applications that you plan to deploy on the instance, the number of users that you expect to use the applications, and also how you expect the load to scale in the future. Remember to also factor in the CPU and memory resources that are necessary for the operating system.
    5. For Boot disk, click Change.
    6. Choose CentOS 7.
    7. For Boot disk type, choose one of the following options:
      • Standard persistent disk
      • SSD persistent disk

      Choose SSD for I/O-intensive applications, where low latency and high throughput are critical. For testing, consider Standard persistent disk to keep costs down.

    8. For Size, specify the size of the boot disk.
    9. Click Select.
    10. Under Identity and API access, for Service Account, select No service account.
    11. For Firewall, choose Allow HTTP traffic.
    12. Click Management, disks, networking, SSH Keys.
    13. Click the Networking tab.
    14. Click Add network interface.
    15. For Network, choose the VPC you previously created for bursting.
    16. For Network Service Tier, click Standard.
    17. Click Done.
    18. Delete any default network interfaces that might have been automatically generated.
    19. Click the Security tab.
    20. For SSH Keys, copy the SSH public key (i.e., id_rsa.pub) of a user account that exists on your site's PBS Server and paste it.
      This user will have sudo rights and will be able to SSH into the VM to install the PBS MoM.
  6. Click Create.
    Creating the virtual machine may take some time.

Installing and Configuring the PBS MoM on the VM

Install the PBS MoM on the virtual machine and configure the VM per your site's environment.

The user name for logging into the virtual machine is determined by the SSH key that was provided during the creation of the virtual machine.

A startup script is provided during the creation of the bursting scenario. This script is ran after a cloud node is burst and is used to set up the system environment so that nodes burst in the cloud can communicate with the PBS Server. A service specifically designed for cloud instance initialization is cloud-init and is utilized by Control when booting cloud nodes, therefore it must be installed on the VM. For more information about installing cloud-init, refer to the Cloudinit Documentation.

  1. Login to your site's PBS Server as the user account (public SSH key) provided during the creation of the VM.
  2. SSH into the virtual machine using the public IP address of the VM:
    ssh IPV4PublicIP
    where IPV4PublicIP is the public IP address of the virtual machine.

    Bursting Virtual Machine
    Figure 2. Bursting Virtual Machine
  3. Enter the command:
    sudo -i
  4. Copy the PBS Professional installation package to the VM.
  5. Using the PBS Professional Installation and Upgrade Guide, install and configure the PBS Professional MOM.
  6. Configure the VM for your site's environment such as mounting file systems, connecting it to the authentication service, installing any applications, etc.
  7. Install cloud-init.
  8. Connect the VM to your PBS Professional server and submit a job to ensure it is working as expected.

Create an OS Image

Create an image of the virtual machine configured per your site's requirements. You can choose to keep the image and use it to update your bursting image with patches, new applications etc. You can bring the instance online, patch and then recreate the image at your discretion. The saving in build and management time for the images will offset the additional storage costs.

You must have already create a virtual machine, installed the PBS MoM on the VM and configured the VM per your site's environment.

GCP documentation can be found at Creating, Deleting, and Deprecating Custom Images.

  1. Login to the GCP console.
  2. Click located in the upper left-hand corner of the web page.
  3. Click Compute Engine > VM instances.
    A list of virtual machines is displayed.
  4. Click next to the virtual machine that was created for cloud bursting.

    Bursting Virtual Machine
    Figure 3. Bursting Virtual Machine
  5. Click Stop.
    It may take some time for the VM to be stopped. Do not proceed until the VM is stopped.
  6. Click Images from the menu located on the left-hand side of the web page.
    CAUTION:
    Do NOT choose Machine images.

    Images
    Figure 4. Images
  7. Click CREATE IMAGE.
  8. Enter the following to create an image:
    1. For Name, enter a name for the image.
      The name can be anything meaningful to your organization, e.g., bursting-image.
    2. For Source, select Disk.
    3. For Source disk, select the previously created virtual machine.
  9. Click Create.
    It may take some time to create the image. Do not proceed until the image is created.
You can now delete the virtual machine so that you are no longer charged for it.

Create a GCP Cloud Bursting Scenario

Create an GCP bursting scenario so that PBS execution nodes can be burst in the cloud.

Open a browser window and login to your GCP console. Open a second browser window and login to Control.

A startup script is provided during the creation of the bursting scenario. This script is ran after a cloud node is burst and is used to set up the system environment so that nodes burst in the cloud can communicate with the PBS Server. Use a cloud-init script for the startup script. For more information about writing cloud-init scripts, refer to the Cloudinit Documentation and Cloud Bursting Startup Script.

  1. Navigate to the Control browser window.
  2. Click the Cloud tab.
  3. Click Bursting from the Infrastructure menu located on the left-hand side of the web page.
  4. Click Add Bursting Scenario.
  5. Enter information about the bursting scenario such as name, cloud provider, cloud account, and geographic location.
    1. For Name, enter a name for the cloud bursting scenario.
    2. For Description, enter a description.
    3. For Cloud Account, choose a previously created GCP cloud account from the list.
    4. For Region, select the same geographic location as was selected for the previously created bursting VPC.
  6. Click Next.

Enter information about the GCP resources necessary to burst in the cloud.

  1. For Domain name, provide the domain name of your organization for the cloud bursting network.
    altair.com, burst.altair.com
  2. For Hostname prefix, enter a prefix for the names of the burst nodes.
    If "node" is entered , the cloud bursting nodes will be available to PBS Professional as node1, node2, node3, etc.
  3. Navigate to the cloud provider browser window.
  4. Click located in the upper left-hand corner of the web page.
  5. Click VPC network > VPC networks.
  6. Click the name of the VPC that was created for cloud bursting.
    VPC network details are displayed.
  7. Copy the name of the VPC network.
  8. Navigate to the Control browser window.
  9. For Network name, paste the name of the VPC network copied from GCP.
  10. Navigate to the cloud provider browser window.
  11. Copy the name of the VPC network subnet.
  12. Navigate to the Control browser window.
  13. For Subnetwork name, paste the name of the VPC network subnet copied from GCP.
  14. Navigate to the cloud provider browser window.
  15. Click located in the upper left-hand corner of the web page.
  16. Click Compute Engine > Images.
  17. Select the image created for cloud bursting.
  18. Click Equivalent REST
  19. Copy the value for the entry called selfLink.
  20. Navigate to the Control browser window.
  21. For OS Image URI, paste the link copied from GCP.
  22. Enable Add public IP to VMs if you need to connect to the cloud bursting nodes from the internet.
    It is recommended to leave this disabled.
  23. For Cloud-init script, click Browse, browse for the startup script file and select it.
    This script will run after a cloud node is burst.
  24. Add public SSH keys for any users that you want to have access to the burst nodes:
    1. Login to the PBS Professional Server.
    2. Copy the public SSH key for the user.
      Public key files are usually stored in /.ssh in the user's home directory.
    3. Navigate to the Control browser window.
    4. Click Add to add the public key.
    5. For Public SSH keys, paste the public SSH key.
    6. Repeat steps a through e to add additional public SSH keys.
    The public keys are copied to each burst node for secure connectivity.
  25. For Idle time before unbursting, enter the minimum time that a cloud node can be idle before it is unburst.

    It is expressed as an integer whose units are in seconds.

  26. For Tags, enter a key-value pair to add a label to the cloud node and press ENTER.
    A tag is a label that is assigned to a cloud node. Tags enable cloud nodes to be categorized in different ways, for example, by purpose, owner, or environment.
    The key-value pair must be entered as key:value. The key must be less than 36 characters and the value must be less than 43 characters. The length of both the key + value should be less than 80 characters. Valid values are alphanumeric, dash (-) and an underscore ( _ ).
    Once the tag is added, it will appear within its own bubble.

    Key-Value Tag
    Figure 5. Key-Value Tag
  27. Repeat the previous step to add more tags.

    Additional Tags
    Figure 6. Additional Tags
    Tip: To remove a tag click on the 'x' located within the tag bubble.
  28. Click Next.
  29. Select the instance types to enable for the bursting scenario by clicking the Enabled checkbox located to the far right of the instance type name.
    If an attempt is made to burst nodes with a disabled instance type, the node is not burst and the below message is displayed in the PBS Server logs (if bursting through PBS), the Cloud logs and through the Cloud component user interface.
    TYPE is not enabled

    Where TYPE is the instance type used when attempting to burst the node.

    Tip: Deselect an instance type by disabling the Enabled checkbox.
  30. Click Instantiate scenario.
Once the scenario created, it is recommended to Add a Token and Add Quotas and Alerts. When adding a token, an API key is generated that is required for bursting. Quotas define bursting limits such as the maximum number of cloud nodes to burst.