• Contents
  • Index

Altair® Panopticon™

 

Livy Spark

Livy is an open source REST interface for interacting with Apache Stark. It supports executing snippets of code or programs such as Scala, Python, Java, and R in a Spark context that runs locally or in Apache Hadoop YARN.

The Livy Spark connector allows you to run these codes and fetch the data in Panopticon Designer (Desktop).

Using Livy Spark

  1. Launch the Connect to Data dialog and then select LivySpark.

The Livy Spark Connection dialog displays.

  1. Provide the Host URL (e.g., http://<IP or name of the machine>:<port number>), User ID, and Password required to connect to the Livy Server.

  2. Select the Kind of connection you will use:

    • Spark

Interactive Scala Spark session

    • Pyspark

Interactive Python Spark session

    • Sparkr

Interactive R Spark session

  1. Set the Request Timeout. 30 is the default.

  2. Set the limit of the Polling Count. 150 is the default.

  3. Set the Polling Frequency. 2 is the default.

  4. Enter or select Browse to select the script that you will use.

  1. Click Test Connection. A confirmation dialog displays to show the connection was successfully established.

  2. Click OK to execute the script and retrieve the records set into Panopticon Designer (Desktop).

The flat record set corresponding to the executed script is returned from the source database and displayed in the Edit Data Table view with the fields listed in the Data Source preview.