Google Cloud PubSub

The Google Cloud PubSub connector allows connection to Google Cloud Pub/Sub’s message bus on a real-time streaming basis. Specifically, the connector allows Panopticon Designer (Desktop) to subscribe to XML, JSON, TEXT or FIX based messages that are published on particular topics. The data format itself is arbitrary, and as a consequence, the connection includes the message definition.

Using Google Cloud PubSub

1. When creating a new data table, select Google Cloud PubSub from the Connect to data dialog. The Google Cloud Pub Sub Connection dialog displays.

   

2. Enter the Service Account Credential JSON Text with the generated JSON key (contains the private key) in the following format:

NOTE: Ensure that when parameterizing the values in the Credential JSON Text, there is no white space as a single line content.

1. Click Fetch to populate the Topic drop-down list. Initially, the first topic in the list is displayed in the Topic drop-down box.

Select a topic.

4. Click Fetch to populate the Subscription Name drop-down list and select a subscription name.

You can also opt to create a subscription by manually entering the value into the Subscription Name list box.

NOTE:

• A subscription name will be automatically generated when it is not entered or selected in the drop-down list.

This subscription will be created for connection and will be deleted as soon as its work is done. For example, when starting a presentation mode, a subscription will be created. Upon quitting the presentation mode, the subscription will then be deleted.

• Pub/Sub can automatically delete inactive subscriptions. This can be done by configuring the minimum required time of inactivity to schedule a subscription for deletion. This time must be longer than the message retention duration.

5. Select any from the following Message Types:

6. • Fix

   • JSON

   • Text

   1. If Text is selected, confirm the Decimal Separator, Text Qualifier, Column Delimiter, and if the first row of the message include column headings.

1. • XML

6. Select either the dot (.) or comma (,) as the Decimal Separator.

7. The schema of the source file is then defined by clicking the Generate Columns… button.

This populates the list of columns, with the data type found from inspecting the first ‘n’ rows of the files.

8. Click Add Column to add columns to the MQ connection that represent sections of the XML/JSON/FIX message.

9. Enter or select the following:

1. • Name

1. • XPath/JsonPath/Fix Tag (for example, price/@stock)

1. • Type (Numeric, Text or Date/Time)

1. • Date/Time format when the Data Type is Time

NOTE:

To parse and format times with higher than millisecond precision, the format string needs to end with a period followed by sequence of lower case Fs. There can be no additional characters following them.

For example: yyyy-MM-dd HH:mm:ss.ffffff

1. • Enabled (determines whether the message field should be processed)

   • Filter (defined parameters that can be used as Filter)

NOTE: The Filter option is only available for JSON, Text and XML Message types

Fix

JSON

Text

XML

You can also opt to click to delete a column in the list.

10. You can opt to save the columns definition into an EXS format by clicking the Save button. The saved EXS files are placed in …\Public\Public Documents\Datawatch Desktop\Column Schemas.

11. Click Load to load the column schema.

12. Expand Real-Time Settings.

Refer to Defining Real-Time Settings for more information.

13. Click OK to confirm the selection and retrieve the record set into Panopticon Designer (Desktop).

Initially, there is no data displayed in the Data Source Preview section of the Edit Data Table window.

Click the Start Preview button to display the data and refresh the values depending on the defined Refresh Period.