Creating Data Extract from JSON
The JSON connector allows the retrieval and processing of JSON files, either from a disk, a Text, or from a defined URL.
Steps:
1. On the New Data Extract page, select JSON in the Connector drop-down list.
2. Select the JSON file source:
· File
You can either:
¨ Upload a data source snapshot by clicking
Upload File then Browse
to
browse to the file source.
After selecting the file, it is displayed with the timestamp of the snapshot.
To
change the data source, click then
Browse
to browse to a new version of
the file.
¨ Link to a data source file by clicking
Link to File and entering a JSON File Path.
Ensure that in a cluster, you need to use a a shared path, or put it on every node and use a path that resolves on every node. You can update its contents whenever you want.
· Text
Then enter the text block to be parsed.
· Web URL
The dialog changes slightly to allow specification of the following:
Property |
Description |
Path |
The absolute path including the http where the JSON file is located. |
Proxy Server URI |
The HTTP Proxy setting that will allow the JSON connector to reach the endpoint |
Headers |
· Headers are separated by a comma · Each Header is entered as Name = Value, where Name and Value can be enclosed in double quotes to allow inclusion of any character except for double quotes · Name and Value can also be left unquoted, in which case they may not include comma or equals characters |
Content Encoding |
Select the Content Encoding with the HTTP Header: None, GZip, Deflate, or GZip and Deflate |
User Id |
The user Id that will be used to connect to the JSON service. |
Password |
The password to connect to the JSON service. |
Http Method |
Select the HTTP Method to map any of the following operations to HTTP requests · GET – retrieve information · POST – create or update an entity · PUT – replace an existing entity · DELETE – remove a request |
Timeout |
The length of time to wait for the server response (10 to 300). Default is 10. |
Request Body |
The Request Body for HTTP POST. |
Content Type |
The required Content Type. Default is application/x-www-form-urlencoded |
3. Enter the Record Path (e.g., myroot.items.item).
4. Select either the dot (.) or comma (,) as the Decimal Separator.
5. Click to the fetch the schema based
on the connection details. Consequently, the list of columns with
the data type found from inspecting the first ‘n’ rows of the input
data source is populated and the Save button is enabled.
6. You can also opt to load or save a copy of the column definition.
7. Click to add columns and specify their properties:
Property |
Description |
Name |
The column name of the source schema. |
Json Path |
The Json Path of the source schema. |
Type |
The data type of the column. Can be a Text, Numeric, or Time |
Date Format |
The format when the data type is Time. |
Enabled |
Determines whether the message field should be processed. |
To
delete a column, check its or all the column entries, check
the topmost
,
then click
.
8. Click to save and display the details
of the data extract.
9. Click then
to display the data preview.