Zendesk Multi Objects Batch Source
Plugin version: 1.2.0
This source reads multiple objects from Zendesk. It extracts reportable data from the multiple Zendesk objects. The Zendesk Multi Object Batch Source plugin enables bulk data extraction from Zendesk. You can configure and execute bulk data transfers from Zendesk without any coding
Examples of objects are Article Comments, Post Comments, Requests Comments, Ticket Comments, Groups, Organizations, Satisfaction Ratings, Tags, Ticket Fields, Ticket Metrics, Ticket Metric Events, Tickets, Users. Tags object lists the 500 most popular tags in the last 60 days.
The data which should be read is specified using object list and filters for those objects.
In addition, for each Object that will be read, this plugin will set pipeline arguments where the key is 'multisink.[ObjectName]' and the value is the schema of the Object.
Configuration
Property | Macro Enabled? | Description |
---|---|---|
Reference Name | No | Required. Name used to uniquely identify this source for lineage, annotating metadata. |
Admin Email | Yes | Required. Zendesk admin email. |
API Token | Yes | Required. Zendesk API token. Can be obtained from the Zendesk Support Admin interface. For API Token generation, see the Zendesk documentation. |
Subdomains | Yes | Required. List of Zendesk Subdomains to read object from. |
Object to Pull | Yes | Optional. Objects to pull from Zendesk API. If not specified, data is pulled from all objects. Default is blank. |
Object to Skip | No | Optional. Objects to skip from Zendesk API. Default is blank. |
Start Date | Yes | Optional. Filter data to include only records that have Zendesk modified date is greater than or equal to the specified date. The date must be provided in the date format: Format: Date, time, and time zone offset Format Syntax: YYYY-MM-DDThh:mm:ss+hh:mm Format Syntax: YYYY-MM-DDThh:mm:ss-hh:mm Format Syntax: YYYY-MM-DDThh:mm:ssZ Start Date is required for batch objects like: Ticket Comments, Organizations, Ticket Metric Events, Tickets, Users. |
End Date | Yes | Optional. Filter data to include only records that have Zendesk modified date is less than the specified date. The date must be provided in the date format: Format: Date, time, and time zone offset Format Syntax: YYYY-MM-DDThh:mm:ss+hh:mm Format Syntax: YYYY-MM-DDThh:mm:ss-hh:mm Format Syntax: YYYY-MM-DDThh:mm:ssZ If you enter an End Date and Start Date, the data is modified within a specific time window. If no value is provided, no upper bound is applied. |
Satisfaction Ratings Score | Yes | Filter Satisfaction Ratings object to include only records that have a Zendesk score equal to the specified score. Only applicable for the Satisfaction Rating object. Default is blank. |
Table Field Name | Yes | Optional. The name of the field that holds the table name. Must not be the name of any table column that will be read. Default is `tablename`. |
Max Retry Count | Yes | Required. Maximum number of retry attempts. Default is 20. |
Connect Timeout | Yes | Required. Maximum time in seconds connection initialization can take. Default is 300. |
Read Timeout | Yes | Required. Maximum time in seconds fetching data from the server can take. Default is 300. |
Use connection (on/off toggle) | No | Whether to use an existing connection. |
Connections (browse connections) | Yes | Choose the existing connection to use. |
Data Type Mapping
Zendesk Data Types | Cloud Data Fusion Schema Data Type |
Boolean | Boolean |
DateTime/Time | String |
Decimal | String |
Int16/int32/int64/long | Long |
String | String |
Array | Array |
Record | Record |
Limitations
The Zendesk Multi Batch plugin supports two types of pagination: offset and time-based pagination. Offset and Time-Based Pagination might result in data duplication.
Data Duplication issues with Zendesk APIs
The Zendesk Multi Batch Source plugin might return duplicate records for offset and time-based exports as mentioned in Zendesk docs. Zendesk documentation
Objects that support offset and time-based exports: Users, Tickets, Ticket Metric Events, Organizations, and Ticket Comments.
To solve the duplication issue, add the Deduplicate plugin from the Analytics list after the Zendesk Multi Batch Source in the pipeline.
Supported Zendesk Objects
Objects Name | Supported Pagination Type | Endpoint URI (https://{subDomain}.zendesk.com/api/v2/) |
---|---|---|
Users | Offset, Time-based, cursor | incremental/users.json |
Tickets | Offset, Time-based, cursor | incremental/tickets.json |
Ticket Metric Events | Offset, Time-based | incremental/ticket_metric_events.json |
Ticket Metrics | Offset | ticket_metrics.json |
Ticket Fields | Offset | ticket_fields.json |
Tags | Offset | tags.json |
Satisfaction Ratings | Offset | satisfaction_ratings.json |
Organizations | Offset, Time-based | incremental/organizations.json |
Groups | Offset | groups.json |
Ticket Comments | Offset, Time-based | incremental/ticket_events.json?include=comment_events |
Request Comments | Offset | requests/{requestId}/comments.json |
Post Comments | Offset | community/users/{userId}/comments.json |
Article Comments | Offset | help_center/users/{userId}/comments.json |
Example
Sample data in Object: Groups
Name | ID | Description |
---|---|---|
Jon Doe | 1 | Test Description |
Raj | 2 | Test Description 2 |
Sample data in Object: Tags
Name | Count |
---|---|
Jon Doe | 5 |
Raj | 6 |
To read data from these two objects, both of them must be entered in the Tables Names field under the Table specification section.
The output of the source will be the following records:
Name | ID | Description | tablename |
---|---|---|---|
Jon Doe | 1 | Test Description | Groups
|
Raj | 2 | Test Description 2 | Groups
|
Name | Price | tablename |
Jon Doe | 5 | Tags
|
Raj | 6 | Tags
|
The plugin will emit two pipeline arguments to provide multi sink plugin with the schema of the output records:
{
"type": "record",
"name": "groups",
"fields": [
{"name": "name","type": ["string","null"]},
{"name": "id","type": ["long","null"]},
{"name": "description","type": ["string","null"]}
]
}
{
"type": "record",
"name": "tags",
"fields": [
{"name": "name","type": ["string", "null"]},
{"name": "count","type": ["long","null"]}
]
}
Created in 2020 by Google Inc.