Zendesk Multi Objects Batch Source

Plugin version: 1.2.0

This source reads multiple objects from Zendesk. It extracts reportable data from the multiple Zendesk objects. The Zendesk Multi Object Batch Source plugin enables bulk data extraction from Zendesk. You can configure and execute bulk data transfers from Zendesk without any coding

Examples of objects are Article Comments, Post Comments, Requests Comments, Ticket Comments, Groups, Organizations, Satisfaction Ratings, Tags, Ticket Fields, Ticket Metrics, Ticket Metric Events, Tickets, Users. Tags object lists the 500 most popular tags in the last 60 days.

The data which should be read is specified using object list and filters for those objects.

In addition, for each Object that will be read, this plugin will set pipeline arguments where the key is 'multisink.[ObjectName]' and the value is the schema of the Object.

Configuration

Property

Macro Enabled?

Description

Property

Macro Enabled?

Description

Reference Name

No

Required. Name used to uniquely identify this source for lineage, annotating metadata.

Admin Email

Yes

Required. Zendesk admin email.

API Token

Yes

Required. Zendesk API token. Can be obtained from the Zendesk Support Admin interface. For API Token generation, see the Zendesk documentation.

Subdomains

Yes

Required. List of Zendesk Subdomains to read object from.

Object to Pull

Yes

Optional. Objects to pull from Zendesk API. If not specified, data is pulled from all objects.

Default is blank.

Object to Skip

No

Optional. Objects to skip from Zendesk API. Default is blank.

Start Date

Yes

Optional.  Filter data to include only records that have Zendesk modified date is greater than or equal to the specified date. The date must be provided in the date format:

Format: Date, time, and time zone offset

Format Syntax: YYYY-MM-DDThh:mm:ss+hh:mm
Example: 1999-01-01T23:01:01+01:00

Format Syntax: YYYY-MM-DDThh:mm:ss-hh:mm
Example: 1999-01-01T23:01:01-08:00

Format Syntax: YYYY-MM-DDThh:mm:ssZ
Example: 1999-01-01T23:01:01Z

Start Date is required for batch objects like: Ticket Comments, Organizations, Ticket Metric Events, Tickets, Users.

End Date

Yes

Optional. Filter data to include only records that have Zendesk modified date is less than the specified date. The date must be provided in the date format:

Format: Date, time, and time zone offset

Format Syntax: YYYY-MM-DDThh:mm:ss+hh:mm
Example: 1999-01-01T23:01:01+01:00

Format Syntax: YYYY-MM-DDThh:mm:ss-hh:mm
Example: 1999-01-01T23:01:01-08:00

Format Syntax: YYYY-MM-DDThh:mm:ssZ
Example: 1999-01-01T23:01:01Z

If you enter an End Date and Start Date, the data is modified within a specific time window. If no value is provided, no upper bound is applied.

Satisfaction Ratings Score

Yes

Filter Satisfaction Ratings object to include only records that have a Zendesk score equal to the specified score. Only applicable for the Satisfaction Rating object.

Default is blank.

Table Field Name

Yes

Optional. The name of the field that holds the table name. Must not be the name of any table column that will be read. 

Default is `tablename`.

Max Retry Count

Yes

Required. Maximum number of retry attempts.

Default is 20.

Connect Timeout

Yes

Required. Maximum time in seconds connection initialization can take.

Default is 300.

Read Timeout

Yes

Required. Maximum time in seconds fetching data from the server can take.

Default is 300.

Use connection (on/off toggle)

No

Whether to use an existing connection.

Connections (browse connections)

Yes

Choose the existing connection to use.

Data Type Mapping

Zendesk Data Types

Cloud Data Fusion Schema Data Type

Boolean

Boolean

DateTime/Time

String

Decimal

String

Int16/int32/int64/long

Long

String

String

Array

Array

Record

Record

Limitations

The Zendesk Multi Batch plugin supports two types of pagination: offset and time-based pagination. Offset and Time-Based Pagination might result in data duplication.

Data Duplication issues with Zendesk APIs

The Zendesk Multi Batch Source plugin might return duplicate records for offset and time-based exports as mentioned in Zendesk docs. Zendesk documentation

Objects that support offset and time-based exports: Users, Tickets, Ticket Metric Events, Organizations, and Ticket Comments.

To solve the duplication issue, add the Deduplicate plugin from the Analytics list after the Zendesk Multi Batch Source in the pipeline.

Supported Zendesk Objects

Objects Name

Supported Pagination Type

Endpoint URI (https://{subDomain}.zendesk.com/api/v2/)

Objects Name

Supported Pagination Type

Endpoint URI (https://{subDomain}.zendesk.com/api/v2/)

Users

Offset, Time-based, cursor

incremental/users.json

Tickets

Offset, Time-based, cursor

incremental/tickets.json

Ticket Metric Events

Offset, Time-based

incremental/ticket_metric_events.json

Ticket Metrics

Offset

ticket_metrics.json

Ticket Fields

Offset

ticket_fields.json

Tags

Offset

tags.json

Satisfaction Ratings

Offset

satisfaction_ratings.json

Organizations

Offset, Time-based

incremental/organizations.json

Groups

Offset

groups.json

Ticket Comments

Offset, Time-based

incremental/ticket_events.json?include=comment_events

Request Comments

Offset

requests/{requestId}/comments.json

Post Comments

Offset

community/users/{userId}/comments.json

Article Comments

Offset

help_center/users/{userId}/comments.json

Example

Sample data in Object: Groups

Name

ID

Description

Name

ID

Description

Jon Doe

1

Test Description

Raj

2

Test Description 2

Sample data in Object: Tags

Name

Count

Name

Count

Jon Doe

5

Raj

6

To read data from these two objects, both of them must be entered in the Tables Names field under the Table specification section.

The output of the source will be the following records:

Name

ID

Description

tablename

Name

ID

Description

tablename

Jon Doe

1

Test Description

Groups

 

Raj

2

Test Description 2

Groups

 

 

 

 

 

 

Name

Price

tablename

Jon Doe

5

Tags

 

Raj

6

Tags

 

 

The plugin will emit two pipeline arguments to provide multi sink plugin with the schema of the output records:

 

{

 "type": "record",

 "name": "groups",

 "fields": [

   {"name": "name","type": ["string","null"]},

   {"name": "id","type": ["long","null"]},

   {"name": "description","type": ["string","null"]}

 ]

}

 

{

 "type": "record",

 "name": "tags",

 "fields": [

   {"name": "name","type": ["string", "null"]}, 

   {"name": "count","type": ["long","null"]}

 ]

}

Created in 2020 by Google Inc.