Zendesk plugins
Introduction
The Zendesk plugins can be used to read one or more Zendesk objects from your Zendesk account. This can allow you to get deeper insights into your customer success by enriching Zendesk data with other data sources.
Use case(s)
User Storie(s)
As a pipeline developer, I would like to fetch a single object from Zendesk, so that I can get deeper insights into my customer success stories.
As a pipeline developer, I would like to fetch multiple objects from Zendesk, so that I do not have to create multiple source nodes merely to replicate all objects to a destination.
As a pipeline developer, I would like to fetch all objects apart from a blacklist from Zendesk, so that I can skip a few objects, but don't have to specify all the remaining ones explicitly.
Plugin Type
Configurables
This integration should be built using two plugins:
Zendesk single object
Used when you want to pull only a single object from Zendesk
Section | User Facing Name | Type | Description | Optional | Constraints |
|---|---|---|---|---|---|
Basic | Admin Email | text | Zendesk's admin email. | No | Should validate email. |
| API Token | text | Zendesk's API token. Can be obtained from the Zendesk Support Admin interface. Check out Zendesk's documentation for API Token generation. | No |
|
| Subdomains | csv | List of Zendesk's Subdomains to read object from. | No | Can have multiple values. |
| Object to Pull | select | Select an object to pull from Zendesk: Comments, Groups, Organizations, Satisfaction Ratings, Tags, Ticket Fields, Ticket Metrics, Ticket Metrics Events, Tickets, Users. | No | Defaults to all. |
|
|
|
|
|
|
| Start Date | textbox | (required by some Zendesk's API Endpoints) | No | Depends on list of the selected objects. |
Zendesk multi-object
Used when you want to pull multiple objects from Zendesk
Section | User Facing Name | Type | Description | Optional | Constraints |
|---|---|---|---|---|---|
Basic | Admin Email | text | Zendesk's admin email. | No | Should validate email. |
| API Token | text | Zendesk's API token. Can be obtained from the Zendesk Support Admin interface. Check out Zendesk's documentation for API Token generation. | No |
|
| Subdomains | csv | List of Zendesk's Subdomains to read objects from. | No | Can have multiple values. |
| Objects to Pull | multi-select | Select a list of objects to pull from Zendesk: Comments, Groups, Organizations, Satisfaction Ratings, Tags, Ticket Fields, Ticket Metrics, Ticket Metrics Events, Tickets, Users | Yes | Defaults to all. |
| Objects to Skip | multi-select | Select a list of objects to skip from Zendesk. All other objects will be pulled. | Yes | Only used when Objects to pull is not specified |
|
|
|
|
|
|
| Start Date | textbox | (required by some Zendesk's API Endpoints) | No | Depends on list of the selected objects. |
The plugin should output one record per row in Zendesk, with an additional field specifying the object where the record came from. Refer to the Salesforce multi-objects plugin for examples
Design / Implementation Tips
v2 is current version of supported Zendesk's API.
Zendesk API has support of Incremental Exports. The API lets export data from specified start_time till end_time.
Plugin will be implemented using Zendesk Java Client by Cloudbees.
Authentication will be performed using Admin Email and API Token.
Output schema will be automatically generated from selected fields.
References
Object Reference
Object Description
Field Name | Type | |||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Comments | ||||||||||||||||||||||||||||||||||||
Article Comments | ||||||||||||||||||||||||||||||||||||
id | number | |||||||||||||||||||||||||||||||||||
url | string | |||||||||||||||||||||||||||||||||||
body | string | |||||||||||||||||||||||||||||||||||
author_id | number | |||||||||||||||||||||||||||||||||||
source_id | number | |||||||||||||||||||||||||||||||||||
source_type | string | |||||||||||||||||||||||||||||||||||
locale | string | |||||||||||||||||||||||||||||||||||
html_url | string | |||||||||||||||||||||||||||||||||||
created_at | datetime | |||||||||||||||||||||||||||||||||||
updated_at | datetime | |||||||||||||||||||||||||||||||||||
vote_sum | number | |||||||||||||||||||||||||||||||||||
vote_count | number | |||||||||||||||||||||||||||||||||||
Post Comments | ||||||||||||||||||||||||||||||||||||
id | number | |||||||||||||||||||||||||||||||||||
url | string | |||||||||||||||||||||||||||||||||||
body | string | |||||||||||||||||||||||||||||||||||
author_id | number | |||||||||||||||||||||||||||||||||||
post_id | number | |||||||||||||||||||||||||||||||||||
official | boolean | |||||||||||||||||||||||||||||||||||
html_url | string | |||||||||||||||||||||||||||||||||||
created_at | datetime | |||||||||||||||||||||||||||||||||||
updated_at | datetime | |||||||||||||||||||||||||||||||||||
vote_sum | number | |||||||||||||||||||||||||||||||||||
vote_count | number | |||||||||||||||||||||||||||||||||||
Ticket Comments | ||||||||||||||||||||||||||||||||||||
id | number | |||||||||||||||||||||||||||||||||||
type | string | |||||||||||||||||||||||||||||||||||
body | string | |||||||||||||||||||||||||||||||||||
html_body | string | |||||||||||||||||||||||||||||||||||
plain_body | string | |||||||||||||||||||||||||||||||||||
public | boolean | |||||||||||||||||||||||||||||||||||
author_id | number | |||||||||||||||||||||||||||||||||||
attachments | list[attachment]
| |||||||||||||||||||||||||||||||||||
Comments
Good point. Yes we should support subdomains.
Can you please check how we handle it in Salesforce and other SaaS plugins?
a. Ok to mention the limitation in the doc. b. Can we not get all the data, in a record? meaning a tag, with its organization, ticket, user, etc
Is there a way to get all?
I think we should use https://developer.zendesk.com/rest_api/docs/support/ticket_metric_events
Can we not get all the data, in a record? meaning a tag, with its organization, ticket, user, etc
Can we not get all the data, in a record? meaning a tag, with its organization, ticket, user, etc
Can we not get all the data, in a record? meaning a tag, with its organization, ticket, user, etc
Can we not get all the data, in a record? meaning a tag, with its organization, ticket, user, etc
Yes, SDK is ok to use
No, please create a developer account at https://developer.zendesk.com to use for testing.
Cool, thanks.
Ok, I'll check it.
b. Yes we can, but we will get 500 most popular tags in the last 60 days.
All of them have API. But in order to get Comments you have to provide id of the object.
For example, Article Comments API looks like this:GET /api/v2/tickets/{ticket_id}/comments.json
In this case we need to get ticket_id or list of ticket_ids from somewhere.
We can add property for it, but it will require Zendesk ids, which is not straightforward to get.Ticket Metrics Events already in list of objects. I'll remove then Ticket Events from the list.
- 9. Yes, we can get object related data only.
I appreciate your help with this.
3.b. That's fine.
4. ticket_id should not be plugin configuration. Asking users to pick ticket ids seems incredibly tedious. Instead, the pattern should be that for a selected user/organization/subdomain, you should pull all ticket_ids, and then comments for each. Is that possible? This is applicable to 6-9 as well, correct? I think optional filters for user or organization is fine. But not for ticket_id. If user or organization is not specified, then pull all in the specified subdomain?
4. Yes, I think it is possible to pull all ticket_ids, and then comments for each if user/organization/subdomain specified. And yes, this is applicable to 6-9 as well.
If user or organization is not specified, then pull all in the specified subdomain?
Yes, this makes sense.
Thank you very much for your help.
2. Unfortunately there is no common solution for Rate Limits in SaaS plugins.
There are few proposals:
Fail pipeline and tell that limit has exceeded
easiest one
not sure that this solution will return any data for users with low limit
Introduce cache to store results that we were able to get before limit exceeded and show warning about limits
there will be some data
need to specify cache size
data integrity is a big question in this case and it's pretty hard to understand the value of such data
Wait until limit is refreshed and try again
will get all data
it is not suitable for all plugins
Zendesk's API sends back information for how long client has to wait before new request
performance of such plugin can be poor in case of low limits
In worst case scenario Zendesk's API has limit 10 requests per minute with 100 documents per request.
Could you please advise which one is better to choose? If you have any ideas please let me know.
Thank you for taking the time to this matter.
Does Zendesk support a bulk API with higher, less restrictive limits?
Zendesk has some sort of bulk API which is called Incremental Exports. It's limit is 10 requests per minute with 1000 documents per request. It is strict for any type of account and can be increased up to 30 requests per minute. Also it doesn't support all objects that were listed.
Ok, then is there a way to (attempt to) switch to the bulk API if we're hitting the API limit, without the user's knowledge? If we hit the rate limit, and bulk API cannot be used, then we can exponentially back-off?
I don't know if its a recommended pattern though. @Albert Shau any thoughts?
Regardless of which api is used, you still need to deal with rate limits. Standard practice is to retry with exponential backoff and some jitter, with some upper bound after which you error out. Of course, the plugin also shouldn't knowingly exceed the rate limit.
I don't see the way to switch between API and bulk API. Bulk API response is time-based while API response is page-based. Also almost all API endpoints don't have sort option. It means that we can't guarantee that results from the first page of API will be included in the first page of bulk API.
Probably we should use bulk API where applicable and standard practice described by @Albert Shau. What do you think about it?
@Bhooshan Mogal could you please do review of the current design version?
Thanks for your attention.
Looks good, but you need to work out the rate limiting strategy with @Albert Shau
From the comment above, I think we're in agreement about exponential backoff
Yes. Thank you for your help.
@Bhooshan Mogal Could you please kindly clarify the following questions:
SubdomainTo request API we need to know Zendesk subdomain. It is possible than one email can have multiple subdomains. For example, account can have multiple Brands and each Brand has it's own subdomain.
Do we need to add support for multiple subdomains?
Rate limits
Zendesk's API has Rate limits for number of requests per minute.
Is there any best practice to handle it?
Tags
Limitation: Lists the 500 most popular tags in the last 60 days, in decreasing popularity.
Do we need to mention it in doc?
It is also possible to get Tags for each ticket, organization, user.
Do we need to add an advanced options in this case? It requires list of ids from Zendesk or some sort of autocomplete widget with requests to Zendesk API.
Comments
There are 4 types of comments available:
Article Comments. Requires article id or user id.
See question about advanced options in Tags.
Post Comments. Requires post id or user id.
See question about advanced options in Tags.
Ticket Comments. Requires ticket id.
See question about advanced options in Tags.
Request Comments. Requires request id.
See question about advanced options in Tags.
Which one to use?
Ticket Events
Unable to find API endpoint for it. There are 2 endpoints:
Ticket Audits endpoint. Audits are a read-only history of all updates to a ticket. Has limitation: Archived tickets are not included in the response.
Ticket Activities endpoint. The Ticket Activities API returns ticket activities by other people affecting the agent making the API request. Has limitation: Lists ticket activities in the last 30 days affecting the agent making the request.
Maybe one of those endpoints can be used?
Ticket Metrics
There is a limitation: Archived tickets are not included in the response.
Do we need to warn about it?
It is possible to get Ticket Metrics for each ticket.
See question about advanced options in Tags.
Ticket Metrics Events
Requires request parameter start_time: The Unix UTC epoch time of the oldest event you're interested in. Example: 1332034771.
Do we need to set any default value and add advanced option for it?
Tickets
It is possible to get Tickets for each organization, user (for user there are 3 possible tickets: requested, ccd, assigned).
Same question about advanced options as for Tags.
Users
It is possible to get Users for each organization, group.
See question about advanced options in Tags.
Java SDK
There is Java SDK. It is third-party SDK by CloudBees with Apache 2.0 Licence. It covers 60-70% of required Zendesk objects and has a bunch of models already mapped on Zendesk API.
Can we use it?
Zendesk account
Is there any Zendesk account that can be used for testing purposes?
Thank you for your attention to this matter.