Plugin version: 2.10.1

The source is used whenever you want to sample tweets from Twitter in real-time using Spark Streaming. For example, you may want to read tweets and store them in a table where they can be accessed by your data scientists to perform experiments.

Samples tweets in real-time through Spark streaming. Output records will have this schema:

field name	type

field name	type
id	long
message	string
lang	nullable string
time	nullable long
favCount	int
rtCount	int
source	nullable string
geoLat	nullable double
geoLong	nullable double
isRetweet	boolean

Configuration

See the Twitter OAuth documentation for more information on obtaining your access token and access token secret. The consumer key and secret are specific to your Twitter app. Login, view your apps, and then click on the relevant app to find the consumer key and secret.

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Reference Name	No	Required. This will be used to uniquely identify this source for lineage, annotating metadata, etc.
Access Token	Yes	Required. Twitter Access Token.
Access Token Secret	Yes	Required. Twitter Access Token Secret.
Consumer Key	Yes	Required. Twitter Consumer Key.
Consumer Secret	Yes	Required. Twitter Consumer Secret.

Example

Property	Value

Property	Value
Reference Name	`tweet`
Access Token	`GetAccessTokenFromTwitter`
Access Token Secret	`GetAccessTokenSecretFromTwitter`
Consumer Key	`GetConsumerKeyFromTwitter`
Consumer Secret	`GetConsumerSecretFromTwitter`

CDAP Documentation

Twitter Tweet Stream Source

Configuration

Example