Twitter

The Qlik Twitter Connector loads data from the Twitter API into your QlikView or Qlik Sense application, allowing you to extract data from Twitter, such as tweets based on a search term or hashtag, and information about users.

You can also use the data in conjunction with other Qlik Web Connectors such as Sentiment140.

Tip:

The Qlik Web Connectors help you connect to different data sources and fetch data in the same way. Learn how to authenticate a data source connection and how to use tables to fetch data.

Connecting to data sources

Ways to access your data

To fully use the connector and all of its tables, you need to have a Twitter account with which to authenticate the connector.

You can click Test Connection to verify that the connection will work with the authentication token you entered. You must also name your connection before you can select data.

Note: You must ensure that your system's clock is accurately set. Deviations can cause an error when authenticating with the Twitter API.

Tables

After authenticating the connector, tables like the following are available.

Table Description
Search (Simple) Returns Tweets based on search term and other parameters.
Search (All Columns) Returns Tweets based on search term and other parameters. Returns more columns than Search (Simple).
UserSearch

Returns information about public Twitter accounts based on a specified topic.

RateLimit Returns the current rate limit.
TweetLookup Returns a tweet for a specific tweet ID.
UserLookUp Returns user information for specified Twitter accounts.
UserLookupByld Returns user information for a specific user ID.
Mentions Returns up to 800 tweets containing the authenticated user's Twitter screen name.
RetweetsOfMe Returns most recent retweets made on the authenticated user's tweets
Followers Returns a list of followers of a specified Twitter account.
FollowerIds Returns a collection of follower IDs of a specified Twitter account.
FriendIds Returns a collection of Twitter account IDs that a specified Twitter account is following.

Following

Returns a list of Twitter accounts that a specified Twitter account is following.
UserTimeline Returns up to 3,200 tweets from a specified Twitter account.
HomeTimeline Returns up to 800 recent tweets of the authenticated user's Twitter account.
DirectMessages Returns direct messages sent and received within the past 30 days by the authenticated Twitter account.
Favorites Returns up to 200 favorited tweets from the authenticated user's Twitter account.
LocationWithTrends Returns a list of locations for trending topics.
TrendsNearLocation Returns a list of trends near a specified location.
TrendsForPlace Returns the top 10 trending topics for a specified location.

Understanding the Twitter search API

It is important to understand the following points about the Twitter Search REST API, which Qlik Web Connectors uses in the background.

The search API is limited to making a maximum of 180 API calls per 15 minutes per authentication token. Each of these API calls will return a maximum of 100 results.

The Search API is not a complete index of all tweets, but instead an index of recent tweets. At the moment that index includes between six to nine days of tweets.

In the background, the Qlik Web Connectors Twitter Connector Search table will make the number of requests specified in the Max Number Of Search Pages parameter, 15 by default, until it reaches the limit of the search index, runs out of search API calls, currently 180 per 15 minute window, or runs out of search results, that is, it has retrieved all the tweets matching the search criteria.

For example, if you have 12 search terms that are each generating 1500 tweets per 15 minutes, Qlik Web Connectors Twitter Connector is be able to gather these tweets. The theoretical maximum would be to retrieve 180 x 100 = 18,000 tweets per 15 minutes or 72,000 tweets per hour.

You can make use of the Since ID parameter to tell Qlik Web Connectors the id of the most recent tweet you have retrieved, and saved into a QVD file or other data store, for a given search query to ensure that the minimum number of API calls necessary are made.

Search API query operators

This table lists the official query operators from Twitter.

Operator Finds tweets...
watching now containing both “watching” and “now”. This is the default operator.
“happy hour” containing the exact phrase “happy hour”.
love OR hate containing either “love” or “hate” (or both).
beer -root containing “beer” but not “root”.
#haiku containing the hashtag “haiku”.
from:alexiskold sent from person “alexiskold”.
to:techcrunch

sent to person “techcrunch”.

@mashable referencing person “mashable”.
superhero since:2010-12-27 containing “superhero” and sent since date “2010-12-27” (year-month-day).
ftw until:2010-12-27 containing “ftw” and sent before the date “2010-12-27”.
movie -scary :) containing “movie”, but not “scary”, and with a positive attitude.
flight :( containing “flight” and with a negative attitude.
traffic ? containing “traffic” and asking a question.
hilarious filter:links containing “hilarious” and linking to URL.
news source:twitterfeed containing “news” and entered via TwitterFeed
Note: You can add RT, for example QlikView RT, to search for retweets only, or -RT, for example QlikView -RT, to exclude retweets.

Twitter Action URLs (Web Intents)

It is possible to use Twitter URLs to respond to tweets or follow users. These can be constructed in your QlikView or Qlik Sense application, which, when clicking, launch Twitter on the appropriate page. The following table shows the URL, with the token to replace, for the popular actions:

Action URL
Reply to a tweet https://twitter.com/intent/tweet?in_reply_to=[TWEET_ID]
Retweet a tweet https://twitter.com/intent/retweet?tweet_id=[TWEET_ID]
Favorite a tweet https://twitter.com/intent/favorite?tweet_id=[TWEET_ID]
Short user profile, with option to Follow https://twitter.com/intent/user?screen_name=[SCREEN_NAME]

For more information, see:

Troubleshooting

You receive an error message that you have reached the API rate limit

Possible cause  

You have exceeded the API limits that are imposed on the Qlik Web Connectors by the Twitter Search API.

Proposed action  

To reduce the impact of reaching the API rate limits, develop your app with the following in mind:

  • Extract only the data you need.
  • Reload one Twitter-based application at a time.
  • Ensure that loops in your script that make API calls will not result in infinite loops.

You can also use the RateLimit table to show you the calls remaining for each resource and the time that they reset.

You receive error code 130 - Twitter is over capacity

Possible cause  

Twitter is temporarily over capacity.

Proposed action  

Wait a short while and then try again.

Did this information help you?

Can you tell us why it did not help you and how we can improve it?