There are already many Drupal modules to display or import Tweets on your website, but most of them come with limitations: they just display the tweets instead of importing them. This makes it hard to filter, especially when you display a specific hashtag. In some cases, it’s pretty handy if you can quickly unpublish an unwanted tweet. The modules that do import tweets have other limitations. Or they create a custom table that makes it hard to display the tweets using Views. Or they only can pull 1 tag or user, instead of a combination of tags.

Let’s describe a current use case: we needed to display tweets combined with news items and blog posts in one overview, ordered by date. So we needed a way to actually store the tweets so we can filter and sort them. After trying some modules we switched to using Feeds. Using Feeds, we could harvest the clients’ Twitter RSS feed (example) and map the title, body and date field on a custom node type ‘Tweet’. This worked fine, but it had some limitations: only a few fields of the actual tweet are part or the RSS feed. It’s not possible to get information about hashtags, links or media being used in the tweet.

The client came up with the wish to display the photo’s being used in the tweet (Twitpic, yfrog, etc) instead of just showing the t.co url. To get this working, we switched to using the JSON feed (example) instead of the RSS feed. With the JSON feed, you can add a parameter to the URL (include_entities=true) to include all this information (more info). The feed also shows to which tweet this is a reply to, the user information, etc, etc. The problem was; how can we harvest JSON with Feeds?

With the help of the magnificent Feeds JSONPath Parser module it actually turned out to be fairly easy! Here’s how:

Create a node type ‘Tweet’

And add fields you want to harvest. We added (apart from the body and title) a field to store the ID, the Links (multiple Links field) and the Hashtags (multiple Term reference field to a vocabulary). If you need more information, just add extra fields.

Create a Feed Importer

1. Download and enable Feeds & Feeds UI, Feeds JSONPath Parser (including JSONPath php plugin which you have to place in the module directory).
2. Then navigate to /admin/structure/feeds/create to add an importer. I named it Tweets JSON (tweets_json).
3. On the next page, you can edit the Importer settings. On Basic settings, I changed Periodic Import to ‘as often as possible’ so the tweets will be fetched on every cron run. Leave Fetcher as is. On Parser select the JSONPath parser.
4. Go to the Node processor settings where you can map the imported feeds to your Content Type. Leave the other settings as is.
5. Then go to Mapping. Here you have to add a mapping for each field you want to import. I added the following mappings:
   If you want to map other fields as well (author, reply_to, etc) you have to add those fields first to your content type before you can map them here.
6. After adding the mappings, click on JSONPath Parser Settings on the left to describe your mappings. Here’s the tricky part.

Add JSON mappings

I won’t go into details of how the JSON structure is set up, but basically all you need to know is the path to each tweet. My use case only needed to import the tweets of a given user account, but obviously you can use every Twitter API request to make the selection you need.

If you use the Devel module (which you should) then the following snippet will give you the information you need:

$tweets = drupal_http_request('https://api.twitter.com/1/statuses/user_timeline.json?user_id=13994362&include_entities=true');
dpm(drupal_json_decode($tweets->data));

Note: to find the Twitter ID for a username, you can use TwIDder.

Here’s an example of the result for the @drupalcon account (the above command):

For the mapping, we need do understand the ‘tree’ of the data and map this tree on the Mappings interface. As you can see, every tweet is placed directly in the root of the request, so we can simply add $.* in the Context field.

Then, for the other fields, I used the following mappings:

• Title - text
• Body - text
• Guid - id
• Field_tweet_urls:url - entities.urls.*.expanded_url
• Field_tweet_hashtags - entities.hashtags.*.text
• Created - created_at
• Field_tweet_id - id

I used the asterix for the URL and Hastags mappings because they are multiple fields (arrays).

Add the Import URL

Now you have created your Feed Importer you only have to add the Import URL. To do so, navigate to /import/add and enter the URL of the feed you want to import.

That’s all! And if you want to import tweets from several users, you can just add another Feed URL using the same importer :)

UPDATE: I've added a feature that you can use as starting point. It contains the content type and the feed importer. 

Tags: 

Drupal
Drupal 7
Planet Drupal
JSON
Tutorial
Twitter
Feeds

Attachment: 

TwitterImporterFeature.zip