A big question I’m asked on a regular basis is if the WP-Drudge theme can use RSS feeds to display content for a news aggregator in WordPress. Right out the the box, it can and does using the built-in RSS Feed widget (more options than the core widget). This takes an RSS feed and displays the headlines just like the rest of your headline links on the site.
Some people want to take this a step further or combine these links with their own manually curated ones. Also, for folks using the interruption page (a page that site visitors get to first before seeing the link posted), this widget skips that and goes straight to the article. In order to have RSS feed links act like a regular Posted Link, we need to create posts out of these items.
To make that happen, there are a number of plugins you can use. I took some time recently to evaluate the main ones and found one that works well with the theme (after installing a simple and free add-on I created), the WP RSS Multi Importer. This plugin accepts feeds, pulls in items, and created blog posts simply and effectively. I have a few qualms over the code quality I saw but, overall, it does what it says it should and includes many different options to control how it happens. The other plugins out there are either not free to create posts or create enormous overhead and performance problems.
There are 2 basic problems with importing and aggregating content via feeds. If you understand what you are getting into (or don’t care), you’re welcome to skip this section.
First, depending on the number of feeds you’re processing, you can create a serious performance issue for your site. This plugin (and other RSS importers) run using something called “wp-cron.” This is an automated process that’s set in mention at regular intervals and started by a site visit. If you have, say, 500 feeds coming in then this process will take a while to run and cause serious slow-downs for a handful of your visitors.
Second, unless you make an effort to point out your sources and link out (which the settings covered below do), then you’re stealing content for your own purposes. Please follow the guidelines below and make sure your feed items are worked properly as from another source. If you’re only posting aggregated content and not making it clear that the content came from another source, then you could easily be excluded from major search engines and denied advertising opportunities. Use this tool at your own risk and do the right thing.
Ok, rant over. Let’s get this thing configured.
We’re actually going to be installing 3 plugins: the WordPress repo plugin mentioned above, another repo plugin to help us keep an eye on our wp-cron processes, and a helper plugin I wrote to make it play nicely with this theme.
Important note: Before you start adding content and working with the RSS Multi-Importer plugin, you will definitely want a viable backup for your database and media content. It’s easy to mis-configure this system and you don’t want to lose any of the work you’ve already put into your site.
Now that we’ve got all the plugins installed and activated, let’s move on to configuring your news aggregator in WordPress.
The Multi Importer plugin has many different options to set so we’ll start by going to wp-admin > Multi Importer > Global Settings. Make the following changes:
The other settings are not too critical but look them over to see if they apply to you.
Next, we’re going set up categories for the feeds. When you create a feed source (RSS link), this plugin will ask you to assign it to a feed category. These categories will be mapped to post categories in the next section of this tutorial. Confused yet?
Basically, each feed can be added to a specific category, then multiple feed categories can be added to post categories (the ones that control the widgets displaying headlines on your site). The easiest way to handle this is just to create a feed category for each of your post categories and map them 1-to-1 (covered below). You could also create just a handful of feed categories and map each one to multiple post categories. Or you can create several different feed categories and map several of them to a single post category.
To clarify the process even further:
OK, I think that’s about as clear as I can make it (feel free to post questions in the comments section below). So we want to set up some feed categories before we configure the post creation and start adding feeds.
Now we have categories that we’ll be adding our feeds to when we’re ready for that. Next step is to activate and configure the Auto Post capability.
This settings page is the longest by far and there are a lot of important options that I want to cover. If I skip an option below, just leave it as the default.
Once these settings are complete, you’re ready to start adding feeds to the system.
If your site is live and posts are set to auto-publish, this step will cause auto-created posts to begin flowing into the system. If you’re not ready for this to happen and want to just see RSS items coming in without showing, uncheck the “Check to Activate this Feature” setting on the Auto Post page first.
That’s it! Just rinse and repeat for all the feeds you want to add and your news aggregator in WordPress is ready.
One thing to note: the feed import process and the feed items to posts process are two different system actions. Once the feed items have been imported, they need to be converted to blog posts separately (still automatic). This can be done manually using the WP Crontrol plugin we added earlier.
The WP Crontrol plugin keeps an eye on automated processes in WordPress and allows you to run them manually if you’d like to check that the process is working. If you’re not seeing posts being created in the system, this is a good place to start troubleshooting.
Go to wp-admin > Tools > Crontrol. In the list that appears, you’re looking for cron processes that start with “wp_rss_multi_event” to identify the automation put in place by this plugin. The process that imports feed items into your site is “wp_rss_multi_event_importfeeds” and the one that turns those feeds into posts is “wp_rss_multi_event_feedtopost.”
The “Next Run” column tells you when that process is scheduled to run next and the “Recurrence” column shows you how often that process is scheduled to run. The latter of the two values were set up in the steps above so you don’t want to alter those here, make sure to do that in the plugin settings. If you just added a feed or two and want that content to show up immediately, click Run Now for “wp_rss_multi_event_importfeeds” first, then “wp_rss_multi_event_feedtopost” to create the latest posts.
It’s important to know that wp-cron relies on two things to run:
If you’re having problems getting these processes to run automatically, there might be another problem preventing these actions from occurring. Google is your friend here, though the fixes might be technical and require either a developer or your hosting company to assist.
If everything is running as expected, we’re ready to see the posts that are coming in.
We want to make sure that posts are coming in as expected so go to wp-admin > Posts > All Posts to see the list. This will include all posts: imported ones and manually created ones, both with links and without.
You’ll see the Outbound Link in the last column and the Featured Image, if one was imported, as well. You should also see the category it was assigned to (hidden on the screenshot above). This category is what will determine what widget it will appear in.
Last step: set up the site to display this imported content on your news aggregator in WordPress.
The RSS Multi-Importer comes with a widget that can be used to display feed items but it will not use the WP-Drudge settings you might currently have setup. In order to do that, you the WP-Drudge Posted Link widget to display posts in the categories they are being added to. Detailed steps to do that are here.
With your feeds loaded to the right category, auto post set up and running, and your Posted Link widgets in place, you should now see imported posts appearing on the site with the same behavior as the rest of the content. You’re still able to create your own posts, both with and without links, and add them to the same categories or different ones. This allows you to combine automatic aggregation with manual curation for increased victory and success.
If you have any questions about integrating this with WP-Drudge or what’s possible using these two together, ask in the comments below, I’m happy to help.