The Calibre ebook manager has a recipe system to generate ebooks from websites/news feeds in order to read them offline.
The following recipe creates an ebook from all StackExchange news feeds:
from calibre.web.feeds.recipes import BasicNewsRecipe import feedparser def get_feeds(url): feed = feedparser.parse(url) entries =  entries.extend( feed["items"] ) sorted_entries = sorted(entries, key=lambda entry: entry["title"]) feeds =  for entry in sorted_entries: feeds.append( "%s/feeds" % entry["id"] ) return feeds class StackExchange(BasicNewsRecipe): title = u'StackExchange recent questions' oldest_article = 10 auto_cleanup = True language = 'en' __author__ = 'Raphaël Pinson' max_articles_per_feed = 100 # SE sites limit the amount of requests per second, per IP delay = 0.05 cover_url = u'http://i1.sndcdn.com/avatars-000011139586-8ft0oi-crop.jpg?04ad178' all_feeds = u'http://stackexchange.com/feeds/sites' feeds = get_feeds(all_feeds) # Use this instead if you only want a few sites instead of all #feeds = [ u'http://tex.stackexchange.com/feeds', # u'http://french.stackexchange.com/feeds', # u'http://christianity.stackexchange.com/feeds', # u'http://stackoverflow.com/feeds' #] def print_version(self, url): return url + '&template=printart'
Example usage, having saved the file as
$ ebook-convert "stackexchange.recipe" stackexchange.epub
You can also add it in the Calibre interface by clicking the "Fetch news" dropdown menu, choosing "Add a custom source" ⇒ "Switch to Advanced mode" and pasting the code.
The recipe can then be used by simply clicking the "Fetch news" button in the interface.
With these default parameters, the generation time can be very long. In order to reduce it (and the size of the generated ebook, you can do the following —independently of each other—:
oldest_articleto a lower value (e.g.
max_articles_per_feedto a lower value (e.g.
- Specify feeds manually instead of using all of them (see commented section);
- Try reducing the delay (to e.g.
0.01). You might get "Too Many Requests for this IP" errors if you lower it too much.