Posted by ark, ,
Since I discovered the joy that is blogs, rss and of course Google Reader I've found one or two feeds that don't quite provide the data I want. Usually they just provide a small summary and a link to the full story, or some crucial part of the post is missing. It started off with woot.com's feed which shows you the woot but not the price (what's the point in that?) so I wrote a small python script that fetched the feed and the price and updated the feed. Then it was b3ta.com's newsletter that had one article per newsletter without the contents of the newsletter, so I changed my script to get that too, and even make a whole bunch of articles with the link be the url that each little snippet points to. Then came snopes.com which was kinda generic, it used to point to a page with the content, so I made it scrape that content and replace the description with the scraped content.

So now I share my script here, hopefull that it'll be useful to someone. I'm not publishing my feeds since my goal isn't to upset the feed publishers/sites. If you make changes or find it useful please let me know.



Posted Saturday 19 November 2005 Share