Google News library for Ruby

I was surprised that Google News has no API, so I built my own: Using HTTParty, I wrote a small Ruby library for Google News. Put this in a ‘googlenews.rb’ file in the /lib/ directory of your Rails application to grab google news stories and topic lists.

Haven’t figured out how to get more than 10 items… Safari seems to be able to with feed://news.google.com?output=rss… but I’m not sure how.

gem "httparty"
require "httparty"

class Googlenews
  include HTTParty
  base_uri 'news.google.com'
  
  def self.items
    options = { :query => { :output => 'rss' } }
    features = self.get('/news', options)
    features['rss']['channel']['item']
  end
  
  def self.extract_topic(item)
    # ncl=duPfj30A5WoxBHMupoOtlciO1jY1M&
    item['description'].match(/ncl=([a-zA-Z0-9]+)&/)[1]
  end

  def self.topic(item)
    options = { :query => { :output => 'rss', :ncl => self.extract_topic(item) } }
    features = self.get('/news/more', options)
    features['rss']['channel']['item']
  end
  

  # store in database (unused)
  # def self.story(item)
    # item['category']
    # item['title']
    # item['guid']
    # item['description']
    # item['link']
    # item['pubDate']

    # t.string :category, :default => ''
    # t.string :title, :default => ''
    # t.string :guid, :default => ''
    # t.text :description, :default => ''
    # t.string :link, :default => ''
  # end

end

Once you’ve dropped this in /lib/, you can use it in a Rails controller like this (I needed JSON, to work with the feed in JavaScript):

class DataController < ApplicationController

  def latest
    render :json => Googlenews.items
  end

end

Update: to get more than 10 items, add a GET parameter ‘num=20’ to the feed URL. This works both for topic and for the main feed. I haven’t confirmed this but I think it’s still limited to 30 items. Again, Safari manages to get more, but perhaps it does repeated queries. Still investigating. Anyways, that makes it:

 def self.items
    options = { :query => { :output => 'rss', :num => '20' } }
    features = self.get('/news', options)
    features['rss']['channel']['item']
  end 

Update: This now has a real home on Github, where it’s called ruby-googlenews.

Old wine jug becomes lamp/biosphere – locals fearful

I decided to rescue an empty bottle of Carlo Rossi and convert it into a self-contained ‘biosphere’, as well as a lamp. I Dremeled through the base of the jar to thread the wire through, and epoxied the whole thing shut. Now stuff is starting to grow!

At the lab, most people have expressed fear and trepidation at what will come of this experiment. The consensus seems to be that it will all die, and moldy putrified goop will become sentient and kill us all. I, on the other hand, continue to be optimistic.

SVG vs. Canvas

Well, SVG doesn’t scale well to large numbers of objects, but Canvas doesn’t scale well to large screens:

Performance with many objects

Here are the results of the first fruitful experiment, which clearly shows that SVG performance degrades quickly (exponentially on Safari?) in the number of objects, but Canvas performance remains at a near-constant low. This makes sense, since Canvas is just a bitmap buffer, while SVG has to maintain additional references to each object that it renders. Also, though not pictured, note that performance in clearing an SVG element also decreases in the number of drawn objects.

Performance on a large rendering context

Via Boris Smus

Serving ‘OSM-JSON’ alongside XML from the OpenStreetMap Rails port

OpenStreetMap.org‘s RESTful API allows anyone to access data on their continually growing collaborative map of the world… in XML. This is great for most applications, but if you’re working in JavaScript (as we are), XML might as well be greek. We need JSON.

To offer OSM-JSON along with of OSM-XML, we added a route to accept a “.format” suffix, and split up the render call based on the params[:format] part of the route:

# /config/routes.rb:46-50

map.connect "api/#{API_VERSION}/geohash/:geohash.:format", :controller => 'api', :action => 'geohash'
map.connect "api/#{API_VERSION}/geohash/:geohash", :controller => 'api', :action => 'geohash'

map.connect "api/#{API_VERSION}/map.:format", :controller => 'api', :action => 'map'
map.connect "api/#{API_VERSION}/map", :controller => 'api', :action => 'map'

Notice we also added a ‘geohash’ route. Whereas the /map call requires a bbox parameter (‘bbox=min_lon,min_lat,max_lon,max_lat’), we can use a geohash (Geohash in JavaScript, Geohash in Rails) which defines a bounding box as a sequence of letters and numbers. This fits Cartagen’s needs well, and since it doesn’t require any parameters, we can page cache it in Rails. (Remember that page caching bypasses Rails entirely, letting Apache handle these cached files at high speed – that saved us when we were on BoingBoing).

Continue reading “Serving ‘OSM-JSON’ alongside XML from the OpenStreetMap Rails port”