Of Blogs, Rails and APIs

This blog (the one you’re reading now) is a custom Ruby on Rails application, built because I either have an insatiable desire to learn new things or a chronic case of Not Invented Here syndrome – take your pick. I recently added support for the MetaWeblog API – with the intention of being able to blog from Flock, a newish browser that has a spiffy, built-in blogging system.

(Alright, I’ll come clean: the main reason is to get the benefits of a fancy WYISKSMWYG HTML editor.) Although… the results don’t work quite the way I wanted (Flock’s fault, I assure you – it works great in MarsEdit, and I’m posting this using BlogJet (which also has bugs, but don’t let that distract you!) The point is: It works well enough for me.

Wait, even that’s not entirely fair. After a dreadful experience using PHP as a SOAP client, I was just telling someone a week or three ago that Web Services in Rails were really, really easy, based on my 50 second skimming of the appropriately titled chapter in AWDWR (I have the 1st Ed - for all I know, the 2nd Ed is a different beast). After saying that, and Googling a bit for supporting evidence, I realized that there wasn’t a lot out there in the way of HOW-TOs and Tutorials for this kind of thing. And, actually, that chapter in AWDWR wasn’t all that helpful.

Finally, a disclaimer: In addition to AWDWR, what follows owes quite a lot to Typo. In all fairness, Typo did it first, and in almost all cases, Typo does it better. That said, except for this post, I haven’t found a very good explanation for how Typo does it, and remember I have that NIH syndrome? Right.

So, here goes.

A Quick Overview

In general, the idea with Web Services is that you’re exposing part of your app to some other app. So, if you have some function “do_cool_stuff(key)” and you want SomeOtherSite.com to be able to use it. If you’re writing a search engine or product database, you might want to let people style or customize the results of a search. If you’re writing a blog engine, you might want to let people post to your blog using third-party apps that do cool WYSIKSMWYG things.

On the technical side, Web Services are all about XML: Your app will receive a POSTed chunk of XML and translate that into magical programming goodness and reply with XML. Per my hypothetical example above,  SomeOtherSite.com might send something like:

<method>
  <name>do_cool_stuff</name>
  <params>
    <param1>
      <name>key</name>
      <value>my_key_value</value>
    </param1>
  </params>
</method>

But you can’t really use just any XML you make up off the top of your head. There are two dominant standards: SOAP and XML-RPC. XML-RPC (spec) seems the simplest to me, it’s actually pretty close to the above. SOAP (spec) is heavier-duty stuff, with namespaces and other features that I’m sure someone will tell me are absolutely vital, but, for the purposes of this blog API business, XML-RPC is all you really need to know.

Getting Grittier

Now we just need an API.

In the big-picture sense: an API is a standard, structured definition of what your Web Service does. Flickr has an API, Amazon has an API, Google has several - and they are each unique to themselves.

The goal here is using a third-party app for posting to a blog. There are already several APIs available that cover blogging – and that’s good, because the odds are low that any given third-party app will be able to use any random API we dream up (yay, standards!)

I took a look at what Flock would support and whittled it down from there, going with  metaWeblog in the end. Blogger is also popular, and there’s also LiveJournal and MovableType has some additions (see here for a thorough, if dated, comparison).

One Leeeeetle Snag

So, I spent a few hours putting together a basic Web Service, but I’m not going to show it to you, because it didn’t work. Well, actually, it worked fine, but nothing else could work with it, which was kinda the whole point, and it had to be re-written largely from scratch.

What happened?

Well, this is one of those things that AWDWR doesn’t explain terribly well. Here goes:

  • When a XML request comes in to your Rails app, Rails uses something called “dispatching” to figure out which Controller and method should handle the request. In the case of WebServices, the simplest (and most limited) “dispatching style” is called Direct Dispatching, which ties all requests on a given Controller (essentially one URL) to a single API. This is normally not a problem, except…
  • The metaWeblog API (at least as implemented by Flock and MarsEdit and every other app I tried) isn’t just one API, it’s actually two APIs: metaWeblog extends the earlier Blogger API with new features. But, rather than replace/rename the earlier, Blogger methods, metaWeblog subsumes them.
  • That is, when you tell your 3rd-party app that your blog API uses the metaWeblog API, the first call it makes is actually a Blogger API call (“blogger.getUsersBlogs” – that method isn’t defined in the metaWeblog spec).
  • This means that you can’t use Direct Dispatching with metaWeblog. Direct Dispatching can’t handle two API calls on the same URL. You need to use Layered Dispatching to accomplish this. That’s okay, it just means this is a slightly more convoluted example.
  • Layered Dispatching allows you to have a single Controller (again, think URL) that can respond to multiple API calls. In this case, we’ll have a single Controller “outside” that can handle both Blogger and metaWeblog service calls.

Alright, Let's Go

First, create a new controller that will act as the entry point for requests:

./script/generate controller outside

When that's done, open up app/controllers/outside_controller.rb and edit it like so:

class OutsideController < ApplicationController
  web_service_dispatching_mode :layered
  web_service_scaffold :invoker
  web_service(:metaWeblog) { MetaWeblogService.new(self) }
  web_service(:blogger) { BloggerService.new(self) }
end

There’s a few odd-ball bits of code here:

  • the first line turns on layered dispatching (direct is the default)
  • the second line sets up a Web Service scaffold - this’ll let us poke around the Web Service in a browser, in case something goes wrong with the 3rd-party app
  • the final two lines configure the dispatcher – method calls that start with “metaWeblog.” will go to MetaWeblogService, while calls that start with “blogger.” will go to BloggerService.
  • Why Service.new(self)? The “services” aren’t technically Controllers, they don’t inherit from Rails’ ActionController, so they don’t get controller-specific methods, like url_for. As you’ll see below, the service will create a reference to the current controller so we retain access to url_for in our service code.

The Blogger Method

It seems I only need one Blogger method to make this work: getUsersBlogs. Per the spec, getUsersBlogs take three parameters (appkey, username and password) and returns an array of structs (with the blogid, blogName and url).

I need three things to make this work:

  • a definition for a struct
  • a definition for a Blogger API that exposes “getUsersBlogs” as a method
  • a method that responds to “getUsersBlogs” with the appropriate struct

So, I’ll call this struct “Blog”:

class Blog < ActionWebService::Struct
    member :blogid, :string
    member :blogName, :string
    member :url, :string
end

That should be pretty straight-forward: Blog has three members, all are strings. Now I need the API:

class BloggerApi < ActionWebService::API::Base
  inflect_names false
  api_method :getUsersBlogs,
    :expects => [ {:appkey => :string}, {:username => :string}, {:password => :string} ],
    :returns => [[Blog]]
end

Theoretically, I could use Rails’ automagical naming to have an API method “get_users_blogs” that maps to “getUsersBlogs”, but I found it frustratingly unreliable in this case. (Whereas, contrary to some, I actually enjoy Rails’ automagical naming elsewhere.) api_method defines and exposes our method, as well as the number and type of parameters we expect, and the kind of response we’ll give. (When I first hit this, I found it a bit odd – Ruby isn’t statically typed, so I’m not used to thinking about the format of my parameters.)

Finally, note the slightly odd typography on the :returns line - this is Rails’ way of saying “I’ll return an array of objects of this kind”. Also, if you already have a Blog object, you may need to explicate the namespacing here (again, Typo serves as an excellent example here, wrapping the above in a module allows you to refer to it in module namespacing style.)

Alright, almost there, we just need the method itself:

class BloggerService < ActionWebService::Base
  web_service_api BloggerApi

  attr_accessor :controller

  def initialize(controller)
    @controller = controller
  end


  def getUsersBlogs(appkey, email, password)
    @author = get_author(email, password)
    [Blog.new(:blogid   => 1, :url => controller.url_for(:author=>@author, :controller=>'blog'), :blogName => @author.short_name.capitalize + "'s Blog")]
  end
 
  def get_author(email, password)
    Author.login(email, password)
  end
end

Lots to go through:

  • web_service_api BloggerApi
    This defines the API that’ll be used for this service – this is the essence of Layered Dispatching in AWS – this calls in the API we defined above.
  • attr_accessor :controller & initialize(controller)
    Remember I said we needed the controller for url_for? When Outside conjures up our BloggerService, we essentially want to capture the controller object for use inside our service object.
  • @author = get_author(email, password)
    In any other Rails controller, I’d probably use a before_filter for this, but I found that AWS puts method parameters in a simple array, which made them considerably more complicated to access than the usual “params[:email]” would have been. In this case, I opted for the (ostensibly) more readable code. YMMV.
  • Speaking of readability, using the Ruby-style quick return with Blog.new is probably not the universally “best” approach, but I only have one blog per user, so this works well enough for me.

If you put all this in a file like “app/apis/blogger_service.rb”, that Rails automagical naming will be able to find it when Outside calls BloggerService.

metaWeblog at last!

Now we come to our first bit of the actual metaWeblog API!

Let’s start with the easiest piece first: getPost. According to the spec, getPost needs three parameters (postid, username and password) and expects a struct (the post in question) in response.

Let’s start with that struct:

class Article < ActionWebService::Struct
  member :description, :string
  member :title, :string
  member :postid, :string
  member :url, :string
  member :link, :string
  member :permaLink, :string
  member :categories, [:string]
  member :dateCreated, :time
end

Most of this should look familiar from before, except:

  • dateCreated is a :time
  • categories is an array of string objects

Also, in my app, posts use the model “post”. I could actually return a Post from getPost, but for two concerns:

  • I’d be exposing the whole Post object in the XML, which may contain bits that aren’t relevant to the 3rd-party app.
  • I’d have to alias the attributes that aren’t named “properly”. For example, I call the body of the post “text” instead of “description”.

It might have been simpler still to call this struct “Post”, wrap it in a Module and use namespacing, but I hate typing so much everytime and I’m not using Article anywhere else, so it fit for me. (Again, see Typo for an example of module wrapping/namespacing.)

I did want to make a translation from Post to Article as simple as possible for the service code, so I added this method to Article:

def self.new_from_post(post, url)
    new(:description=>post.text,
        :title=>post.title,
        :postid=>post.id,
        :categories=>post.tags.collect(&:label),
        :url => url,
        :link => url,
        :permaLink => url,
        :dateCreated=>post.created_at)
end

The only real trick here, was passing in URL from the service, since the struct can’t “see” url_for (though I could probably stand to learn some namespacing tricks).

Right. Ready for the API definition?

class MetaWeblogApi < ActionWebService::API::Base
  inflect_names false
  api_method :getPost,
              :expects => [{:postid=>:string}, {:username=>:string}, {:password=>:string}],
              :returns => [Article] 
end

The only thing I’d point out here, is that my Ruby-trained eyes are still adjusting to the way AWS treats return values – I read this and think “Returns an array of Article”, but that’s not what it returns. (It returns one Article, go back to the getUsersBlogs code, above, for an example of an array of structs.)

I suppose, in theory, the service could return one, two or three values, any of which could be an array. At the very least, bracketing everything up into an array is certainly a convenient way to transport everything to api_method. Well, anyway, I’ll adapt…

Only thing left to do is write the method, most of which will look a lot like the BloggerService code, above. I’m going to zoom in on the new stuff:

class MetaWeblogService < ActionWebService::Base
[…]
  def getPost(postid, email, password)
    @author = get_author(email, password)
    post = Post.find(postid)
    if @author && post && (@author == post.author)
      Article.new_from_post(post, url_for_post(post))
    else
      raise "Bad user".inspect
    end
  end

  def url_for_post(post)
    controller.url_for(:author=>@author,
                       :controller=>'blog',
                       :action=>'view',
                       :id=>post.id,
                       :only_path => false)
  end
[…]
end

Again, a lot of this should look familiar from the BloggerService code, above. I’ll skip the repeat material:

  • getPost basically works like this: “Get this author. Get the requested post. Got that? And, you are that post’s author, right? Cool, here’s your Pos– er, Article.”
  • url_for_post – I found myself writing this over and over again in the other methods (see below) and you know, DRY, so… Oh, I’m using some fancy routing tricks on this blog. In “standard” Rails routing, you’d expect a URL like “/josh/view/123” to use a controller named “josh”. But, that would mean separate stacks of code for my two authors – which is a maintenance nightmare (trust me, I did that for a few weeks - ugh.)

Okay… Here's what I've done with the other metaWeblog methods:

  • getRecentPosts is very similar to getPost. In fact, if I were a smarter person, I’d probably have written getRecentPosts to re-use the getPosts code, but I’m not.
  • getCategories will require a new struct: Category. Luckily, this is a very simple struct, just three members, all of which are strings.
  • I’m not implementing deletePost. Not that I’m paranoid, but the idea here was a quick, rich editor – not an external app to manage the whole blog. If I want to delete something, I’ll come to the blog the “old-fashioned way”.
  • I’m also cheating on the media functions. I am lazy, but Flock (my primary target, here) already integrates with Flickr, where I’d rather store these things, anyway. (Though, if Liz ever prevails, I’ll probably add uploading eventually…)
  • newPost and editPost are pretty similar to each other, and there are no real “surprises” in either, so I’ll leave them for the complete code (below).

You can see the finished code (for the Web Service, anyway) on Pastie:

And… that’s basically it. You now have a Web Service. AWS handles all the XML and actually implements SOAP and XML-RPC for you. If you fire your browser at http://localhost:3000/outside/invoker , the scaffolding will walk you through interacting with your methods. (Note: I haven’t explored hooking this up with Rails’ test suite, though the docs assure me this is possible.)

As I implied above (waaaaay above), I haven’t gotten this to work in Flock, yet. Flock apparently has a bug that eats my blog configuration, which pretty effectively prevents it from working. But, I’ve used MarsEdit (a slick, Mac-only editor from the NetNewsWire people) and BlogJet (a Windows editor), and both (apart from some app-specific quirks) work fine.

One final note on configuring the 3rd-party clients: When entering the entry URL or endpoint URL or whatever they call it, you want to end with “api”, like so: http://blogs.thewehners.net/outside/api . If you’re seeing errors in your logs like “No action responded to index”, that’s probably why.

Permalink • Posted in: ruby on rails, tech stuff, webComments (3)

Comments

Bill Oct 26, 2006

Thank you so, so much for this! It was just what I was looking for.

One note, though. In MarsEdit, my category list was showing a bunch of blanks. I added "categoryName" to the getCategories response, and then it gave me the names of my categories.

Thanks again!

Joshua Feb 25, 2007

Paul --

Two possibilities:

1. I haven't tested the above in Rails 1.2.x yet, it's possible that it introduces new "quirks". Do you know which version of Rails is included in your copy of InstantRails?

2. Double-check that the file app/apis/meta_weblog_service.rb exists, and contains the MetaWeblogService-related classes.

Brian Apr 8, 2007

Just can't get layered AWS to work. Always missing a contant. Using scaffold to try to see the web service - as you listed, but not dice... Seeing the actual source that ran would be great and the location of the files if that matters.