Earlier this week, we tried to figure out the cleanest and easiest way to get our Rails app to accept incoming JSON requests. Up until recently, developers were able to use various Rails plugins for this purpose, such as the json_request plugin.
Luckily, it turns out that full support for JSON was added to Rails in April, making it a first class citizen along with XML and regular URL-encoded form fields. This functionality will be officially released in Rails 2.1, but in addition to Edge Rails, it is already included in Rails 2.0.991, which is available from the Ruby on Rails Gem Repository. You can install this pre-release via:
sudo gem update rails --source http://gems.rubyonrails.org
Using this functionality is really simple. Let’s say we have created the following scaffolded Rails app with a Book resource, perhaps to manage your library:
rails library cd library script/generate scaffold book title:string author:string isbn:string price:decimal rake db:migrate
As you know, you can now access the books controller in the browser via http://localhost:3000/books, and use the “New Book” link to create a new book via the scaffolded form that Rails provides. But you can also create books via JSON (or XML, for that matter). In fact, we will try XML first, which has been natively supported in Rails for a while:
curl -H "Content-Type:text/xml" -H "Accept:text/xml" \ -d "<book><title>Posted via XML</title><author>Ex Emel</author><isbn>1234567890</isbn><price>34.99</price></book>" \ http://localhost:3000/books
Note that I am setting both the Content-Type and Accept header to “text/xml”, indicating that the incoming request consists of XML and that we would like to receive an XML-formatted response as well. The response looks like this:
<?xml version="1.0" encoding="UTF-8"?> <book> <author>Ex Emel</author> <created-at type="datetime">2008-05-26T05:58:38Z</created-at> <id type="integer">2</id> <isbn>1234567890</isbn> <price type="decimal">34.99</price> <title>Posted via XML 2</title> <updated-at type="datetime">2008-05-26T05:58:38Z</updated-at> </book>
Now let’s try the same thing in JSON:
curl -H "Content-Type:application/json" -H "Accept:application/json" \ -d "{\"book\":{\"title\":\"Posted via JSON\", \"author\":\"Jason Bourne\", \"isbn\":1234567890, \"price\":49.95}}" \ http://localhost:3000/books
As you can verify in the browser, this request was successful and had the desired effect. However, unlike the XML case, there was no response this time. This is because our controller does not know how to render JSON results yet. Looking at the create method in the BooksController, we notice that the responds_to block contains entries for HTML and XML, but not for JSON. Simply copy the XML lines and replace all occurrences of xml with json. The updated method should look like this:
def create @book = Book.new(params[:book]) respond_to do |format| if @book.save flash[:notice] = 'Book was successfully created.' format.html { redirect_to(@book) } format.xml { render :xml => @book, :status => :created, :location => @book } format.json { render :json => @book, :status => :created, :location => @book } else format.html { render :action => "new" } format.xml { render :xml => @book.errors, :status => :unprocessable_entity } format.json { render :json => @book.errors, :status => :unprocessable_entity } end end end
If you run the same curl command again, you should now get the following response:
{"book": {"isbn": 1234567890, "updated_at": "2008-05-26T06:11:33Z",
"title": "Posted via JSON", "price": 49.95, "author": "Jason Bourne",
"id": 4, "created_at": "2008-05-26T06:11:33Z"}}One important thing to note is that both the incoming and outgoing JSON contain an outermost element called book. This is in fact required for resource based JSON requests to work. The same is true for XML, but since XML requires an enclosing element (unlike JSON, which can be “naked”), it is perhaps less obvious in this case. The outermost JSON element should always have the same name as the resource it corresponds to.
A few notes on testing JSON requests:
We initially banged our heads against the wall trying to figure out how to convince our functional test to pass JSON-formatted parameters in the post method, including various hacks to override different settings on the @request object. Admittedly it had been a while since I had seriously used Rails the last time, but it later dawned on me that we were going about this the wrong way. Functional tests (in Rails, anyways… let’s not talk about its confusing and non-standard test terminology) bypass most of the actual HTTP request handling and are not meant to test this aspect of an application. They essentially pick up at the point where the controller has received its (already parsed) parameters in the params hash, regardless of whether these originated from an XML, JSON, or URL-encoded form request.
Since JSON support is implemented by Rails (and thus covered by its own unit tests), it probably does not make sense to focus too much on testing this general functionality in the individual application. But if you do want to test JSON requests, you can use integration tests for this purpose. The post method in integration tests is more low level and simulates the actual HTTP request, along with the parameter parsing.
So in our library example, we might use an integration test case such as the one below to specifically test creating a book via JSON:
def test_create_via_json assert_difference('Book.count') do post '/books/create', '{"book":{"title":"Posted via JSON", "author":"Jason Bourne", "isbn":1234567890, "price":49.95}}', {'Content-Type' => 'application/json', 'Accept' => 'application/json'} end end
Native JSON support in Rails is definitely a useful feature. In fact, I was fairly surprised that this wasn’t already implemented until recently. But now that it’s here, it should come in very handy.
[...] I mentioned in a previous blog post, Rails 2.1 natively supports incoming JSON requests. Unfortunately, it still struggles with JSON data containing non-ASCII [...]
Question: I’ve noticed an “Authentication token” in the form for creating a new object. If I am sending the json to create a new object from a different server without this authentication token, will the creation of my object fail?
I’m hoping you’ll be able to provide an answer… if not, I guess I’ll just have to try it and see what happens… :-\
-Steve
I’m not 100% sure, but I believe that Rails only verifies requests using the authentication token when using html, not with json or xml.
Well I tried it out. Works fine!! I should have been able to figure that out from your post anyway. Heh sorry about that.
But now I have another, hopefully more useful question: Can I return only certain attributes? I don’t need the timestamps in the response, and I’m trying to keep it as slim as possible…
Or would I have to do something like: format.json { render :json => {:author=>@book.author,:title=>@book.title} }
Steve, check out the serialization docs.
You should be able to do something like this:
format.json { render :json => @book.to_json(:except => [:created_at, :updated_at]) }Or the other way round, using
nly:
format.json { render :json => @book.to_json(:only => [:author, :title]) }Alternatively, you could override to_json in your Book model and have it exclude / include the necessary attributes. This may be preferable if you're rendering books to json in multiple places and don't want to duplicate the same
nly / :except options everywhere.
This… is… freaking… awesome!!! Thank you Mr DigitalHobbit
Ruby on Rails is great…
OK this is only tangentially related, and I’m sorry to ask here, but dang if I couldn’t find any other blog post that seemed appropriate…
Can you point me to some doc(s) about creating an XMLRPC server in Rails? Seems like it should be a fairly basic task, but my numerous google searches turned up nothing recent or useful…
@Steve:
Glad you’re enjoying Rails.
Regarding XML-RPC: I never had to implement this myself in Rails, and the Rails community’s focus has definitely been on REST. ActionWebservice used to provide this functionality but was discontinued. However, it looks like someone created a fork on GitHub that works with Rails 2.1 and 2.2, supports both XML-RPC and SOAP, and includes various improvements.
ActionWebservice on Github
Blog post announcing the release
Hope this works for you.
Interesting… my searches eventually turned up: http://blog.multiplay.co.uk/2008/10/serving-xml-rpc-from-rails-2x But I can’t tell if this is a gem/plugin or included with Rails 2.x (as the comment seems to imply)…
What do you make of this xmlrpc4r? http://www.fantasy-coders.de/ruby/xmlrpc4r/
I swear, this is my last question on this particular post. I don’t want to impose on your hobbit hospitality…
Well for what it’s worth, I got ActionWebService working today. It was actually relatively painless, although I wish it had better documentation. Maybe I’ll add something to their Wiki on Github once I’m done implementing…
Hey Steve, glad to hear you got ActionWebService working!
Just wanted to leave a note that I think the test can be better written as:
def test_create_via_json assert_difference(’Book.count’) do post :create, :book => { :title => “Posted via JSON”, :author => “Jason Bourne”, :isbn => “1234567890, :price => 49.95}, :format => :json end end
That makes the test agnostic of the format so you could easily do an enumeration of multiple post format’s (i.e. [:json, :html].each do |format| … :format => format}) and not have to change the test.
This is very useful indeed, I have a related question if you’d be kind enough to point me to the right direction in answering it that’d be great.
I am trying to write a script that uses curl to post data to a tiny webapp that uses authlogic to provide the username/password protection. Does curl support that? I mean, I am guessing one can do a post with username and password (in raw text?) to login, but will curl and rails “remember” that it has been logged in? or do I have to supply username and password for each request that I do?
Thank You!
Nik, are you writing a Ruby script or a shell script? If you’re using Ruby, why not just use Net::HTTP instead of shelling out to curl?
By itself, neither of these solutions will keep track of the logged in user. If you really need this, you probably need to implement cookies, as these are necessary to keep track of the user session. Otherwise, you may be able to get away with including the basic auth credentials in each request.