Consulting. Training. Development.

Lightning JSON in Rails

Rendering JSON is pretty easy in Rails:

1

render json: @statuses

This works well if there are small number of records to be returned. But what happens when we need to return 10,000 records at once? Things slow down dramatically and the most time-consuming parts are JSON serialization and database operations.

Include only required attributes

The first obvious thing is generating JSON with only attributes that we need, e.g.:

1

render json: @statuses, methods: [:latitude, :longitude, :timestamp, :virtual_odometer]

Tidying JSON gives over 20% performance:

1
2

default    5.940000   0.080000   6.020000 (  6.094221)
attrs      4.820000   0.010000   4.830000 (  4.932337)

Select only required columns

Second, we should consider selecting only required columns when we don’t need all of them.

1

render json: @statuses.select([:latitude, :longitude, :timestamp, :virtual_odometer])

It’ll help us to avoid transferring a huge amount of data to the application from the database and gives 2x speedup:

1
2
3

default    5.940000   0.080000   6.020000 (  6.094221)
attrs      4.820000   0.010000   4.830000 (  4.932337)
select     2.170000   0.020000   2.190000 (  2.222277)

Don’t instantiate ActiveRecord objects if possible

Let’s implement a method to return “lightning” array of hashes instead of ActiveRecord instances:

1
2
3
4
5
6
7

def self.lightning
  connection.select_all(select([:latitude, :longitude, :timestamp, :virtual_odometer]).arel).each do |attrs|
    attrs.each_key do |attr|
      attrs[attr] = type_cast_attribute(attr, attrs)
    end
  end
end

It works the same way as pluck but returns array of hashes instead of array of single column values. Invoke a new method in controller:

1

render json: @statuses.lightning

Using lightweight hashes makes JSON rendering 2x faster:

1
2
3
4

default    5.940000   0.080000   6.020000 (  6.094221)
attrs      4.820000   0.010000   4.830000 (  4.932337)
select     2.170000   0.020000   2.190000 (  2.222277)
lightning  1.120000   0.010000   1.130000 (  1.148763)

Use the fastest JSON dumper

There are several JSON libraries available:

  • JSON The default JSON gem with C-extensions (ships with Ruby 1.9)
  • Yajl Yet Another JSON Library by Brian Lopez
  • Oj Optimized JSON by Peter Ohler

It’s a good idea to use the fastest dumper of them:

1
2
3

json       0.810000   0.020000   0.830000 (  0.841307)
yajl       0.760000   0.020000   0.780000 (  0.809903)
oj         0.640000   0.010000   0.650000 (  0.666230)

So we prefer using Oj dumper:

1

render json: Oj.dump(@statuses.lightning, mode: :compat)

Summarized benchmark results are:

1
2
3
4
5
6
7
8

   user     system      total        real
default    5.940000   0.080000   6.020000 (  6.094221)
attrs      4.820000   0.010000   4.830000 (  4.932337)
select     2.170000   0.020000   2.190000 (  2.222277)
lightning  1.120000   0.010000   1.130000 (  1.148763)
json       0.810000   0.020000   0.830000 (  0.841307)
yajl       0.760000   0.020000   0.780000 (  0.809903)
oj         0.640000   0.010000   0.650000 (  0.666230)

Comments