Daysleeper

Scaling rails

Scaling isn't easy, it never was. Its not easy with any dynamic language. So, twitter as a rails application is facing some issues of scaling. So while I was sleeping peacefully, a war was going on, in the community. Alex, the twitter developer shares his thoughts, http://www.radicalbehavior.com/5-question-interview-with-twitter-developer-alex-payne/ and DHH retorts back, and criticizes arms-crossed approach, http://www.loudthinking.com/arc/000608.html.

So, what shall a rails developer learn from this. Well, just the basics that - any fool can code web pages, its not rocket science for sure. But how will he react when problems like these strike him? LiveJournal invented memcached, Google invented GFS, Wikipedia does this by aggressive caching. You can't get a out of box, scaling solution. This is the precise reason, Why I have been adamant about hiring people, who know ruby better, who can dig into the internals and come up with shiny code.

Dr Nic, comes with a solution, http://drnicwilliams.com/2007/04/12/magic-multi-connections-a-facility-in-rails-to-talk-to-more-than-one-database-at-a-time/, which is not exactly the same. But lets see. Oh and dev.rubyonrails.org is a python page, which had same problem with trac.

fscking cool stuff from ActiveSupport

Some rails stuff from ActiveSupport

>> a = "Hemant Kumar"
=> "Hemant Kumar"
>> a.at(4)
=> "n"
>> a.from(4)
=> "nt Kumar"
>> a.first(4)
=> "Hema"
>> a.last(4)
=> "umar"
>> a = "2007/01/01 9:11:25PM"
=> "2007/01/01 9:11:25PM"
>> a.to_time
=> Mon Jan 01 21:11:25 UTC 2007
>> a.to_date
=> #<Date: 4908203/2,0,2299161>
>> a = "camels"
=> "camels"
>> a.pluralize
=> "camels"
>> a = "user_comments"
=> "user_comments"
>> a.camelize
=> "UserComments"
>> "UserComments".underscore
=> "user_comments"
>> "user_comments".dasherize
=> "user-comments"
>> "UserComments".demodulize
=> "UserComments"
>> "UserComments".tableize
=> "user_comments"
>> "UserComments".classify
=> "UserComment"
>> "UserComments".humanize
=> "Usercomments"
>> "UserComments".foreign_key
=> "user_comments_id"
>> "User".constantize
=> User
>> "user_comments".each_char {|char| print char}
=> nil
>> "640 <lol>".starts_with? 640
=> true
>> "640 <lol>".ends_with? "lol>"
=> true
>> "अ".is_utf8?
=> true
>> class Foobar
>>   cattr_reader :name
>>   def foo
>>     p "#{@name}"
>>   end
>> end
=> nil
>> foo = Foobar
=> Foobar
>> foo.name = "Hemant"
"NoMethodError: undefined method `name=' for Foobar:Class
  from (irb):45"
>> foo.name
=> nil
>> class << foo
>>   cattr_accessor :age
>> end
=> [:age]
>> foo.age = 10
=> 10
>> foo.age
=> 10
>> foo.class_variables
=> ["@@name"]
>> foo.instance_variables
=> ["@inheritable_attributes"]
>> class Blah
>>   class_inheritable_writer :age
>> end
=> [:age]
>> class Bleh < Blah
>> end
TypeError: superclass mismatch for class Bleh
  from (irb):80
>> class Beam
>> end
=> nil
>> class Blem
>> end
=> nil
>> " ".blank?
=> true
>> "       ".blank?
=> true
>> a = nil
=> nil
>> a.blank?
=> true
>> a = []
=> []
>> a.blank?
=> true
>> a = {}
=> {}
>> a.blank?
=> true
>> a = User.find(:all,:select => "id,login")
=> [#<User:0xb6f84128 ]
>> a.class
=> Array
>> a.index_by(&:id)
=> {71=>#<User:0xb6f84024>}
>> a.index_by(&:id).hash
=> -612671648
>> c = a.index_by(&:id)
=> {71=>#<User:0xb6f84024 @attributes={"id"=>"71"}}
>> c.class
=> Hash
>> a = [1,2,3,4]
=> [1, 2, 3, 4]
>> a.sum
=> 10
>> a = {:name => "Hemant Kumar",:age => 10}
=> {:name=>"Hemant Kumar", :age=>10}
>> a.to_xml
=> "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<hash>\n  <name>Hemant Kumar</name>\n  <age type=\"integer\">10</age>\n</hash>\n"
>> b = "<person><name>Hemant</name><age>10</age></persone>"
=> "<person><name>Hemant</name><age>10</age></persone>"
>> b = "<person><name>Hemant</name><age>10</age></person>"
=> "<person><name>Hemant</name><age>10</age></person>"
>> Hash.from_xml b
=> {"person"=>{"name"=>"Hemant", "age"=>"10"}}
>> c = Hash.from_xml b
=> {"person"=>{"name"=>"Hemant", "age"=>"10"}}
>> a
=> {:name=>"Hemant Kumar", :age=>10}
>> a.diff c
=> {:name=>"Hemant Kumar", :age=>10, "person"=>{"name"=>"Hemant", "age"=>"10"}}
>> a.stringify_keys
=> {"name"=>"Hemant Kumar", "age"=>10}
>> d = a.diff c
=> {:name=>"Hemant Kumar", :age=>10, "person"=>{"name"=>"Hemant", "age"=>"10"}}
>> d.symbolize_keys
=> {:name=>"Hemant Kumar", :person=>{"name"=>"Hemant", "age"=>"10"}, :age=>10}
>> a.reverse_merge c
=> {:name=>"Hemant Kumar", :age=>10, "person"=>{"name"=>"Hemant", "age"=>"10"}}
>> a = 23
=> 23
>> a.even?
=> false
>> a.odd?
=> true
>> a.multiple_of 23
"NoMethodError: undefined method `multiple_of' for 23:Fixnum
  from (irb):123"
>> a.multiple_of? 23
=> true
>> 1.ordinalize
=> "1st"
>> module Fleh
>> mattr_writer :age
>> end
=> [:age]
>> Fleh.age = 10
=> 10
>>

Things your uncle didn't tell you about rails 1.2

Hmm, well with new year all the rails programmers have been chattering away about rails 1.2 and how it will do laundry for them and get christmas presents too( href )

Here is something, where I would list gotchas and stupid things that can stop you from going full steam ahead:

• Take a deep breadth and accept that memcache session handler is broken in development mode. As a quick fix you can use pstore for session storage in development mode. Update: Turns out that I was a complete idiot and was storing entire model in session and since now with Rails 1.2, only model which is required in the current controller context is reloaded in development mode, I didn’t have class definition for model in entire app and hence all the funny errors. just doing a require 'user' in application.rb fixed the issue. Cheers to Rails core team for helping me out on IRC.

• Now, rails reloads all your classes/modules for each new requests in development mode. Well, so what, its been doing that for years. So here is catch, it does this for /lib and /vendor/plugins also. And sometimes this can cause quite some trouble.
  http://dev.rubyonrails.org/ticket/6738
  So the simplest solution is to put this in your environment.rb file:
  

    config.load_once_paths = ["#{RAILS_ROOT}/lib"]

  
  Now, what it does is, it forces modules/classes defined in /lib to load only once.


• Well, rails routes are now a lot more stricter and doesn’t do too many assumptions. For example:
  

  >> rs.recognize_path "/forums/index"
  => {:controller=>"forums", :action=>"index"}
  >> rs.recognize_path "/forums"
  => {:controller=>"forums", :action=>"index"}
  >> rs.recognize_path "/forums/index/foo"
  => {:controller=>"forums", :action=>"index", :id=>"foo"}
  >> rs.recognize_path "/forums/index/foo.bar"
  ActionController::RoutingError: no route found to match "/forums/index/foo.bar" with {}
  >> 

  
  Well as you can see, params[:id] doesn’t like parameters with dots anymore and you must set an explicit route for this:
  

    map.connect('/forums/index/:id',
                :controller => 'forums',
                :action => 'index',
                :id => /.*/)

  
  Update: Please note, a better regexp would be required to properly match the ID, because otherwise it would gobble up everything after index. Its eager matching you see!!


• How to upgrade yourself:
  

  
  gem install rails --source http://gems.rubyonrails.org --include-dependencies
  

  
  And go to your project directory and do
  

  
  rake rails:update
  

  
  to update scipts/javascript/css rails generated for you.


• Update : with Rails 1.1.6, when you say request.domain you would normally get just the domain name but with rails 1.2.1 both request.host and request.host return the same thing by default. It means you will have to resort to these sort of ugly hacks:
  

    def get_subdomain url
      domain_string = url.split("://").last
      domain_array = domain_string.downcase.split('.')[0..1]
      country_codes = %w[au br ca cn fr de hk in it jp kr mx mg es tw uk us]
      country = domain_array.find { |x| country_codes.include? x}
      return country
    end

• There are quite few other stuffs and you should see:
  [1]
  http://weblog.rubyonrails.org/2006/11/23/rails-1-2-release-candidate-1
  [2] http://weblog.rubyonrails.org/2006/11/26/1-2-new-in-activerecord
  [3]
  http://weblog.rubyonrails.org/2006/11/26/1-2-new-in-actionpack
  [4] http://weblog.rubyonrails.org/2006/11/26/1-2-new-in-activesupport

On Testing in Rails

Dynamic fixtures are nice, but recently I found that It doesn’t work so well with timestamp entries, so if you are trying something like this in your fixture:

  message:
    pub_date: <%= Time.now %>
 

it won’t work. The simple solution is:

  message:
    pub_date: <%= Time.now.to_s(:db) %>

Another testing nightmare could be testing your RJS templates or embedded RJS code in controller. Normal asserts won’t work.

Kevin has written nice plugin that solves the problem of testing RJS. Its quite elegant:

  assert_rjs :alert, 'Hi!'
  assert_rjs :assign, 'a', '2'
  assert_rjs :call, 'foo', 'bar', 'baz'
  assert_rjs :draggable, 'draggable_item'
  assert_rjs :drop_receiving, 'receiving_item'
  assert_rjs :hide, "post_1", "post_2", "post_3"
  assert_rjs :insert_html, :bottom, 'posts'
  assert_rjs :redirect_to, :action => 'list'
  assert_rjs :remove, "post_1", "post_2", "post_3"
  assert_rjs :replace, 'completely_replaced_div'
  assert_rjs :replace, 'completely_replaced_div', '<p>This replaced the div</p>'
  assert_rjs :replace, 'completely_replaced_div', /replaced the div/
  assert_rjs :replace_html, 'replaceable_div', "This goes inside the div"
  assert_rjs :show, "post_1", "post_2", "post_3"
  assert_rjs :sortable, 'sortable_item'
  assert_rjs :toggle, "post_1", "post_2", "post_3"
  assert_rjs :visual_effect, :highlight, "posts", :duration => '1.0'

Read about Arts here

problems with Typo

Here are some funny things about Typo:

1. Sidebar is a component and it uses a ugly rails hack to suppress extensive logging. Making it difficult to debug sidebar plugins.

2. Code for implementing multiple blogs on single app is just crap.

3. Code itself is kinda funny, difficult to understand. Lambda chaining in full glory.

4. Scott is a busy googler, hardly has time to maintain it.

Solutions:

1. Someone should take ownership or fork the project.

2. Remove component thingy.

3. Remove ugly hacks.

I have done some of above and in process may be I have made Typo code still uglier.
Cheers.

Monit configurations files

Here are some monit configurations file that might be useful in creating monit daemons that monitor the processes:

Config file

Another config file

Ruby on Rails and EJB

Google Trend has a nice comparison chart on RoR and EJB(Enterprise Java Beans).

A new matrix avatar

Well...i wanted to show this image directly..but somehow image upload is not working on this typo.So, i am stuck with just posting the link:
http://www.loudthinking.com/lt-files/linuxjournal.jpg

fcgid and Apache Rewrites

I am planning to write regular nice tutorials on rails..let's see if time permits to do that.

Whenever you are using any Alias directive to refer your Virtual Host root in Apache, make sure that you have a corresponding Rewrite rule set in .htaccess file.

Cookies and autologin in rails using login engine

Everyone seems to have and idea on what the heck are persistent cookies...but what happens when they stop working?

What happens when you need a more secure approach than encrypting username and password for autologin.I spend the night, finding answers to these small problems.