Remote Work

Nov 7, 2013

At ClearCove we have been working remotely for 15 years. We have worked with clients on other continents, or in the next town. For certain kinds of jobs, this is a great model where everybody wins: The client gets access to the best talent for the job, we get to establish work/life balance and live exactly where we want to be, and the environment wins by one more person who doesn’t have to commute to work every day.

37Signals is an example for a company that gets it. Their tools facilitate remote work through asynchronous communications, they wrote a book about it, and now they have a job board for remote jobs.

Link: Remote Jobs: Design, Programming, Rails, Executive, and more. via weworkremotely.com

How to increase max stack size for Ruby 2.0 when experiencing (SystemStackError) “stack level too deep”

Oct 11, 2013

We process a lot of HTML documents using Nokogiri. On large HTML files we sometimes get the (SystemStackError) “stack level too deep” exception. This happens because Nokogiri recursively processes the dom, and on large documents it exceeds the maximum stack size of 1024 KB on our system.

In Ruby versions before Dec. 2012, the Ruby stack size was limited by the C stack size which can be changed using the ‘ulimit -s’ command. In Ruby 2.0 the stack size is now limited via its own environment variable RUBY_THREAD_VM_STACK_SIZE.

We address the problem by setting this env variable to a higher value when launching our ruby processes.

Here is some code to try this yourself:

First launch a Ruby console

irb

Then inspect your current max stack size

irb(main):052:0> ENV['RUBY_THREAD_VM_STACK_SIZE']
=> "1048576"

You can also look at all the default settings for the VM:

irb(main):001:0> RubyVM::DEFAULT_PARAMS
=> {:thread_vm_stack_size=>1048576,
    :thread_machine_stack_size=>1048576,
    :fiber_vm_stack_size=>131072,
    :fiber_machine_stack_size=>524288}

Now run the program below and modify the upper limit from 7600 to 7700. You will see it work with 7600 and exit with SystemStackError for 7700:

upper_limit = 7600 # 7600 works, 7700 doesn't

def so(arg, count)
  r = if count < upper_limit
    ('A') + so(arg, count + 1)
  else
    arg
  end
  r
end

so('a', 0)

In order to make it work with 7700, quit your irb and re-launch it with modified max stack size:

export RUBY_THREAD_VM_STACK_SIZE=2000000
irb

Link: [ruby-cvs:45648] ko1:r38478 (trunk): * vm.c: support variable VM/Machi via permalink.gmane.org

How to deduplicate rows in PostgreSQL

Sep 5, 2013

Use the following query to keep only one row and delete all duplicates of that row.

Note: this is non-standard SQL and works only with PostgreSQL.

The following snippet finds all rows in the users table that have identical email and organization_id. It keeps the oldest of the rows (via users.id > u2.id) and deletes the newer duplicates.

DELETE FROM users USING users u2
 WHERE users.email = u2.email
 AND users.organization_id = u2.organization_id
 AND users.id > u2.id;

Read the documentation for PostgreSQL’s USING clause in combination with DELETE.