Perl programs are powerful workers that works well especially good for data conversion tasks. One of my $client_project requires importing a big data set that's available in either Access DB file or in CSV files. I thought: maybe a perl program can solve this quickly.

And the reason for that is of course, the almighty CPAN.

The approach to the solution is quite simple:

  • some program converts the data set into yaml files
  • write a script/runner -based Rails script to load those yaml files, and then create records with ActiveRecord, or alternatively, loading those yaml files as fixtures.

Thanks to the fact that YAML is in Ruby core library.

I found that Text::CSV::Slurp is very handy, especially when CSV field values contains newlines.

This approach has been working for two of my $client_projects and I still feel happy and confidence with it. One only needs to do one data import, the other needs to do it in a daily bases (to adapt legacy system output) and it's up and running smoothly.

It's not that it's impossible to be done with Ruby, but it can be very easily done with Perl. Plus perl is built-in on the Linux server, and I happened to be very familiar with the Perl language - I see no reason that I must do it with only Ruby.



blog comments powered by Disqus