Frankly, that’s easy! WordPress has a functionality to export the site’s content to a single XML file containing all the published Categories, Tags, Posts, Pages and Comments. To do this (WordPress v2.9.2) click Tools > Export and save the file. In previous versions of the software I believe it’s under the Manage menu.

I’m aware I could import the data directly from the WordPress database (to wherever it goes in the end) but let’s imagine we can’t. Anyway, database access would be tediously slow and inefficient to test against and implement.

A quick google for “import wordpress xml ruby” threw up nothing helpful so I turned to the Ruby XML libraries. John Nunemaker “feverishly posts everything he learns” at railstips.org and has two articles of use here:

• Parsing XML with hpricot
• Parsing XML with Ruby

The latter deals with three different ruby xml libraries and compares their speed, ease of use and how nice their names are to say. He puts REXML, hpricot and libxml-ruby. I’ll save you the pleasure of reading the article (if you like) and ccv John’s summary:

“Libxml is blisteringly fast, [but] Hpricot has cooler name, REXML and Hpricot both feel easier to use out of the box”

And there you go. Hpricot it is!

Now to get the data into Ruby. After a quick glance at the rubytips article and The RDocs I put together this code as a starting point:

cats_hierarchy={}
(doc/"wp:category").each do |category|
    cat_name = category.at("wp:category_nicename").innerHTML
    cat_parent = category.at("wp:category_parent").innerHTML

    if cats_hierarchy.include? cat_parent
        cats_hierarchy[cat_parent] = cat_name
    else
        cats_hierarchy[cat_name] = []
    end
end

cats = cats_hierarchy.to_a.flatten

That gives me two each to use Ruby objects each containing all of my category data: a hash which preserves the hierarchy of the structure and all the names in a linear array.

?> cats = cats_hierarchy.to_a.flatten.uniq
=> ["route66", nil, "rails", "american-2008", "reciprocal-affection", "hope-for-the-future", "code", "blog", "review-blog", "rant", "brands", "projects", "yab_shop", "textpattern", "meaningful-labor", "giants", "accessibility", "root", "charity-project", "apple", "xhtml", "america-2006-route-66", "ruby", "learning", "america-2007", "uncategorized", "iphone", "america-2008"]

?> cats_hierarchy
=> {"route66"=>nil, "rails"=>nil, "american-2008"=>nil, "reciprocal-affection"=>nil, "hope-for-the-future"=>nil, "code"=>nil, "blog"=>"review-blog", "rant"=>nil, "brands"=>nil, "projects"=>nil, "yab_shop"=>nil, "textpattern"=>nil, "meaningful-labor"=>nil, "giants"=>nil, "accessibility"=>nil, "root"=>nil, "charity-project"=>nil, "apple"=>nil, "xhtml"=>nil, "america-2006-route-66"=>nil, "ruby"=>nil, "learning"=>nil, "america-2007"=>nil, "uncategorized"=>nil, "iphone"=>nil, "america-2008"=>nil}

And so we have the starting point to getting this WordPress exported XML data into a Ruby application.

More soon.

For a recent client project I’ve used a Dreamhost unlimited account, which for value compared with the resources available and the fact that you don’t have to do any building or setting up of the server environment makes it an easy win for a site that’s not going to have a huge amount of traffic or a large amount of processing.

Post-launch I got to work putting together a basic search engine and here’s a quick run through of the steps it took to get a very simple Sphinx instance working on Dreamhost, and a few hurdles thrown in the way by various googled articles.

Development Environment

Using the guide from FG install Sphinx locally:

curl -O http://sphinxsearch.com/downloads/sphinx-0.9.8-rc2.tar.gz
tar zxvf sphinx-0.9.8-rc2.tar.gz
cd sphinx-0.9.8-rc2
./configure
make
sudo make install

then install the TS plugin into your application

script/plugin install git://github.com/freelancing-god/thinking-sphinx.git

Any problems with that, check out the FG page linked.

Getting a basic search going

Following tutorials such as the Sphinx Railscast will get you there pretty quick.

In your searchable model you need to define an index

class Page < ActiveRecord::Base
  define_index do
    indexes :title
    indexes :long
    indexes :short
  end ...

Run the indexer and start the Sphinx instance:

rake thinking_sphinx:index
rake thinking_sphinx:start

After this you'll be able to search on your object. So using script/console

@searched_pages = Page.search("query")

will return what you're looking for!

Setting up Dreamhost

First things first you need to install Sphinx in your local area, as posted by Hugh Evans:

cd ~/
mkdir -p local
wget http://sphinxsearch.com/downloads/sphinx-0.9.8.1.tar.gz
tar -xzf sphinx-0.9.8.1.tar.gz
cd sphinx-0.9.8.1/
./configure --prefix=$HOME/local/ --exec-prefix=$HOME/local/
make
make install

then set up the PATHs

echo "export PATH=\"$PATH:~/local/bin\"" >> ~/.bash_profile
source ~/.bash_profile

You can choose to set up a CRON task at this point too, but I'm not going into that.

Also at this point in the there's talk of using Sphinx being anti TOS in DH's eyes... but we'll see does the process get killed or not!

Configuring Sphinx for DH

Create a file called sphinx.yml in the RAILS_ROOT/config/ folder.

Because Dreamhost uses an externally referenced MySQL server instead of localhost you need to set up the sql_* parameters:

  sql_host: "mysql.YOURDOMAIN"
  sql_port: 3306
  sql_user: "USER"
  sql_password: "PASSWORD"
  sql_database: "DATABASE"

And because you installed Sphinx in your local area:

  bin_path: '/home/YOURUSERNAME/local/bin'

Finally, after setting whatever memory/fine tuning settings you wish/require set up the locations for the Sphinx files:

  config_file: "/home/YOURUSERNAME/DOMAIN.co.uk/shared/production.sphinx.conf"
  searchd_log_file: "/home/YOURUSERNAME/DOMAIN.co.uk/shared/log/searchd.log"
  query_log_file: "/home/YOURUSERNAME/DOMAIN.co.uk/shared/log/searchd.query.log"
  pid_file: "/home/YOURUSERNAME/DOMAIN.co.uk/shared/log/searchd.production.pid"
  searchd_file_path: "/home/YOURUSERNAME/DOMAIN.co.uk/shared/db/sphinx"

That should be you ready to start deploying.

Deploying

Using Git + Capistrano for deployment (and Passenger for the http server) my deploy.rb's namespace area looks like this:

namespace :deploy do
  task :restart do
    after_symlink
    restart_sphinx
    run "touch #{deploy_to}/current/tmp/restart.txt"
  end

  task :start do
    # nothing  (this avoids the 'spin' script issue)
  end

  desc "Re-establish symlinks"
  task :after_symlink do
    run <<-CMD
      rm -fr #{release_path}/db/sphinx &&
      ln -nfs #{shared_path}/db/sphinx #{release_path}/db/sphinx
    CMD
  end

  desc "Stop the sphinx server"
  task :stop_sphinx , :roles => :app do
    run "cd #{current_path} && rake thinking_sphinx:stop RAILS_ENV=production"
  end

  desc "Start the sphinx server"
  task :start_sphinx, :roles => :app do
    run "cd #{current_path} && rake thinking_sphinx:configure RAILS_ENV=production && rake thinking_sphinx:index RAILS_ENV=production && rake thinking_sphinx:start RAILS_ENV=production"
  end

  desc "Restart the sphinx server"
  task :restart_sphinx, :roles => :app do
    stop_sphinx
    start_sphinx
  end

end

There's probably a neater way to do this, but basically this makes sure Sphinx's indexes and conf files live in the shared deployment folder.

I recommend you try all this in a staging area first, obviously... and you can use Dreamhost's control panel to set up a staging subdomain with a new database in whatever fashion you prefer.

Any problems with this script flag them up, please! This is as much for my future reference as you googlies out there.

Leopard has PHP and Apache fairly up to date, and probably MySQL as well, but I decided to get MySQL up to scratch using the Universal Binary along with a few command line instructions available here (guide and links to downloads).

I didn’t use this script myself, and at a year old it may have a few imperfections, but HiveLogic have a fairly well rounded Ruby/Rails install going on.

Finally, WordPress to go on your LAMP stack will be a lot less painful using this guide for Tech Recipes.

Any suggestions additions or replacements for this list? Drop a comment!

I think it might prove useful: http://www.unethicalblogger.com/blog_categories/git will be taking a read in my spare time.