Lincoln Stoll

Fog and the Vcloud API

2011-07-11T00:00:00+00:00

Fog and the Vcloud API

19/05/2011

This post is located at http://techblog.realestate.com.au/2011/07/controlling-vcloud-with-ruby-via-fog/ , redirecting…

Ninefold API Action with Fog

2011-05-19T00:00:00+00:00

Ninefold API Action with Fog

19/05/2011

I’ve been working on adding support for Ninefold to Fog, and the code is now in a somewhat beta state. It is now possible to create a machine via the library, and set up networking to access it - then tear it all down.

I want to have a look!

I’ve created a sample Rake script to demonstrate this, it can be found at https://gist.github.com/ee5b65a8ed778ce61963. Intstructions on how to use it are included in the gist.

This will start up a server, allocate it a public IP, and map the SSH port. Then when prompted, it will then tear this environment down.

I want to use it myself!

Currently, the code hasn’t been submitted upstream, so you will need to work directly against the git repo. To do this, add this to your Gemfile:

gem 'fog', :git => "https://github.com/ninefold/fog.git", :branch => "ninefold"

You can then bundle install. This is currently a moving target, so you may want to ‘bundle update fog’ often to ensure you’re up to date.

See the gist for an example on how to interact with the API

You can also check out the repository, build install it, but this will override your system install - I’d reccomend using bundler.

If you want an interactive shell to experiment with the API:

git clone https://github.com/ninefold/fog.git ninefold-fog
cd ninefold-fog
bundle install

Then set your API credentials in ~/.fog like in the gist

bundle exec bin/fog

You can then interact with the Ninefold API via Ninefold:compute. Some examples:

>> Ninefold[:compute].servers.all

>> Ninefold[:compute].list_networks

>> Ninefold[:compute].list_templates(:templatefilter => 'executable')

>> Ninefold[:compute].list_service_offerings

>> Ninefold[:compute].servers.new(:templateid => 421, :serviceofferingid => 67, :zoneid => 1)

How does this all work anyway?

Via fog, items can be exposed in two ways. Requests, which map directly to the API. For naming convention reasons, we map the CamelCase API methods to underscore_methods. There are also Models, which map items at a higher level, and follow a set of fog conventions to make code somewhat portable across cloud providers.

At all times it will be handy to keep the API reference handy as we are wrapping that, it can be found at https://ninefold.com/support/display/SPT/API+Documentation

The best way to see the requests in action is via the tests at https://github.com/ninefold/fog/tree/ninefold/tests/compute/requests/ninefold

For interacting via the models, see the example gist and the fog documentation

I’m having problems!

If you find any bugs or have any feature requests, please log them at https://github.com/ninefold/fog/issues. If you just want to have a chat about how this works, or how you can use it, feel free to contact me.

Compass with Rails 3 on Heroku

2010-06-08T00:00:00+00:00

Compass with Rails 3 on Heroku

8/6/2010

I’ve noticed plenty of posts around the net on how to get this to work, but none of them worked as is for me on a Rails 3 Beta 3 app using Compass and the Compass 960 plugin

Here’s my cobbled together solution - it’s also a little simpler than some, as it doesn’t try and do things different for heroku and local dev - it keeps it the same, and seems to work OK.

1) Make sure you are using the latest gems

2) Make sure config/compass.rb has any plugins referenced, and set it’s css_dir to tmp/stylesheets - this is writable on heroku.

3) Create a initializer in config/initializers/stylesheets.rb , and set it to serve the generated stylesheets from tmp/stylesheets under /stylesheets . This has to be loaded before the Rack::Sendfile middleware, otherwise it will try to X-Sendfile the CSS file which doesn’t work on Heroku, rather than just rendering it’s contents.

Google App Engine XMPP in Play Framework

2010-05-10T00:00:00+00:00

Google App Engine XMPP in Play Framework

8/6/2010

Google’s documentation for receiving XMPP messages on App Engine involves running the HttpServletRequest through XMPPServiceFactory.parseMessage to extract the details, however on Play you don’t get access to the HttpServletRequest, so you can’t use their message parser. Fortunately the message is really just POSTed form data, so you can easily extract it in a handler bound to “POST /_ah/xmpp/message/chat/” like this:

Displaying your Location from Twitter on your Site

2010-05-08T00:00:00+00:00

Displaying your Location from Twitter on your Site

8/6/2010

I was giving this site a little tweak, and one thing I wanted to add was my current location to my contact details - I move around a lot, and getting calls at stupid hours was begging to get a little annoying.

The catch was, I didn’t want to have to update this information manually all the time - it would get tiresome quickly, and I would end up letting it go stale. I usually geotag tweets semi-regularly from my phone, so I figured that would be a great place to get the data.

I sat down, and whipped up a quick Sinatra application that looks for the latest tweet with coordinates, extracts them, and uses the google reverse geocoding service to determine the city name. To make it a little less stalkerish I kept the city as the finest level of granularity. This is then exported as a simple little JSON-P Service, as well as a redirect to a map of the location.

I deployed it to Heroku like usual, I love how simple they make it to get an app up and running. The running app can be found at http://whereami.lds.li, and the code is on github at http://github.com/lstoll/whereami

I integrate it in to this site with the following script:

Ubuntu 10.04 Vagrant Base Boxes

2010-05-07T00:00:00+00:00

Ubuntu 10.04 Vagrant Base Boxes

07/05/2010

I’ve made a pair of base boxes for Ubuntu 10.04 Server (Lucid Lynx), one 32 bit, one 64 bit.

They are set up as per the guide, with chef 0.8.10 installed for provisioning. It’s using the packaged ruby, with a custom installed rubygems as /usr/bin/gem. It also has the 3.1.6 additions, and is set up with the normal public SSH keys

You can add them by running:

32 Bit

$ vagrant box add lucid32 http://s3.lds.li/vagrant/lucid32.box
64 Bit

$ vagrant box add lucid64 http://s3.lds.li/vagrant/lucid64.box

I intend to update them occasionally as new versions of chef are released, will package updates as needed.

Sync File Creation Date with EXIF Date on OS X

2010-05-07T00:00:00+00:00

Sync File Creation Date with EXIF Date on OS X

19/05/2011

So I could order my images in the Finder and use Smart Folders based on the created date of the image, I hacked this together to update the file creation date to match the EXIF date. For Mac OS X only.

See http://gist.github.com/310413 for all the code.

Simple node.js based http router

2010-05-07T00:00:00+00:00

Simple node.js based http router

19/05/2011

Can be used to do a lookup based on the URL or incoming connection details, and route to the appropriate backend

Bulk deleting tumblr posts

2009-11-05T00:00:00+00:00

Bulk deleting tumblr posts

19/05/2011

Because tumblr doesn’t let you change the default blog, I wanted to clear out the old original blog I had to re-use for this site. Which brought me to the other problem - tumblr provide no way to batch delete posts, and will over 1,000 items in there I wasn’t going to sit around and delete each one by hand. So here’s a quick script I wrote, I hope it’s useful to someone.

Enabling iPhone 3.0 Tethering and MMS on Three UK (Updated)

2009-06-14T00:00:00+00:00

Enabling iPhone 3.0 Tethering and MMS on Three UK (Updated)

14/06/2009 – London, UK

Update:
I’ve created TetherMe, a site for tethering your iPhone. Support for Australian Carriers (Telstra, Optus, Vodafone, Three) plus a few UK carriers, and Swisscom. Will add more soon, or you can add your own APN. You can find the site at http://tetherme.lstoll.net

The iPhone 3.0 Software is almost upon us, with some features I’ve been looking forward to – including tethering and MMS. Unfortunately, the tethering feature has been limited to those carriers that officially carry the iPhone, and wish to allow their users to use it. This is unfortunate for those of us who have paid to unlock our phone, and are using an alternate carrier – as the feature never gets enabled this. There is a way around this however – a .mobileconfig file that sets the APN’s up correctly. If you’re using 3 UK, browse to this page on your phone, and click on this link (Or you can save it, and mail to your phone). This will provision your phone, and enable tethering. Works great with the 5 pounds a month unlimited internet add-on. For those on other carriers, take a look at the 3UKTether.mobileconfig file here: http://github.com/lstoll/iphone-carrier-settings/tree/master, it’s pretty easy to set it up yourself. Then, just email the file to your phone, and open it. Enabling MMS is easier – On your phone, go to Settings → General → Network → Cellular Data, and enter the following details in the dialog:

MMS APN: three.co.uk
MMSC: mms.um.three.co.uk:10021/mmsc
MMS Proxy: mms.three.co.uk:8799

After I entered this I had to restart my phone – once it came back, MMS was working. Check by making sure there is a camera icon in the new message box, like this:

and try sending a message.

Announcing the rb-gae-support gem

2009-04-14T00:00:00+00:00

Announcing the rb-gae-support gem

14/04/2009 – Salzburg, Austria

Update:
Some googlers have gone out and done a much more complete verion of this, so this project is now defunct. see http://code.google.com/p/appengine-jruby/

Recently Google announced Java support for Google App Engine – which opened the floodgates for me. I’ve always loved the idea of GAE, but have never been that in to Python. Now however not only can I work in Java, but also other JVM languages, including one of my favorites, Ruby. GAE is not without limitations though, so some things have to be done a little differently especially HTTP access. It also provides some useful services, like User authentication and Memcached. To make this all easier, I’ve started on a gem called rb-gae-support. At the moment it is very alpha, however it monkey patches enough Net:HTTP to get it working on GAE (tested with a few gems). It also provides easier wrappers for the User and Memcache services, to make it fit in a little better.

It’s still in active development, so expect it to become much more stable and have more functionality soon. Patches are always welcome too! I’m also working on a template to make it easier to get started with and develop GAE apps, that will avoid all the restarts in the dev env. Hopefully, I’ll wrap this into some kind of generator.

For now, you can install the gem



  sudo gem install rb-gae-support

, see the RDoc at http://rb-gae-support.rubyforge.org and grab the code from http://github.com/lstoll/rb-gae-support. The template app I’m working on can be found at http://github.com/lstoll/rack-gae-template.

So please try it out, and let me know how you go!

Triple J on your iPhone.

2009-03-14T00:00:00+00:00

Triple J on your iPhone.

14/03/2009 – Islington, London, UK.

A month or so ago, I set up a little site that let’s you stream Triple J to the iPhone. I’ve since been using it more, so I’ve decided to finally release the code, and push the site a little more.

Firstly, you can visit the site on your iPhone at r.lstoll.net. Don’t try it on desktop Safari – it seems to hang. It currently only has Triple J, however I can add more stations on request – providing they have a MP3 stream available.

I’ve also pushed the code up to http://hg.lstoll.net/r.lstoll.net. It’s a pretty basic Sinatra application, pulling the config out of a yaml file. It could probably use some improvement, but it works so I’m happy. It demonstrates a custom body object to stream data back to a user, you can see it in lib/radio_streamer.rb. Unfortunately some ruby servers don’t support streaming the response, notably thin and passenger < 2.1.1, so if you plan on playing with it I’d recommend using mongrel or the latest passenger.

Another interesting point in the code is that it’s a Sinatra app, developed in Netbeans – including running and debugging, something which there isn’t too much info out there about. If you’re trying to develop Sinatra apps in Netbeans, it’s worth a look at the layout.

The code definitely needs some enhancements – currently each user stream opens a new stream to the source, which isn’t terribly efficient, plus stream transcoding so WM and AAC sources can be added would be good too.

Darwin Streaming Server on Ubuntu.

2009-03-13T00:00:00+00:00

Darwin Streaming Server on Ubuntu.

13/03/2009 – Islington, London, UK.

Update 3 (11/11/2009):
I’ve moved the download links to http://github.com/lstoll/dss/downloads – please see there for more information.

Update 2 (24/10/2009):
I’ve got a 64 bit version of the package available now as well. Please note it has had very little testing – please try it out, and report any bugs you find. You can download it here: http://github.com/lstoll/dss

Update:
There is now an issue tracker at http://github.com/lstoll/dss/issues

I’ve made available a pre-built .dpkg for running the latest version of Darwin Streaming Server on Ubuntu. This has been tested on 8.04, will probably work on other versions. Currently only supports 32bit versions – if you only have servers running 64bit Ubuntu, you can use xen to run a 32bit instance under this. It is mostly the Apple released code, with a few linux compatibility patches from the mailing list and some changes by myself to make it work behind Cisco load balancers, and to be properly packed for Ubuntu (including startup items). It’s been running in production for a while, so should be fairly stable. You can download the binary package (you will need to gunzip it first) from http://github.com/lstoll/dss/downloads, access the source at http://github.com/lstoll/dss/, and report any issues relating to this build to me lstoll@lstoll.net. This isn’t a replacement for the Apple bug tracker – things should also be reported there as well.

Stop Leopard changing your hostname

2009-03-11T00:00:00+00:00

Stop Leopard changing your hostname

11/03/2009 – Islington, London, UK.

One annoying thing in Leopard is that by default it is happy to take on the DNS (or other name resolution) provided hostname, which messes up the terminal prompt, and makes a whole bunch of applications assume that is your hostname, which is annoying when it’s something like “77-101-168-198”. This hasn’t been a problem for me in the past as I’ve usually been on a network behind NAT, but here I’m on a direct cable connection, and it is all over the place. To fix this, run the following (replacing ‘linc’ with your desired hostname)

  sudo hostname linc.local
  sudo scutil --set HostName linc.local
  sudo scutil --set LocalHostName linc

Once this is done, the name should stick permanently.

Shell Meme

2008-04-18T00:00:00+00:00

Shell Meme

18/04/2008 – Sydney, NSW, AU.

I can’t remember where I saw this first…

I ran

history|awk '{a[$2]++} END{for(i in a){printf "%5d\t%s\n ",a[i],i}}'|sort -rn|head

and got this:

100	cd
62	ls
60	git
28	ssh
26	hg
19	mate
17	psql
15	sudo
15	mv
14	rake

Looks like I’ve spent too much time looking for things, and not enough time working on them.

Passenger (A.K.A. mod_rails_)

2008-04-12T00:00:00+00:00

Passenger (A.K.A. mod_rails_)

12/04/2008 – Sydney, NSW, AU.

Passenger is now available for download – http://www.modrails.com . If you haven’t heard of it, is basically an apache module for running rails apps – nice and easily. I’ve just moved it over to this site, and it was incredibly easy. The install on my ubuntu server went like this:

sudo apt-get install apache2-prefork-dev libapr1-dev
sudo gem install passenger
passenger-install-apache2-module

I then added the apache config lines to the end of the apache config, and set my site up like:

<VirtualHost *:80>
ServerName lstoll.net
DocumentRoot /home/xxx/sites/lstoll.net/current/public
ServerAdmin xxx@xxx.net
ErrorLog /home/xxx/logs/lstoll.net_error.log
CustomLog /home/xxx/logs/lstoll.net_access.log combined
</VirtualHost>

Then, when apache started up it automatically started the rails app, and it just worked. It’s simple to use with capistrano, here’s my deploy tasks from deploy.rb:

namespace :deploy do
  desc "Custom restart task for passenger"
  task :restart, :roles => :app, :except => { :no_release => true } do
    run "touch #{deploy_to}/current/tmp/restart.txt"
  end
 
  desc "Custom start task for passenger"
  task :start, :roles => :app do
    # Don't need to do anything, it's automatic
  end
 
  desc "Custom stop task for passenger"
  task :stop, :roles => :app do
    # currently no way I know of to 'stop' the app
  end
 
end

Once that is done, capistrano can flag to passenger to restart the app.

A bit of casual benchmarking shows it’s fast too. I’m very happy this is out there, it makes running rails apps as easy as PHP – saving all the messing around with mongrel/thin/monit. It also works with rails caching, so so more messing around with rewrite rules to get cached files to be read right off disk. This should hopefully make rails better on shared hosts, and lower the barrier of entry. Only downside is that this is currently rails only – so for merb/camping/anything else the old way is still needed. I’m hoping that soon passenger will support rack, or mod_rack/mod_rubinius will come out, so this ease is brought to all ruby frameworks.

Stand-alone ActiveRecord Migrations

2008-04-07T00:00:00+00:00

Stand-alone ActiveRecord Migrations

4/7/2008 – Sydney

Update: Todd Huss has created a github project based on this – probably worth a look. You can find it here

Currently working on a merb project using datamapper – nice framework, but the auto-migrate feature is fairly limited – So I decided to hook up ActiveRecord’s migrations into the app. This code could be equally as easily adapted in to anything else.

First, I created a folder for the migrations to live in:

mkdir -p db/ar_migrations

I then appended a few tasks ripped out of Rails and slightly tweaked to the Rakefile:

These will read from database.yml in config/ , so make sure you’ve got datamapper set up.

Lastly, Lets create an example migration to create the datamapper sessions table in db/ar_migrations/001_create_dm_session_table.rb:

Once this is done, you can run rake db:migrate , and the DB will get the sessions table added. The db:schema:load task is also there, so you should use this when you first deploy your app to a new server. The naming is also consistent with rails, so capistrano should interact with it out of the box, with the exception that the environment to load/migrate needs to be called MERB_ENV instead of RAILS_ENV

When creating additional migrations you will need to follow the ‘proper’ naming scheme: the filename should start with an incrementing three digit number, and the rest should be a ‘flattened’ version of the class name. (i.e migration class AddPostTable becomes add_post_table).

Installing MySQL gem on Leopard with MacPorts MySQL

2007-10-30T00:00:00+00:00

Installing MySQL gem on Leopard with MacPorts MySQL

30/10/2007 – Sydney, NSW, AU.

The MySQL Gem won’t install directly on a Leopard machine using a Macports install of mysql (or any other MySQL install method I can think of) because by default gem will try built a universal gem, however the MySQL install will only have libraries for your current platform. So we need to force the architecture the gem is built for.

To install on a Intel Machine:

sudo bash -c "ARCHFLAGS='-arch i386' gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5"

To install on a PowerPC Machine:

sudo bash -c "ARCHFLAGS='-arch i386' gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5"

Make sure you select the newest ruby (not mswin32) version.
If you are not using a Macports MySQL Install, find where mysql_config lives and update the above command.

More info on the bundled Ruby under Leopard can be found here: http://trac.macosforge.org/projects/ruby/wiki/WhatsNewInLeopard

Locale Errors on Ubuntu

2007-08-07T00:00:00+00:00

Locale Errors on Ubuntu

7/8/2007 – Sydney, NSW, AU.

I set up a new Ubuntu XEN VM, and installed a few packages.. then started getting errors like this all over the place:

perl: warning: Setting locale failed.

perl: warning: Please check that your locale settings:

LANGUAGE = (unset),

LC_ALL = (unset),

LANG = 'en_AU.UTF-8'

are supported and installed on your system.

perl: warning: Falling back to the standard locale ().

locale: Cannot set LC_CTYPE to default locale: No such file or directory

locale: Cannot set LC_MESSAGES to default locale: No such file or directory

locale: Cannot set LC_ALL to default locale: No such file or directory

Seems that at some point all the entries get screwed up for setting locales. I ran sudo apt-get install language-pack-en-base , and then everything was fineso this ease is brought to all ruby frameworks.

VMWare Server Console Username/Password on OpenSUSE 10.2 x64

2007-05-12T00:00:00+00:00

VMWare Server Console Username/Password on OpenSUSE 10.2 x64

12/5/2007 – Sydney, NSW, AU.

I tried installing VMware Server 1.0.3 onto OpenSuse 10.2, and came across an issue – It wouldn’t let me log in as root (or anything), telling me that I had an invalid username or password. To get it working I had to use yast to install pam-modules-32bit, then edit /etc/vmware/pam.d/vmware-authd , and replace \%pamdir\% with /usr/lib/vmware/lib/libpam.so.0/security . You should then be right to go, straight away!