I've got the Power!!

---

W00t! I’ve just received my invite into the Powerset Labs community! I’m totally stoked up as I’m been waiting for quite a few months now to get my hands on some of Powersets next generation search.

I’ve only taken a quick look around the labs site but it’s pretty obvious that some decent amount of thought went into it’s development. One of the nice features in the discussion forums is the ability to mod discussions ala digg, but obviously I can pretty much guarantee that the comments will be a lot more constructive than those found on the former site mentioned.

I’m off to play now. Don’t be suprised if I can’t be found for a few days;).

Meta

Posted by jonathan 7 months ago

0 comments »

Uploading and feedback with Nginx

---

I was in the office the other day, you know doing the usual drinking coffee and unleashing my usual carry on’esque humour to those unfortunate enough to be listening. When the conversation turned to file upload strategies and progress reporting solutions in Rails.

Now for various reasons that have been talked about by other people before, Rails as a solution to uploading large files was instantly rejected. The usual answer to this problem is to use a bank of merb’s and feed back progress to the client. It was during this conversation that Mr Tanner brought up the subject of the Nginx upload progress module.

Now we’re big fan’s of Nginx because it’s fast, lightweight and easy on resources. So I immediately looked into this. This module basically counts the number of bytes it’s received, and allows the progress of the upoad to be retrieved by the client via AJAX and JSON. After the file upload is complete Nginx then pushes the payload to the upstream server. This get’s around the problems people have had using Nginx with merb/upload progress. The upload progress feedback is pushed to Nginx instead which for me would allow more flexibility in my choice of what the upstream server/application will be. Merb’s great. It’s fast and lightweight, but if you’re using it just file uploads then it could be a bit of a waste. It would be simple to write a simple mongrel handler to deal with the file uploads or use the soon to be released Wisteria a Ruby micro framework by Kirk Haines.

Wisteria is a micro framework that doesn’t try and do anywhere near as much as Rails or Merb and because of this it’s managed to perform approx. 2000 req/s for a simple hello world application. These figures make this a framework to definitely consider in the future if you’re doing large file uploads or dealing with requests that need high performance but also need non of the polish that merb or Rails provide.

Of course there is the problem of validation if you want to verify both the upload payload and any other fields such as descriptions, tags etc. In my architectures I always push off as much as I can in a non synchronous way. So in the domain of a popular video sharing site that has to accept uploaded videos, with a description, some tags and then transcode the video into the appropriate format. I would probably validate the incoming data, push this onto a message queue (albeit without the actual file payload of course) to be dealt with by another process when resources are available.

Now the conundrum for me is to do with validation. In the lightweight process that’s dealing with the file upload I probably don’t want to be loading the entire model of the ORM I’m using such as ActiveRecord or DataMapper. However I do want to use the validations that are part of the model. Wouldn’t it be great if I could just mixin my validations into my ORM model and share them also with a lightweight DAO that I’m just using a a temporary container? This way I wouldn’t be duplicating business logic.

To my knowledge the AR validations are too tightly coupled with AR for me to do this in a lightweight manner. However I’m sure that I could do this with DataMapper because of the way it was developed using BDD. I’ll experiment more tonight to see if this is indeed feasible.

Meta

Posted by jonathan 7 months ago

3 comments »

Symbol shortcut

---

I just found this out today which is pretty sweet. Sometimes I need a symbol but compose it using a string and then call to_sym on that. Turns out there’s a shorter more concise way of doing this:

   @ivar2 = "_two" 
   :"my_string#{@ivar2}" 

This returns a symbol :my_string_two. Pretty cool I thought.

Meta

Posted by jonathan 7 months ago

2 comments »

Validate with DataMapper

---

So, a week or two ago I was modelling something with DataMapper and noticed that I needed some validation that went beyond the built-in validation macros. Now I could of just created my own validation macro as DataMapper makes this incredibly easy to do so, but I just needed something to get my specs to pass and go green.

Being a long time ActiveRecord user I immediately brought up the API documentation and went looking for the validates method. In AR, if you have something that doesn’t fit into one of the validation macros you simply override the validate method and push your errors onto the errors array. After a bit of poking around in the docs I couldn’t find anything that would fit my needs so I loaded up the DataMapper source code in TextMate and went searching. In order to make it easy for us AR old timers to switch over to DataMapper the authors were kind enough to create an AR impersonation module that wraps a few DataMapper methods in AR like methods. One of the is the save method and that’s where I noticed that DM called the valid? method before persisting the database session.

Now I have no idea if this is the official or the best way to do it but seeing this I realised that simply overriding the valid? method in my model, pushing my errors onto the errors array then calling super seemed to work fine for me. Now I’ve got to warn you that this relies on the internal workings of DataMapper and this could break in the future so use with caution. Of course you’re already writing specs/tests so this shouldn’t be too much of a problem:). By the way when adding errors if you have no attribute to bind the error to then passing in the context of :general is the recommended way of doing this.

Meta

Posted by jonathan 7 months ago

0 comments »

Subtle difference between DataMapper & ActiveRecord

---

So today I was messing around with DataMapper and something weird happened. I was porting some of the model code over from the Restful_auth plugin by Rick Olson to DataMapper when I hit a bug, or so I thought.

I’d was trying to change the user’s password and then calling the save method, but each time I did so it would return false. Now the first thing I did was to check the errors attribute on the model, which strangely was empty. At first I scratched my head for a minute puzzled at what was going on when suddenly it occurred to me the foolishness of what I was doing.

The thing is that DataMapper is smarter than ActiveRecord.. Alot smarter. One of the clever things it does is only update rows in the database that have changed unlike AR which saves the entire record to the DB even if only one field has changed. Not very performant. Another thing DataMapper does which is related to this is that it will persist the model to the database if any attribute changes that it manages when you explicitly call save otherwise it won’t do anything and just return false. Aha! There’s a hint. Now the model code from the Restful_auth plugin uses a ‘virtual’ attribute called password which is basically the password in a state of plaintext before it’s encrypted, assigned to the crypted_password attribute and saved. Because the password attribute on the model is not persisted to the database and therefore not strictly managed by DataMapper the model was never marked as dirty when I changed the password attribute. In fact because the model wasn’t marked as dirty the before_save callback also wasn’t being called either so the crypted_password attribute which was managed by DataMapper was not altered either. As explained before, this wouldn’t have been a problem if I’d used AR as it’s pretty dumb when it saves.

So the lesson of this tale is that you should treat DataMapper with more respect, beware of all it’s subtleties and not treat it like it’s impaired cousin ActiveRecord.

Meta

Posted by jonathan 8 months ago

0 comments »