No F*cking Idea

Common answer to everything

Building ORM/ODM Using Virtus for MongoDB Part 1

| Comments

This blog post is start of series on building ORM/ODM libraries for Mongodb in different languages. I am a big fan of mongoid http://mongoid.org great Ruby ODM for mongodb. But i am even bigger fan of Mongodb http://www.mongodb.org/. Some time ago i saw post by my friend from http://www.lunarlogicpolska.com/ about virtus http://solnic.eu/2011/06/06/virtus—attributes-for-your-plain-ruby-objects.html (it’s really worth reading) and i thought “it’s nice”. Today i have spent two hours to cook this starter project because i want to learn more about virtus and building ODM’s ORM’s is not so common topic across web.

Aim

Building fully featured ODM in two hours from scratch is more then you can expect from me. Goal if first part is to make something that will map ruby PORO’s to mongodb collections, be able to save them, find one or many in database and destroy. This looks like a lot but we will do it slowly and the implementation will be very basic. Our ODM will be named “Muppet”. Name describes the project :).

Virtus

Virtus handles properties for Plain Old Ruby Objects and this is all we need to have. This eliminates a lot of boilerplate code we would have to write to make anything work. PORO Powah!

Mongo

mongo is a Mongodb native ruby driver. It has really good api and is easy to use. For most things in this post i just proxied calls to it :).

Database!

Be sure to have mongodb working ;).

First step: Building a gem

All about building gem you can find in my previous blog post here http://no-fucking-idea.com/blog/2012/04/11/building-gem-with-bundler/ Be sure to add rspec :) i used it to describe tests for this project.

Second step: Describing api

I like do develop things in TDD/BDD style so first thing for me description of api i wanted to implement during this tutorial. All this specs are stripped to minimum to enhance readability.

spec/muppet_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

require 'spec_helper'

class User
  include Virtus
  include Muppet

  attribute :name, String
  attribute :age, Integer
  attribute :birthday, DateTime
end

describe Muppet do

  describe "#configuration" do

    it "should be configurable" do
      lambda do
        Muppet.configure \{ \{
          host: "localhost",
          port: 27017,
          database: "test"
        }}
        Muppet.config.should be_a(Hash)
        Muppet.config[:host].should eql("localhost")
      end.should_not raise_error
    end

    it "should by default point to localhost:27017" do
      lambda do
        Muppet.config[:host].should eql("localhost")
        Muppet.config[:port].should eql(27017)
      end
    end

  end

  describe "#connection" do

    it "should connect to mongodb" do
      lambda do
        Muppet.connect!
      end.should_not raise_error
    end

  end

  describe "#inserting" do

    it "should insert values to database" do
      pending
      lambda do
        user = User.new(name: "Jakub", age: 27 )
        user.save.should be_true
      end.should change(User, :count)
    end

  end

  describe "#quering" do

    it "should query all documents from collection" do
      pending
      User.find_one.should be_a(User)
    end

    it "should query first document from collection" do
      pending
      users = User.find
      users.should be_a(Array)
      users[0].should be_a(User)
    end

  end

  describe "#destroy" do

    it "should destroy document" do
      pending
      lambda do
        User.find_one.destroy
      end.should change(User, :count).by(-1)
    end

  end

end

I defined them in order how i wanted to implement this. First configuration and connection to db. Next inserting and querying and destroying as last thing. Also at top i have simple User object with virtus and muppet included.

Third step: layouting Muppet!

At this moment we should have ready specs, that fail hard throwing errors on unknown tokens. Its good. What i did is simple lib/muppet.rb defines the module we will include into out PORO’s.

muppet.rb
1
2
3
4
5
6
7

require "mongo"
require "muppet/version"
require "muppet/setup"

module Muppet
  extend Muppet::Setup
end

in lib/muppet/ we will have components of our project. I we already know we will start with setup as defined in muppet.rb so lets create file lib/muppet/setup and configuration and connection to mongodb.

lib/muppet/setup.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

module Muppet
  module Setup
    @@configuration = {
      host: "localhost",
      port: 27017,
      database: "test"
    }

    def configure(&user_config)
      @@configuration.merge!(user_config.call)
    end

    def config
      @@configuration
    end

    def connect!
      @@connection = Mongo::Connection.new(@@configuration[:host],
        @@configuration[:port])
      @@database = @@connection.db(@@configuration[:database])
    end

    def database
      @@database
    end

    def connection
      @@connection
    end

  end
end

Here i defined default configuration and few method to access vital part of our config like database , connection. Most important part is connect! this method uses mongo gem to establish connection to mongodb store it into connection variable and set the database we will be working on. I wanted to make few things explicit so i used some name redundancy. (later on i learned that i did not even need config method)

With this working we can run out rspecs and if mongodb is up we should see all green and few yellows! Good it works!

Forth step: support for quering and inserting

Now lets remove pending marks from specs that are in “describes” quering and inserting. This will be the heart of our ODM. We will define how he should save and load object from database. Before this we will have to update out lib/muppet.rb to include things we will use.

lib/muppet.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14

require "mongo"
require "muppet/version"
require "muppet/setup"
require "muppet/document"

module Muppet
  extend Muppet::Setup
  include Muppet::Document::InstanceMethods

  def self.included(base)
    base.extend( Muppet::Document::ClassMethods )
  end

end

Ok many new things. I created lib/muppet/document.rb module with the stuff we will put into class and instance definitions in the moment of inclusion. As we can see in definition of User in our test cases we will include Muppet so all the instance methods like (save, destroy) will have to be defined in separated module then class methods like (find_one, find). In document.rb we can see how it is implemented.

lib/muppet/document.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

module Muppet
  module Document
    module ClassMethods

      def collection_name
        self.to_s
      end

      def collection
        Muppet.database.collection(collection_name)
      end

      def find_one(opts = {})
        result = collection.find_one(opts)
        return nil unless result
        self.new(result)
      end

      def find(opts = {})
        results = collection.find(opts)
        results.map{|result| self.new(result) }
      end

      def count
        collection.count
      end

    end

    module InstanceMethods

      def save
        self.class.collection.insert(self.attributes)
      end

      def destroy
        self.class.collection.remove(self.attributes)
      end

    end

  end
end

I could not think of a way to implement it in a more simple way. We have to “sections” first is class methods where we define

  • collection_name this method says to us what is the collection name (we can override it in model)
  • collection uses database to return this collection object for us to use.
  • find_one, find, count are methods that we proxy in a dirty explicit way (but without using method missing) things to mongo native driver.

Only thing we do here is to wrap things that we get back into self.new object. So we can mimic AR/Mongoid api and when doing User.find we will get back array of all users. not array of hashes :).

Instance methods are only two save and destroy save is raw and simple, takes all attributes from using virtues and saves them using mongo native driver. destroy acts in the same way.

Now we can run the specs and see all is green. We have a basic ODM where we can add documents, map and query them to PORO’s and remove.

Summary

I really enjoyed writing this code and blog post :). You can find code for this blog post here https://github.com/JakubOboza/muppet

This code is buggy and even specs needs to be enhanced but this is a good start for building new features on top of it. In future parts i want to implement, updating, proxy objects, relations, embedded objects, dirty tracking(probably using https://github.com/solnic/virtus-dirty_tracking) and few other mechanism that will enable us to make a fully functional ODM from “Muppet”

-Cheers

Comments