Performing Updates With CSMongo

I got some good questions earlier today about how to perform updates on records in CSMongo. Rather than doing a short answer in a comment I figured a full post about it might be more helpful. While playing with this I made some changes to the code so if you have a previous version then you’re going to want to update to the newest version.

CSMongo has a couple ways available to update records right now. One is after you have made a selection from a database and the other is a query sent to the database.

Update After Selection

If you’ve used LINQ to SQL before then you know that you can load database objects into memory, modify them and then submit the changes and like magic your records are updated… maybe not *real* magic but I was still impressed the first time I saw it…

CSMongo allows for a similar approach when performing updates on documents loaded from the database. For example, here is how we could load a set of users, modify them and then issue back our changes.

//connect to the database
using (MongoDatabase database = new MongoDatabase(connectionString)) {

    //load a set of records
    MongoCollection collection = database.GetCollection("drivers");
    
    //select a set of records
    var drivers = collection.Find()
        .Greater("age", 16)
        .Select();

    //make some changes
    drivers.Apply(new {
        canDrive = true
    });

    //submit the changes
    collection.SubmitChanges();

}

This code load in a set of records and saves their reference to the collection (which is also managed by the MongoDatabase class in case you were wondering). This allows you to make changes to your object in multiple places and then call SubmitChanges to apply everything you’ve done.

When the record is first loaded a hash is created of the object which is used to check for changes which means that if you don’t change anything, or if values are set but not actually different then the update request is never sent.

It is also important to realize that MongoCollection.SubmitChanges() only sends updates for the collection that is called on whereas MongoDatabase.SubmitChanges() checks all of the collections that have been loaded and attempts to apply their changes. This is actually one of the advantages to using the MongoDatabase to create instances of your MongoCollection since it can automatically handle checking for changes.

In this last example we don’t actually use any of the information in the record which makes loading it into memory first rather pointless which leads us into the next type of update.

Immediate Updates

Sometimes when you want to change records you don’t want to have to load them first. Sometimes you simply want to perform and update for a bunch of matching records that meet a certain condition. In that instance you can issue an update immediately from a MongoQuery.

The example we used above is a good example where sending an update would be better than loading the records first. There isn’t a lot that changes but what happens in the background is much different.

//connect to the database
using (MongoDatabase database = new MongoDatabase(connectionString)) {

    //issue the query directly from the database level
    database.From("drivers")
        .Greater("age", 16)
        .Set(new {
            canDrive = true
        });

}

You can also perform an update directly from the MongoCollection by using the Find() command, which starts a MongoQuery that can be used for the update.

You may notice that this example doesn’t have a call to SubmitChanges() in it — That’s because when I say immediate then by golly, I mean right away! In fact, if I remember correctly ever command on the MongoDatabase level is evaluated immediately.

Anyways, CSMongo is still early in development so if anyone has some additional ideas how to handle these updates then I’m certainly interested in what you think.

CSMongo – An Introduction

Last night I did a short screencast explaining the basics of using Documents in CSMongo. The screencast goes over adding, changing and removing values along with getting access to the values you assign.

CSMongo On GitHub

I’ve been hammering away at the keyboard for a few weeks now and my beta version of CSMongo is now live on GitHub!

I’ve written a lot of documentation in for the project in the Wiki section on the site, so if you’re interested you can start reading the details of how the code works.

In this post I’ll just go over some of the more interesting things the CSMongo does to make working with Mongo easier than normal.

Anonymous Types and Dynamic Creation

Working with documents, including those with multiple levels, is a snap with CSMongo. You have many ways to make changes but each of these ways also allow you to create entirely new sections of your document.

MongoDocument document = new MongoDocument();

//add some new items
document += new {
    name = "Hugo",
    age = 30,
    settings = new {
        font = "arial",
        color = "orange"
    }
};

//set values for a lower level field
document.Set("browser.urls", new object [] {
    new { site = "google.com", favorite = false },
    new { site = "hugoware.net", favorite = true, icon = "blue-icon.ico" }
});

//or create new elements with an index path
document["blog.url"] = "https://somewebguy.wordpress.com";

//or use existing objects as a template
Uri address = new Uri("http://www.herdingcode.com");
document += address;

Nifty Anonymous Type Mapping Method TM

One thing that I didn’t like was always having to refer to elements using a string. I had a moment late one night … er… maybe it was early one moring… like around 1AM or so — Why can’t I use a return anonymous type value to act as a template for the values to use. Here is an example of what I put together…

//grab a document - Assume that the value returned
//looks like the JSON example below
MongoDocument document = database.Find("users").SelectOne();
//{
//    name : "Jimmy",
//    age : 50,
//    website : "http://www.hugoware.net"
//    settings : {
//        font : "arial",
//        color : "red"
//    }
//}

//provide an anonymous type as the template to return
//Notice the values provide match in some places but not 
//in other areas
var value = document.Get(new {
    name = "no name",
    age = -1,
    admin = false,
    favorite = Mongo.Null,
    settings = new {
        color = "black",
        version = 0
    }
});

//now we have an anonymous type with the values
//it could find and then the default values for the
//sections it couldn't find or was missing
value.name; // "Jimmy"
value.age; // 50
value.admin; // false (fallback)
value.favorite; // null (fallback)
value.settings.color; // "red"
value.settings.version; // 0

This means that we can refactor our code without needing to update a bunch of string values. You can even use this mapping option directly after a query so all of the values are converted into the new type automatically.

Lazy Collections

When you’re making changes to a MongoDatabase, for the most part all of the commands are immediate – meaning they connect to the database right away and perform their work. However, most commands inside of a MongoCollection don’t execute until you call the SubmitChanges method. This can allow you to queue up a few inserts and deletes or make changes to documents that have been loaded into memory and then update them all at once. The code below illustrates how it works.

//Inserting Documents
//=================================
MongoCollection collection = database.GetCollection("users");

//add a few documents
collection.InsertOnSubmit(new { name = "Hugo" });
collection.InsertOnSubmit(new { name = "Mary" });
collection.InsertOnSubmit(new { name = "Gwyn" });
collection.InsertOnSubmit(new { name = "Cassie" });

//nothing has been inserted yet, do it now
collection.SubmitChanges();

//Performing Updates
//=================================
MongoCollection collection = database.GetCollection("users");
var documents = collection.Find()
    .Greater("age", 50)
	.Select();
	
//make each of the changes
foreach(MongoDocument document in documents) {
    document += { isOld = true };
}

//documents are only changed in memory
//send the update to the server
collection.SubmitChanges();

//Deleting Documents
//=================================
MongoCollection collection = database.GetCollection("users");
var documents = collection.Find()
    .Greater("age", 50)
	.Select();
	
//mark each for deletion
foreach(MongoDocument document in documents) {
    collection.DeleteOnSubmit(document);
}

//documents are only changed in memory
//send the update to the server
collection.SubmitChanges();

Of course, you can perform a blanket update statement using a query from the MongoDatabase object itself, but this is a good example of how a MongoCollection can do it in a lazy way.

It is also worth mentioning that if you use the GetCollection method to work with a collection then it is also monitored by the database. This means that if you call the MongoDatabase method SubmitChanges then it will check each of the collections it is tracking and submit their changes automatically.

Try It Out!

If you’ve haven’t been sure if you want to try out MongoDB yet then now is a great time! The links below can help you get started!

Getting Started
MongoDB Website
Using MongoDB With Visual Studio

Source Code
CSMongo on GitHub
CSMongo Wiki Help on GitHub

CSMongo Driver – Part 1

My new project CSMongo, a Mongo driver for .NET, is now online! Check it out!

Lately, I’ve been spending a lot of my time writing a driver for Mongo in C#. The primary goal was to make it comfortable to work with dynamic objects in a static language — which is a project I’ve worked on before in the past.

I still have a lot of code to write but I’m getting close to a working version that I can get out the door. This post discusses how my Mongo driver works currently works.

Dynamic In Static-Land

The next version of .NET introduces the dynamic keyword but until then I needed to build a class that worked with C# and was dynamic enough to work with Mongo. The main hurdle is that a ‘dynamic’ isn’t just a dictionary of key/value pairs. Instead, an object can be nested several level deep.

Additionally, data types between static languages and dynamic languages are also handled differently. Think about numbers for a second. In C# you have single, double, float, decimal, etc. – But Javascript? Yeah, you have Number — A heck of a lot easier to deal with but can cause problems when trying to pull a value back out of your object.

Below are some samples how you can create Mongo document with this driver.

//Creating And Assigning Vaues
//===============================
MongoDocument document = new MongoDocument(new {
	name = "Hugo",
	age = 29,
	admin = true
	});
	
//make changes and assign new values using +=
document += new {
	age = 30,
	settings = new {
		color = "red",
		width = 700
	},
	permissions = new string[] {
		"read", "write", "delete"
	}
};

//make changes using the path to a field
document["name"] = "Hugoware";
document["settings.color"] = "blue";

//or create new objects entirely
document["settings.urls.favorite"] = "http://hugoware.net";

//remove fields using -= and a string or string array
document -= "name";
document -= "settings.color";

//or remove multiple items
document -= new string[] { "age", "permissions" };

//also use typical methods to perform same changes
document.Set("name", new {
	first = "Hugo",
	last = "Bonacci"
});

document.Remove("settings", "name", "age");


//Merging multiple documents
//===============================

MongoDocument first = new MongoDocument();
MongoDocument second = new MongoDocument(new {
	name = "Hugo"
});

//merge either of the two ways listed
first += second; /* or */ first.Merge(second);
string name = first["name"]; // "Hugo"

This class allows for a lot of flexability to write to an object and make changes to it without needing to know about the object.

Going back to data types, each object has a value handler that manages working with the type and performing conversions. You are also able to provide a default value in case the property requested doesn’t exist or can’t be converted.

Here are a few more examples of accessing values types.

//Accessing Typical Values
//===============================

//create an empty object and access not real values
MongoDocument document = new MongoDocument();

//accessing by default property returns an object
//NOTE: Using Get() is better for accessing values
int a = (int)document["not.real"]; // *crash* null
int? b = document["not.real"] as int?; // null

//accessing by Get<T>() returns default(T)
int c = document.Get<int>("not.real"); // 0

//or define a fallback value
int d = document.Get<int>("not.real", 55); // 55

//next examples are assigned '54.342' which defaults to 'double'
document.Set("value", 54.342");

//access by default property
double e = (double)document["value"]; // 54.342

//or use the Get method that allows a cast request type
double f = document.Get<double>("value"); // 54.342
int g = document.Get<int>("value"); // 54
bool h = document.Get<bool>("value"); // true
string i = document.Get<string>("value"); // "54.342"

//Accessing Uncommon Values
//===============================

doc["count"] = long.MaxValue;
int a = (int)doc["count"]; // *crash*
int b = doc.Get<int>("count"); // 0
long c = doc.Get<int>("count"); // 0
long d = doc.Get<long>("count"); // 9223372036854775807

doc["count"] = uint.MaxValue;
uint e = doc.Get<uint>("count"); // 4294967295
long f = doc.Get<long>("count"); // 4294967295
decimal h = doc.Get<decimal>("count"); // 4294967295
int g = doc.Get<int>("count", int.MaxValue); // 0 (no default!)

As you can see there are a lot of different ways data types like this could play out. Failed conversions tend to be easier to detect than incorrect conversions. I suspect this will improve over time.

If you don’t like the way conversions are handled for a certain type, or if you have a custom class you want special rules created for, can define your own custom DataTypes that handle the formatting, parsing and saving to the database.

Database Queries

If you’ve grown accustomed to SQL queries then you’re going to be in a world of hurt with Mongo. They make sense after a bit of time but not like the somewhat human-readable format of a standard SQL query.

In this driver I’ve attempted to to make a LINQ-ish style query builder. This builder is used in a variety of places depending on the purpose (such as selection as opposed to updating).

Here are some query examples.

MongoDatabase database = new MongoDatabase("100.0.0.1", "website");

//simple chained query
database.From("users")
    .Less("age", 50)
    .EqualTo("group", "admins")
    .Select();

//anonymous types for parameters
database.From("users")
    .In("permissions", "write", "delete")
    .Select(new {
        take = 30
    });

//updating without a record using a
//query selector and update arguments
database.From("users")
    .Match("name", "^(a|b|c)", RegexOptions.IgnoreCase)
    .Update(new {
        enabled = false,
        status = "Disabled by Admin"
    });

Mongo queries appear to be missing several pretty important features that might make it a little more difficult for you to find records. I figured out how to use jLinq directly within Mongo so I’m going to see if version two might include a little jLinq support – How cool would that be?

I’m hoping I’ll have the first version of my driver released sometime this week so check back for more news on the progress!

jLinq in MongoDB (Oh snap!!)

My new project CSMongo, a Mongo driver for .NET, is now online! Check it out!

Yeah, you read that right – jLinq in MongoDB!

Lately I’ve been working on a Mongo database driver and I got to thinking — Since I can use Javascript with MongoDB then I wonder if I could use jLinq to write queries? As it turns out the answer is yes!

… and if you don’t know what jLinq is… – jLinq is a javascript query language, similar to the LINQ project created by Microsoft, for JSON data which was designed to run within web pages using Javascript

Getting Started

I’m not really sure how you can get a Javascript library to load directly into MongoDB but for now I was simply copying and pasting the existing jLinq packed library into the command line and allowing it to evaluate the script — and that is it! You can use jLinq right away!

You should be aware though that you can’t just evaluate results directly. You actually need to call the toArray() function on each of them to see the results. So for example, your query would look something like…

jLinq.from(db.users.find().toArray()).startsWith("name", "h").select();

You don’t need to actually do anything with the results since they are automatically displayed on the screen for you.

So far everything works. I could do standard queries, use the OR operator, reorder records, join different databases, perform comparisons against sub-properties — really everything I’ve tried has worked just the way I’d expect it to!

Granted, database and collection names are going to be different, here are some interesting queries you can try with your database.

//Selects records where the name starts with a, b or c (case-insensitive)
jLinq.from(db.users.find().toArray())
    .startsWith("name", "a")
    .or("b")
    .or("c")
    .select();

//selects users older than 50 and are administrators then orders them by their names
jLinq.from(db.users.find().toArray())
    .greater("age", 50)
    .is("admin")
    .orderBy("name")
    .select();

//performs a join against another database to get locations
//and then queries the location to find users that live in 'texas'
//and then selects only the name of the person and location
jLinq.from(db.users.find().toArray())
    .join(db.locations.find().toArray(), "location", "locationId", "id")
    .equals("location.name", "texas")
    .select(function(rec) {
        return {
            person: rec.name,
            location: rec.location.name
        };
    });

More On jLinq

Unfortunately, it isn’t possible to go over all of the cool stuff you can do with jLinq – but here are some screencasts and blog posts that you can read that might help you get started.

jLinq Project Page
Screencast 1 – Getting Started
Screencast 2 – Extending jLinq (Custom commands)
Screencast 3 – Modifying Live Data In A Query (Joins, Assignment)
Screencast 4 – jLinq 2.2.1 (Updates)

Anyways, for now I’m not sure how to integrate jLinq into Mongo queries but keep an eye for future blog posts as I find out more.

Using MongoDB With Visual Studio

My new project CSMongo, a Mongo driver for .NET, is now online! Check it out!

I recently found out about MongoDB and decided to check it out and so far I’ve been really impressed. The most interesting thing about MongoDB that it doesn’t work like a normal database — there aren’t really any schemas for any of the “tables”. In fact, you can make changes to the structure of any record at any time without affecting the rest of the “table”.

Getting Started

Here are a few steps I used to get MongoDB running and testable from my Visual Studio. To keep my main computer clean I used a virtual instance of Ubuntu to host the “database server”.

This part might take a little bit of time but start by downloading VirtualBox and Ubuntu 9.10 (You don’t need to download Mongo just yet). Once everything is downloaded install and configure Ubuntu but don’t start it up right away (you need to configure some stuff).

You’re going to want to make sure that your PC can connect to the virtual instance on the standard MongoDB port (unless you change it of course). If not, open a command prompt (on the host system) and then run the following commands (from the VirtualBox directory).

VBoxManage setextradata UbuntuDev "VBoxInternal/Devices/pcnet/0/LUN#0/Config/MongoDB/HostPort" 27017
VBoxManage setextradata UbuntuDev "VBoxInternal/Devices/pcnet/0/LUN#0/Config/MongoDB/GuestPort" 27017
VBoxManage setextradata UbuntuDev "VBoxInternal/Devices/pcnet/0/LUN#0/Config/MongoDB/Protocol" TCP

This example uses the same name (UbuntuDev) that I used for the screenshot example above. Make sure you use the correct name when setting yours up.

It is also worth mentioning that I had to use a Bridged Connection for my Network connection so I’d get an IP address from my wireless. But, if you aren’t on a wireless network… say, like your in-laws house for several hours… you can use Host Only Adapter so you can keep working… just sayin’…

Inside Ubuntu

Once you’ve successfully setup your Ubuntu system go on and download the MongoDB binaries. I don’t know the correct location to install these files so I placed them inside /etc/mongodb.

If you’ve never used Linux before then warm up your typing fingers and open a terminal window (Applications > Accessories > Terminal). Start typing in the following commands…

sudo bash
mkdir /data
mkdir /data/db
mkdir /etc/mongodb
chmod a=rwx /etc/mongodb

Note: This is certainly not the recommended security setup for this folder but for our testing purposes it is sufficient.

At this point we can revert back to our lazy Windows ways and drag the contents of the .tgz file into the /etc/mongodb directory. Once we have the contents copied over switch back to the terminal window and then type…

sudo /etc/mongodb/bin/mongod

And you should see something like the screenshot below…

Once you see this message you should be ready to test from Visual Studio but you can always test it from Ubuntu by opening a new Terminal window and typing…

sudo /etc/mongodb/bin/mongo

Which allows you to enter commands and make changes to the database similar to the online demo on their website.

Connecting Via Visual Studio

I haven’t found much for C# code to connect to Mongo but there is currently a project hosted on GitHub that allows you to perform queries and edit documents. It is a mess of code – but it *does* at least work… mostly… (nothing personal guys) 😉 Here is a simple example of how to use the code…

//connect to the source (IP address of virtual)
Mongo mongo = new Mongo("100.0.0.1");
mongo.Connect();

//get the database and 'table'
Database website = mongo.getDB("website");
IMongoCollection users = website.GetCollection("users");

//make a new document of information
Document user = new Document();
user["name"] = "Hugoware";
user["age"] = 29;
user["isAdmin"] = false;

//then save the document
users.Insert(user);

If you still have your Terminal window up then you might have noticed messages listed in response to your update. If you still have the MongoDB command line up you can view your changes by entering in a few commands. For example, to see the database I just created in the sample above I would enter…

use website
db.users.find()

And I would get a response similar to this…

What Is Next?

Personally, I think MongoDB is going to end up being huge. The main problem I see for C# developers is that MongoDB really favors Dynamic Languages which isn’t really a strong suit of the language.

Right now I’m working on my own driver to talk to Mongo that heavily relies on my AnonymousType code (anybody remember that old stuff?). It is still early on in the project so if you’re interested in helping feel free to contact me.