jsshell 3.0

This weekend I finished up an upgrade for jsshell to version 3.0. I also put together a new jsshell landing page to help users get started.

New Stuff In 3.0

1. jQuery and jLinq Upgrade: Both libraries have been upgraded to their newest versions. jQuery 1.4.4 and jQuery UI 1.8.7. jLinq has been upgraded to 3.0.2.
2. Auto Complete: Previously, jsshell would show css classes, IDs and tags that were on the page as a ‘auto-suggest” popup but you couldn’t select anything on the list. Now, jsshell allows you use TAB to select the currently highlighted item. You can use up and down to change the highlighted item.
3. More Auto Complete Sources: jsshell now shows suggestions for jQuery and jLinq commands, current IDs, tag names and css classes on the page and now some Javascript keywords and common CSS properties.
4. Remember Last Command: jsshell now remembers the last successfully executed command. If you browse to a new page or refresh the view, the last command you ran will still be waiting for you.
5. Access ‘file:///’ URL: Previously, Google denied the app from being released because it had access to the file URL. It seems they have changed the way this rule is handled so this version now includes the local file url.
6. A ‘Run’ Button – Finally…: The only way to run commands before was to press CTRL+Enter. Now there is a button that does the same thing.

Special thanks to @stevehebert for his great suggestions on improvements for jsshell.

Please feel free to let me know if anything isn’t behaving as expected.

Teaching Software Development With Games

Since an early age I was always interested in doing game development. I’ve made several efforts at developing games doing both the art and the code. I can honestly say that it was my interest in games that led me to become a software developer as a career.

Games have always been a great way to teach. You can see it applied to many different topics and to many different age groups.

So this last week I started on a project to do just that using Canvas and Javascript. The general idea behind the game is to build a better ‘bot’ to pilot your ship. The ships fight it out and the last one standing is the winner.

The logic for the ships, how they move, when they shoot, are all actually written by developers and then plug into the game.

Sounds complicated? Well that is up to the developer. In order to create a ship you only need to create a simple Javascript class.

var yourBotName = function() {
    
    //a visual name for your bot
    this.name = "Display name";

    //the preferred weapons for your bot
    this.weapons = ["lazer", "burst", "tracker"];

    // a method to update your bot
    this.update = function(gameState, botState) {
        //do something here
    };
};

And there you have it! Your bot is ready to go!

The developer is passed game and bot state information that they can use to determine how to behave. This means a developer can write their code to behave anyway they like. That means if you write good code then your bot will do great – if you write bad code (the kind that causes exceptions or tries to cheat) then your bot is penalized.

I have some more work to do to get the game in a web ready format (meaning people can upload their own bots, challenge others, etc…). But so far the game is coming along great.

Below is a quick video showing how the game works and what you can expect. Enjoy!

[Watch the preview video]

Null Instance Extension Methods

Checking for null can be time consuming and add a lot of extra noise to the code you write. In some ways it can reduce the readability of your code and obscure the purpose of your functions.

For example, check out all of the null checks required for this bit of code.

public static int GetVersion() {
	
	//check if this exists
	if (!File.Exists(PATH_APP_DETAILS)) { return DEFAULT_VERSION; }

	//try and load the document
	XDocument document = null;
	try {
		document = XDocument.Load(PATH_APP_DETAILS);
	}
	catch {
		return DEFAULT_VERSION;
	}
	if (document == null) { return DEFAULT_VERSION; }
	
	//find the element
	XElement release = document.XPathSelectElement("app/release");
	if (release == null) { return DEFAULT_VERSION; }
	
	//get the value
	XAttribute version = release.Attribute("version");
	if (version == null) { return DEFAULT_VERSION; }
	
	//convert the version
	int current = int.TryParse(version.Value, out current) ? current : DEFAULT_VERSION;
	return current;
	
}

Yikes… that is a lot of code for such a small task but what else can you do? You can’t just call methods on something that is null or your app is going to blow up… or can you?

Extension methods feel magical the first time you see them in action but ultimately they are just static methods that can be invoked from anywhere in your project.

But here is the neat thing — Since an extension method isn’t tied to the instance of a class then they can be invoked without an instance of the class.

Sound confusing? Here is an example of what I mean…

//create an extension method
public static class StringExtensions {
	public static void Test(this string value) {
		Console.WriteLine("Hello from a null test!");
	}
}

//and then call it
string nothing = null;
nothing.Test(); //ok!
nothing.ToUpper(); //crash

You’ll notice that the call to Test worked fine – If you put a break point in the method and check the value argument then you’ll notice it is actually null! We were able to call the method Test even though there wasn’t an instance of a string.

So how can this help us? Well let’s take the same XML problem from earlier in the post and create some extension methods that don’t care if the instance being invoked is null or not.

//helper extension methods for XML documents
public static class XmlExtensions {
	
	//loads an xml document to use
	public static XDocument LoadDocument(this XDocument document, string path) {
		if (!File.Exists(path)) { return null; }
		try { return XDocument.Load(path); }
		catch { return null; }
	}
	
	//uses an XPath to find an element
	public static XElement GetElement(this XDocument document, string path) {
		return document is XDocument ? document.XPathSelectElement(path) : null;
	}
	
	//returns the value of an attribute
	public static string GetAttribute(this XElement element, string name) {
		XAttribute attribute = element is XElement ? element.Attribute(name) : null;
		return attribute is XAttribute ? attribute.Value : string.Empty;
	}
}

Now let’s take a look at how we can solve the same problem as before but with these new extension methods…

public static int GetVersion() {
	
	//open the document (Mono)
	XDocument document = document.LoadDocument(PATH_APP_DETAILS);

	//It appears you have to do it like this in Visual Studio
	//XDocument document = ((XDocument)null).LoadDocument(PATH_APP_DETAILS);
	
	//find the element
	XElement release = document.GetElement("app/release");
	
	//get the version
	string version = release.GetAttribute("version");
	
	//convert and return the value
	int current = int.TryParse(version, out current) ? current : DEFAULT_VERSION;
	return current;
}

As far as I can tell, no matter what you throw at this code, you will either get the default version or the value in the XML document. This definitely reduces the lines of code count… but the clarity gains might be debatable.

1. A developer modifying this code might not realize the value could be null or could be a real object. If they call an instance method and the value ends up being null then the application is going to crash.
2. Some developers are going to scratch their heads when they read the line XDocument document = document.LoadDocument(...). Invoking a method on a value you’re declaring and should be null and returning the result it to itself? Huh?
3. The more forgiving your code is then the more prone to external errors it will be, which in turn will introduce subtle runtime errors in your applications. Sometimes it’s a better idea just to throw an exception and keep the program from running.

So, while this approach has benefits it also has pitfalls. Make sure to evaluate the risks before using something like this in your code. If implemented carefully this could reduce complexity in your application…

…or at least make for a neat parlor trick the next time you’re with your dev team.

3 Things To Know Before Using jLinq

Since I released the beta code for the new jLinq I’ve had several of the same questions show up. Here are some answers to help you get started.

Where Is ‘Manual’, Query Commands Are ‘Auto-pilot’

I’ve had a few questions where people start trying to use the where command and find that it isn’t solving anything.

As far as I can tell, a lot of other LINQ style Javascript libraries use Where as their query command and have the developer manually provide the comparisons. Granted, I don’t know the other frameworks very well (hint, I wrote my own) but a lot of the code samples look something like this.

var results = lib.From(data)
    .Where(function(record) { return record.name.substr(0, 1) == "a"; })
    .OrderBy(function(record) { return record.name; })
    .Select();

I’m sure that these libraries are doing a lot of stuff for you in the background, but the where command in itself seems like placing too much work on the developer.

jLinq approaches this differently. Instead, you’ll find many common query methods included as part of the library. These methods also evaluate the data types to determine the best way to query the property.

Here is an example, consider you have some data that looks like this.

var data = [{
    name: "James",
    permissions: ["read", "write", "delete"]
},
/* snip... */];

Instead of checking these records manually, you can use the same command for both properties and they will handle the data differently for each.

//with string types
jlinq.from(data).contains("name", "e").select(); //matches the 4th 'e' in 'James'

//with array types
jlinq.from(data).contains("permissions", "write").select(); //matches the 2nd 'write'

This reduces the typing you do since you no longer have to perform manual comparisons.

Of course, jLinq does include a where command that uses the same syntax as the example above (anonymous function), but it is recommended to create extension methods instead of using the where command.

Operators Are Query Commands Too

Operators are an important part of jLinq. Sometimes you’d like to match on more than one condition or ignore records that do not meet a certain condition. You can use operators as individual commands in your queries.

//Examples only - Shorter syntax explained in Section 3

jlinq.from(data)
    .starts("first", "a")
    .or()
    .starts("first", "b");

jlinq.from(data)
    .not()
    .starts("first", "d")

jLinq takes operators a step further by creating operator aliases for all of the query commands in your library. This means that instead of only having the query command starts you actually have 6 – starts, orStarts, andStarts, notStarts, orNotStarts, andNotStarts. The queries above can be rewritten as.

//Examples only - Shorter syntax explained in Section 3

jlinq.from(data)
    .starts("first", "a")
    .orStarts("first", "b")
    .select()

jlinq.from(data)
    .notStarts("first", "d");

It is worth noting that and is the implied default unless you provide an or – the and versions of the query commands are for aethestetics only. 🙂

DRY — I Repeat, DRY!

If you aren’t familiar with the term DRY, it means “Don’t Repeat Yourself”. The previous samples reduced some of our typing, but jLinq can actually take it a step further.

jLinq uses field and command memorizing, meaning after you say it once it is implied until you say something different.

jlinq.from(data)
    .starts("first", "a")
    .or("b")
    .or("c");

jlinq.from(data)
    .starts("first", "a")
    .ends("b")
    .or("last", "a");

The first example memorizes the command being used is starts and the field being queried is first. After that, the remaining calls to ‘or’ will repeat the previous command and field unless something else is used.

The second example shows how you can mix up commands and field names without needing to repeat yourself. Basically, here is what the query is doing.

1. Records with first names starting with ‘a’…
2. then records with first names ending with ‘b’…
3. then records with last names ending with ‘a’

You can see that jLinq makes it easy to avoid repeating the same commands and field names to reduce the typing.

This is just some of the first basics to understand about jLinq. I’ll be blogging more (and writing documentation) for the next few days. Feel free to contact me with questions.

jLinq Reloaded

[jLinq Beta is now online – go check it out!]

My first ‘open source’ project I ever released was a Javascript library that allowed you perform LINQ style queries with arrays of objects. The library supported operators, extension methods and even memorized commands to reduce typing.

Overall, it did a good job of making it easier to select information that normally would need to be handled with a for loop. Unfortunately, it turns out jLinq is rather slow. I recently blogged about a performance evaluation that came back with less than flattering results.

However, this is exactly where ‘open source’ helps make better software. If I hadn’t released my code then I might never know how it could be improved. This is the sort of thing that inspires software developers to improve their code — and that is exactly what is did for me.

So for the past few days I’ve been completely rewriting jLinq from the ground up and the gains have been incredible to say the least.

1. About 60% smaller (around 8kb compressed)
2. 99% faster queries
3. Easier to extend library
4. Minty fresh scent (your results may vary)

If you’re looking for a download link then you’re going to have to contact me instead. The code should be released soon but for now I’m still writing tests and preparing to rerelease the project with a brand new site (and probably a new name).

[http://github.com/hugoware/jlinq-beta/]

There has been a lot of interest already in seeing the new code so I’ve started a new repository on github.com. This is beta code so I suspect I’ll have errors in it. Please feel free to contact me to log an issue on the site.

If you’re interested in how jLinq improved so much then read on…

jLinq… But On Caffeine

My first theory about why jLinq was so slow had to do with the use of evals in my code.

At the time, I didn’t understand you could pass functions as arguments, I certainly had no idea what enclosures were, so jLinq would build an entire query in a string and then eval it into a method that could be invoked against each record.

I also used eval to get values from records instead of just referring to them by their names as an index. I don’t know why eval is so much slower, but the difference is undeniable.

//using index names
var fast = function(obj, path) {
    path = (path+"").split(/\./g);
    var index = 0;
    while(obj != null && index < path.length) {
        obj = obj[path[index++]];
    }
    return obj;
};

//being lazy with an eval
var slow = function(obj, path) {
    return eval("obj."+path);
};

//looping 100,000 times
loop(100000, function() { fast(data, "name.first"); }); //128ms
loop(100000, function() { slow(data, "name.first"); }); //6.8 minutes

So what kind of improvement do these two changes result in? Big ones.

Single Argument Query (850 Records)
Previously: ~430ms
Now: ~10ms

Multi-Argument Query with Strings (850 Records)
Previously: ~2730ms
Now: ~35ms

Simple Join (850 Records)
Previously: ~6200ms
Now: ~135ms

The list of improvements goes on but I’ll talk about them more in later blog posts.

So What Is Next?

It’s screaming fast and much smaller but unfortunately it is too new to even suggest that it is put into production. I’ll definitely feel more confident in the code after I’ve finished the tests.

But I also plan to rebrand the library this time. Why?

As it turns out, the name LINQ has caused some confusion with non-.NET developers. I’ve heard that some developers see the name and say ‘Well, I don’t use .NET so I don’t need this.’, which is clearly not the response I’d like to hear.

jLinq isn’t only for web pages. In fact, the neat thing about jLinq is that it can be plugged into anything running Javascript and it works great. I’d like to see jLinq be exposed to more developers.

So, with a new name, new site and all brand new code I think there is a chance to redefine what jLinq can do for the community.

The code is mostly finished so now it’s time for the hard part… coming up with a good name…

Quirks Of Using For Each

Collections can be tricky sometimes when you’re writing code. This is especially true when you need to modify them while you’re looping through them. Let’s look at a couple examples that can get you into trouble.

//create a typical list of items
List<string> items = new List<string> {
    "first", "second", "third", "fourth"
};

//now clear the list
foreach (string item in items) {
    items.Remove(item);
}

Of course, this won’t work. Why?

InvalidOperationException: Collection was modified; enumeration operation may not execute

Once our loop has started the boundaries of the collection have been defined. Adding or removing items will change those boundaries and create errors. Naturally, most languages aren’t going to like this and will crash.

Fortunately, we do get an Exception message (which is the best kind of error message). However, we should be more concerned about subtle problems that could arise if we change a collection while looping through it. For example, consider the ForEach method attached to Generic Lists…

//get a list of the employees that 
//should be deleted from the system
List<string> firedEmployees = new List<string> {
    "jeff", "blake", "steve"
};

//and delete them
firedEmployees.ForEach(employee => {
    EmployeeSystem.Delete(employee);
    firedEmployees.Remove(employee);
});

Everyone deleted? Nope…

I don’t know about you, but I was surprised to find that one employee remained on this list after the code was run (the second employee at that). ForEach made sure to keep track of our collection as it changed but unfortunately it created an undesirable result.

To take it a step further, if you were to add a new item each time then you’d end up with an infinite loop, an error that would be caught sooner but a problem all the same.

This straight forward code has potential for some serious problems.

So, abandon the framework! Move to Java! .NET is doomed!

…Or we can just write an extension method to take care of both problems at the same time.

public static class EnumerableExtensions {

    /// <summary>
    /// Safely enumerates through a collection
    /// </summary>
    public static IEnumerable<T> Each<T>(this IEnumerable<T> collection, Action<T> action) {

        //move the collection into its own list and
        //perform the work on each item
        collection.ToList().ForEach(action);
        return collection;

    }

}

The fix itself isn’t very impressive. One additional call to ToList() and we’re pretty much okay. But this does keep the original collection safe while we enumerate through the protected one.

Almost Sorta Real Dynamic in .NET

This post discusses the basics of writing your own dynamic type – If you’re interested in the finished code make sure to download the source.

The new dynamic keyword is a great way to create and modify a dynamic object in your .NET code – For example, check out this bit of code.

dynamic counter = new {
    sent = 0,
    received = 0
};
counter.sent++;

… Oh… wait… That doesn’t actually work…

That’s because even though you have an object declared as dynamic, it is still just the same old object it always was under the hood. Maybe not the dynamic utopia we were expecting. Seems like plain ol’ Reflection with a slightly cleaner syntax.

Bummer…

However, Microsoft was awesome enough to actually allow us to create our own dynamic types by simply inheriting from the DynamicObject class. This post looks at how we can create a semi-dynamic object by handling the binding methods ourselves.

[Source Code: Dynamic.cs]

Our goal is to create an object that we can add, remove and change values on the fly. It ends up being a little easier than we expect. Let’s look at the minimum code we are going to need to get started.

public class SimpleDynamic : DynamicObject {

    //contains the values to use
    private Dictionary<string, object> _Values = new Dictionary<string, object>();

    //handles getting values from our object
    public override void TryGetMember(GetMemberBinder binder, out object result) {
        //...
    }

    //handles assigning values to our object
    public override bool TrySetMember(SetMemberBinder binder, object value) {
        //...
    }

}

Note: If you downloaded the source code above you might be wondering why these examples don’t look the same. The source code is the complete Dynamic class while this post only goes over the basics of using the DynamicObject class.

In order to create our dynamic object we must handle getting and setting values. We can use a simple Dictionary<string,object> to hold our values until we need them.

//handles getting values from our object
public override void TryGetMember(GetMemberBinder binder, out object result) {
    return this._Values.TryGetValue(binder.name, out result);
}

//handles assigning values to our object
public override bool TrySetMember(SetMemberBinder binder, object value) {
    this._Values.Remove(binder.name);
    if (value is object) { this._Values.Add(binder.name, value); }
    return true;
}

Not much code, but now our dynamic object is a little more dynamic than it was before. We can now work with properties a lot easier.

//create the new object
dynamic simple = new SimpleDynamic();

//add and update some values
simple.red = "#f00";
simple.green = "#0f0";
simple.count = 10;
simple.add = new Action<int>(value => simple.count += value);

//and check the values
Console.WriteLine(simple.red); // #f00
Console.WriteLine(simple.green); // #0f0
Console.WriteLine(simple.count); // 10

//invoke methods
simple.add(5);
Console.WriteLine(simple.count); // 15

Of course this class isn’t that helpful since it’s only regular Dictionary without string indexes but it is interesting to see how you can make a ‘dynamic’ object with only a few lines of code.

As for the included source code, it has a few extra features that this class doesn’t have.

• Use Anonymous Types directly to build out entire objects.
• Support for the += operator.
• Enumeration through member names.
• String indexes (for both single and nested properties).

Here are a few examples the Dynamic.cs class can do.

//create using anonymous types (and nested anonymous types)
dynamic info = new Dynamic(new {
    name = "Hugo",
    age = 30,
    settings = new {
        color = "orange",
        homepage = "website.com"
    }});

//add values using +=
info.create = new { value = false; }
info.nested.value.allowed = 55;

//enumeration of properties
foreach(string prop in info) {
    Console.WriteLine("{0}: {1}", prop, info[prop]);
}

I’d caution against using this a lot in your projects, but it is really interesting to see the kind of control Microsoft has built into the new DynamicObject class.

[Source Code: Dynamic.cs]

jsshell – Reputable Software??

Yes, it seems impossible so I took a screenshot. My little jsshell project has made it onto a pretty ‘notable’ list of Google extensions.

If you don’t know what jsshell then you might watch the introduction screencast.

Well… yeah, it isn’t spelled correctly… but I promise it is the same one.

In any case, you can check out the list on the Google site for yourself and possibly pick up some of the other awesome plugins while you’re at it. Don’t wait too long since it is only a matter of time before they realize their mistake 🙂

So, what better way to celebrate than to go ahead and open-source the code?

As usual, you’ll find my code out on git-hub, since its, ya’know, awesome… GitHub that is…

[Source Code]
http://github.com/hugoware/jsshell

Autocomplete? Yeah, But How?

Unless you’re really hardcore you probably use an IDE with some sort of autocomplete feature. It doesn’t have to be the super charged intellisense you find in Visual Studio – It could be as simple as the autocomplete feature in Notepad++ (which I love by the way…)

In any case, I got to wondering today – How do other people use their autocomplete features?

Best I could figure there was probably about 4 different ways you could approach using autocomplete…

1. Type just long enough to see the item you want and then use the arrow keys to correct item.
2. Keep typing, regardless of when it appears on the list, until it becomes the selected item.
3. Use both pretty much equally.
4. Auto complete is for noobs…

Yeah, it isn’t a very deep, through provoking sort of question but it is somewhat interesting to think about. Maybe there is some sort of deeper psychological meaning behind our choices… or not…

[Poll] : So, how do you use your autocomplete feature?

Stop, Think, Fix Errors… (Repeat!)

“Don’t change your code unless you know how it will fix your problem”. If I had to pick a quote to put a ‘squiggly line and my name’ next to, then this would be it.

I’ve seen developers stare blankly at an exception message, comment out a line of code or two and then try and run their code again hoping for the best.

This, of course, rarely fixes anything. Quite often the line of the error message has little to do with the actual problem. Instead, this introduces new errors into your code and causes the problem to take longer to resolve.

Even if you guess and make an exception message go away you still don’t know what caused the error to begin with. Instead, you have a new error that is quietly causing problems and is now much harder to find. They might be ugly, but exception messages are the easiest errors to fix… well, after compiler errors that is…

So, how can you “know” your change will fix the problem? Here are some of the recommendations I make to other developers.

Stop and Think… Carefully

The bug isn’t going anywhere and each time you guess incorrectly you add a new error to your code.

It seems like such a silly thing to even mention, but I’ve seen plenty of developers take a cursory glance at an exception message and then just start changing stuff. It doesn’t even matter if you’re in a “break-fix” situation – you should always carefully consider the problem before changing your code.

Read the error, start following up the stack and think about what could cause this result. Think about the conditions that would cause the error or how you could force the error to happen again.

If you don’t know why your change will fix the code then you aren’t fixing anything.

Understand the Debugger

You don’t need to know what a memory dump is for or how read binary manually to use debugger. Just simply understanding how to set break points and stepping into, over and out of code will probably cover 99% of the problems you encounter.

It might seem daunting at first but don’t ignore the debugger because logging or message boxes are easier to write. Once you get a hang of it, the debugger will most likely become the greatest tool in your future bug squashing adventures.

Isolate and Resolve the Specific Problem

It can be difficult at times to figure out why a section of code doesn’t work when it sits in the middle of a larger application. Sometimes its better to create an empty project and work with a smaller section of the program by itself.

For example, if you don’t understand how a part of your framework is supposed to behave, pull it into a separate program, for example a console application, and work with it until you are comfortable with the results. Then when you integrate your changes back into the main project you can be confident that any remaining errors aren’t part of the newly introduced code.

Look At Open Source Code

If you’ve written some code that is consistently causing problems then consider looking for a framework or open source project instead.

This isn’t a matter of being a good or bad developer. It certainly has nothing to do with being unable to figure out the problem for yourself. Quite simply, code that consistently has errors is probably a difficult problem.

Public frameworks and open source projects are typically owned by other developers that understand the problem better than others. They have taken the time to abstract away the specifics of a problem and build a clean API for other developers to use.

These developers are also typically eager to improve their code and will listen to feedback. Even if you don’t use their project there’s a good chance that you will learn something by simply reading their code.

To sum it up, don’t just change code and hope for the best. Stop, think, ask yourself how does this fix the problem?.

How do you approach fixing problems in your code?