jLinq Reloaded

[jLinq Beta is now online – go check it out!]

My first ‘open source’ project I ever released was a Javascript library that allowed you perform LINQ style queries with arrays of objects. The library supported operators, extension methods and even memorized commands to reduce typing.

Overall, it did a good job of making it easier to select information that normally would need to be handled with a for loop. Unfortunately, it turns out jLinq is rather slow. I recently blogged about a performance evaluation that came back with less than flattering results.

However, this is exactly where ‘open source’ helps make better software. If I hadn’t released my code then I might never know how it could be improved. This is the sort of thing that inspires software developers to improve their code — and that is exactly what is did for me.

So for the past few days I’ve been completely rewriting jLinq from the ground up and the gains have been incredible to say the least.

1. About 60% smaller (around 8kb compressed)
2. 99% faster queries
3. Easier to extend library
4. Minty fresh scent (your results may vary)

If you’re looking for a download link then you’re going to have to contact me instead. The code should be released soon but for now I’m still writing tests and preparing to rerelease the project with a brand new site (and probably a new name).

[http://github.com/hugoware/jlinq-beta/]

There has been a lot of interest already in seeing the new code so I’ve started a new repository on github.com. This is beta code so I suspect I’ll have errors in it. Please feel free to contact me to log an issue on the site.

If you’re interested in how jLinq improved so much then read on…

jLinq… But On Caffeine

My first theory about why jLinq was so slow had to do with the use of evals in my code.

At the time, I didn’t understand you could pass functions as arguments, I certainly had no idea what enclosures were, so jLinq would build an entire query in a string and then eval it into a method that could be invoked against each record.

I also used eval to get values from records instead of just referring to them by their names as an index. I don’t know why eval is so much slower, but the difference is undeniable.

//using index names
var fast = function(obj, path) {
    path = (path+"").split(/\./g);
    var index = 0;
    while(obj != null && index < path.length) {
        obj = obj[path[index++]];
    }
    return obj;
};

//being lazy with an eval
var slow = function(obj, path) {
    return eval("obj."+path);
};

//looping 100,000 times
loop(100000, function() { fast(data, "name.first"); }); //128ms
loop(100000, function() { slow(data, "name.first"); }); //6.8 minutes

So what kind of improvement do these two changes result in? Big ones.

Single Argument Query (850 Records)
Previously: ~430ms
Now: ~10ms

Multi-Argument Query with Strings (850 Records)
Previously: ~2730ms
Now: ~35ms

Simple Join (850 Records)
Previously: ~6200ms
Now: ~135ms

The list of improvements goes on but I’ll talk about them more in later blog posts.

So What Is Next?

It’s screaming fast and much smaller but unfortunately it is too new to even suggest that it is put into production. I’ll definitely feel more confident in the code after I’ve finished the tests.

But I also plan to rebrand the library this time. Why?

As it turns out, the name LINQ has caused some confusion with non-.NET developers. I’ve heard that some developers see the name and say ‘Well, I don’t use .NET so I don’t need this.’, which is clearly not the response I’d like to hear.

jLinq isn’t only for web pages. In fact, the neat thing about jLinq is that it can be plugged into anything running Javascript and it works great. I’d like to see jLinq be exposed to more developers.

So, with a new name, new site and all brand new code I think there is a chance to redefine what jLinq can do for the community.

The code is mostly finished so now it’s time for the hard part… coming up with a good name…

Using Code Behinds For MVC Templates

I’ve done quite a few posts about using WebControls in MVC mostly as experiments to see if it was possible to bring WebForm functionality into a MVC application. It wasn’t really that I wanted to use WebForms in MVC but rather I still really like Controls having logical functionality. Sometimes a View is a little more complicated than simply inserting a value into the correct location on the page.

In my last blog post I gave an example that seemed impossible to do all within the render phase of the page but was easy to accomplish with a few WebControls. My controls didn’t have to dump out horrible markup or use the ViewState – They we’re simple and easy to use controls the rendered exactly the content I wanted to display.

So I’ve been working on a templating framework for MVC that can fit into an existing project without needing to start over that makes use of CSS Friendly Control Adapters – Remember those things?

Here is an example – (this is just some demo code and is missing some parts)

using System;
using System.Reflection;
using System.Web.UI;
using System.Web.UI.Adapters;

namespace MvcControlAdapters {

    /// <summary>
    /// Handles rendering controls to the page without IDs 
    /// and ensuring that DataBinding takes place on the page
    /// </summary>
    public class MvcControlAdapters : ControlAdapter {

        //perform data binding for the control
        protected override void OnLoad(EventArgs e) {
            (this.Control.Parent ?? this.Control).DataBind();
            base.OnLoad(e);
        }

        //create the HTML output for this control
        protected override void Render(HtmlTextWriter writer) {

            //clear this ID from view
            this.Control.ID = null;

            //check for an 'actualId'
            AttributeCollection attributes = this._GetAttributes();
            if (attributes is AttributeCollection) {
                attributes["id"] = attributes["_id"] ?? null;
                attributes.Remove("_id");
            }

            //perform normal rendering
            base.Render(writer);

        }

        //finds the attributes for a control (if any)
        private AttributeCollection _GetAttributes() {
            PropertyInfo property = this.Control.GetType().GetProperty("Attributes");
            return property is PropertyInfo
                ? property.GetValue(this.Control, null) as AttributeCollection
                : null;
        }

    }

}

Don’t compile and run just yet – You need to also add this XML to a new Browser File in your App_Browsers folder.

<browsers>
    <browser refID="Default">
        <controlAdapters>
            <adapter controlType="System.Web.UI.HtmlControls.HtmlControl" 
                adapterType="MvcWebControls.MvcControlAdapter" />
            <adapter controlType="System.Web.UI.Control" 
                adapterType="MvcWebControls.MvcControlAdapter" />
            <!-- snip... -->
        </controlAdapters>
    </browser>
</browsers>

The general idea behind this code is to attach an adapter onto any WebControl on the page and then either remove or replace the ID property with the correct value. This allows for the element to be referenced from the code behind but prevents the excessive ID from being displayed.

Keep in mind this is just a sample and won’t work exactly the way you’d like (for example, this doesn’t do anything about ‘name’ attributes on input tags).

In order to use this code we’re going to want to add a ViewPage that has a code behind that we can work with. You could add a script marked as runat=server at the top of the page but I don’t recommend it (which I’ll explain more of in a moment).

So here is a sample of what a page would look like now…

<%@ Page Language="C#" 
    CodeBehind="Index.aspx.cs" 
    Inherits="MvcAdapters.Views.Home.Index"  %>

<html>
    <body>
        <h1 runat="server" id="section" />
        <div runat="server" id="description" _id="desc" />
        <a runat="server" id="link" title="Check out the products!" >Learn More!</a>
    </body>
</html>

And then the code behind…

using System;
using System.Web.Mvc;
using MvcAdapters.Models;

namespace MvcAdapters.Views.Home {
    
    //the view for the page - STILL uses a Model
    public partial class Index : ViewPage<Product> {

        //prepare the content for the page
        protected override void OnLoad(EventArgs e) {

            this.section.InnerText = this.Model.Name;
            this.description.InnerHtml = this.Model.Description;
            this.link.HRef = this.Url.Action("Details", "Products", new { id = this.Model.ID });

        }

    }

}

And is finally rendered as…

<html> 
    <body> 
        <h1>Cheese Crackers &lt;3</h1> 
        <div id="desc">The best cheese crackers <strong>you've ever tasted</strong> - GUARANTEED!</div> 
        <a href="/Products/Details/18924" title="Check out the products!">Learn More!</a> 
    </body> 
</html> 

Which means we can assign the values for the model to the correct element on the page. We use properties like InnerText which will save us from calling Html.Encode or make logical changes to styles and attributes for elements without a mess of server side code blocks.

This might seem like an excessive amount of setup to populate content onto a page but it does have its advantages.

1. Refactoring is easier – If a model is changed then the modification can be easily pushed into the rest of the project without requiring you to dig though server code blocks. (yes, you have precompilation but you’re still the one who has to change them.)
2. View logic is cleaner – It is easier to write code to handle View logic outside of the view itself. For example, using a Visible property instead of an if-then statement wrapping HTML elements.

Of course, this might not make sense for every View, but the nice thing about this approach is that is works without needing to make a lot of changes to a project. This might make more sense in larger more complex views. Fortunately, an approach like this doesn’t require that you File->New Project to get it started.

Anyways, I’m still trying to decide what makes the most sense in this project so I’ll post more code as it moves along. Enjoy!

Software Design For Audiences – General versus Specific

So here is the problem — I’ve got limited key presses on my keyboard to make something awesome happen — even then, the awesome factor is debatable. I won’t be around forever so I need to build something the right way with as few mistakes as possible.

Which leads me to a problem I’ve found with every project I’ve worked on — Who is my target audience?

I know I’m not the first person to run into it, but I still have a hard time committing to a design until I’ve passed this hurdle — which I’m still stuck on as I write this.

General versus Specific

It’s a tough question. On one hand you can try to target a very broad audience — for example Yahoo Answers. You might get a lot more traffic because you cater to a much larger group of internet users. However, at the same time it seems that the more general you are, then the more competition you have — most likely with more programming resources on their side.

On the other hand you could try to hit a specific audience — for example StackOverflow.com. Same general idea, but more attractive to a specific group of people because they know that the content they are going to find is going to coincide with their interests. But unless your that specific group cares then you’re left with a product that no one cares about.

Is Specific Less Competition?

I was speaking with a guy where I work at the other day and we started talking about one of his friends who made it big in software development. He made big bucks selling a software package he had designed and was living it up in style while he worked on his next project. What was the software? Office management tools? A Facebook/Myspace style application?

No — Surprisingly it was software to manage State Fairs — You know, those carnivals that show up in your city from time to time. The thing was that there wasn’t any software out there already. Instead, the market was completely empty and ready for someone to swoop in and grab up all the customers.

Does “specific” targets equal to “less competition” — I’m not really sure, but it certainly seems that way.

Still Stuck

I’m still stuck trying to decide what to do in this situation. Do I go for the bigger audience, knowing that I have a higher amounts of competition — or the specific group where I have a much higher chance missing my mark?

What do you think — which audience should be the target of programmers with limited resources?

jLinq – Going Online!

This blog is focused on being about general development along with a lot of discussion about my projects I’m currently working on. I’ve cleared out all of my other blogs and created this one so it can be a broader discussion about development than the others could be.

jLinq is the main project I’ll be posting about for now. jLinq is similar to LINQ for .NET, but it is used in Javascript. It’s especially useful for working with medium sized arrays of information that won’t be changing.

Most of the time people use AJAX to make calls to their server and download the information they need. jLinq allows you use many of the same functions you use in LINQ, but against existing data on the page. 

One of the most important parts of jLinq was that I wanted it to be easy to extend. In fact, at one point the whole jLinq library was wiped and started over and built around an extensibility framework! I wanted the extensibility to work so well that even the core functions of jLinq was built around it!

I’ll blog more about using jLinq, but for now the best thing you can do is try it out on my website! Please leave me feedback about what you think!

Documentation is still in progress, but there are a lot of handy tutorials that will get you started!