Hugoware

The product of a web developer with a little too much caffeine

Search Results

Waiter, There Is Some jQuery In My jLinq…

leave a comment »

jLinq occasionally gets confused as a plug-in for jQuery. Unfortunately, jLinq doesn’t do much to help performing queries in jLinq. I’ve written some extension methods before to help link the two together but they were packaged with other projects (and not really ever released)

So this weekend I put together a few helpful extension methods and released them on GitHub.

Naturally, you’ll need the core jLinq library and the jLinq.jQuery library along with jQuery. Since a lot of jQuery uses chained methods you’ll need to use the special invoke syntax for jLinq queries (just an array).

Here are some of the commands you can use.

  • jlinq.$([selector]) : Starts a new query from document.body with the starting selector (if any).
  • $(selector).query([selector]) : Starts a new query from the current jQuery object. Allows an optional selector.
  • .include(selector, [jQuery Object]) : Includes the matches of the provided selector into the current query. You can use a different jQuery object as the source (otherwise it defaults to the object that started the query).
  • .get([selector]) : Converts the current matches in the query to a jQuery object. Allows an optional selector to further filter the results.

Just as a reminder, in order to invoke methods within a query you need to use an array where the first value is the name of the method and following values are the arguments. These arguments are all where the field name would normally be. Here are a few examples in case you need them.


//Example 1: Non-method query
jlinq.from(data).contains(
    "fieldName", //the field to pull the value from
    "check for this" //the argument for contains
    );

// Example 2 : No arguments
// function test() { ... }
jlinq.from(data).contains(
    ["test"], //no arguments, but still an invoke
    "check for this" //the argument for contains
    );

// Example 3 : With arguments
// function test(arg1, arg2) { ... }
jlinq.from(data).contains(
    ["test", 33, false], //the method with arguments
    "check for this" //the argument for contains
    );

So what do these new methods look like? Well here are a few examples to get you started!


//finds all .result elements containing red or green
//and is also case-insensitive
jlinq.$("div.results")
    .contains(["text"], "red")
    .or("green")
    .get()
    .appendTo(document.body);

//finds all links pointing to MP3 files then sorts and styles them
var mp3Links = jlinq.$("a")
    .endsWith(["attr","href"], ".mp3")
    .sort(["text"])
    .get()
    .css({
        color:"#f00",
        fontWeight:"bold"
    });

//highlights all of the matching markup with 'Joe Smith'
$("div.result").query()
    .contains(["html"], "Joe Smith")
    .each(function(rec) { 
        var markup = rec.html()
            .replace(/Joe Smith/g, 
                "<span class='highlight'>Joe Smith</span>");
        rec.html(markup); 
        });

Now you can enjoy the awesomeness of jQuery along with the semi-usefulness of jLinq! You have access to ‘or’-ing, case insensitivity, sorting — It is all there!

This code is all brand new so I don’t recommend using it anywhere important just yet, but it is fun to see how jLinq can make queries a little bit easier.

Written by hugoware

September 26, 2010 at 10:18 pm

Using jLinq With JSON APIs

leave a comment »

I’ve had a few e-mails asking how to perform a query on JSON data from an external source, for example Flickr Services. This post we will go over accessing Flickr data but also performing a secondary query on the information returned.

You can download a .zip file at the end of the post with the files used in the examples.

Let’s take a look at what we’re going to create…

You can see we’ve included a search box along with a few additional check boxes with options we can use for the secondary query. These options aren’t available for the API call, but we can intercept the records and apply additional filters before we use them on the page.

To start, we need to create an extension method for downloading the data from the Flickr APIs. jLinq allows you to create extension methods and plug them directly into the framework.

//extend jLinq to search Flickr
jlinq.flickr = function(search, action) {
    search = escape(search);
    $.getJSON(
        "http"+"://api.flickr.com/services/feeds/photos_public.gne?"+
        "tagmode=any&format=json&jsoncallback=?&tags="+search, 
        function(data) {
            var query = jlinq.from(data.items);
            action(query);
        });
};

This example creates a method named flickr that resides with the rest of the jLinq library. The method accepts a keyword to search with and an anonymous function that will be used to work with the data. This is because we have to wait for the server to return a response.

This means you would use the function with a search phrase and a jLinq query to perform.

//start a new jLinq query
jlinq.flickr("cake", function(query) {

    //perform a normal query and use the data
    query.contains("title", "red")
        .sort("title");
});

At this point we need to wire up all of the UI elements on the page. I won’t go into detail on this part but you can always refer to the example for help if you get stuck.

So lets take a step back and take a look at how the filtering is done for the records. You’ll notice that we can selectively perform certain parts of the query depending on which check boxes are selected.

//sample.js - line 31     - - - - - -
//if excluding titles with digits
if (flickr.ui.filterNumbers.is(":checked")) {
    query.notMatch("title", /\d/g);
}

//if requiring a photo to have a title
if (flickr.ui.requireTitle.is(":checked")) { 
    query.notEmpty("title"); 
}

//if requiring sorting of titles
if (flickr.ui.sortTitle.is(":checked")) { 
    query.sort("title"); 
}

//snip...

You’ll notice that you don’t have to chain the entire query together. You can do sections at a time. This means you can decide when certain parts of the script run and when they are skipped.

Now, we can write queries against Flickr and also perform our own secondary queries using jLinq. In this case we are sorting data and filtering out records that wasn’t part of the query to the server.

Of course, this is a simple example but is does show some of the neat ways you can you can use jLinq to help you work with JSON data.

[Sample Files]
http://hugoware.com/examples/jlinq-flickr.zip

Written by hugoware

August 26, 2010 at 1:12 am

3 Things To Know Before Using jLinq

leave a comment »

Since I released the beta code for the new jLinq I’ve had several of the same questions show up. Here are some answers to help you get started.

Where Is ‘Manual’, Query Commands Are ‘Auto-pilot’

I’ve had a few questions where people start trying to use the where command and find that it isn’t solving anything.

As far as I can tell, a lot of other LINQ style Javascript libraries use Where as their query command and have the developer manually provide the comparisons. Granted, I don’t know the other frameworks very well (hint, I wrote my own) but a lot of the code samples look something like this.

var results = lib.From(data)
    .Where(function(record) { return record.name.substr(0, 1) == "a"; })
    .OrderBy(function(record) { return record.name; })
    .Select();

I’m sure that these libraries are doing a lot of stuff for you in the background, but the where command in itself seems like placing too much work on the developer.

jLinq approaches this differently. Instead, you’ll find many common query methods included as part of the library. These methods also evaluate the data types to determine the best way to query the property.

Here is an example, consider you have some data that looks like this.

var data = [{
    name: "James",
    permissions: ["read", "write", "delete"]
},
/* snip... */];

Instead of checking these records manually, you can use the same command for both properties and they will handle the data differently for each.

//with string types
jlinq.from(data).contains("name", "e").select(); //matches the 4th 'e' in 'James'

//with array types
jlinq.from(data).contains("permissions", "write").select(); //matches the 2nd 'write'

This reduces the typing you do since you no longer have to perform manual comparisons.

Of course, jLinq does include a where command that uses the same syntax as the example above (anonymous function), but it is recommended to create extension methods instead of using the where command.

Operators Are Query Commands Too

Operators are an important part of jLinq. Sometimes you’d like to match on more than one condition or ignore records that do not meet a certain condition. You can use operators as individual commands in your queries.

//Examples only - Shorter syntax explained in Section 3

jlinq.from(data)
    .starts("first", "a")
    .or()
    .starts("first", "b");

jlinq.from(data)
    .not()
    .starts("first", "d")

jLinq takes operators a step further by creating operator aliases for all of the query commands in your library. This means that instead of only having the query command starts you actually have 6 – starts, orStarts, andStarts, notStarts, orNotStarts, andNotStarts. The queries above can be rewritten as.

//Examples only - Shorter syntax explained in Section 3

jlinq.from(data)
    .starts("first", "a")
    .orStarts("first", "b")
    .select()

jlinq.from(data)
    .notStarts("first", "d");

It is worth noting that and is the implied default unless you provide an or – the and versions of the query commands are for aethestetics only. 🙂

DRY — I Repeat, DRY!

If you aren’t familiar with the term DRY, it means “Don’t Repeat Yourself”. The previous samples reduced some of our typing, but jLinq can actually take it a step further.

jLinq uses field and command memorizing, meaning after you say it once it is implied until you say something different.

jlinq.from(data)
    .starts("first", "a")
    .or("b")
    .or("c");

jlinq.from(data)
    .starts("first", "a")
    .ends("b")
    .or("last", "a");

The first example memorizes the command being used is starts and the field being queried is first. After that, the remaining calls to ‘or’ will repeat the previous command and field unless something else is used.

The second example shows how you can mix up commands and field names without needing to repeat yourself. Basically, here is what the query is doing.

  1. Records with first names starting with ‘a’…
  2. then records with first names ending with ‘b’…
  3. then records with last names ending with ‘a’

You can see that jLinq makes it easy to avoid repeating the same commands and field names to reduce the typing.

This is just some of the first basics to understand about jLinq. I’ll be blogging more (and writing documentation) for the next few days. Feel free to contact me with questions.

Written by hugoware

August 11, 2010 at 9:54 pm

jLinq Reloaded

with 5 comments

[jLinq Beta is now online – go check it out!]

My first ‘open source’ project I ever released was a Javascript library that allowed you perform LINQ style queries with arrays of objects. The library supported operators, extension methods and even memorized commands to reduce typing.

Overall, it did a good job of making it easier to select information that normally would need to be handled with a for loop. Unfortunately, it turns out jLinq is rather slow. I recently blogged about a performance evaluation that came back with less than flattering results.

However, this is exactly where ‘open source’ helps make better software. If I hadn’t released my code then I might never know how it could be improved. This is the sort of thing that inspires software developers to improve their code — and that is exactly what is did for me.

So for the past few days I’ve been completely rewriting jLinq from the ground up and the gains have been incredible to say the least.

  1. About 60% smaller (around 8kb compressed)
  2. 99% faster queries
  3. Easier to extend library
  4. Minty fresh scent (your results may vary)

If you’re looking for a download link then you’re going to have to contact me instead. The code should be released soon but for now I’m still writing tests and preparing to rerelease the project with a brand new site (and probably a new name).

[http://github.com/hugoware/jlinq-beta/]

There has been a lot of interest already in seeing the new code so I’ve started a new repository on github.com. This is beta code so I suspect I’ll have errors in it. Please feel free to contact me to log an issue on the site.

If you’re interested in how jLinq improved so much then read on…

jLinq… But On Caffeine

My first theory about why jLinq was so slow had to do with the use of evals in my code.

At the time, I didn’t understand you could pass functions as arguments, I certainly had no idea what enclosures were, so jLinq would build an entire query in a string and then eval it into a method that could be invoked against each record.

I also used eval to get values from records instead of just referring to them by their names as an index. I don’t know why eval is so much slower, but the difference is undeniable.

//using index names
var fast = function(obj, path) {
    path = (path+"").split(/\./g);
    var index = 0;
    while(obj != null && index < path.length) {
        obj = obj[path[index++]];
    }
    return obj;
};

//being lazy with an eval
var slow = function(obj, path) {
    return eval("obj."+path);
};

//looping 100,000 times
loop(100000, function() { fast(data, "name.first"); }); //128ms
loop(100000, function() { slow(data, "name.first"); }); //6.8 minutes

So what kind of improvement do these two changes result in? Big ones.

Single Argument Query (850 Records)
Previously: ~430ms
Now: ~10ms

Multi-Argument Query with Strings (850 Records)
Previously: ~2730ms
Now: ~35ms

Simple Join (850 Records)
Previously: ~6200ms
Now: ~135ms

The list of improvements goes on but I’ll talk about them more in later blog posts.

So What Is Next?

It’s screaming fast and much smaller but unfortunately it is too new to even suggest that it is put into production. I’ll definitely feel more confident in the code after I’ve finished the tests.

But I also plan to rebrand the library this time. Why?

As it turns out, the name LINQ has caused some confusion with non-.NET developers. I’ve heard that some developers see the name and say ‘Well, I don’t use .NET so I don’t need this.’, which is clearly not the response I’d like to hear.

jLinq isn’t only for web pages. In fact, the neat thing about jLinq is that it can be plugged into anything running Javascript and it works great. I’d like to see jLinq be exposed to more developers.

So, with a new name, new site and all brand new code I think there is a chance to redefine what jLinq can do for the community.

The code is mostly finished so now it’s time for the hard part… coming up with a good name…

Written by hugoware

August 8, 2010 at 8:53 pm

jLinq Performance Analyzed

with 2 comments

After writing this post I went back and rewrote jLinq from the ground up to improve performance.

So here is something interesting. Another developer named Dan Stocker has been working on a Javascript sorting library named jOrder and used jLinq for comparing benchmarks. The results are interesting to say the least.

Dan has a good write up of the results in the wiki for his project.

I’m disappointed, but in a good way. I thought jLinq was ‘good enough’. It performed the way it should and that was good enough for me. These results tell a much different story.

jLinq probably performs well in typical scenarios, whereas these benchmarks are for medium to large arrays queried thousands of times. I’ve used it in several projects with great results.

Regardless, jLinq clearly performs slower than it should. Reviewing the jLinq code I can think of a few potential pain points.

  1. The entire jLinq object and all of the commands are generated each time a query is started. jLinq there should be a way to cache a standard jLinq object and only rebuild it when needed.
  2. jLinq uses a lot of evals that probably could have been avoided if I understood Javascript better at the time. This undoubtedly is going to cause a significant performance hit.

jLinq was written just about a year and a half ago and I’ve since learned some of the finer parts of Javascript which would most likely improve the performance of the library. In fact, seeing this review has really inspired me to get into back into the project and see what kind of improvements I can make.

I appreciate Dan showing these results to me. Sharing your code with others is a great way to get feedback about what you write… even if it is a kick in the butt.

Written by hugoware

August 4, 2010 at 8:43 am

jLinq in MongoDB (Oh snap!!)

with 9 comments

My new project CSMongo, a Mongo driver for .NET, is now online! Check it out!

Yeah, you read that right – jLinq in MongoDB!

Lately I’ve been working on a Mongo database driver and I got to thinking — Since I can use Javascript with MongoDB then I wonder if I could use jLinq to write queries? As it turns out the answer is yes!

… and if you don’t know what jLinq is… – jLinq is a javascript query language, similar to the LINQ project created by Microsoft, for JSON data which was designed to run within web pages using Javascript

Getting Started

I’m not really sure how you can get a Javascript library to load directly into MongoDB but for now I was simply copying and pasting the existing jLinq packed library into the command line and allowing it to evaluate the script — and that is it! You can use jLinq right away!

You should be aware though that you can’t just evaluate results directly. You actually need to call the toArray() function on each of them to see the results. So for example, your query would look something like…

jLinq.from(db.users.find().toArray()).startsWith("name", "h").select();

You don’t need to actually do anything with the results since they are automatically displayed on the screen for you.

So far everything works. I could do standard queries, use the OR operator, reorder records, join different databases, perform comparisons against sub-properties — really everything I’ve tried has worked just the way I’d expect it to!

Granted, database and collection names are going to be different, here are some interesting queries you can try with your database.

//Selects records where the name starts with a, b or c (case-insensitive)
jLinq.from(db.users.find().toArray())
    .startsWith("name", "a")
    .or("b")
    .or("c")
    .select();

//selects users older than 50 and are administrators then orders them by their names
jLinq.from(db.users.find().toArray())
    .greater("age", 50)
    .is("admin")
    .orderBy("name")
    .select();

//performs a join against another database to get locations
//and then queries the location to find users that live in 'texas'
//and then selects only the name of the person and location
jLinq.from(db.users.find().toArray())
    .join(db.locations.find().toArray(), "location", "locationId", "id")
    .equals("location.name", "texas")
    .select(function(rec) {
        return {
            person: rec.name,
            location: rec.location.name
        };
    });

More On jLinq

Unfortunately, it isn’t possible to go over all of the cool stuff you can do with jLinq – but here are some screencasts and blog posts that you can read that might help you get started.

jLinq Project Page
Screencast 1 – Getting Started
Screencast 2 – Extending jLinq (Custom commands)
Screencast 3 – Modifying Live Data In A Query (Joins, Assignment)
Screencast 4 – jLinq 2.2.1 (Updates)

Anyways, for now I’m not sure how to integrate jLinq into Mongo queries but keep an eye for future blog posts as I find out more.

Written by hugoware

February 14, 2010 at 11:45 pm

jLinq Update (2.2.1)

with 10 comments

I’m off from work this week so the first thing I did was get some work done on jLinq. I’ve had a few bugs I’ve needed to sort out along with a new feature I wanted to implement. I’m also going to be working on documentation this week. Previously, I had created a Wiki and was hoping that people might add to the documentation… but instead spam bots pretty much ruined it so I’ll be starting over from scratch… oh well..

New Features

jLinq itself doesn’t really have a new feature but instead a new way to get the jLinq library. Instead of simply downloading a standard jLinq library you can use the new online jLinq Framework Generator to select only the functionality that you want to include. You can still download the basic pack but this option gives you a little more control what goes into your version of jLinq.

Changes

The most notable change for jLinq is that all of the operator commands have been included as standard functions for the framework. Before, these functions were actually methods extended onto the framework. Since these functions are required for jLinq to work I’ve moved them so that they are in every jLinq framework by default.

Bug Fixes

orderBy with joined records would return incorrect results
jLinq uses eval to figure out the values of field names since you can provide methods, array indexes or whatever. The code used to sort the values wasn’t getting the values correctly so the sorted result was always wrong.

using 0 in certain comparisons would return incorrect results
Ah, this one was fun. So check out the code below…

var values = [ 0 ];
var index = 0;
var a = !values[index];
var b = values[index] == null;

So what is a and what is b? If you said true and false then give yourself a cookie.

jLinq uses the number of arguments passed to help determine if anything is being memorized (like the field name). jLinq makes a quick pass to select all values until it finds a null value… or at least that is what I meant for it to do. Unfortunately, I wasn’t thinking when I wrote that method and didn’t check explicitly for null.

Feedback

If you find any problems with jLinq or have any suggestions, please leave a comment or contact me directly.

Written by hugoware

November 23, 2009 at 9:03 pm

Double Whammy – jLinq Release And Screencast!

with 2 comments

Now that you’re all excited time for some disappointment – it is just a minor update to fix a few bugs and to add one new feature.

But instead of just blogging about it I’ve also done a new screencast to talk about the updates and new feature in this release.

  • Dynamic Queries (using OR) – Before, if you used “OR” at the start of a query then you’d get an exception. jLinq would try and tag “||” at the start of your query which would cause the whole thing to collapse. Now, jLinq correctly handles this and prevents the error. In turn, writing dynamic queries just got a whole lot easier!.
  • Range Comparisons With Dates – Version 2.2.0 introduced smarter comparisons by evaluating the types of values being passed in instead of expecting only one type. However, in the change, the range comparisons like greater, less, between, etc, would all incorrectly evaluate Dates as strings… bummer.
  • Bitwise Flag Comparisons – jLinq 2.2.1 introduces a new query method called has which performs bitwise flag comparisons on records. It even works with regular ol’ numbers by converting them before performing the comparisons.

Anyways, go check out he new screen cast now and tell me what you think!


screencast-4-play

Bonus: CamStudio Icon

camStudioExample

Last night while I was setting up CamStudio for this screencast I was looking at their UGLY icon they use for their program — I just couldn’t stand it being in my beautiful RocketDock – so I designed a new one for them. If you’re looking for an improved logo then you can download the PNG for the CamStudio icon.

Written by hugoware

September 9, 2009 at 2:18 am

jLinq Screencast #3 – Modifying Records During A Query

with 3 comments

Sometimes when you get your records they aren’t exactly what you we’re needing, or sometimes some of the fields need to be formatted before they can be queried against. Fortunately, jLinq comes with that functionality built into it.

This screencast goes over several of the methods you have available to you that allow records to be formatted before they are selected.

  • each(delegate(record)): Performs a loop through each of the values in the array. This code can be used to modify records, create new properties or anything you need it to do.
  • attach(alias, delegate(record)): Performs the method passed in on each of the records in the array and attaches the result to the record using the alias provided.
  • join(records, alias, foreignKey, primaryKey): Joins two arrays together using the alias and the keys to match the values. Joins tend to be slow when using large arrays, so use caution.


screencast3

If you have any recommendations or requests for the next screencast, please let me know and I’ll see what I can do.

Side note: Never improvise on a screen cast – nothing like having to redo a screen cast twice because you say something that isn’t true and then realize it while recording. 🙂

Written by hugoware

August 24, 2009 at 2:21 am