How to filter the results of JSCOM requests on the server side

In the past I already discussed the internals of the Managed Client Object Model and the ECMAScript Object Model (a.k.a. JSCOM) several times. In this post I illustrate an unofficial way of extending the JSCOM.

WARNING: As I wrote above, the method I describe here is totally unofficial and unsupported. It’s only an experiment that depends on unpublished features of the JSCOM, like internal functions that can be changed or even removed without prior notice. Please, take my words seriously, and do not apply this method in a productive environment.

Assuming you are working with the Managed Client Object Model, you can submit queries, that are evaluated on the server side, and return only results that fulfill the conditions. For example, the request below returns only the filterable, non-hidden fields of a list (sample taken from MSDN):

clientContext.Load(oList,
    list => list.Fields.Where(
    field => field.Hidden == false
        && field.Filterable == true));
clientContext.ExecuteQuery();

The most important benefit of this syntax is that you can reduce the network traffic caused by returning irrelevant data (in this case fields) that we would have to filter on the client side without this feature.

Unfortunately, this feature seems to be missing in the ECMAScript Object Model, the Include keyword supports only limiting the members of the object to be returned (the query below returns only the Title and Id properties of each lists), but does not help to filter the results at all (e.g. you can not filter lists based on their properties).

clientContext.load(collList, ‘Include(Title, Id)’);

Since I’m working recently quite a lot with JSCOM (mainly in context of Project Server 2013), I was curious what happens when we use this feature from the Managed Client Object Model, and how we could inject the functionality into JSCOM.

Note: Yes, I know that REST makes it possible (via the $filter query option) to filter the items returned, however it has its own limitation, like the lack of the batch requests – e.g. aggregating requests on the client side and sending them in batches via executeQueryAsync – that is (at least, IMHO) one of the best features of JSCOM.

So I’ve created a simple .NET console application that submits the following query (first without, next with filtering) to the server:

ClientContext ctx = new ClientContext(“http://sp2013”);
// without filtering
// var listQuery = ctx.Web.Lists;
// with filtering
var listQuery = ctx.Web.Lists.Where(l => l.Title == "Images");
var list = ctx.LoadQuery(listQuery);
ctx.ExecuteQuery();

and captured the network traffic using Fiddler in both cases. The screenshot below highlights the difference between the two queries:

The equivalent version of the no-filter query in JavaScript:

As in the case of the .NET version, this query submits the request without the QueryableExpression. If we want to enable filtering in case of JSCOM, we should first find where the Query part of the request is assembled. Using a simple search in the .js files shows that this task is performed in the SP.ClientQueryInternal.prototype.$2s_1 method in the SP.Runtime.js.

The default implementation of this method is illustrated below (code taken from SP.Runtime.debug.js):

We should create our custom override of the $2s_1 method that injects the QueryableExpression part of the query into the ChildItemQuery block (see a similar implementation in the $2w_0 method of the SP.ClientObjectPropertyConditionalScope.prototype in SP.Runtime.debug.js). For example (to filter child items with Title “Images”):

If we define this method in our page as illustrated above, all request sent to the server will filter the child items using the condition defined in the override (Title EQ “Images”). That is probably not what we would like to achieve. It would be better to limit the filtering for example to the context the request was sent from.

I’ve found, that the calling SP.ClientContext object is available as this.$0_1.

First, I‘ve created a filterBase function that writes the filtering part of the request stream based on the condition, filter name and filter value parameters:

To keep our former example condition, we can call filterBase from a new function filterLists:

We assign the filtering function in our getLists method to the context:

In the new version of the of the $2s_1 method override we check if the current context has a filtering method assigned to, and if one is found, it is called:

OK, we are one step further now, but wouldn’t it be nice, if we could query more objects in the same context (and the same batch) and have the option to set different filters for each one?

I’ve found that the object path property of the client object to be queried is available in the $2s_1 method via this.$G_0.

In this case we should override the $2s_1 method like this:

The object path property is available as myObject.$5_0.$e_0 property of the client object (for example, lists.$5_0.$e_0), so we can rewrite our methods as illustrated below. In this case we extend our example with a further object (groups) that should be filtered by the filterGroups method (OwnerTitle property of the group EQ "DevSite Owners"). The getList function was also renamed to getObjects.

The following Fiddler screenshot shows the resulting query:

This kind of overrides provides already a quite flexible “framework” for filtering objects on the server side using the JSCOM, making this important feature available for JavaScript developers as well. It would be nice if Microsoft would provide a similar, but official solution to this issue in a forthcoming service pack.

We can see from this experiment as well, that there is much more power available in the XML / JSON communication protocol behind the client object models, than it is made public by this APIs, so there is yet place for improvements. You can find valuable information related to the SharePoint Client Query Protocol here.