Updated on 2018-01-17
Dependency free, fast, lightweight JSON parsing and light query
See the Update section at the end for new features.
Nowadays, with LINQ being de riguer among C# developers, I scarcely have to explain the value of tuple types, object querying, and tree processing.
But what of using features like this in deep, systems level code, or code that you'd like to port to run elsewhere, where LINQ isn't available? What if I want to use nice little dynamic object trees, say, within my parser generators, and search query objects, where I don't want to have consumers of the generated code require LINQ or fancy AST object models just to do error recovery in a parser? What if I want to create lower level calls for querying JSON based web services and I don't want all the baggage that usually entails? Or what if my code is running on a phone and I don't have access to all the fancy stuff?
What if all I need is a little object model, and just a little bit of query? What if I just need a way to simply use tuples and trees without all of the mess?
And what if I want to be able to use LINQ with my tuples and such if I want to?
The main drawback of modern .NET language features is the sheer amount of infrastructure and overhead just to do some very simple tasks. Like the above.
What if we had, say, a bare bones JSON parser that builds out simple System.Object trees using lists and dictionaries for the branches?
And yet, there isn't consistent support of this everywhere even now, even with the latest version of Silverlight installed, since Microsoft decided to alter JSON functionality in later versions of it.
And anyway, who needs a bunch of extra dependencies if all you need is just enough to cover 80%? And worse, what if you have to target .NET 1.x? There's nearly nothing available for JSON for that.
A familiarity with JSON is necessary, a rough familiarity with LINQ, ASTs (abstract syntax trees) and tuples is helpful simply to understand the problem.
JsonUtility.cs is a single class for handling basic JSON functionality.
It handles the core JSON datatypes - boolean, (null), number, string, array and object/dictionary.
It can parse from strings or streams, and write to string and streams, including pretty printing.
It contains a basic, very simple query mechanism, called Get. Get allows you to query by fields or indices only, but allows you to query a path of them in one operation.
The code is capable of parsing with no backtracking or seeking so it can work from a forward only stream, but currently, it must build the complete object model in memory before returning from a parse, at which point the tree can be queried. Adding visitor support to circumvent this limitation may be in the works for the future, but for now, let's remember to keep it simple. The advantage is, modifying the in memory tree is simple.
If you'd like JsonUtility to support 2.0 features like generics, be sure to include the NET20 define in your project. Doing so will cause the routines to prefer returning 2.0 Lists and Dictionaries, and will parse using typed character enumerators where applicable. The performance gain is measurable, but not mind blowing. The example code I've provided that uses JsonUtility will happily use either/or as well. Either way, the actual JSON object model returned is not typed, but the 2.0 version uses IList
Make sure to include the magic.
using Grimoire;The following is optional, but I like to do it. If you don't want to support .NET 1.x, just remove the conditional preprocessors and only include the 2.0 definitions. You don't even need to use the definitions like JObject but they make it clear when your source var is part of a JSON tree.
using JNumber=System.Double;
using JString=System.String;
using JBoolean=System.Boolean;
#if NET20 || NET40
using JArray=System.Collections.Generic.List<object>;
using JObject=System.Collections.Generic.Dictionary<string,object>;
using JField=System.Collections.Generic.KeyValuePair<string,object>;
#else
using JArray=System.Collections.ArrayList;
using JObject=System.Collections.Hashtable;
using JField=System.Collections.DictionaryEntry;
#endifAs you can see, parsing is simple. The input can be a string or a TextReader, or any enumerable character source like a character array. Here, we use it to parse the response body of a JSON base web service:
object json;
WebRequest req = WebRequest.Create(url);
using (WebResponse wr = req.GetResponse ())
using (Stream wrs = wr.GetResponseStream ())
using (StreamReader r = new StreamReader (wrs))
json=JsonUtility.Parse (r);
// get the count of the array at json .seasons
int sc = ((JArray)JsonUtility.Get (json, "seasons")).Count;It's also respectably fast. In terms of parsing to an in memory tree, it should hold its own against other high performance JSON implementations for .NET.
Once you have a tree, not only can you do simple field selects using Get, you can also write sub-trees, either to a TextWriter, a StringBuilder, or to a String, optionally pretty printing the result. This is also respectably fast, regardless of which method you use, though writing to a stream may be the fastest simply because it doesn't necessarily require a copy to be made in memory of what was written.
You may notice JsonUtility.cs has a lot of duplicated code in it. This is because JsonUtility itself is a prototype of a parser generator output, as part of a larger project I am creating. JsonUtility started as a test case to make some design decisions around performance, including what the tradeoff would be using typed and untyped enumerators under real world conditions. The generator as I said, is part of a separate project, and the types of parsers it produces varies widely. This code is a desirable output for an LL grammar that can be made using recursive descent. The goal was speed of parsing and creating the in memory tree, while making the code regular enough that it could be baked into a code generation feature.
Most of the parsing code is extremely straightforward, it may be slightly less precise than the standard. It should parse all compliant JSON trees, and reject most, if not all poorly formed JSON as early as possible after it encounters it. The error handling is not very detailed, but may be improved in a future release. It will produce valid JSON strings in every case so long as the input tree is valid, meaning no circular references, and no non-native JSON datatypes.
You'll see that the recursive descent parsing methods are labeled ParseXXXXX and take an enumerator as an input.
The base object Parse(#enumerator e) method parses boolean, and null terminals directly in the routine, while dispatching whitespace skipping, string processing, number processing, array and object processing to their respective SkipWhitespace, ParseString, ParseNumber, ParseArray, and ParseObject methods.
The one parsing method that breaks this pattern is ParseCombineDigits, which is a custom method for parsing a number which combines accumulation of digits and building the number into one step for better performance. The method parses a series of digits such as "342" into a double value. When you call it with a different "part" parameter, you're telling it to parse different parts of the floating point value you're working on, to wit, for 123.456E+7, ParseCombineDigits will be called 3 times, with the part parameter as 0 for "123", 1 for "456", and again with a part of 2 for "7", each time modifying the accumulator value during the parse such that the final result contains the combined values into a proper 64-bit floating point value. This method is used by ParseNumber.
The Get methods are quite simple. They merely take the current object and allow for indirection into one of the children, based on the object type. It uses type comparison (minor reflection, but not type interrogation) to determine how to proceed. The Get methods simply repeat this in a series if more than one field parameter is passed, allowing for path traversal using recursion.
One of the nice things about having burdenless JSON in my code is it allows me to readily employ tuples and trees without devoting a lot of extra worry about design requirements and dependencies. I can just drop it in and go, and it has a very small footprint. This has led me to using JSON even for things like method parameters in C# where appropriate, versus building structs for complex classes. Being able to use micrographs to represent chunks of a problem domain without all of the extra work is really helpful, and viewing any of these JSON trees in a debugger is straightforward as well. It has changed the way I write code under certain circumstances.
Because there has been at least some interest in this code, I whipped up NET/DLR dynamic support and a JsonNode wrapper in JsonNode.cs you can use to ease some of the basic use. It's a read-only access to the JSON for now but allows for some sugar.
I also did something I had been meaning to do - add a token type to the JSON grammar to allow field names to be specified without quotes. so you can do "{x:5,y:1}" and it will parse. Now an object field key is either a string or an identifier that can begin with an underscore or a letter, and can contain any letter or number or underscore or hyphen in subsequent characters. Approxamately, this regex [A-Za-z_][A-Za-z-_0-9]* but the letters and digits can be any unicode letter or number.
// assumes NET40 is #defined
...
dynamic dom = JsonNode.Parse("{left:{left:10,op:'+',right:5},op:'-',right:2}");
Console.WriteLine("dynamic ref {0}, type = {1}",
(int)dom.left.right,
dom.left.right.NodeType
);or:
var dom = JsonNode.Parse("{left:{left:10,op:'+',right:5},op:'-',right:2}");
Console.WriteLine("{0}, type = {1}",
(int)dom["left","right"],
dom["left","right"].NodeType
);as well as the lower level JsonUtility.cs methods (which unlike JsonNode.cs don't have a NodeType feature).
object json = JsonUtility.Parse("{left:{left:10,op:'+',right:5},op:'-',right:2}");
Console.WriteLine("{0}, type = ?",
JsonUtility.Get(json,"left","right")
);JsonNode, slightly expanded JSON grammar, and DLR dynamic support