Compared to ASN.1/DER or some TLV encoding/decoding, JSON is a little bit CPU expansive, and the data encoded uses more bytes. Compared to XML, it is quick and light :-). So it is a good compromise between all the following criteria : size, easy to encode/decode, human readable.
Here the encoding specifications: http://json.org/
This encoding is very famous and each language support it. Why an article for Lua ? Lua provides many implementations. Some implementation are very CPU expansive, and I will provides some tests, and a way to use JSON with Lua in HAProxy without sacrificing the response times.
Here the list of Lua JSON libraries: http://lua-users.org/wiki/JsonModules. There is a lot of libraries. I can't test each one. I cheat testing two library I already tested.
- DKJSON: http://dkolf.de/src/dkjson-lua.fsl/home
- CJSON: https://www.kyne.com.au/~mark/software/lua-cjson.php
DKJSON is a pure Lua implementation. Its main advantage is to be compilation less and all the libary is embedded in one file. CJSON is a C bindings for Lua, its main advantage is a quick encoding/decoding.
Compiling CJSON
You can find this library already compiled on your distro, or maybe using LuaRocks (I never this system, and I'm afraid to discover it). It is very simple to compile it yourself. The package doesn't contains "./configure" or complicated things. Just a Makefile with some path options.
Why compiling the package yourself ? HAProxy uses Lua 5.3, and Lua 5.3 is not yet available on all distro.
$ wget https://www.kyne.com.au/~mark/software/download/lua-cjson-2.1.0.tar.gz
$ tar xf lua-cjson-2.1.0.tar.gz
$ cd lua-cjson-2.1.0
$ make LUA_INCLUDE_DIR=/home/thierry/temp/lua-5.3.4/src/
cc -c -O3 -Wall -pedantic -DNDEBUG -I/home/thierry/temp/lua-5.3.4/src/ -fpic -o lua_cjson.o lua_cjson.c
In file included from lua_cjson.c:43:0:
/home/thierry/temp/lua-5.3.4/src/lua.h:93:9: warning: ISO C90 does not support 'long long' [-Wlong-long]
/home/thierry/temp/lua-5.3.4/src/lua.h:96:9: warning: ISO C90 does not support 'long long' [-Wlong-long]
cc -c -O3 -Wall -pedantic -DNDEBUG -I/home/thierry/temp/lua-5.3.4/src/ -fpic -o strbuf.o strbuf.c
cc -c -O3 -Wall -pedantic -DNDEBUG -I/home/thierry/temp/lua-5.3.4/src/ -fpic -o fpconv.o fpconv.c
Compare the to libraries
I present a comparative protocol. The way is encoding Lua struct as JSON message in a loop. We perform this encoding about 500 000 times.
Why 500 000 ? because I do some tests: high value gives more precision for the calculus of one encode() or one decode() function. Value highest than 500 000 requires a long test (more than one minute) and I'm not patient :-). 500 000 is a good intermediate value.
There is the test code:
package.cpath = package.cpath .. ";lua-cjson-2.1.0/?.so"
require('print_r')
cjson = require('cjson')
dkjson = require('dkjson')
lua = {}
lua['entry1'] = 123
lua['entry2'] = 456.789
lua['entry3'] = "string"
lua['entry4'] = {'a', 'b', 'c', 'd', 'e', '...'}
lua['entry5'] = {}
lua['entry5']['entry1'] = 1123
lua['entry5']['entry2'] = 1456.789
lua['entry5']['entry3'] = "Another string"
lua['entry5']['entry4'] = {'a', 'b', 'c', 'd', 'e', '...'}
enc = "{\"entry4\":[\"a\",\"b\",\"c\",\"d\",\"e\",\"...\"],\"entry2\":456.789,\"entry5\":{\"entry3\":\"Another string\",\"entry4\":[\"a\",\"b\",\"c\",\"d\",\"e\",\"...\"],\"entry2\":1456.789,\"entry1\":1123},\"entry1\":123,\"entry3\":\"string\"}"
-- Validate encoding:
--print(cjson.encode(lua))
--print(dkjson.encode(lua))
-- Validate decoding
--print_r(cjson.decode(enc))
--print_r(dkjson.decode(enc))
for i = 0, 500000 do
-- Test CJSON
--local j = cjson.encode(lua)
-- Test DKJSON
--local j = dkjson.encode(lua)
end
First we validate the right encoding for the two libraries. We uncomment the lines starting by print(..). The script return two independent JSON messages. Note the flags "-s" with jq. It read independent JSON message on its input and concatenate it an array.
$ ./lua json_test.lua | jq -s .
[
{
"entry4": [
"a",
"b",
"c",
"d",
"e",
"..."
],
"entry5": {
"entry4": [
"a",
"b",
"c",
"d",
"e",
"..."
],
"entry1": 1123,
"entry2": 1456.789,
"entry3": "Another string"
},
"entry3": "string",
"entry2": 456.789,
"entry1": 123
},
{
"entry4": [
"a",
"b",
"c",
"d",
"e",
"..."
],
"entry5": {
"entry4": [
"a",
"b",
"c",
"d",
"e",
"..."
],
"entry1": 1123,
"entry2": 1456.789,
"entry3": "Another string"
},
"entry3": "string",
"entry2": 456.789,
"entry1": 123
}
]
$ ./lua json_test.lua
(table) table: 0x25b65e0 [
"entry2": (number) 456.789
"entry5": (table) table: 0x25b66b0 [
"entry1": (number) 1123.0
"entry2": (number) 1456.789
"entry4": (table) table: 0x25b23c0 [
1: (string) "a"
2: (string) "b"
3: (string) "c"
4: (string) "d"
5: (string) "e"
6: (string) "..."
]
"entry3": (string) "Another string"
]
"entry3": (string) "string"
"entry1": (number) 123.0
"entry4": (table) table: 0x25b6620 [
1: (string) "a"
2: (string) "b"
3: (string) "c"
4: (string) "d"
5: (string) "e"
6: (string) "..."
]
]
(table) table: 0x25caf80 [
METATABLE: (table) table: 0x25cae70 [
"__jsontype": (string) "object"
]
"entry2": (number) 456.789
"entry5": (table) table: 0x25cb5f0 [
METATABLE: (table) table: 0x25cae70 [
"__jsontype": (string) "object"
]
"entry1": (number) 1123
"entry2": (number) 1456.789
"entry4": (table) table: 0x25cb780 [
METATABLE: (table) table: 0x25caee0 [
"__jsontype": (string) "array"
]
1: (string) "a"
2: (string) "b"
3: (string) "c"
4: (string) "d"
5: (string) "e"
6: (string) "..."
]
"entry3": (string) "Another string"
]
"entry3": (string) "string"
"entry1": (number) 123
"entry4": (table) table: 0x25cb090 [
METATABLE: (table) table: 0x25caee0 [
"__jsontype": (string) "array"
]
1: (string) "a"
2: (string) "b"
3: (string) "c"
4: (string) "d"
5: (string) "e"
6: (string) "..."
]
]
For comparing the two libraries, we need to comment/uncomment the encode() lines. There is the results:
$ # DKJSON
$ time ./lua json_test.lua
real 0m18.273s
user 0m18.260s
sys 0m0.000s
$ # CJSON
$ time ./lua json_test.lua
real 0m2.038s
user 0m2.036s
sys 0m0.000s
$ # DKJSON
$ time ./lua json_test.lua
real 0m26.461s
user 0m26.436s
sys 0m0.004s
$ # CJSON
$ time ./lua json_test.lua
real 0m1.895s
user 0m1.892s
sys 0m0.000s
| DKJSON | CJSON | Comment | |
|---|---|---|---|
| Encoding time measured | 18.273s | 2.038s | CJSON is 9 times quicly than DKJSON |
| One encoding (÷ 500000) | 36.546µs | 4.072µs | |
| Decoding per second | 27 363 | 245 339 | |
| Decoding time measured | 26.461s | 1.895 | CJSON is 14 times quicly than DKJSON |
| One encoding (÷ 500000) | 52.922µs | 3.79µs | |
| Encoding per second | 18 896 | 263 852 |
So, Is the speed is not a criteria for your application, you can use DKJSON because it is easier to embbed. If the speed is important, use CJSON. Note that the compilation of CJSON is a one-time action, once is compiled, the usage of the two libraries is the same.
You can test the other JSON libraries with this protocol and share the result in the comments. I will update the array. Note that the result will be not comparable because the test must be performed with the same CPU.
No comments:
Post a Comment