re
/
sjson
forked from mirror/sjson
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

first commit

This commit is contained in:
Josh Baker 2016-10-18 15:52:46 -07:00
commit 4e588076ed
6 changed files with 757 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.travis.yml Normal file
View File

@ -0,0 +1 @@
language: go

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2016 Josh Baker
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

192
README.md Normal file
View File

@ -0,0 +1,192 @@
<p align="center">
<img
src="logo.png"
width="240" height="78" border="0" alt="SJSON">
<br>
<a href="https://travis-ci.org/tidwall/sjson"><img src="https://img.shields.io/travis/tidwall/sjson.svg?style=flat-square" alt="Build Status"></a>
<a href="https://godoc.org/github.com/tidwall/sjson"><img src="https://img.shields.io/badge/api-reference-blue.svg?style=flat-square" alt="GoDoc"></a>
</p>
<p align="center">set a json value quickly</a></p>
SJSON is a Go package the provides a **very fast** and simple way to set a value in a json document. The reason for this library it to provide efficient json updating for the [SummitDB](https://github.com/tidwall/summitdb) project.
For quickly retrieving json values check out the [GJSON](https://github.com/tidwall/gjson).
Getting Started
===============
Installing
----------
To start using SJSON, install Go and run `go get`:
```sh
$ go get -u github.com/tidwall/sjson
```
This will retrieve the library.
Set a value
-----------
Set sets the value for the specified path.
A path is in dot syntax, such as "name.last" or "age".
This function expects that the json is well-formed and validates.
Invalid json will not panic, but it may return back unexpected results.
Invalid paths may return an error.
```go
package main
import "github.com/tidwall/sjson"
const json = `{"name":{"first":"Janet","last":"Prichard"},"age":47}`
func main() {
value, _ := sjson.Set(json, "name.last", "Anderson")
println(value)
}
```
This will print:
```json
{"name":{"first":"Janet","last":"Anderson"},"age":47}
```
Path syntax
-----------
A path is a series of keys separated by a dot.
The dot and colon characters can be escaped with '\'.
```json
{
"name": {"first": "Tom", "last": "Anderson"},
"age":37,
"children": ["Sara","Alex","Jack"],
"fav.movie": "Deer Hunter",
"friends": [
{"first": "James", "last": "Murphy"},
{"first": "Roger", "last": "Craig"}
]
}
```
```
"name.last" >> "Anderson"
"age" >> 37
"children.1" >> "Alex"
"friends.1.last" >> "Craig"
```
The `-1` key can be used to append a value to an existing array:
```
"children.-1" >> appends a new value to the end of the children array
```
Normally number keys are used to modify arrays, but it's possible to force a numeric object key by using the colon character:
```json
{
"users":{
"2313":{"name":"Sara"},
"7839":{"name":"Andy"}
}
}
```
A colon path would look like:
```
"users.:2313.name" >> "Sara"
```
Supported types
---------------
Pretty much any type is supported:
```go
sjson.Set(`{"key":true}`, "key", nil)
sjson.Set(`{"key":true}`, "key", false)
sjson.Set(`{"key":true}`, "key", 1)
sjson.Set(`{"key":true}`, "key", 10.5)
sjson.Set(`{"key":true}`, "key", "hello")
sjson.Set(`{"key":true}`, "key", map[string]interface{}{"hello":"world"})
```
When a type is not recognized, SJSON will fallback to the `encoding/json` Marshaller.
Examples
--------
Set a value from empty document:
```go
value, _ := sjson.Set("", "name", "Tom")
println(value)
// Output:
// {"name":"Tom"}
```
Set a nested value from empty document:
```go
value, _ := sjson.Set("", "name.last", "Anderson")
println(value)
// Output:
// {"name":{"last":"Anderson"}}
```
Set a new value:
```go
value, _ := sjson.Set(`{"name":{"last":"Anderson"}}`, "name.first", "Sara")
println(value)
// Output:
// {"name":{"first":"Sara","last":"Anderson"}}
```
Update an existing value:
```go
value, _ := sjson.Set(`{"name":{"last":"Anderson"}}`, "name.last", "Smith")
println(value)
// Output:
// {"name":{"first":"Sara","last":"Smith"}}
```
Set a new array value:
```go
value, _ := sjson.Set(`{"friends":["Andy","Carol"]}`, "friends.2", "Sara")
println(value)
// Output:
// {"friends":["Andy","Carol","Sara"]
```
Append an array value by using the `-1` key in a path:
```go
value, _ := sjson.Set(`{"friends":["Andy","Carol"]}`, "friends.-1", "Sara")
println(value)
// Output:
// {"friends":["Andy","Carol","Sara"]
```
Append an array value that is past the end:
```go
value, _ := sjson.Set(`{"friends":["Andy","Carol"]}`, "friends.4", "Sara")
println(value)
// Output:
// {"friends":["Andy","Carol",null,null,"Sara"]
```
## Contact
Josh Baker [@tidwall](http://twitter.com/tidwall)
## License
SJSON source code is available under the MIT [License](/LICENSE).

BIN
logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

392
sjson.go Normal file
View File

@ -0,0 +1,392 @@
// Package sjson provides setting json values.
package sjson
import (
jsongo "encoding/json"
"reflect"
"strconv"
"unsafe"
"github.com/tidwall/gjson"
)
type errorType struct {
msg string
}
func (err *errorType) Error() string {
return err.msg
}
type pathResult struct {
part string // current key part
path string // remaining path
force bool // force a string key
more bool // there is more path to parse
}
func parsePath(path string) (pathResult, error) {
var r pathResult
if len(path) > 0 && path[0] == ':' {
r.force = true
path = path[1:]
}
for i := 0; i < len(path); i++ {
if path[i] == '.' {
r.part = path[:i]
r.path = path[i+1:]
r.more = true
return r, nil
}
if path[i] == '*' || path[i] == '?' {
return r, &errorType{"wildcard characters not allowed in path"}
} else if path[i] == '#' {
return r, &errorType{"array access character not allowed in path"}
}
if path[i] == '\\' {
// go into escape mode. this is a slower path that
// strips off the escape character from the part.
epart := []byte(path[:i])
i++
if i < len(path) {
epart = append(epart, path[i])
i++
for ; i < len(path); i++ {
if path[i] == '\\' {
i++
if i < len(path) {
epart = append(epart, path[i])
}
continue
} else if path[i] == '.' {
r.part = string(epart)
r.path = path[i+1:]
r.more = true
return r, nil
} else if path[i] == '*' || path[i] == '?' {
return r, &errorType{"wildcard characters not allowed in path"}
} else if path[i] == '#' {
return r, &errorType{"array access character not allowed in path"}
}
epart = append(epart, path[i])
}
}
// append the last part
r.part = string(epart)
return r, nil
}
}
r.part = path
return r, nil
}
// appendStringify makes a json string and appends to buf.
func appendStringify(buf []byte, s string) []byte {
for i := 0; i < len(s); i++ {
if s[i] < ' ' || s[i] > 0x7f || s[i] == '"' {
b, _ := jsongo.Marshal(s)
return append(buf, b...)
}
}
buf = append(buf, '"')
buf = append(buf, s...)
buf = append(buf, '"')
return buf
}
// appendBuild builds a json block from a json path.
func appendBuild(buf []byte, array bool, paths []pathResult, raw string, stringify bool) []byte {
if !array {
buf = appendStringify(buf, paths[0].part)
buf = append(buf, ':')
}
if len(paths) > 1 {
n, numeric := atoui(paths[1])
if numeric {
buf = append(buf, '[')
buf = appendRepeat(buf, "null,", n)
buf = appendBuild(buf, true, paths[1:], raw, stringify)
buf = append(buf, ']')
} else {
buf = append(buf, '{')
buf = appendBuild(buf, false, paths[1:], raw, stringify)
buf = append(buf, '}')
}
} else {
if stringify {
buf = appendStringify(buf, raw)
} else {
buf = append(buf, raw...)
}
}
return buf
}
// atoui does a rip conversion of string -> unigned int.
func atoui(r pathResult) (n int, ok bool) {
if r.force {
return 0, false
}
for i := 0; i < len(r.part); i++ {
if r.part[i] < '0' || r.part[i] > '9' {
return 0, false
}
n = n*10 + int(r.part[i]-'0')
}
return n, true
}
// appendRepeat repeats string "n" times and appends to buf.
func appendRepeat(buf []byte, s string, n int) []byte {
for i := 0; i < n; i++ {
buf = append(buf, s...)
}
return buf
}
// trim does a rip trim
func trim(s string) string {
for len(s) > 0 {
if s[0] <= ' ' {
s = s[1:]
continue
}
break
}
for len(s) > 0 {
if s[len(s)-1] <= ' ' {
s = s[:len(s)-1]
continue
}
break
}
return s
}
func appendRawPaths(buf []byte, jstr string, paths []pathResult, raw string, stringify bool) ([]byte, error) {
var err error
res := gjson.Get(jstr, paths[0].part)
if res.Index > 0 {
if len(paths) > 1 {
buf = append(buf, jstr[:res.Index]...)
buf, err = appendRawPaths(buf, res.Raw, paths[1:], raw, stringify)
if err != nil {
return nil, err
}
buf = append(buf, jstr[res.Index+len(res.Raw):]...)
return buf, nil
}
buf = append(buf, jstr[:res.Index]...)
if stringify {
buf = appendStringify(buf, raw)
} else {
buf = append(buf, raw...)
}
buf = append(buf, jstr[res.Index+len(res.Raw):]...)
return buf, nil
}
n, numeric := atoui(paths[0])
isempty := true
for i := 0; i < len(jstr); i++ {
if jstr[i] > ' ' {
isempty = false
break
}
}
if isempty {
if numeric {
jstr = "[]"
} else {
jstr = "{}"
}
}
jsres := gjson.Parse(jstr)
if jsres.Type != gjson.JSON {
return nil, &errorType{"json must be an object or array"}
}
var comma bool
for i := 1; i < len(jsres.Raw); i++ {
if jsres.Raw[i] <= ' ' {
continue
}
if jsres.Raw[i] == '}' || jsres.Raw[i] == ']' {
break
}
comma = true
break
}
switch jsres.Raw[0] {
default:
return nil, &errorType{"json must be an object or array"}
case '{':
buf = append(buf, '{')
buf = appendBuild(buf, false, paths, raw, stringify)
if comma {
buf = append(buf, ',')
}
buf = append(buf, jsres.Raw[1:]...)
return buf, nil
case '[':
var appendit bool
if !numeric {
if paths[0].part == "-1" && !paths[0].force {
appendit = true
} else {
return nil, &errorType{"array key must be numeric"}
}
}
if appendit {
njson := trim(jsres.Raw)
if njson[len(njson)-1] == ']' {
njson = njson[:len(njson)-1]
}
buf = append(buf, njson...)
if comma {
buf = append(buf, ',')
}
buf = appendBuild(buf, true, paths, raw, stringify)
buf = append(buf, ']')
return buf, nil
}
buf = append(buf, '[')
ress := jsres.Array()
for i := 0; i < len(ress); i++ {
if i > 0 {
buf = append(buf, ',')
}
buf = append(buf, ress[i].Raw...)
}
if len(ress) == 0 {
buf = appendRepeat(buf, "null,", n-len(ress))
} else {
buf = appendRepeat(buf, ",null", n-len(ress))
if comma {
buf = append(buf, ',')
}
}
buf = appendBuild(buf, true, paths, raw, stringify)
buf = append(buf, ']')
return buf, nil
}
}
func set(jstr, path, raw string, stringify bool) ([]byte, error) {
// parse the path, make sure that it does not contain invalid characters
// such as '#', '?', '*'
if path == "" {
return nil, &errorType{"path cannot be empty"}
}
paths := make([]pathResult, 0, 4)
r, err := parsePath(path)
if err != nil {
return nil, err
}
paths = append(paths, r)
for r.more {
if r, err = parsePath(r.path); err != nil {
return nil, err
}
paths = append(paths, r)
}
njson, err := appendRawPaths(nil, jstr, paths, raw, stringify)
if err != nil {
return nil, err
}
return njson, nil
}
// Set sets a json value for the specified path.
// A path is in dot syntax, such as "name.last" or "age".
// This function expects that the json is well-formed, and does not validate.
// Invalid json will not panic, but it may return back unexpected results.
// An error is returned if the path is not valid.
//
// A path is a series of keys seperated by a dot.
//
// {
// "name": {"first": "Tom", "last": "Anderson"},
// "age":37,
// "children": ["Sara","Alex","Jack"],
// "friends": [
// {"first": "James", "last": "Murphy"},
// {"first": "Roger", "last": "Craig"}
// ]
// }
// "name.last" >> "Anderson"
// "age" >> 37
// "children.1" >> "Alex"
//
func Set(json, path string, value interface{}) (string, error) {
jsonh := *(*reflect.StringHeader)(unsafe.Pointer(&json))
jsonbh := reflect.SliceHeader{Data: jsonh.Data, Len: jsonh.Len}
jsonb := *(*[]byte)(unsafe.Pointer(&jsonbh))
res, err := SetBytes(jsonb, path, value)
return string(res), err
}
// SetRaw sets a raw json value for the specified path. The works the same as
// Set except that the value is set as a raw block of json. This allows for setting
// premarshalled json objects.
func SetRaw(json, path, value string) (string, error) {
res, err := set(json, path, value, false)
return string(res), err
}
// SetRawBytes sets a raw json value for the specified path.
// If working with bytes, this method preferred over SetRaw(string(data), path, value)
func SetRawBytes(json []byte, path string, value []byte) ([]byte, error) {
jstr := *(*string)(unsafe.Pointer(&json))
vstr := *(*string)(unsafe.Pointer(&value))
return set(jstr, path, vstr, false)
}
// SetBytes sets a json value for the specified path.
// If working with bytes, this method preferred over Set(string(data), path, value)
func SetBytes(json []byte, path string, value interface{}) ([]byte, error) {
jstr := *(*string)(unsafe.Pointer(&json))
var res []byte
var err error
switch v := value.(type) {
default:
b, err := jsongo.Marshal(value)
if err != nil {
return nil, err
}
raw := *(*string)(unsafe.Pointer(&b))
res, err = set(jstr, path, raw, false)
case string:
res, err = set(jstr, path, v, true)
case []byte:
raw := *(*string)(unsafe.Pointer(&v))
res, err = set(jstr, path, raw, true)
case bool:
if v {
res, err = set(jstr, path, "true", false)
} else {
res, err = set(jstr, path, "false", false)
}
case int8:
res, err = set(jstr, path, strconv.FormatInt(int64(v), 10), false)
case int16:
res, err = set(jstr, path, strconv.FormatInt(int64(v), 10), false)
case int32:
res, err = set(jstr, path, strconv.FormatInt(int64(v), 10), false)
case int64:
res, err = set(jstr, path, strconv.FormatInt(int64(v), 10), false)
case uint8:
res, err = set(jstr, path, strconv.FormatUint(uint64(v), 10), false)
case uint16:
res, err = set(jstr, path, strconv.FormatUint(uint64(v), 10), false)
case uint32:
res, err = set(jstr, path, strconv.FormatUint(uint64(v), 10), false)
case uint64:
res, err = set(jstr, path, strconv.FormatUint(uint64(v), 10), false)
case float32:
res, err = set(jstr, path, strconv.FormatFloat(float64(v), 'f', -1, 64), false)
case float64:
res, err = set(jstr, path, strconv.FormatFloat(float64(v), 'f', -1, 64), false)
}
return res, err
}

151
sjson_test.go Normal file
View File

@ -0,0 +1,151 @@
package sjson
import (
"encoding/hex"
"math/rand"
"testing"
"time"
)
func TestInvalidPaths(t *testing.T) {
var err error
_, err = SetRaw(`{"hello":"world"}`, "", `"planet"`)
if err == nil || err.Error() != "path cannot be empty" {
t.Fatalf("expecting '%v', got '%v'", "path cannot be empty", err)
}
_, err = SetRaw("", "name.last.#", "")
if err == nil || err.Error() != "array access character not allowed in path" {
t.Fatalf("expecting '%v', got '%v'", "array access character not allowed in path", err)
}
_, err = SetRaw("", "name.last.\\1#", "")
if err == nil || err.Error() != "array access character not allowed in path" {
t.Fatalf("expecting '%v', got '%v'", "array access character not allowed in path", err)
}
_, err = SetRaw("", "name.las?t", "")
if err == nil || err.Error() != "wildcard characters not allowed in path" {
t.Fatalf("expecting '%v', got '%v'", "wildcard characters not allowed in path", err)
}
_, err = SetRaw("", "name.la\\s?t", "")
if err == nil || err.Error() != "wildcard characters not allowed in path" {
t.Fatalf("expecting '%v', got '%v'", "wildcard characters not allowed in path", err)
}
_, err = SetRaw("", "name.las*t", "")
if err == nil || err.Error() != "wildcard characters not allowed in path" {
t.Fatalf("expecting '%v', got '%v'", "wildcard characters not allowed in path", err)
}
_, err = SetRaw("", "name.las\\a*t", "")
if err == nil || err.Error() != "wildcard characters not allowed in path" {
t.Fatalf("expecting '%v', got '%v'", "wildcard characters not allowed in path", err)
}
}
const (
setRaw = 1
setBool = 2
setInt = 3
setFloat = 4
setString = 5
)
func testRaw(t *testing.T, kind int, expect, json, path string, value interface{}) {
var json2 string
var err error
switch kind {
default:
json2, err = Set(json, path, value)
case setRaw:
json2, err = SetRaw(json, path, value.(string))
}
if err != nil {
t.Fatal(err)
} else if json2 != expect {
t.Fatalf("expected '%v', got '%v'", expect, json2)
}
var json3 []byte
switch kind {
default:
json3, err = SetBytes([]byte(json), path, value)
case setRaw:
json3, err = SetRawBytes([]byte(json), path, []byte(value.(string)))
}
if err != nil {
t.Fatal(err)
} else if string(json3) != expect {
t.Fatalf("expected '%v', got '%v'", expect, string(json3))
}
}
func TestBasic(t *testing.T) {
testRaw(t, setRaw, `[{"hiw":"planet","hi":"world"}]`, `[{"hi":"world"}]`, "0.hiw", `"planet"`)
testRaw(t, setRaw, `[true]`, ``, "0", `true`)
testRaw(t, setRaw, `[null,true]`, ``, "1", `true`)
testRaw(t, setRaw, `[1,null,true]`, `[1]`, "2", `true`)
testRaw(t, setRaw, `[1,true,false]`, `[1,null,false]`, "1", `true`)
testRaw(t, setRaw,
`[1,{"hello":"when","this":[0,null,2]},false]`,
`[1,{"hello":"when","this":[0,1,2]},false]`,
"1.this.1", `null`)
testRaw(t, setRaw,
`{"a":1,"b":{"hello":"when","this":[0,null,2]},"c":false}`,
`{"a":1,"b":{"hello":"when","this":[0,1,2]},"c":false}`,
"b.this.1", `null`)
testRaw(t, setRaw,
`{"a":1,"b":{"hello":"when","this":[0,null,2,null,4]},"c":false}`,
`{"a":1,"b":{"hello":"when","this":[0,null,2]},"c":false}`,
"b.this.4", `4`)
testRaw(t, setRaw,
`{"b":{"this":[null,null,null,null,4]}}`,
``,
"b.this.4", `4`)
testRaw(t, setRaw,
`[null,{"this":[null,null,null,null,4]}]`,
``,
"1.this.4", `4`)
testRaw(t, setRaw,
`{"1":{"this":[null,null,null,null,4]}}`,
``,
":1.this.4", `4`)
testRaw(t, setRaw,
`{":1":{"this":[null,null,null,null,4]}}`,
``,
"\\:1.this.4", `4`)
testRaw(t, setRaw,
`{":\1":{"this":[null,null,null,null,{".HI":4}]}}`,
``,
"\\:\\\\1.this.4.\\.HI", `4`)
testRaw(t, setRaw,
`{"b":{"this":{"😇":""}}}`,
``,
"b.this.😇", `""`)
testRaw(t, setRaw,
`[ 1,2 ,3]`,
` [ 1,2 ] `,
"-1", `3`)
testRaw(t, setInt, `[1234]`, ``, `0`, int64(1234))
testRaw(t, setFloat, `[1234.5]`, ``, `0`, float64(1234.5))
testRaw(t, setString, `["1234.5"]`, ``, `0`, "1234.5")
testRaw(t, setBool, `[true]`, ``, `0`, true)
}
// TestRandomData is a fuzzing test that throws random data at SetRaw
// function looking for panics.
func TestRandomData(t *testing.T) {
var lstr string
defer func() {
if v := recover(); v != nil {
println("'" + hex.EncodeToString([]byte(lstr)) + "'")
println("'" + lstr + "'")
panic(v)
}
}()
rand.Seed(time.Now().UnixNano())
b := make([]byte, 200)
for i := 0; i < 2000000; i++ {
n, err := rand.Read(b[:rand.Int()%len(b)])
if err != nil {
t.Fatal(err)
}
lstr = string(b[:n])
SetRaw(lstr, "zzzz.zzzz.zzzz", "123")
}
}