re
/
buntdb
forked from mirror/buntdb
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

initial commit

This commit is contained in:
Josh Baker 2016-07-19 15:15:00 -07:00
commit 7875d65f2a
6 changed files with 2483 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.travis.yml Normal file
View File

@ -0,0 +1 @@
language: go

20
LICENSE Normal file
View File

@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright (c) 2016 Josh Baker
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

314
README.md Normal file
View File

@ -0,0 +1,314 @@
<p align="center">
<img
src="logo.png"
width="307" height="150" border="0" alt="BuntDB">
<br>
<a href="https://travis-ci.org/tidwall/buntdb"><img src="https://travis-ci.org/tidwall/buntdb.svg?branch=master" alt="Build Status"></a>
<img src="https://img.shields.io/badge/coverage-96%25-green.svg?style=flat" alt="Code Coverage">
<a href="https://godoc.org/github.com/tidwall/buntdb"><img src="https://godoc.org/github.com/tidwall/buntdb?status.svg" alt="GoDoc"></a>
<img src="https://img.shields.io/badge/version-0.1.0-green.svg" alt="Version">
</p>
====
BuntDB is a low-level, in-memory, key/value store in pure Go.
It persists to disk, is ACID compliant, and uses locking for multiple
readers and a single writer. It supports custom indexes and geospatial
data. It's ideal for projects that need a dependable database and favor
speed over data size.
The desire to create BuntDB stems from the need for a new embeddable
database for [Tile38](https://github.com/tidwall/tile38). One that can work
both as a performant [Raft Store](https://github.com/tidwall/raft-boltdb),
and a Geospatial database.
Much of the API is inspired by the fantastic [BoltDB](https://github.com/boltdb/bolt),
an amazing key/value store that can handle terrabytes of data on disk.
Features
========
- In-memory database for [fast reads and writes](https://github.com/tidwall/raft-boltdb#benchmarks)
- Embeddable with a [simple API](https://godoc.org/github.com/tidwall/buntdb)
- [Spatial indexing](#spatial-indexes) for up to 4 dimensions; Useful for Geospatial data
- - Create [custom indexes](#custom-indexes) for any data type
- [Built-in types](#built-in-types) that are easy to get up & running; String, Uint, Int, Float
- Flexible [iteration](#iterating) of data; ascending, descending, and ranges
- Durable append-only file format. Adopts the [Redis AOF](http://redis.io/topics/persistence) process
- Option to evict old items with an [expiration](#data-expiration) TTL
- Tight codebase, under 1K loc using the `cloc` command.
- ACID semantics with locking [transactions](#transactions) that support rollbacks
Getting Started
===============
## Installing
To start using BuntDB, install Go and run `go get`:
```sh
$ go get github.com/tidwall/buntdb
```
This will retrieve the library.
## Opening a database
The primary object in BuntDB is a `DB`. To open or create your
database, use the `buntdb.Open()` function:
```go
package main
import (
"log"
"github.com/tidwall/buntdb"
)
func main() {
// Open the data.db file. It will be created if it doesn't exist.
db, err := buntdb.Open("data.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
...
}
```
It's important to note that BuntDB does not currently support file locking, so avoid accessing the database from multiple processes.
## Transactions
All reads and writes must be performed from inside a transaction. BuntDB can have one write transaction opened at a time, but can have many concurrent read transactions. Each transaction maintains a stable view of the database. In other words, once a transaction has begun, the data for that transaction cannot be changed by other transactions.
Transactions run in a function that exposes a `Tx` object, which represents the transaction state. While inside a transaction, all database operations should be performed using this object. You should never access the origin `DB` object while inside a transaction. Doing so may have side-effects, such as blocking your application.
When a transaction fails, it will roll back, and revert all chanages that ocurred to the database during that transaction. There's a single return value that you can use to close the transaction. For read/write transactions, returning an error this way will force the transaction to roll back. When a read/write transaction succeeds all changes are persisted to disk.
### Read-only Transactions
A read-only transaction should be used when you don't need to make changes to the data. The advantage of a read-only transaction is that there can be many running concurrently.
```go
err := db.View(func(tx *buntdb.Tx) error {
...
return nil
})
```
### Read/write Transactions
A read/write transaction is used when you need to make changes to your data. There can only be one read/write transaction running at a time. So make sure you close it as soon as you are done with it.
```go
err := db.Update(func(tx *buntdb.Tx) error {
...
return nil
})
```
## Setting and getting key/values
To set a value you must open a read/write tranasction:
```go
err := db.Update(func(tx *buntdb.Tx) error {
err := tx.Set("mykey", "myvalue", nil)
return err
})
```
To get the value:
```go
err := db.View(func(tx *buntdb.Tx) error {
val, err := tx.Get("mykey")
if err != nil{
return err
}
fmt.Printf("value is %s\n", val)
return nil
})
```
Getting non-existent values will case an `ErrNotFound` error.
### Iterating
All keys/value pairs are ordered in the database by the key. To iterate over the keys:
```go
err := db.View(func(tx *buntdb.Tx) error {
err := tx.Ascend("", func(key, value string) bool{
fmt.Printf("key: %s, value: %s\n", key, value)
})
return err
})
```
There is also `AscendGreaterOrEqual`, `AscendLessThan`, `AscendRange`, `Descend`, `DescendLessOrEqual`, `DescendGreaterThan`, and `DescendRange`. Please see the [documentation](https://godoc.org/github.com/tidwall/buntdb) for more information on these functions.
## Custom Indexes
Initially all data is stored in a single [B-tree](https://en.wikipedia.org/wiki/B-tree) with each item having one key and one value. All of these items are ordered by the key. This is great for quickly getting a value from a key or [iterating](#iterating) over the keys.
You can also create custom indexes that allow for ordering and [iterating](#iterating) over values. A custom index also uses a B-tree, but it's more flexible because it allows for custom ordering.
For example, let's say you want to create an index for ordering names:
```go
db.CreateIndex("names", "*", buntdb.IndexString)
```
This will create an index named `names` which stores and sorts all values. The second parameter is a pattern that is used to filter on keys. A `*` wildcard argument means that we want to accept all keys. `IndexString` is a built-in function that performs case-insensitive ordering on the values
Now you can add various names:
```go
db.Update(func(tx *buntdb.Tx) error {
tx.Set("user:0:name", "tom", nil)
tx.Set("user:1:name", "Randi", nil)
tx.Set("user:2:name", "jane", nil)
tx.Set("user:4:name", "Janet", nil)
tx.Set("user:5:name", "Paula", nil)
tx.Set("user:6:name", "peter", nil)
tx.Set("user:7:name", "Terri", nil)
return nil
})
```
Finally you can iterate over the index:
```go
db.View(func(tx *buntdb.Tx) error {
tx.Ascend("names", func(key, val string) bool {
fmt.Printf(buf, "%s %s\n", key, val)
return true
})
return nil
})
```
The output should be:
```
user:2:name jane
user:4:name Janet
user:5:name Paula
user:6:name peter
user:1:name Randi
user:7:name Terri
user:0:name tom
```
The pattern parameter can be used to filter on keys like this:
```go
db.CreateIndex("names", "user:*", buntdb.IndexString)
```
Now only items with keys that have the prefix `user:` will be added to the `names` index.
### Built-in types
Along with `IndexString`, there is also `IndexInt`, `IndexUint`, and `IndexFloat`.
These are built-in types for indexing. You can choose to use these or create your own.
So to create an index that is numerically ordered on an age key, we could use:
```go
db.CreateIndex("ages", "user:*:age", buntdb.IndexInt)
```
And then add values:
```go
db.Update(func(tx *buntdb.Tx) error {
tx.Set("user:0:age", "35", nil)
tx.Set("user:1:age", "49", nil)
tx.Set("user:2:age", "13", nil)
tx.Set("user:4:age", "63", nil)
tx.Set("user:5:age", "8", nil)
tx.Set("user:6:age", "3", nil)
tx.Set("user:7:age", "16", nil)
return nil
})
```
```go
db.View(func(tx *buntdb.Tx) error {
tx.Ascend("ages", func(key, val string) bool {
fmt.Printf(buf, "%s %s\n", key, val)
return true
})
return nil
})
```
The output should be:
```
user:6:name 3
user:5:name 8
user:2:name 13
user:7:name 16
user:0:name 35
user:1:name 49
user:4:name 63
```
### Spatial Indexes
BuntDB has support for spatial indexes by storing rectangles in an [R-tree](https://en.wikipedia.org/wiki/R-tree). An R-tree is organized in a similar manner as a [B-tree](https://en.wikipedia.org/wiki/B-tree), and both are balanaced trees. But, an R-tree is special because it can operate on data that is in multiple dimensions. This is super handy for Geospatial applications.
To create a spatial index use the `CreateSpatialIndex` function:
```go
db.CreateSpatialIndex("fleet", "fleet:*:pos", buntdb.IndexRect)
```
Then `IndexRect` is a built-in function that converts rect strings to a format that the R-tree can use. It's easy to use this function out of the box, but you might find it better to create a custom one that renders from a different format, such as [Well-known text](https://en.wikipedia.org/wiki/Well-known_text) or [GeoJSON](http://geojson.org/).
To add some lon,lat points to the `fleet` index:
```go
db.Update(func(tx *buntdb.Tx) error {
tx.Set("fleet:0:pos", "[-115.567 33.532]", nil)
tx.Set("fleet:1:pos", "[-116.671 35.735]", nil)
tx.Set("fleet:2:pos", "[-113.902 31.234]", nil)
return nil
})
```
And then you can run the `Intersects` function on the index:
```go
db.View(func(tx *buntdb.Tx) error {
tx.Intersects("fleet", "[-117 30],[-112 36]", func(key, val string) bool {
...
return true
})
return nil
})
```
This will get all three positions.
The bracket syntax `[-117 30],[-112 36]` is unique to BuntDB, and it's how the built-in rectangles are processed, but you are not limited to this syntax. Whatever Rect function you choose to use during `CreateSpatialIndex` will be used to process the paramater, in this case it's `IndexRect`.
### Data Expiration
Items can be automatically evicted by using the `SetOptions` object in the `Set` function to set a `TTL`.
```go
db.Update(func(tx *buntdb.Tx) error {
tx.Set("mykey", "myval", &buntdb.SetOptions{Expires:true, TTL:time.Second})
return nil
})
```
Now `mykey` will automatically be deleted after one second. You can remove the TTL by setting the value again with the same key/value, but with the options parameter set to nil.
## Contact
Josh Baker [@tidwall](http://twitter.com/tidwall)
## License
BuntDB source code is available under the MIT [License](/LICENSE).

1357
buntdb.go Normal file
View File

File diff suppressed because it is too large Load Diff

791
buntdb_test.go Normal file
View File

@ -0,0 +1,791 @@
package buntdb
import (
"bufio"
"bytes"
"errors"
"fmt"
"io/ioutil"
"math/rand"
"os"
"strings"
"testing"
"time"
)
func TestBackgroudOperations(t *testing.T) {
os.RemoveAll("data.db")
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
for i := 0; i < 1000; i++ {
if err := db.Update(func(tx *Tx) error {
for j := 0; j < 200; j++ {
tx.Set(fmt.Sprintf("hello%d", j), "planet", nil)
}
tx.Set("hi", "world", &SetOptions{Expires: true, TTL: time.Second / 2})
return nil
}); err != nil {
t.Fatal(err)
}
}
n := 0
db.View(func(tx *Tx) error {
n, _ = tx.Len()
return nil
})
if n != 201 {
t.Fatalf("expecting '%v', got '%v'", 201, n)
}
time.Sleep(time.Millisecond * 1500)
db.Close()
db, err = Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
n = 0
db.View(func(tx *Tx) error {
n, _ = tx.Len()
return nil
})
if n != 200 {
t.Fatalf("expecting '%v', got '%v'", 200, n)
}
}
func TestVariousTx(t *testing.T) {
os.RemoveAll("data.db")
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
if err := db.Update(func(tx *Tx) error {
tx.Set("hello", "planet", nil)
return nil
}); err != nil {
t.Fatal(err)
}
errBroken := errors.New("broken")
if err := db.Update(func(tx *Tx) error {
tx.Set("hello", "world", nil)
return errBroken
}); err != errBroken {
t.Fatalf("did not correctly receive the user-defined transaction error.")
}
var val string
db.View(func(tx *Tx) error {
val, _ = tx.Get("hello")
return nil
})
if val == "world" {
t.Fatal("a rollbacked transaction got through")
}
if val != "planet" {
t.Fatal("expecting '%v', got '%v'", "planet", val)
}
if err := db.Update(func(tx *Tx) error {
tx.db = nil
if _, _, err := tx.Set("hello", "planet", nil); err != ErrTxClosed {
t.Fatal("expecting a tx closed error")
}
if _, err := tx.Delete("hello"); err != ErrTxClosed {
t.Fatal("expecting a tx closed error")
}
if _, err := tx.Get("hello"); err != ErrTxClosed {
t.Fatal("expecting a tx closed error")
}
tx.db = db
tx.writable = false
if _, _, err := tx.Set("hello", "planet", nil); err != ErrTxNotWritable {
t.Fatal("expecting a tx not writable error")
}
if _, err := tx.Delete("hello"); err != ErrTxNotWritable {
t.Fatal("expecting a tx not writable error")
}
tx.writable = true
if _, err := tx.Get("something"); err != ErrNotFound {
t.Fatalf("expecting not found error")
}
if _, err := tx.Delete("something"); err != ErrNotFound {
t.Fatalf("expecting not found error")
}
tx.Set("var", "val", &SetOptions{Expires: true, TTL: 0})
if _, err := tx.Get("var"); err != ErrNotFound {
t.Fatalf("expecting not found error")
}
if _, err := tx.Delete("var"); err != ErrNotFound {
tx.unlock()
t.Fatalf("expecting not found error")
}
return nil
}); err != nil {
t.Fatal(err)
}
// test for invalid commits
if err := db.Update(func(tx *Tx) error {
// we are going to do some hackery
defer func() {
if v := recover(); v != nil {
if v.(string) != "managed tx commit not allowed" {
t.Fatal(v.(string))
}
}
}()
tx.commit()
return nil
}); err != nil {
t.Fatal(err)
}
// test for invalid commits
if err := db.Update(func(tx *Tx) error {
// we are going to do some hackery
defer func() {
if v := recover(); v != nil {
if v.(string) != "managed tx rollback not allowed" {
t.Fatal(v.(string))
}
}
}()
tx.rollback()
return nil
}); err != nil {
t.Fatal(err)
}
// test for closed transactions
if err := db.Update(func(tx *Tx) error {
tx.db = nil
return nil
}); err != ErrTxClosed {
t.Fatal("expecting tx closed error")
}
db.mu.Unlock()
// test for invalid writes
if err := db.Update(func(tx *Tx) error {
tx.writable = false
return nil
}); err != ErrTxNotWritable {
t.Fatal("expecting tx not writable error")
}
db.mu.Unlock()
// test for closed transactions
if err := db.View(func(tx *Tx) error {
tx.db = nil
return nil
}); err != ErrTxClosed {
t.Fatal("expecting tx closed error")
}
db.mu.RUnlock()
// flush to unwritable file
if err := db.Update(func(tx *Tx) error {
tx.Set("var1", "val1", nil)
tx.db.file.Close()
return nil
}); err == nil {
t.Fatal("should not be able to commit when the file is closed")
}
db.file, err = os.OpenFile("data.db", os.O_CREATE|os.O_RDWR, 0666)
if err != nil {
t.Fatal(err)
}
if _, err := db.file.Seek(0, 2); err != nil {
t.Fatal(err)
}
db.bufw = bufio.NewWriter(db.file)
db.CreateIndex("blank", "*", nil)
// test scanning
if err := db.Update(func(tx *Tx) error {
tx.Set("nothing", "here", nil)
return nil
}); err != nil {
t.Fatal(err)
}
if err := db.View(func(tx *Tx) error {
s := ""
tx.Ascend("", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "hello:planet\nnothing:here\n" {
t.Fatal("invalid scan")
}
tx.db = nil
err = tx.Ascend("", func(key, val string) bool { return true })
if err != ErrTxClosed {
tx.unlock()
t.Fatal("expecting tx closed error")
}
tx.db = db
err = tx.Ascend("na", func(key, val string) bool { return true })
if err != ErrNotFound {
t.Fatal("expecting not found error")
}
err = tx.Ascend("blank", func(key, val string) bool { return true })
if err != nil {
t.Fatal(err)
}
s = ""
tx.AscendLessThan("", "liger", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "hello:planet\n" {
t.Fatal("invalid scan")
}
s = ""
tx.Descend("", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "nothing:here\nhello:planet\n" {
t.Fatal("invalid scan")
}
s = ""
tx.DescendLessOrEqual("", "liger", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "hello:planet\n" {
t.Fatal("invalid scan")
}
s = ""
tx.DescendGreaterThan("", "liger", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "nothing:here\n" {
t.Fatal("invalid scan")
}
s = ""
tx.DescendRange("", "liger", "apple", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "hello:planet\n" {
t.Fatal("invalid scan")
}
return nil
}); err != nil {
t.Fatal(err)
}
// test some spatial stuff
db.CreateSpatialIndex("spat", "rect:*", IndexRect)
db.CreateSpatialIndex("junk", "rect:*", nil)
db.Update(func(tx *Tx) error {
tx.Set("rect:1", "[10 10],[20 20]", nil)
tx.Set("rect:2", "[15 15],[25 25]", nil)
tx.Set("shape:1", "[12 12],[25 25]", nil)
s := ""
tx.Intersects("spat", "[5 5],[13 13]", func(key, val string) bool {
s += key + ":" + val + "\n"
return true
})
if s != "rect:1:[10 10],[20 20]\n" {
t.Fatal("invalid scan")
}
tx.db = nil
err := tx.Intersects("spat", "[5 5],[13 13]", func(key, val string) bool {
return true
})
if err != ErrTxClosed {
t.Fatal("expecting tx closed error")
}
tx.db = db
err = tx.Intersects("", "[5 5],[13 13]", func(key, val string) bool {
return true
})
if err != nil {
t.Fatal(err)
}
err = tx.Intersects("na", "[5 5],[13 13]", func(key, val string) bool {
return true
})
if err != ErrNotFound {
t.Fatal("expecting not found error")
}
err = tx.Intersects("junk", "[5 5],[13 13]", func(key, val string) bool {
return true
})
if err != nil {
t.Fatal(err)
}
n, err := tx.Len()
if err != nil {
t.Fatal(err)
}
if n != 5 {
t.Fatal("expecting %v, got %v", 5, n)
}
tx.db = nil
_, err = tx.Len()
if err != ErrTxClosed {
t.Fatal("expecting tx closed error")
}
tx.db = db
return nil
})
// test after closing
db.Close()
if err := db.Update(func(tx *Tx) error { return nil }); err != ErrDatabaseClosed {
t.Fatalf("should not be able to perform transactionso on a closed database.")
}
}
func TestNoExpiringItem(t *testing.T) {
item := &dbItem{key: "key", val: "val"}
if !item.expiresAt().Equal(maxTime) {
t.Fatal("item.expiresAt() != maxTime")
}
if min, max := item.Rect(nil); min != nil || max != nil {
t.Fatal("item min,max should both be nil")
}
}
// test database format loading
func TestDatabaseFormat(t *testing.T) {
// should succeed
func() {
resp := strings.Join([]string{
"*3\r\n$3\r\nset\r\n$4\r\nvar1\r\n$4\r\n1234\r\n",
"*3\r\n$3\r\nset\r\n$4\r\nvar2\r\n$4\r\n1234\r\n",
"*2\r\n$3\r\ndel\r\n$4\r\nvar1\r\n",
"*5\r\n$3\r\nset\r\n$3\r\nvar\r\n$3\r\nval\r\n$2\r\nex\r\n$2\r\n10\r\n",
}, "")
os.RemoveAll("data.db")
ioutil.WriteFile("data.db", []byte(resp), 0666)
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
}()
testBadFormat := func(resp string) {
os.RemoveAll("data.db")
ioutil.WriteFile("data.db", []byte(resp), 0666)
db, err := Open("data.db")
if err == nil {
db.Close()
os.RemoveAll("data.db")
t.Fatalf("invalid database should not be allowed")
}
}
testBadFormat("*3\r")
testBadFormat("*3\n")
testBadFormat("*a\r\n")
testBadFormat("*2\r\n")
testBadFormat("*2\r\n%3")
testBadFormat("*2\r\n$")
testBadFormat("*2\r\n$3\r\n")
testBadFormat("*2\r\n$3\r\ndel")
testBadFormat("*2\r\n$3\r\ndel\r\r")
testBadFormat("*0\r\n*2\r\n$3\r\ndel\r\r")
testBadFormat("*1\r\n$3\r\nnop\r\n")
testBadFormat("*1\r\n$3\r\ndel\r\n")
testBadFormat("*1\r\n$3\r\nset\r\n")
testBadFormat("*5\r\n$3\r\nset\r\n$3\r\nvar\r\n$3\r\nval\r\n$2\r\nxx\r\n$2\r\n10\r\n")
testBadFormat("*5\r\n$3\r\nset\r\n$3\r\nvar\r\n$3\r\nval\r\n$2\r\nex\r\n$2\r\naa\r\n")
}
func TestInsertsAndDeleted(t *testing.T) {
os.RemoveAll("data.db")
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
db.CreateIndex("any", "*", IndexString)
db.CreateSpatialIndex("rect", "*", IndexRect)
if err := db.Update(func(tx *Tx) error {
tx.Set("item1", "value1", &SetOptions{Expires: true, TTL: time.Second})
tx.Set("item2", "value2", nil)
tx.Set("item3", "value3", &SetOptions{Expires: true, TTL: time.Second})
return nil
}); err != nil {
t.Fatal(err)
}
// test replacing items in the database
if err := db.Update(func(tx *Tx) error {
if _, _, err := tx.Set("item1", "nvalue1", nil); err != nil {
return err
}
if _, _, err := tx.Set("item2", "nvalue2", nil); err != nil {
return err
}
if _, err := tx.Delete("item3"); err != nil {
return err
}
return nil
}); err != nil {
t.Fatal(err)
}
}
// test index compare functions
func TestIndexCompare(t *testing.T) {
if !IndexFloat("1.5", "1.6") {
t.Fatalf("expected true, got false")
}
if !IndexInt("-1", "2") {
t.Fatalf("expected true, got false")
}
if !IndexUint("10", "25") {
t.Fatalf("expected true, got false")
}
if !IndexBinary("Hello", "hello") {
t.Fatalf("expected true, got false")
}
if IndexString("hello", "hello") {
t.Fatalf("expected false, got true")
}
if IndexString("Hello", "hello") {
t.Fatalf("expected false, got true")
}
if IndexString("hello", "Hello") {
t.Fatalf("expected false, got true")
}
if !IndexString("gello", "Hello") {
t.Fatalf("expected true, got false")
}
if IndexString("Hello", "gello") {
t.Fatalf("expected false, got true")
}
if Rect(IndexRect("[1 2 3 4],[5 6 7 8]")) != "[1 2 3 4],[5 6 7 8]" {
t.Fatalf("expected '%v', got '%v'", "[1 2 3 4],[5 6 7 8]", Rect(IndexRect("[1 2 3 4],[5 6 7 8]")))
}
if Rect(IndexRect("[1 2 3 4]")) != "[1 2 3 4]" {
t.Fatalf("expected '%v', got '%v'", "[1 2 3 4]", Rect(IndexRect("[1 2 3 4]")))
}
if Rect(nil, nil) != "" {
t.Fatalf("expected '%v', got '%v'", "", Rect(nil, nil))
}
if Point(1, 2, 3) != "[1 2 3]" {
t.Fatalf("expected '%v', got '%v'", "[1 2 3]", Point(1, 2, 3))
}
}
// test opening a folder.
func TestOpeningAFolder(t *testing.T) {
os.RemoveAll("dir.tmp")
os.Mkdir("dir.tmp", 0700)
defer os.RemoveAll("dir.tmp")
db, err := Open("dir.tmp")
if err == nil {
db.Close()
t.Fatalf("opening a directory should not be allowed")
}
}
// test opening an invalid resp file.
func TestOpeningInvalidDatabaseFile(t *testing.T) {
os.RemoveAll("data.db")
ioutil.WriteFile("data.db", []byte("invalid\r\nfile"), 0666)
defer os.RemoveAll("data.db")
db, err := Open("data.db")
if err == nil {
db.Close()
t.Fatalf("invalid database should not be allowed")
}
}
// test closing a closed database.
func TestOpeningClosedDatabase(t *testing.T) {
os.RemoveAll("data.db")
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
if err := db.Close(); err != nil {
t.Fatal(err)
}
if err := db.Close(); err != ErrDatabaseClosed {
t.Fatal("should not be able to close a closed database")
}
}
func TestVariousIndexOperations(t *testing.T) {
os.RemoveAll("data.db")
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
// test creating an index with no index name.
err = db.CreateIndex("", "", nil)
if err == nil {
t.Fatal("should not be able to create an index with no name")
}
// test creating an index with a name that has already been used.
err = db.CreateIndex("hello", "", nil)
if err != nil {
t.Fatal(err)
}
err = db.CreateIndex("hello", "", nil)
if err == nil {
t.Fatal("should not be able to create a duplicate index")
}
db.Update(func(tx *Tx) error {
tx.Set("user:1", "tom", nil)
tx.Set("user:2", "janet", nil)
tx.Set("alt:1", "from", nil)
tx.Set("alt:2", "there", nil)
tx.Set("rect:1", "[1 2],[3 4]", nil)
tx.Set("rect:2", "[5 6],[7 8]", nil)
return nil
})
// test creating an index after adding items. use pattern matching. have some items in the match and some not.
if err := db.CreateIndex("string", "user:*", IndexString); err != nil {
t.Fatal(err)
}
// test creating a spatial index after adding items. use pattern matching. have some items in the match and some not.
if err := db.CreateSpatialIndex("rect", "rect:*", IndexRect); err != nil {
t.Fatal(err)
}
// test dropping an index
if err := db.DropIndex("hello"); err != nil {
t.Fatal(err)
}
// test dropping an index with no name
if err := db.DropIndex(""); err == nil {
t.Fatal("should not be allowed to drop an index with no name")
}
// test dropping an index with no name
if err := db.DropIndex("na"); err == nil {
t.Fatal("should not be allowed to drop an index that does not exist")
}
// test retrieving index names
names, err := db.Indexes()
if err != nil {
t.Fatal(err)
}
if strings.Join(names, ",") != "rect,string" {
t.Fatalf("expecting '%v', got '%v'", "rect,string", strings.Join(names, ","))
}
// test creating an index after closing database
if err := db.Close(); err != nil {
t.Fatal(err)
}
if err := db.CreateIndex("new-index", "", nil); err != ErrDatabaseClosed {
t.Fatal("should not be able to create an index on a closed database")
}
// test getting index names after closing database
if _, err := db.Indexes(); err != ErrDatabaseClosed {
t.Fatal("should not be able to get index names on a closed database")
}
// test dropping an index after closing database
if err := db.DropIndex("rect"); err != ErrDatabaseClosed {
t.Fatal("should not be able to drop an index on a closed database")
}
}
func test(t *testing.T, a, b bool) {
if a != b {
t.Fatal("failed, bummer...")
}
}
func TestPatternMatching(t *testing.T) {
test(t, wildcardMatch("hello", "hello"), true)
test(t, wildcardMatch("hello", "h*"), true)
test(t, wildcardMatch("hello", "h*o"), true)
test(t, wildcardMatch("hello", "h*l*o"), true)
test(t, wildcardMatch("hello", "h*z*o"), false)
test(t, wildcardMatch("hello", "*l*o"), true)
test(t, wildcardMatch("hello", "*l*"), true)
test(t, wildcardMatch("hello", "*?*"), true)
test(t, wildcardMatch("hello", "*"), true)
test(t, wildcardMatch("hello", "h?llo"), true)
test(t, wildcardMatch("hello", "h?l?o"), true)
test(t, wildcardMatch("", "*"), true)
test(t, wildcardMatch("", ""), true)
test(t, wildcardMatch("h", ""), false)
test(t, wildcardMatch("", "?"), false)
}
func TestBasic(t *testing.T) {
rand.Seed(time.Now().UnixNano())
os.RemoveAll("data.db")
db, err := Open("data.db")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll("data.db")
defer db.Close()
// create a simple index
db.CreateIndex("users", "fun:user:*", IndexString)
// create a spatial index
db.CreateSpatialIndex("rects", "rect:*", IndexRect)
if true {
db.Update(func(tx *Tx) error {
tx.Set("fun:user:0", "tom", nil)
tx.Set("fun:user:1", "Randi", nil)
tx.Set("fun:user:2", "jane", nil)
tx.Set("fun:user:4", "Janet", nil)
tx.Set("fun:user:5", "Paula", nil)
tx.Set("fun:user:6", "peter", nil)
tx.Set("fun:user:7", "Terri", nil)
return nil
})
// add some random items
start := time.Now()
if err := db.Update(func(tx *Tx) error {
for _, i := range rand.Perm(100) {
tx.Set(fmt.Sprintf("tag:%d", i+100), fmt.Sprintf("val:%d", rand.Int()%100+100), nil)
}
return nil
}); err != nil {
t.Fatal(err)
}
if false {
println(time.Now().Sub(start).String(), db.keys.Len())
}
// add some random rects
if err := db.Update(func(tx *Tx) error {
tx.Set("rect:1", Rect([]float64{10, 10}, []float64{20, 20}), nil)
tx.Set("rect:2", Rect([]float64{15, 15}, []float64{24, 24}), nil)
tx.Set("rect:3", Rect([]float64{17, 17}, []float64{27, 27}), nil)
return nil
}); err != nil {
t.Fatal(err)
}
}
// verify the data has been created
buf := &bytes.Buffer{}
db.View(func(tx *Tx) error {
tx.Ascend("users", func(key, val string) bool {
fmt.Fprintf(buf, "%s %s\n", key, val)
return true
})
err = tx.AscendRange("", "tag:170", "tag:172", func(key, val string) bool {
fmt.Fprintf(buf, "%s\n", key)
return true
})
if err != nil {
t.Fatal(err)
}
err = tx.AscendGreaterOrEqual("", "tag:195", func(key, val string) bool {
fmt.Fprintf(buf, "%s\n", key)
return true
})
if err != nil {
t.Fatal(err)
}
err = tx.AscendGreaterOrEqual("", "rect:", func(key, val string) bool {
if !strings.HasPrefix(key, "rect:") {
return false
}
min, max := IndexRect(val)
fmt.Fprintf(buf, "%s: %v,%v\n", key, min, max)
return true
})
expect := make([]string, 2)
n := 0
tx.Intersects("rects", "[0 0],[15 15]", func(key, val string) bool {
if n == 2 {
t.Fatalf("too many rects where received, expecting only two")
}
min, max := IndexRect(val)
s := fmt.Sprintf("%s: %v,%v\n", key, min, max)
if key == "rect:1" {
expect[0] = s
} else if key == "rect:2" {
expect[1] = s
}
n++
return true
})
for _, s := range expect {
buf.WriteString(s)
}
return nil
})
res := `
fun:user:2 jane
fun:user:4 Janet
fun:user:5 Paula
fun:user:6 peter
fun:user:1 Randi
fun:user:7 Terri
fun:user:0 tom
tag:170
tag:171
tag:195
tag:196
tag:197
tag:198
tag:199
rect:1: [10 10],[20 20]
rect:2: [15 15],[24 24]
rect:3: [17 17],[27 27]
rect:1: [10 10],[20 20]
rect:2: [15 15],[24 24]
`
res = strings.Replace(res, "\r", "", -1)
if strings.TrimSpace(buf.String()) != strings.TrimSpace(res) {
t.Fatalf("expected [%v], got [%v]", strings.TrimSpace(res), strings.TrimSpace(buf.String()))
}
}
func testRectStringer(min, max []float64) error {
nmin, nmax := IndexRect(Rect(min, max))
if len(nmin) != len(min) {
return fmt.Errorf("rect=%v,%v, expect=%v,%v", nmin, nmax, min, max)
}
for i := 0; i < len(min); i++ {
if min[i] != nmin[i] || max[i] != nmax[i] {
return fmt.Errorf("rect=%v,%v, expect=%v,%v", nmin, nmax, min, max)
}
}
return nil
}
func TestRectStrings(t *testing.T) {
test(t, Rect(IndexRect(Point(1))) == "[1]", true)
test(t, Rect(IndexRect(Point(1, 2, 3, 4))) == "[1 2 3 4]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[1 2],[1 2]")))) == "[1 2]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[1 2],[2 2]")))) == "[1 2],[2 2]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[1 2],[2 2],[3]")))) == "[1 2],[2 2]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[1 2]")))) == "[1 2]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[1.5 2 4.5 5.6]")))) == "[1.5 2 4.5 5.6]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[1.5 2 4.5 5.6 -1],[]")))) == "[1.5 2 4.5 5.6 -1],[]", true)
test(t, Rect(IndexRect(Rect(IndexRect("[]")))) == "[]", true)
test(t, Rect(IndexRect(Rect(IndexRect("")))) == "", true)
if err := testRectStringer(nil, nil); err != nil {
t.Fatal(err)
}
if err := testRectStringer([]float64{}, []float64{}); err != nil {
t.Fatal(err)
}
if err := testRectStringer([]float64{1}, []float64{2}); err != nil {
t.Fatal(err)
}
if err := testRectStringer([]float64{1, 2}, []float64{3, 4}); err != nil {
t.Fatal(err)
}
if err := testRectStringer([]float64{1, 2, 3}, []float64{4, 5, 6}); err != nil {
t.Fatal(err)
}
if err := testRectStringer([]float64{1, 2, 3, 4}, []float64{5, 6, 7, 8}); err != nil {
t.Fatal(err)
}
if err := testRectStringer([]float64{1, 2, 3, 4, 5}, []float64{6, 7, 8, 9, 0}); err != nil {
t.Fatal(err)
}
}

BIN
logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 94 KiB