re
/
go-sqlcipher
forked from mirror/go-sqlcipher
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Upgrade the latest version of go-sqlite3 and sqlcipher.

This commit is contained in:
xeodou 2019-02-25 02:39:29 +08:00
parent 7f9cd31998 ad30583d83
commit efb0ea6075
46 changed files with 34564 additions and 13508 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
.gitignore vendored
View File

@ -2,3 +2,13 @@
*.exe *.exe
*.dll *.dll
*.o *.o
# VSCode
.vscode
# Exclude from upgrade
upgrade/*.c
upgrade/*.h
# Exclude upgrade binary
upgrade/upgrade

20
.travis.yml
View File

@ -7,18 +7,24 @@ addons:
packages: packages:
- libssl-dev - libssl-dev
env:
- GOTAGS=
- GOTAGS=trace
- GOTAGS=vtable
go: go:
- 1.7.x
- 1.8.x
- 1.9.x - 1.9.x
- 1.10.x
- 1.11.x
- master - master
before_install: before_install:
- |
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
brew update
fi
- go get github.com/smartystreets/goconvey
- go get github.com/mattn/goveralls - go get github.com/mattn/goveralls
- go get golang.org/x/tools/cmd/cover - go get golang.org/x/tools/cmd/cover
script: script:
- $HOME/gopath/bin/goveralls -repotoken PfqH9iFyzW3daUxflzllSougjTxvFwQZE - $HOME/gopath/bin/goveralls -repotoken PfqH9iFyzW3daUxflzllSougjTxvFwQZE
- go test -race -v . -tags "$GOTAGS" - go test -race -v . -tags ""
- go test -race -v . -tags "libsqlite3"
- go test -race -v . -tags "sqlite_allow_uri_authority sqlite_app_armor sqlite_foreign_keys sqlite_fts5 sqlite_icu sqlite_introspect sqlite_json sqlite_secure_delete sqlite_see sqlite_stat4 sqlite_trace sqlite_userauth sqlite_vacuum_incr sqlite_vtable sqlite_unlock_notify"
- go test -race -v . -tags "sqlite_vacuum_full"

474
README.md
View File

@ -6,35 +6,59 @@ SQLCipher driver conforming to the built-in database/sql interface and using the
which is which is
`3.20.1` `3.26.0`
Working with sqlcipher version which is Working with sqlcipher version which is
`3.4.2` `4.0.1`
It's wrapper with It's wrapper with
* [go-sqlite3](https://github.com/mattn/go-sqlite3) sqlite3 driver for go that using database/sql. * [go-sqlite3](https://github.com/mattn/go-sqlite3) sqlite3 driver for go that using database/sql.
* [SQLCipher](https://github.com/sqlcipher/sqlcipher) SQLCipher is an SQLite extension that provides 256 bit AES encryption of database files. * [SQLCipher](https://github.com/sqlcipher/sqlcipher) SQLCipher is an SQLite extension that provides 256 bit AES encryption of database files.
* Using [openssl](https://github.com/openssl/openssl) as the 256 bit AES encryption. * Using [openssl](https://github.com/openssl/openssl) as the 256 bit AES encryption.
*Have't build test in a windows machine or linux machine*
**Working in my macbook-air**
```
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.3.0
Thread model: posix
OpenSSL 0.9.8zd 8 Jan 2015 Supported Golang version: See .travis.yml
built on: Mar 9 2015
platform: darwin64-x86_64-llvm
options: bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) blowfish(idx)
compiler: -arch x86_64 -fmessage-length=0 -pipe -Wno-trigraphs -fpascal-strings -fasm-blocks -O3 -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DMD32_REG_T=int -DOPENSSL_NO_IDEA -DOPENSSL_PIC -DOPENSSL_THREADS -DZLIB -mmacosx-version-min=10.6
OPENSSLDIR: "/System/Library/OpenSSL"
[This package follows the official Golang Release Policy.](https://golang.org/doc/devel/release.html#policy)
### Upgrade
Due to the [go-sqlite3](https://github.com/mattn/go-sqlite3) project change its way to load the `PRAGMA` variables. Setting the encrypting key won't work for the existing database anymore. But you can load the encrypt key by setting with query parameter `_key`, like:
```golang
b, err = sql.Open("sqlite3", databasefile +"?_key=password")
``` ```
Installation To upgrade SQLCipher from 3.x to 4.x, please take a look of:
------------ 1. https://www.zetetic.net/sqlcipher/sqlcipher-api/#cipher_migrate
2. [Upgrading to SQLCipher 4](https://discuss.zetetic.net/t/upgrading-to-sqlcipher-4/3283)
### Overview
- [Installation](#installation)
- [API Reference](#api-reference)
- [Connection String](#connection-string)
- [Features](#features)
- [Compilation](#compilation)
- [Android](#android)
- [ARM](#arm)
- [Cross Compile](#cross-compile)
- [Google Cloud Platform](#google-cloud-platform)
- [Linux](#linux)
- [Alpine](#alpine)
- [Fedora](#fedora)
- [Ubuntu](#ubuntu)
- [Mac OSX](#mac-osx)
- [Windows](#windows)
- [Errors](#errors)
- [User Authentication](#user-authentication)
- [Compile](#compile)
- [Usage](#usage)
- [Extensions](#extensions)
- [Spatialite](#spatialite)
- [FAQ](#faq)
- [License](#license)
# Installation
This package can be installed with the go get command: This package can be installed with the go get command:
@ -44,15 +68,233 @@ _go-sqlcipher_ is *cgo* package.
If you want to build your app using go-sqlcipher, you need gcc. If you want to build your app using go-sqlcipher, you need gcc.
However, if you install _go-sqlcipher_ with `go install github.com/xeodou/go-sqlcipher`, you don't need gcc to build your app anymore. However, if you install _go-sqlcipher_ with `go install github.com/xeodou/go-sqlcipher`, you don't need gcc to build your app anymore.
Documentation ***Important: because this is a `CGO` enabled package you are required to set the environment variable `CGO_ENABLED=1` and have a `gcc` compile present within your path.***
-------------
# API Reference
API documentation can be found here: http://godoc.org/github.com/xeodou/go-sqlcipher API documentation can be found here: http://godoc.org/github.com/xeodou/go-sqlcipher
Examples can be found under the `./_example` directory Examples can be found under the [examples](./_example) directory
FAQ # Connection String
---
When creating a new SQLite database or connection to an existing one, with the file name additional options can be given.
This is also known as a DSN string. (Data Source Name).
Options are append after the filename of the SQLite database.
The database filename and options are seperated by an `?` (Question Mark).
Options should be URL-encoded (see [url.QueryEscape](https://golang.org/pkg/net/url/#QueryEscape)).
This also applies when using an in-memory database instead of a file.
Options can be given using the following format: `KEYWORD=VALUE` and multiple options can be combined with the `&` ampersand.
This library supports dsn options of SQLite itself and provides additional options.
Boolean values can be one of:
* `0` `no` `false` `off`
* `1` `yes` `true` `on`
| Name | Key | Value(s) | Description |
|------|-----|----------|-------------|
| UA - Create | `_auth` | - | Create User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Username | `_auth_user` | `string` | Username for User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Password | `_auth_pass` | `string` | Password for User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Crypt | `_auth_crypt` | <ul><li>SHA1</li><li>SSHA1</li><li>SHA256</li><li>SSHA256</li><li>SHA384</li><li>SSHA384</li><li>SHA512</li><li>SSHA512</li></ul> | Password encoder to use for User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Salt | `_auth_salt` | `string` | Salt to use if the configure password encoder requires a salt, for User Authentication, for more information see [User Authentication](#user-authentication) |
| Auto Vacuum | `_auto_vacuum` \| `_vacuum` | <ul><li>`0` \| `none`</li><li>`1` \| `full`</li><li>`2` \| `incremental`</li></ul> | For more information see [PRAGMA auto_vacuum](https://www.sqlite.org/pragma.html#pragma_auto_vacuum) |
| Busy Timeout | `_busy_timeout` \| `_timeout` | `int` | Specify value for sqlite3_busy_timeout. For more information see [PRAGMA busy_timeout](https://www.sqlite.org/pragma.html#pragma_busy_timeout) |
| Case Sensitive LIKE | `_case_sensitive_like` \| `_cslike` | `boolean` | For more information see [PRAGMA case_sensitive_like](https://www.sqlite.org/pragma.html#pragma_case_sensitive_like) |
| Defer Foreign Keys | `_defer_foreign_keys` \| `_defer_fk` | `boolean` | For more information see [PRAGMA defer_foreign_keys](https://www.sqlite.org/pragma.html#pragma_defer_foreign_keys) |
| Foreign Keys | `_foreign_keys` \| `_fk` | `boolean` | For more information see [PRAGMA foreign_keys](https://www.sqlite.org/pragma.html#pragma_foreign_keys) |
| Ignore CHECK Constraints | `_ignore_check_constraints` | `boolean` | For more information see [PRAGMA ignore_check_constraints](https://www.sqlite.org/pragma.html#pragma_ignore_check_constraints) |
| Immutable | `immutable` | `boolean` | For more information see [Immutable](https://www.sqlite.org/c3ref/open.html) |
| Journal Mode | `_journal_mode` \| `_journal` | <ul><li>DELETE</li><li>TRUNCATE</li><li>PERSIST</li><li>MEMORY</li><li>WAL</li><li>OFF</li></ul> | For more information see [PRAGMA journal_mode](https://www.sqlite.org/pragma.html#pragma_journal_mode) |
| Locking Mode | `_locking_mode` \| `_locking` | <ul><li>NORMAL</li><li>EXCLUSIVE</li></ul> | For more information see [PRAGMA locking_mode](https://www.sqlite.org/pragma.html#pragma_locking_mode) |
| Mode | `mode` | <ul><li>ro</li><li>rw</li><li>rwc</li><li>memory</li></ul> | Access Mode of the database. For more information see [SQLite Open](https://www.sqlite.org/c3ref/open.html) |
| Mutex Locking | `_mutex` | <ul><li>no</li><li>full</li></ul> | Specify mutex mode. |
| Query Only | `_query_only` | `boolean` | For more information see [PRAGMA query_only](https://www.sqlite.org/pragma.html#pragma_query_only) |
| Recursive Triggers | `_recursive_triggers` \| `_rt` | `boolean` | For more information see [PRAGMA recursive_triggers](https://www.sqlite.org/pragma.html#pragma_recursive_triggers) |
| Secure Delete | `_secure_delete` | `boolean` \| `FAST` | For more information see [PRAGMA secure_delete](https://www.sqlite.org/pragma.html#pragma_secure_delete) |
| Shared-Cache Mode | `cache` | <ul><li>shared</li><li>private</li></ul> | Set cache mode for more information see [sqlite.org](https://www.sqlite.org/sharedcache.html) |
| Synchronous | `_synchronous` \| `_sync` | <ul><li>0 \| OFF</li><li>1 \| NORMAL</li><li>2 \| FULL</li><li>3 \| EXTRA</li></ul> | For more information see [PRAGMA synchronous](https://www.sqlite.org/pragma.html#pragma_synchronous) |
| Time Zone Location | `_loc` | auto | Specify location of time format. |
| Transaction Lock | `_txlock` | <ul><li>immediate</li><li>deferred</li><li>exclusive</li></ul> | Specify locking behavior for transactions. |
| Writable Schema | `_writable_schema` | `Boolean` | When this pragma is on, the SQLITE_MASTER tables in which database can be changed using ordinary UPDATE, INSERT, and DELETE statements. Warning: misuse of this pragma can easily result in a corrupt database file. |
## DSN Examples
```
file:test.db?cache=shared&mode=memory
```
# Features
This package allows additional configuration of features available within SQLite3 to be enabled or disabled by golang build constraints also known as build `tags`.
[Click here for more information about build tags / constraints.](https://golang.org/pkg/go/build/#hdr-Build_Constraints)
### Usage
If you wish to build this library with additional extensions / features.
Use the following command.
```bash
go build --tags "<FEATURE>"
```
If you want to build the project without the `libcrypto`, you could specific the openssl library by using the command.
```bash
CGO_ENABLE=0 CGO_LDFLAGS="-L/usr/local/opt/openssl/lib" CGO_CPPFLAGS="-I/usr/local/opt/openssl/include" go build build _example/encrypto/encrypto.go
```
For available features see the extension list.
When using multiple build tags, all the different tags should be space delimted.
Example:
```bash
go build --tags "icu json1 fts5 secure_delete"
```
### Feature / Extension List
| Extension | Build Tag | Description |
|-----------|-----------|-------------|
| Additional Statistics | sqlite_stat4 | This option adds additional logic to the ANALYZE command and to the query planner that can help SQLite to chose a better query plan under certain situations. The ANALYZE command is enhanced to collect histogram data from all columns of every index and store that data in the sqlite_stat4 table.<br><br>The query planner will then use the histogram data to help it make better index choices. The downside of this compile-time option is that it violates the query planner stability guarantee making it more difficult to ensure consistent performance in mass-produced applications.<br><br>SQLITE_ENABLE_STAT4 is an enhancement of SQLITE_ENABLE_STAT3. STAT3 only recorded histogram data for the left-most column of each index whereas the STAT4 enhancement records histogram data from all columns of each index.<br><br>The SQLITE_ENABLE_STAT3 compile-time option is a no-op and is ignored if the SQLITE_ENABLE_STAT4 compile-time option is used |
| Allow URI Authority | sqlite_allow_uri_authority | URI filenames normally throws an error if the authority section is not either empty or "localhost".<br><br>However, if SQLite is compiled with the SQLITE_ALLOW_URI_AUTHORITY compile-time option, then the URI is converted into a Uniform Naming Convention (UNC) filename and passed down to the underlying operating system that way |
| App Armor | sqlite_app_armor | When defined, this C-preprocessor macro activates extra code that attempts to detect misuse of the SQLite API, such as passing in NULL pointers to required parameters or using objects after they have been destroyed. <br><br>App Armor is not available under `Windows`. |
| Disable Load Extensions | sqlite_omit_load_extension | Loading of external extensions is enabled by default.<br><br>To disable extension loading add the build tag `sqlite_omit_load_extension`. |
| Foreign Keys | sqlite_foreign_keys | This macro determines whether enforcement of foreign key constraints is enabled or disabled by default for new database connections.<br><br>Each database connection can always turn enforcement of foreign key constraints on and off and run-time using the foreign_keys pragma.<br><br>Enforcement of foreign key constraints is normally off by default, but if this compile-time parameter is set to 1, enforcement of foreign key constraints will be on by default |
| Full Auto Vacuum | sqlite_vacuum_full | Set the default auto vacuum to full |
| Incremental Auto Vacuum | sqlite_vacuum_incr | Set the default auto vacuum to incremental |
| Full Text Search Engine | sqlite_fts5 | When this option is defined in the amalgamation, versions 5 of the full-text search engine (fts5) is added to the build automatically |
| International Components for Unicode | sqlite_icu | This option causes the International Components for Unicode or "ICU" extension to SQLite to be added to the build |
| Introspect PRAGMAS | sqlite_introspect | This option adds some extra PRAGMA statements. <ul><li>PRAGMA function_list</li><li>PRAGMA module_list</li><li>PRAGMA pragma_list</li></ul> |
| JSON SQL Functions | sqlite_json | When this option is defined in the amalgamation, the JSON SQL functions are added to the build automatically |
| Secure Delete | sqlite_secure_delete | This compile-time option changes the default setting of the secure_delete pragma.<br><br>When this option is not used, secure_delete defaults to off. When this option is present, secure_delete defaults to on.<br><br>The secure_delete setting causes deleted content to be overwritten with zeros. There is a small performance penalty since additional I/O must occur.<br><br>On the other hand, secure_delete can prevent fragments of sensitive information from lingering in unused parts of the database file after it has been deleted. See the documentation on the secure_delete pragma for additional information |
| Secure Delete (FAST) | sqlite_secure_delete_fast | For more information see [PRAGMA secure_delete](https://www.sqlite.org/pragma.html#pragma_secure_delete) |
| Tracing / Debug | sqlite_trace | Activate trace functions |
| User Authentication | sqlite_userauth | SQLite User Authentication see [User Authentication](#user-authentication) for more information. |
# Compilation
This package requires `CGO_ENABLED=1` ennvironment variable if not set by default, and the presence of the `gcc` compiler.
If you need to add additional CFLAGS or LDFLAGS to the build command, and do not want to modify this package. Then this can be achieved by using the `CGO_CFLAGS` and `CGO_LDFLAGS` environment variables.
## Android
This package can be compiled for android.
Compile with:
```bash
go build --tags "android"
```
For more information see [#201](https://github.com/mattn/go-sqlite3/issues/201)
# ARM
To compile for `ARM` use the following environment.
```bash
env CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ \
CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 \
go build -v
```
Additional information:
- [#242](https://github.com/mattn/go-sqlite3/issues/242)
- [#504](https://github.com/mattn/go-sqlite3/issues/504)
# Cross Compile
This library can be cross-compiled.
In some cases you are required to the `CC` environment variable with the cross compiler.
Additional information:
- [#491](https://github.com/mattn/go-sqlite3/issues/491)
- [#560](https://github.com/mattn/go-sqlite3/issues/560)
# Google Cloud Platform
Building on GCP is not possible because Google Cloud Platform does not allow `gcc` to be executed.
Please work only with compiled final binaries.
## Linux
To compile this package on Linux you must install the development tools for your linux distribution.
To compile under linux use the build tag `linux`.
```bash
go build --tags "linux"
```
If you wish to link directly to libsqlite3 then you can use the `libsqlite3` build tag.
```
go build --tags "libsqlite3 linux"
```
### Alpine
When building in an `alpine` container run the following command before building.
```
apk add --update gcc musl-dev
```
### Fedora
```bash
sudo yum groupinstall "Development Tools" "Development Libraries"
```
### Ubuntu
```bash
sudo apt-get install build-essential
```
## Mac OSX
OSX should have all the tools present to compile this package, if not install XCode this will add all the developers tools.
Required dependency
```bash
brew install sqlite3
```
For OSX there is an additional package install which is required if you whish to build the `icu` extension.
This additional package can be installed with `homebrew`.
```bash
brew upgrade icu4c
```
To compile for Mac OSX.
```bash
go build --tags "darwin"
```
If you wish to link directly to libsqlite3 then you can use the `libsqlite3` build tag.
```
go build --tags "libsqlite3 darwin"
```
Additional information:
- [#206](https://github.com/mattn/go-sqlite3/issues/206)
- [#404](https://github.com/mattn/go-sqlite3/issues/404)
## Windows
The golang code is copy from [go-sqlite3](https://github.com/mattn/go-sqlite3) The golang code is copy from [go-sqlite3](https://github.com/mattn/go-sqlite3)
If you have some issue, maybe you can find from https://github.com/mattn/go-sqlite3/issues If you have some issue, maybe you can find from https://github.com/mattn/go-sqlite3/issues
@ -61,51 +303,209 @@ Here is some help from go-sqlite3 project.
* Want to build go-sqlite3 with libsqlite3 on my linux. * Want to build go-sqlite3 with libsqlite3 on my linux.
Use `go build --tags "libsqlite3 linux"` 1) Install a Windows `gcc` toolchain.
2) Add the `bin` folders to the Windows path if the installer did not do this by default.
3) Open a terminal for the TDM-GCC toolchain, can be found in the Windows Start menu.
4) Navigate to your project folder and run the `go build ...` command for this package.
* Want to build go-sqlite3 with libsqlite3 on OS X. For example the TDM-GCC Toolchain can be found [here](https://sourceforge.net/projects/tdm-gcc/).
Install sqlite3 from homebrew: `brew install sqlite3` ## Errors
Use `go build --tags "libsqlite3 darwin"` - Compile error: `can not be used when making a shared object; recompile with -fPIC`
* Want to build go-sqlite3 with icu extension. When receiving a compile time error referencing recompile with `-FPIC` then you
are probably using a hardend system.
Use `go build --tags "icu"` You can compile the library on a hardend system with the following command.
Available extensions: `json1`, `fts5`, `icu` ```bash
go build -ldflags '-extldflags=-fno-PIC'
```
* Can't build go-sqlite3 on windows 64bit. More details see [#120](https://github.com/mattn/go-sqlite3/issues/120)
- Can't build go-sqlite3 on windows 64bit.
> Probably, you are using go 1.0, go1.0 has a problem when it comes to compiling/linking on windows 64bit. > Probably, you are using go 1.0, go1.0 has a problem when it comes to compiling/linking on windows 64bit.
> See: https://github.com/mattn/go-sqlite3/issues/27 > See: https://github.com/mattn/go-sqlite3/issues/27
* Getting insert error while query is opened. - `go get github.com/mattn/go-sqlite3` throws compilation error.
`gcc` throws: `internal compiler error`
Remove the download repository from your disk and try re-install with:
```bash
go install github.com/mattn/go-sqlite3
```
# User Authentication
This package supports the SQLite User Authentication module.
## Compile
To use the User authentication module the package has to be compiled with the tag `sqlite_userauth`. See [Features](#features).
## Usage
### Create protected database
To create a database protected by user authentication provide the following argument to the connection string `_auth`.
This will enable user authentication within the database. This option however requires two additional arguments:
- `_auth_user`
- `_auth_pass`
When `_auth` is present on the connection string user authentication will be enabled and the provided user will be created
as an `admin` user. After initial creation, the parameter `_auth` has no effect anymore and can be omitted from the connection string.
Example connection string:
Create an user authentication database with user `admin` and password `admin`.
`file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin`
Create an user authentication database with user `admin` and password `admin` and use `SHA1` for the password encoding.
`file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin&_auth_crypt=sha1`
### Password Encoding
The passwords within the user authentication module of SQLite are encoded with the SQLite function `sqlite_cryp`.
This function uses a ceasar-cypher which is quite insecure.
This library provides several additional password encoders which can be configured through the connection string.
The password cypher can be configured with the key `_auth_crypt`. And if the configured password encoder also requires an
salt this can be configured with `_auth_salt`.
#### Available Encoders
- SHA1
- SSHA1 (Salted SHA1)
- SHA256
- SSHA256 (salted SHA256)
- SHA384
- SSHA384 (salted SHA384)
- SHA512
- SSHA512 (salted SHA512)
### Restrictions
Operations on the database regarding to user management can only be preformed by an administrator user.
### Support
The user authentication supports two kinds of users
- administrators
- regular users
### User Management
User management can be done by directly using the `*SQLiteConn` or by SQL.
#### SQL
The following sql functions are available for user management.
| Function | Arguments | Description |
|----------|-----------|-------------|
| `authenticate` | username `string`, password `string` | Will authenticate an user, this is done by the connection; and should not be used manually. |
| `auth_user_add` | username `string`, password `string`, admin `int` | This function will add an user to the database.<br>if the database is not protected by user authentication it will enable it. Argument `admin` is an integer identifying if the added user should be an administrator. Only Administrators can add administrators. |
| `auth_user_change` | username `string`, password `string`, admin `int` | Function to modify an user. Users can change their own password, but only an administrator can change the administrator flag. |
| `authUserDelete` | username `string` | Delete an user from the database. Can only be used by an administrator. The current logged in administrator cannot be deleted. This is to make sure their is always an administrator remaining. |
These functions will return an integer.
- 0 (SQLITE_OK)
- 23 (SQLITE_AUTH) Failed to perform due to authentication or insufficient privileges
##### Examples
```sql
// Autheticate user
// Create Admin User
SELECT auth_user_add('admin2', 'admin2', 1);
// Change password for user
SELECT auth_user_change('user', 'userpassword', 0);
// Delete user
SELECT user_delete('user');
```
#### *SQLiteConn
The following functions are available for User authentication from the `*SQLiteConn`.
| Function | Description |
|----------|-------------|
| `Authenticate(username, password string) error` | Authenticate user |
| `AuthUserAdd(username, password string, admin bool) error` | Add user |
| `AuthUserChange(username, password string, admin bool) error` | Modify user |
| `AuthUserDelete(username string) error` | Delete user |
### Attached database
When using attached databases. SQLite will use the authentication from the `main` database for the attached database(s).
# Extensions
If you want your own extension to be listed here or you want to add a reference to an extension; please submit an Issue for this.
## Spatialite
Spatialite is available as an extension to SQLite, and can be used in combination with this repository.
For an example see [shaxbee/go-spatialite](https://github.com/shaxbee/go-spatialite).
# FAQ
- Getting insert error while query is opened.
> You can pass some arguments into the connection string, for example, a URI. > You can pass some arguments into the connection string, for example, a URI.
> See: [#39](https://github.com/mattn/go-sqlite3/issues/39) > See: [#39](https://github.com/mattn/go-sqlite3/issues/39)
* Do you want to cross compile? mingw on Linux or Mac? - Do you want to cross compile? mingw on Linux or Mac?
> See: [#106](https://github.com/mattn/go-sqlite3/issues/106) > See: [#106](https://github.com/mattn/go-sqlite3/issues/106)
> See also: http://www.limitlessfx.com/cross-compile-golang-app-for-windows-from-linux.html > See also: http://www.limitlessfx.com/cross-compile-golang-app-for-windows-from-linux.html
* Want to get time.Time with current locale - Want to get time.Time with current locale
Use `_loc=auto` in SQLite3 filename schema like `file:foo.db?_loc=auto`. Use `_loc=auto` in SQLite3 filename schema like `file:foo.db?_loc=auto`.
* Can I use this in multiple routines concurrently? - Can I use this in multiple routines concurrently?
Yes for readonly. But, No for writable. See [#50](https://github.com/mattn/go-sqlite3/issues/50), [#51](https://github.com/mattn/go-sqlite3/issues/51), [#209](https://github.com/mattn/go-sqlite3/issues/209), [#274](https://github.com/mattn/go-sqlite3/issues/274). Yes for readonly. But, No for writable. See [#50](https://github.com/mattn/go-sqlite3/issues/50), [#51](https://github.com/mattn/go-sqlite3/issues/51), [#209](https://github.com/mattn/go-sqlite3/issues/209), [#274](https://github.com/mattn/go-sqlite3/issues/274).
* Why is it racy if I use a `sql.Open("sqlite3", ":memory:")` database? - Why I'm getting `no such table` error?
Why is it racy if I use a `sql.Open("sqlite3", ":memory:")` database?
Each connection to :memory: opens a brand new in-memory sql database, so if Each connection to :memory: opens a brand new in-memory sql database, so if
the stdlib's sql engine happens to open another connection and you've only the stdlib's sql engine happens to open another connection and you've only
specified ":memory:", that connection will see a brand new database. A specified ":memory:", that connection will see a brand new database. A
workaround is to use "file::memory:?mode=memory&cache=shared". Every workaround is to use "file::memory:?mode=memory&cache=shared". Every
connection to this string will point to the same in-memory database. See connection to this string will point to the same in-memory database.
[#204](https://github.com/mattn/go-sqlite3/issues/204) for more info.
For more information see
* [#204](https://github.com/mattn/go-sqlite3/issues/204)
* [#511](https://github.com/mattn/go-sqlite3/issues/511)
- Reading from database with large amount of goroutines fails on OSX.
OS X limits OS-wide to not have more than 1000 files open simultaneously by default.
For more information see [#289](https://github.com/mattn/go-sqlite3/issues/289)
- Trying to execute a `.` (dot) command throws an error.
Error: `Error: near ".": syntax error`
Dot command are part of SQLite3 CLI not of this library.
You need to implement the feature or call the sqlite3 cli.
* Print some waring messages like `warning: 'RAND_add' is deprecated: first deprecated in OS X 10.7` * Print some waring messages like `warning: 'RAND_add' is deprecated: first deprecated in OS X 10.7`

2
_example/encrypto/encrypto.go
View File

@ -14,7 +14,7 @@ import (
) )
func main() { func main() {
db, err := sql.Open("sqlite3", "users.db") db, err := sql.Open("sqlite3", "users.db?key=123456")
if err != nil { if err != nil {
fmt.Println(err) fmt.Println(err)
} }

18
callback.go
View File

@ -77,6 +77,12 @@ func updateHookTrampoline(handle uintptr, op int, db *C.char, table *C.char, row
callback(op, C.GoString(db), C.GoString(table), rowid) callback(op, C.GoString(db), C.GoString(table), rowid)
} }
//export authorizerTrampoline
func authorizerTrampoline(handle uintptr, op int, arg1 *C.char, arg2 *C.char, arg3 *C.char) int {
callback := lookupHandle(handle).(func(int, string, string, string) int)
return callback(op, C.GoString(arg1), C.GoString(arg2), C.GoString(arg3))
}
// Use handles to avoid passing Go pointers to C. // Use handles to avoid passing Go pointers to C.
type handleVal struct { type handleVal struct {
@ -331,8 +337,18 @@ func callbackRetText(ctx *C.sqlite3_context, v reflect.Value) error {
return nil return nil
} }
func callbackRetNil(ctx *C.sqlite3_context, v reflect.Value) error {
return nil
}
func callbackRet(typ reflect.Type) (callbackRetConverter, error) { func callbackRet(typ reflect.Type) (callbackRetConverter, error) {
switch typ.Kind() { switch typ.Kind() {
case reflect.Interface:
errorInterface := reflect.TypeOf((*error)(nil)).Elem()
if typ.Implements(errorInterface) {
return callbackRetNil, nil
}
fallthrough
case reflect.Slice: case reflect.Slice:
if typ.Elem().Kind() != reflect.Uint8 { if typ.Elem().Kind() != reflect.Uint8 {
return nil, errors.New("the only supported slice type is []byte") return nil, errors.New("the only supported slice type is []byte")
@ -352,7 +368,7 @@ func callbackRet(typ reflect.Type) (callbackRetConverter, error) {
func callbackError(ctx *C.sqlite3_context, err error) { func callbackError(ctx *C.sqlite3_context, err error) {
cstr := C.CString(err.Error()) cstr := C.CString(err.Error())
defer C.free(unsafe.Pointer(cstr)) defer C.free(unsafe.Pointer(cstr))
C.sqlite3_result_error(ctx, cstr, -1) C.sqlite3_result_error(ctx, cstr, C.int(-1))
} }
// Test support code. Tests are not allowed to import "C", so we can't // Test support code. Tests are not allowed to import "C", so we can't

42585
sqlite3-binding.c
View File

File diff suppressed because it is too large Load Diff

1511
sqlite3-binding.h
View File

File diff suppressed because it is too large Load Diff

894
sqlite3.go
View File

File diff suppressed because it is too large Load Diff

120
sqlite3_func_crypt.go Normal file
View File

@ -0,0 +1,120 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
import (
"crypto/sha1"
"crypto/sha256"
"crypto/sha512"
)
// This file provides several different implementations for the
// default embedded sqlite_crypt function.
// This function is uses a caesar-cypher by default
// and is used within the UserAuthentication module to encode
// the password.
//
// The provided functions can be used as an overload to the sqlite_crypt
// function through the use of the RegisterFunc on the connection.
//
// Because the functions can serv a purpose to an end-user
// without using the UserAuthentication module
// the functions are default compiled in.
//
// From SQLITE3 - user-auth.txt
// The sqlite_user.pw field is encoded by a built-in SQL function
// "sqlite_crypt(X,Y)". The two arguments are both BLOBs. The first argument
// is the plaintext password supplied to the sqlite3_user_authenticate()
// interface. The second argument is the sqlite_user.pw value and is supplied
// so that the function can extract the "salt" used by the password encoder.
// The result of sqlite_crypt(X,Y) is another blob which is the value that
// ends up being stored in sqlite_user.pw. To verify credentials X supplied
// by the sqlite3_user_authenticate() routine, SQLite runs:
//
// sqlite_user.pw == sqlite_crypt(X, sqlite_user.pw)
//
// To compute an appropriate sqlite_user.pw value from a new or modified
// password X, sqlite_crypt(X,NULL) is run. A new random salt is selected
// when the second argument is NULL.
//
// The built-in version of of sqlite_crypt() uses a simple Caesar-cypher
// which prevents passwords from being revealed by searching the raw database
// for ASCII text, but is otherwise trivally broken. For better password
// security, the database should be encrypted using the SQLite Encryption
// Extension or similar technology. Or, the application can use the
// sqlite3_create_function() interface to provide an alternative
// implementation of sqlite_crypt() that computes a stronger password hash,
// perhaps using a cryptographic hash function like SHA1.
// CryptEncoderSHA1 encodes a password with SHA1
func CryptEncoderSHA1(pass []byte, hash interface{}) []byte {
h := sha1.Sum(pass)
return h[:]
}
// CryptEncoderSSHA1 encodes a password with SHA1 with the
// configured salt.
func CryptEncoderSSHA1(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha1.Sum(p)
return h[:]
}
}
// CryptEncoderSHA256 encodes a password with SHA256
func CryptEncoderSHA256(pass []byte, hash interface{}) []byte {
h := sha256.Sum256(pass)
return h[:]
}
// CryptEncoderSSHA256 encodes a password with SHA256
// with the configured salt
func CryptEncoderSSHA256(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha256.Sum256(p)
return h[:]
}
}
// CryptEncoderSHA384 encodes a password with SHA384
func CryptEncoderSHA384(pass []byte, hash interface{}) []byte {
h := sha512.Sum384(pass)
return h[:]
}
// CryptEncoderSSHA384 encodes a password with SHA384
// with the configured salt
func CryptEncoderSSHA384(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha512.Sum384(p)
return h[:]
}
}
// CryptEncoderSHA512 encodes a password with SHA512
func CryptEncoderSHA512(pass []byte, hash interface{}) []byte {
h := sha512.Sum512(pass)
return h[:]
}
// CryptEncoderSSHA512 encodes a password with SHA512
// with the configured salt
func CryptEncoderSSHA512(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha512.Sum512(p)
return h[:]
}
}
// EOF

57
sqlite3_func_crypt_test.go Normal file
View File

@ -0,0 +1,57 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
import (
"fmt"
"strings"
"testing"
)
// TestCryptEncoders to increase coverage
func TestCryptEncoders(t *testing.T) {
tests := []struct {
enc string
salt string
expected string
}{
{"sha1", "", "d033e22ae348aeb5660fc2140aec35850c4da997"},
{"sha256", "", "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"},
{"sha384", "", "9ca694a90285c034432c9550421b7b9dbd5c0f4b6673f05f6dbce58052ba20e4248041956ee8c9a2ec9f10290cdc0782"},
{"sha512", "", "c7ad44cbad762a5da0a452f9e854fdc1e0e7a52a38015f23f3eab1d80b931dd472634dfac71cd34ebc35d16ab7fb8a90c81f975113d6c7538dc69dd8de9077ec"},
{"ssha1", "salt", "9bc7aa55f08fdad935c3f8362d3f48bcf70eb280"},
{"ssha256", "salt", "f9a81477552594c79f2abc3fc099daa896a6e3a3590a55ffa392b6000412e80b"},
{"ssha384", "salt", "9ed776b477fcfc1b5e584989e8d770f5e17d98a7643546a63c2b07d4ab00f1348f6b8e73103d3a23554f727136e8c215"},
{"ssha512", "salt", "3c4a79782143337be4492be072abcfe979dd703c00541a8c39a0f3df4bab2029c050cf46fddc47090b5b04ac537b3e78189e3de16e601e859f95c51ac9f6dafb"},
}
for _, e := range tests {
var fn func(pass []byte, hash interface{}) []byte
switch e.enc {
case "sha1":
fn = CryptEncoderSHA1
case "ssha1":
fn = CryptEncoderSSHA1(e.salt)
case "sha256":
fn = CryptEncoderSHA256
case "ssha256":
fn = CryptEncoderSSHA256(e.salt)
case "sha384":
fn = CryptEncoderSHA384
case "ssha384":
fn = CryptEncoderSSHA384(e.salt)
case "sha512":
fn = CryptEncoderSHA512
case "ssha512":
fn = CryptEncoderSSHA512(e.salt)
}
h := fn([]byte("admin"), nil)
if strings.Compare(fmt.Sprintf("%x", h), e.expected) != 0 {
t.Fatalf("Invalid %s hash: expected: %s; got: %x", strings.ToUpper(e.enc), e.expected, h)
}
}
}

3
sqlite3_go18.go
View File

@ -1,10 +1,9 @@
// +build cgo
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build cgo
// +build go1.8 // +build go1.8
package sqlite3 package sqlite3

1
sqlite3_go18_test.go
View File

@ -80,7 +80,6 @@ func randStringBytes(n int) string {
} }
func initDatabase(t *testing.T, db *sql.DB, rowCount int64) { func initDatabase(t *testing.T, db *sql.DB, rowCount int64) {
t.Logf("Executing db initializing statements")
for _, query := range testTableStatements { for _, query := range testTableStatements {
_, err := db.Exec(query) _, err := db.Exec(query)
if err != nil { if err != nil {

2
sqlite3_libsqlite3.go
View File

@ -2,6 +2,7 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build libsqlite3 // +build libsqlite3
package sqlite3 package sqlite3
@ -10,6 +11,7 @@ package sqlite3
#cgo CFLAGS: -DUSE_LIBSQLITE3 #cgo CFLAGS: -DUSE_LIBSQLITE3
#cgo linux LDFLAGS: -lsqlite3 #cgo linux LDFLAGS: -lsqlite3
#cgo darwin LDFLAGS: -L/usr/local/opt/sqlite/lib -lsqlite3 #cgo darwin LDFLAGS: -L/usr/local/opt/sqlite/lib -lsqlite3
#cgo openbsd LDFLAGS: -lsqlite3
#cgo solaris LDFLAGS: -lsqlite3 #cgo solaris LDFLAGS: -lsqlite3
*/ */
import "C" import "C"

1
sqlite3_load_extension.go
View File

@ -2,6 +2,7 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build !sqlite_omit_load_extension // +build !sqlite_omit_load_extension
package sqlite3 package sqlite3

1
sqlite3_omit_load_extension.go → sqlite3_load_extension_omit.go
View File

@ -2,6 +2,7 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build sqlite_omit_load_extension // +build sqlite_omit_load_extension
package sqlite3 package sqlite3

15
sqlite3_opt_allow_uri_authority.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_allow_uri_authority
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ALLOW_URI_AUTHORITY
#cgo LDFLAGS: -lm
*/
import "C"

16
sqlite3_opt_app_armor.go Normal file
View File

@ -0,0 +1,16 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !windows
// +build sqlite_app_armor
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_API_ARMOR
#cgo LDFLAGS: -lm
*/
import "C"

15
sqlite3_opt_foreign_keys.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_foreign_keys
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_DEFAULT_FOREIGN_KEYS=1
#cgo LDFLAGS: -lm
*/
import "C"

0
sqlite3_fts3_test.go → sqlite3_opt_fts3_test.go
View File

3
sqlite3_fts5.go → sqlite3_opt_fts5.go
View File

@ -2,7 +2,8 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build fts5
// +build sqlite_fts5 fts5
package sqlite3 package sqlite3

6
sqlite3_icu.go → sqlite3_opt_icu.go
View File

@ -2,12 +2,16 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build icu
// +build sqlite_icu icu
package sqlite3 package sqlite3
/* /*
#cgo LDFLAGS: -licuuc -licui18n #cgo LDFLAGS: -licuuc -licui18n
#cgo CFLAGS: -DSQLITE_ENABLE_ICU #cgo CFLAGS: -DSQLITE_ENABLE_ICU
#cgo darwin CFLAGS: -I/usr/local/opt/icu4c/include
#cgo darwin LDFLAGS: -L/usr/local/opt/icu4c/lib
#cgo openbsd LDFLAGS: -lsqlite3
*/ */
import "C" import "C"

15
sqlite3_opt_introspect.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_introspect
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_INTROSPECTION_PRAGMAS
#cgo LDFLAGS: -lm
*/
import "C"

3
sqlite3_json1.go → sqlite3_opt_json1.go
View File

@ -2,7 +2,8 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build json1
// +build sqlite_json sqlite_json1 json1
package sqlite3 package sqlite3

15
sqlite3_opt_secure_delete.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_secure_delete
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_SECURE_DELETE=1
#cgo LDFLAGS: -lm
*/
import "C"

15
sqlite3_opt_secure_delete_fast.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_secure_delete_fast
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_SECURE_DELETE=FAST
#cgo LDFLAGS: -lm
*/
import "C"

15
sqlite3_opt_stat4.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_stat4
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_STAT4
#cgo LDFLAGS: -lm
*/
import "C"

85
sqlite3_opt_unlock_notify.c Normal file
View File

@ -0,0 +1,85 @@
// Copyright (C) 2018 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
#ifdef SQLITE_ENABLE_UNLOCK_NOTIFY
#include <stdio.h>
#include <sqlite3-binding.h>
extern int unlock_notify_wait(sqlite3 *db);
int
_sqlite3_step_blocking(sqlite3_stmt *stmt)
{
int rv;
sqlite3* db;
db = sqlite3_db_handle(stmt);
for (;;) {
rv = sqlite3_step(stmt);
if (rv != SQLITE_LOCKED) {
break;
}
if (sqlite3_extended_errcode(db) != SQLITE_LOCKED_SHAREDCACHE) {
break;
}
rv = unlock_notify_wait(db);
if (rv != SQLITE_OK) {
break;
}
sqlite3_reset(stmt);
}
return rv;
}
int
_sqlite3_step_row_blocking(sqlite3_stmt* stmt, long long* rowid, long long* changes)
{
int rv;
sqlite3* db;
db = sqlite3_db_handle(stmt);
for (;;) {
rv = sqlite3_step(stmt);
if (rv!=SQLITE_LOCKED) {
break;
}
if (sqlite3_extended_errcode(db) != SQLITE_LOCKED_SHAREDCACHE) {
break;
}
rv = unlock_notify_wait(db);
if (rv != SQLITE_OK) {
break;
}
sqlite3_reset(stmt);
}
*rowid = (long long) sqlite3_last_insert_rowid(db);
*changes = (long long) sqlite3_changes(db);
return rv;
}
int
_sqlite3_prepare_v2_blocking(sqlite3 *db, const char *zSql, int nBytes, sqlite3_stmt **ppStmt, const char **pzTail)
{
int rv;
for (;;) {
rv = sqlite3_prepare_v2(db, zSql, nBytes, ppStmt, pzTail);
if (rv!=SQLITE_LOCKED) {
break;
}
if (sqlite3_extended_errcode(db) != SQLITE_LOCKED_SHAREDCACHE) {
break;
}
rv = unlock_notify_wait(db);
if (rv != SQLITE_OK) {
break;
}
}
return rv;
}
#endif

93
sqlite3_opt_unlock_notify.go Normal file
View File

@ -0,0 +1,93 @@
// Copyright (C) 2018 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build cgo
// +build sqlite_unlock_notify
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_UNLOCK_NOTIFY
#include <stdlib.h>
#include <sqlite3-binding.h>
extern void unlock_notify_callback(void *arg, int argc);
*/
import "C"
import (
"fmt"
"math"
"sync"
"unsafe"
)
type unlock_notify_table struct {
sync.Mutex
seqnum uint
table map[uint]chan struct{}
}
var unt unlock_notify_table = unlock_notify_table{table: make(map[uint]chan struct{})}
func (t *unlock_notify_table) add(c chan struct{}) uint {
t.Lock()
defer t.Unlock()
h := t.seqnum
t.table[h] = c
t.seqnum++
return h
}
func (t *unlock_notify_table) remove(h uint) {
t.Lock()
defer t.Unlock()
delete(t.table, h)
}
func (t *unlock_notify_table) get(h uint) chan struct{} {
t.Lock()
defer t.Unlock()
c, ok := t.table[h]
if !ok {
panic(fmt.Sprintf("Non-existent key for unlcok-notify channel: %d", h))
}
return c
}
//export unlock_notify_callback
func unlock_notify_callback(argv unsafe.Pointer, argc C.int) {
for i := 0; i < int(argc); i++ {
parg := ((*(*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.uint)(nil))]*[1]uint)(argv))[i])
arg := *parg
h := arg[0]
c := unt.get(h)
c <- struct{}{}
}
}
//export unlock_notify_wait
func unlock_notify_wait(db *C.sqlite3) C.int {
// It has to be a bufferred channel to not block in sqlite_unlock_notify
// as sqlite_unlock_notify could invoke the callback before it returns.
c := make(chan struct{}, 1)
defer close(c)
h := unt.add(c)
defer unt.remove(h)
pargv := C.malloc(C.sizeof_uint)
defer C.free(pargv)
argv := (*[1]uint)(pargv)
argv[0] = h
if rv := C.sqlite3_unlock_notify(db, (*[0]byte)(C.unlock_notify_callback), unsafe.Pointer(pargv)); rv != C.SQLITE_OK {
return rv
}
<-c
return C.SQLITE_OK
}

222
sqlite3_opt_unlock_notify_test.go Normal file
View File

@ -0,0 +1,222 @@
// Copyright (C) 2018 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_unlock_notify
package sqlite3
import (
"database/sql"
"fmt"
"os"
"sync"
"testing"
"time"
)
func TestUnlockNotify(t *testing.T) {
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
dsn := fmt.Sprintf("file:%s?cache=shared&mode=rwc&_busy_timeout=%d", tempFilename, 500)
db, err := sql.Open("sqlite3", dsn)
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
_, err = db.Exec("CREATE TABLE foo(id INTEGER, status INTEGER)")
if err != nil {
t.Fatal("Failed to create table:", err)
}
tx, err := db.Begin()
if err != nil {
t.Fatal("Failed to begin transaction:", err)
}
_, err = tx.Exec("INSERT INTO foo(id, status) VALUES(1, 100)")
if err != nil {
t.Fatal("Failed to insert null:", err)
}
_, err = tx.Exec("UPDATE foo SET status = 200 WHERE id = 1")
if err != nil {
t.Fatal("Failed to update table:", err)
}
wg := sync.WaitGroup{}
wg.Add(1)
timer := time.NewTimer(500 * time.Millisecond)
go func() {
<-timer.C
err := tx.Commit()
if err != nil {
t.Fatal("Failed to commit transaction:", err)
}
wg.Done()
}()
rows, err := db.Query("SELECT count(*) from foo")
if err != nil {
t.Fatal("Unable to query foo table:", err)
}
if rows.Next() {
var count int
if err := rows.Scan(&count); err != nil {
t.Fatal("Failed to Scan rows", err)
}
}
if err := rows.Err(); err != nil {
t.Fatal("Failed at the call to Next:", err)
}
wg.Wait()
}
func TestUnlockNotifyMany(t *testing.T) {
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
dsn := fmt.Sprintf("file:%s?cache=shared&mode=rwc&_busy_timeout=%d", tempFilename, 500)
db, err := sql.Open("sqlite3", dsn)
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
_, err = db.Exec("CREATE TABLE foo(id INTEGER, status INTEGER)")
if err != nil {
t.Fatal("Failed to create table:", err)
}
tx, err := db.Begin()
if err != nil {
t.Fatal("Failed to begin transaction:", err)
}
_, err = tx.Exec("INSERT INTO foo(id, status) VALUES(1, 100)")
if err != nil {
t.Fatal("Failed to insert null:", err)
}
_, err = tx.Exec("UPDATE foo SET status = 200 WHERE id = 1")
if err != nil {
t.Fatal("Failed to update table:", err)
}
wg := sync.WaitGroup{}
wg.Add(1)
timer := time.NewTimer(500 * time.Millisecond)
go func() {
<-timer.C
err := tx.Commit()
if err != nil {
t.Fatal("Failed to commit transaction:", err)
}
wg.Done()
}()
const concurrentQueries = 1000
wg.Add(concurrentQueries)
for i := 0; i < concurrentQueries; i++ {
go func() {
rows, err := db.Query("SELECT count(*) from foo")
if err != nil {
t.Fatal("Unable to query foo table:", err)
}
if rows.Next() {
var count int
if err := rows.Scan(&count); err != nil {
t.Fatal("Failed to Scan rows", err)
}
}
if err := rows.Err(); err != nil {
t.Fatal("Failed at the call to Next:", err)
}
wg.Done()
}()
}
wg.Wait()
}
func TestUnlockNotifyDeadlock(t *testing.T) {
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
dsn := fmt.Sprintf("file:%s?cache=shared&mode=rwc&_busy_timeout=%d", tempFilename, 500)
db, err := sql.Open("sqlite3", dsn)
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
_, err = db.Exec("CREATE TABLE foo(id INTEGER, status INTEGER)")
if err != nil {
t.Fatal("Failed to create table:", err)
}
tx, err := db.Begin()
if err != nil {
t.Fatal("Failed to begin transaction:", err)
}
_, err = tx.Exec("INSERT INTO foo(id, status) VALUES(1, 100)")
if err != nil {
t.Fatal("Failed to insert null:", err)
}
_, err = tx.Exec("UPDATE foo SET status = 200 WHERE id = 1")
if err != nil {
t.Fatal("Failed to update table:", err)
}
wg := sync.WaitGroup{}
wg.Add(1)
timer := time.NewTimer(500 * time.Millisecond)
go func() {
<-timer.C
err := tx.Commit()
if err != nil {
t.Fatal("Failed to commit transaction:", err)
}
wg.Done()
}()
wg.Add(1)
go func() {
tx2, err := db.Begin()
if err != nil {
t.Fatal("Failed to begin transaction:", err)
}
defer tx2.Rollback()
_, err = tx2.Exec("DELETE FROM foo")
if err != nil {
t.Fatal("Failed to delete table:", err)
}
err = tx2.Commit()
if err != nil {
t.Fatal("Failed to commit transaction:", err)
}
wg.Done()
}()
rows, err := tx.Query("SELECT count(*) from foo")
if err != nil {
t.Fatal("Unable to query foo table:", err)
}
if rows.Next() {
var count int
if err := rows.Scan(&count); err != nil {
t.Fatal("Failed to Scan rows", err)
}
}
if err := rows.Err(); err != nil {
t.Fatal("Failed at the call to Next:", err)
}
wg.Wait()
}

289
sqlite3_opt_userauth.go Normal file
View File

@ -0,0 +1,289 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_userauth
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_USER_AUTHENTICATION
#cgo LDFLAGS: -lm
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
static int
_sqlite3_user_authenticate(sqlite3* db, const char* zUsername, const char* aPW, int nPW)
{
return sqlite3_user_authenticate(db, zUsername, aPW, nPW);
}
static int
_sqlite3_user_add(sqlite3* db, const char* zUsername, const char* aPW, int nPW, int isAdmin)
{
return sqlite3_user_add(db, zUsername, aPW, nPW, isAdmin);
}
static int
_sqlite3_user_change(sqlite3* db, const char* zUsername, const char* aPW, int nPW, int isAdmin)
{
return sqlite3_user_change(db, zUsername, aPW, nPW, isAdmin);
}
static int
_sqlite3_user_delete(sqlite3* db, const char* zUsername)
{
return sqlite3_user_delete(db, zUsername);
}
static int
_sqlite3_auth_enabled(sqlite3* db)
{
int exists = -1;
sqlite3_stmt *stmt;
sqlite3_prepare_v2(db, "select count(type) from sqlite_master WHERE type='table' and name='sqlite_user';", -1, &stmt, NULL);
while ( sqlite3_step(stmt) == SQLITE_ROW) {
exists = sqlite3_column_int(stmt, 0);
}
sqlite3_finalize(stmt);
return exists;
}
*/
import "C"
import (
"errors"
"unsafe"
)
const (
SQLITE_AUTH = C.SQLITE_AUTH
)
var (
ErrUnauthorized = errors.New("SQLITE_AUTH: Unauthorized")
ErrAdminRequired = errors.New("SQLITE_AUTH: Unauthorized; Admin Privileges Required")
)
// Authenticate will perform an authentication of the provided username
// and password against the database.
//
// If a database contains the SQLITE_USER table, then the
// call to Authenticate must be invoked with an
// appropriate username and password prior to enable read and write
//access to the database.
//
// Return SQLITE_OK on success or SQLITE_ERROR if the username/password
// combination is incorrect or unknown.
//
// If the SQLITE_USER table is not present in the database file, then
// this interface is a harmless no-op returnning SQLITE_OK.
func (c *SQLiteConn) Authenticate(username, password string) error {
rv := c.authenticate(username, password)
switch rv {
case C.SQLITE_ERROR, C.SQLITE_AUTH:
return ErrUnauthorized
case C.SQLITE_OK:
return nil
default:
return c.lastError()
}
}
// authenticate provides the actual authentication to SQLite.
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authenticate(username, password string) int {
// Allocate C Variables
cuser := C.CString(username)
cpass := C.CString(password)
// Free C Variables
defer func() {
C.free(unsafe.Pointer(cuser))
C.free(unsafe.Pointer(cpass))
}()
return int(C._sqlite3_user_authenticate(c.db, cuser, cpass, C.int(len(password))))
}
// AuthUserAdd can be used (by an admin user only)
// to create a new user. When called on a no-authentication-required
// database, this routine converts the database into an authentication-
// required database, automatically makes the added user an
// administrator, and logs in the current connection as that user.
// The AuthUserAdd only works for the "main" database, not
// for any ATTACH-ed databases. Any call to AuthUserAdd by a
// non-admin user results in an error.
func (c *SQLiteConn) AuthUserAdd(username, password string, admin bool) error {
isAdmin := 0
if admin {
isAdmin = 1
}
rv := c.authUserAdd(username, password, isAdmin)
switch rv {
case C.SQLITE_ERROR, C.SQLITE_AUTH:
return ErrAdminRequired
case C.SQLITE_OK:
return nil
default:
return c.lastError()
}
}
// authUserAdd enables the User Authentication if not enabled.
// Otherwise it will add a user.
//
// When user authentication is already enabled then this function
// can only be called by an admin.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authUserAdd(username, password string, admin int) int {
// Allocate C Variables
cuser := C.CString(username)
cpass := C.CString(password)
// Free C Variables
defer func() {
C.free(unsafe.Pointer(cuser))
C.free(unsafe.Pointer(cpass))
}()
return int(C._sqlite3_user_add(c.db, cuser, cpass, C.int(len(password)), C.int(admin)))
}
// AuthUserChange can be used to change a users
// login credentials or admin privilege. Any user can change their own
// login credentials. Only an admin user can change another users login
// credentials or admin privilege setting. No user may change their own
// admin privilege setting.
func (c *SQLiteConn) AuthUserChange(username, password string, admin bool) error {
isAdmin := 0
if admin {
isAdmin = 1
}
rv := c.authUserChange(username, password, isAdmin)
switch rv {
case C.SQLITE_ERROR, C.SQLITE_AUTH:
return ErrAdminRequired
case C.SQLITE_OK:
return nil
default:
return c.lastError()
}
}
// authUserChange allows to modify a user.
// Users can change their own password.
//
// Only admins can change passwords for other users
// and modify the admin flag.
//
// The admin flag of the current logged in user cannot be changed.
// THis ensures that their is always an admin.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authUserChange(username, password string, admin int) int {
// Allocate C Variables
cuser := C.CString(username)
cpass := C.CString(password)
// Free C Variables
defer func() {
C.free(unsafe.Pointer(cuser))
C.free(unsafe.Pointer(cpass))
}()
return int(C._sqlite3_user_change(c.db, cuser, cpass, C.int(len(password)), C.int(admin)))
}
// AuthUserDelete can be used (by an admin user only)
// to delete a user. The currently logged-in user cannot be deleted,
// which guarantees that there is always an admin user and hence that
// the database cannot be converted into a no-authentication-required
// database.
func (c *SQLiteConn) AuthUserDelete(username string) error {
rv := c.authUserDelete(username)
switch rv {
case C.SQLITE_ERROR, C.SQLITE_AUTH:
return ErrAdminRequired
case C.SQLITE_OK:
return nil
default:
return c.lastError()
}
}
// authUserDelete can be used to delete a user.
//
// This function can only be executed by an admin.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authUserDelete(username string) int {
// Allocate C Variables
cuser := C.CString(username)
// Free C Variables
defer func() {
C.free(unsafe.Pointer(cuser))
}()
return int(C._sqlite3_user_delete(c.db, cuser))
}
// AuthEnabled checks if the database is protected by user authentication
func (c *SQLiteConn) AuthEnabled() (exists bool) {
rv := c.authEnabled()
if rv == 1 {
exists = true
}
return
}
// authEnabled perform the actual check for user authentication.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// 0 - Disabled
// 1 - Enabled
func (c *SQLiteConn) authEnabled() int {
return int(C._sqlite3_auth_enabled(c.db))
}
// EOF

152
sqlite3_opt_userauth_omit.go Normal file
View File

@ -0,0 +1,152 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !sqlite_userauth
package sqlite3
import (
"C"
)
// Authenticate will perform an authentication of the provided username
// and password against the database.
//
// If a database contains the SQLITE_USER table, then the
// call to Authenticate must be invoked with an
// appropriate username and password prior to enable read and write
//access to the database.
//
// Return SQLITE_OK on success or SQLITE_ERROR if the username/password
// combination is incorrect or unknown.
//
// If the SQLITE_USER table is not present in the database file, then
// this interface is a harmless no-op returnning SQLITE_OK.
func (c *SQLiteConn) Authenticate(username, password string) error {
// NOOP
return nil
}
// authenticate provides the actual authentication to SQLite.
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authenticate(username, password string) int {
// NOOP
return 0
}
// AuthUserAdd can be used (by an admin user only)
// to create a new user. When called on a no-authentication-required
// database, this routine converts the database into an authentication-
// required database, automatically makes the added user an
// administrator, and logs in the current connection as that user.
// The AuthUserAdd only works for the "main" database, not
// for any ATTACH-ed databases. Any call to AuthUserAdd by a
// non-admin user results in an error.
func (c *SQLiteConn) AuthUserAdd(username, password string, admin bool) error {
// NOOP
return nil
}
// authUserAdd enables the User Authentication if not enabled.
// Otherwise it will add a user.
//
// When user authentication is already enabled then this function
// can only be called by an admin.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authUserAdd(username, password string, admin int) int {
// NOOP
return 0
}
// AuthUserChange can be used to change a users
// login credentials or admin privilege. Any user can change their own
// login credentials. Only an admin user can change another users login
// credentials or admin privilege setting. No user may change their own
// admin privilege setting.
func (c *SQLiteConn) AuthUserChange(username, password string, admin bool) error {
// NOOP
return nil
}
// authUserChange allows to modify a user.
// Users can change their own password.
//
// Only admins can change passwords for other users
// and modify the admin flag.
//
// The admin flag of the current logged in user cannot be changed.
// THis ensures that their is always an admin.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authUserChange(username, password string, admin int) int {
// NOOP
return 0
}
// AuthUserDelete can be used (by an admin user only)
// to delete a user. The currently logged-in user cannot be deleted,
// which guarantees that there is always an admin user and hence that
// the database cannot be converted into a no-authentication-required
// database.
func (c *SQLiteConn) AuthUserDelete(username string) error {
// NOOP
return nil
}
// authUserDelete can be used to delete a user.
//
// This function can only be executed by an admin.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// C.SQLITE_OK (0)
// C.SQLITE_ERROR (1)
// C.SQLITE_AUTH (23)
func (c *SQLiteConn) authUserDelete(username string) int {
// NOOP
return 0
}
// AuthEnabled checks if the database is protected by user authentication
func (c *SQLiteConn) AuthEnabled() (exists bool) {
// NOOP
return false
}
// authEnabled perform the actual check for user authentication.
//
// This is not exported for usage in Go.
// It is however exported for usage within SQL by the user.
//
// Returns:
// 0 - Disabled
// 1 - Enabled
func (c *SQLiteConn) authEnabled() int {
// NOOP
return 0
}
// EOF

619
sqlite3_opt_userauth_test.go Normal file
View File

@ -0,0 +1,619 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_userauth
package sqlite3
import (
"database/sql"
"fmt"
"os"
"testing"
)
var (
conn *SQLiteConn
create func(t *testing.T, username, password string) (file string, err error)
createWithCrypt func(t *testing.T, username, password, crypt, salt string) (file string, err error)
connect func(t *testing.T, f string, username, password string) (file string, db *sql.DB, c *SQLiteConn, err error)
connectWithCrypt func(t *testing.T, f string, username, password string, crypt string, salt string) (file string, db *sql.DB, c *SQLiteConn, err error)
authEnabled func(db *sql.DB) (exists bool, err error)
addUser func(db *sql.DB, username, password string, admin int) (rv int, err error)
userExists func(db *sql.DB, username string) (rv int, err error)
isAdmin func(db *sql.DB, username string) (rv bool, err error)
modifyUser func(db *sql.DB, username, password string, admin int) (rv int, err error)
deleteUser func(db *sql.DB, username string) (rv int, err error)
)
func init() {
// Create database connection
sql.Register("sqlite3_with_conn",
&SQLiteDriver{
ConnectHook: func(c *SQLiteConn) error {
conn = c
return nil
},
})
create = func(t *testing.T, username, password string) (file string, err error) {
var db *sql.DB
file, db, _, err = connect(t, "", username, password)
db.Close()
return
}
createWithCrypt = func(t *testing.T, username, password, crypt, salt string) (file string, err error) {
var db *sql.DB
file, db, _, err = connectWithCrypt(t, "", "admin", "admin", crypt, salt)
db.Close()
return
}
connect = func(t *testing.T, f string, username, password string) (file string, db *sql.DB, c *SQLiteConn, err error) {
conn = nil // Clear connection
file = f // Copy provided file (f) => file
if file == "" {
// Create dummy file
file = TempFilename(t)
}
db, err = sql.Open("sqlite3_with_conn", "file:"+file+fmt.Sprintf("?_auth&_auth_user=%s&_auth_pass=%s", username, password))
if err != nil {
defer os.Remove(file)
return file, nil, nil, err
}
// Dummy query to force connection and database creation
// Will return ErrUnauthorized (SQLITE_AUTH) if user authentication fails
if _, err = db.Exec("SELECT 1;"); err != nil {
defer os.Remove(file)
defer db.Close()
return file, nil, nil, err
}
c = conn
return
}
connectWithCrypt = func(t *testing.T, f string, username, password string, crypt string, salt string) (file string, db *sql.DB, c *SQLiteConn, err error) {
conn = nil // Clear connection
file = f // Copy provided file (f) => file
if file == "" {
// Create dummy file
file = TempFilename(t)
}
db, err = sql.Open("sqlite3_with_conn", "file:"+file+fmt.Sprintf("?_auth&_auth_user=%s&_auth_pass=%s&_auth_crypt=%s&_auth_salt=%s", username, password, crypt, salt))
if err != nil {
defer os.Remove(file)
return file, nil, nil, err
}
// Dummy query to force connection and database creation
// Will return ErrUnauthorized (SQLITE_AUTH) if user authentication fails
if _, err = db.Exec("SELECT 1;"); err != nil {
defer os.Remove(file)
defer db.Close()
return file, nil, nil, err
}
c = conn
return
}
authEnabled = func(db *sql.DB) (exists bool, err error) {
err = db.QueryRow("select count(type) from sqlite_master WHERE type='table' and name='sqlite_user';").Scan(&exists)
return
}
addUser = func(db *sql.DB, username, password string, admin int) (rv int, err error) {
err = db.QueryRow("select auth_user_add(?, ?, ?);", username, password, admin).Scan(&rv)
return
}
userExists = func(db *sql.DB, username string) (rv int, err error) {
err = db.QueryRow("select count(uname) from sqlite_user where uname=?", username).Scan(&rv)
return
}
isAdmin = func(db *sql.DB, username string) (rv bool, err error) {
err = db.QueryRow("select isAdmin from sqlite_user where uname=?", username).Scan(&rv)
return
}
modifyUser = func(db *sql.DB, username, password string, admin int) (rv int, err error) {
err = db.QueryRow("select auth_user_change(?, ?, ?);", username, password, admin).Scan(&rv)
return
}
deleteUser = func(db *sql.DB, username string) (rv int, err error) {
err = db.QueryRow("select auth_user_delete(?);", username).Scan(&rv)
return
}
}
func TestUserAuthCreateDatabase(t *testing.T) {
f, db, c, err := connect(t, "", "admin", "admin")
if err != nil && c == nil && db == nil {
t.Fatal(err)
}
defer db.Close()
defer os.Remove(f)
enabled, err := authEnabled(db)
if err != nil || !enabled {
t.Fatalf("UserAuth not enabled: %s", err)
}
e, err := userExists(db, "admin")
if err != nil {
t.Fatal(err)
}
if e != 1 {
t.Fatal("UserAuth: admin does not exists")
}
a, err := isAdmin(db, "admin")
if err != nil {
t.Fatal(err)
}
if !a {
t.Fatal("UserAuth: User is not administrator")
}
}
func TestUserAuthLogin(t *testing.T) {
f1, err := create(t, "admin", "admin")
if err != nil {
t.Fatal(err)
}
defer os.Remove(f1)
f2, db2, c2, err := connect(t, f1, "admin", "admin")
if err != nil {
t.Fatal(err)
}
defer db2.Close()
if f1 != f2 {
t.Fatal("UserAuth: Database file mismatch")
}
// Test lower level authentication
err = c2.Authenticate("admin", "admin")
if err != nil {
t.Fatalf("UserAuth: *SQLiteConn.Authenticate() Failed: %s", err)
}
// Test Login Failed
_, _, _, err = connect(t, f1, "admin", "invalid")
if err == nil {
t.Fatal("Login successful while expecting to fail")
}
if err != ErrUnauthorized {
t.Fatal(err)
}
err = c2.Authenticate("admin", "invalid")
if err == nil {
t.Fatal("Login successful while expecting to fail")
}
if err != ErrUnauthorized {
t.Fatal(err)
}
}
func TestUserAuthAddAdmin(t *testing.T) {
f, db, c, err := connect(t, "", "admin", "admin")
if err != nil && c == nil && db == nil {
t.Fatal(err)
}
defer db.Close()
defer os.Remove(f)
// Add Admin User through SQL call
rv, err := addUser(db, "admin2", "admin2", 1)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
// Check if user was created
exists, err := userExists(db, "admin2")
if err != nil {
t.Fatal(err)
}
if exists != 1 {
t.Fatal("UserAuth: 'admin2' does not exists")
}
// Check if user was created as an Administrator
admin, err := isAdmin(db, "admin2")
if err != nil {
t.Fatal(err)
}
if !admin {
t.Fatal("UserAuth: 'admin2' is not administrator")
}
// Test *SQLiteConn
err = c.AuthUserAdd("admin3", "admin3", true)
if err != nil {
t.Fatal(err)
}
// Check if user was created
exists, err = userExists(db, "admin2")
if err != nil {
t.Fatal(err)
}
if exists != 1 {
t.Fatal("UserAuth: 'admin3' does not exists")
}
// Check if the user was created as an Administrator
admin, err = isAdmin(db, "admin3")
if err != nil {
t.Fatal(err)
}
if !admin {
t.Fatal("UserAuth: 'admin3' is not administrator")
}
}
func TestUserAuthAddUser(t *testing.T) {
f1, db1, c, err := connect(t, "", "admin", "admin")
if err != nil && c == nil && db == nil {
t.Fatal(err)
}
defer os.Remove(f1)
// Add user through SQL call
rv, err := addUser(db1, "user", "user", 0)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
// Check if user was created
exists, err := userExists(db1, "user")
if err != nil {
t.Fatal(err)
}
if exists != 1 {
t.Fatal("UserAuth: 'user' does not exists")
}
// Check if user was created as an Administrator
admin, err := isAdmin(db1, "user")
if err != nil {
t.Fatal(err)
}
if admin {
t.Fatal("UserAuth: 'user' is administrator")
}
// Test *SQLiteConn
err = c.AuthUserAdd("user2", "user2", false)
if err != nil {
t.Fatal(err)
}
// Check if user was created
exists, err = userExists(db1, "user2")
if err != nil {
t.Fatal(err)
}
if exists != 1 {
t.Fatal("UserAuth: 'user2' does not exists")
}
// Check if the user was created as an Administrator
admin, err = isAdmin(db1, "user2")
if err != nil {
t.Fatal(err)
}
if admin {
t.Fatal("UserAuth: 'user2' is administrator")
}
// Reconnect as normal user
db1.Close()
_, db2, c2, err := connect(t, f1, "user", "user")
if err != nil {
t.Fatal(err)
}
defer db2.Close()
// Try to create admin user while logged in as normal user
rv, err = addUser(db2, "admin2", "admin2", 1)
if err != nil {
t.Fatal(err)
}
if rv != SQLITE_AUTH {
t.Fatal("Created admin user while not allowed")
}
err = c2.AuthUserAdd("admin3", "admin3", true)
if err != ErrAdminRequired {
t.Fatal("Created admin user while not allowed")
}
// Try to create normal user while logged in as normal user
rv, err = addUser(db2, "user3", "user3", 0)
if err != nil {
t.Fatal(err)
}
if rv != SQLITE_AUTH {
t.Fatal("Created user while not allowed")
}
err = c2.AuthUserAdd("user4", "user4", false)
if err != ErrAdminRequired {
t.Fatal("Created user while not allowed")
}
}
func TestUserAuthModifyUser(t *testing.T) {
f1, db1, c1, err := connect(t, "", "admin", "admin")
if err != nil && c1 == nil && db == nil {
t.Fatal(err)
}
defer os.Remove(f1)
// Modify Password for current logged in admin
// through SQL
rv, err := modifyUser(db1, "admin", "admin2", 1)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to modify password for admin")
}
// Modify password for current logged in admin
// through *SQLiteConn
err = c1.AuthUserChange("admin", "admin3", true)
if err != nil {
t.Fatal(err)
}
// Modify Administrator Flag
// Because we are current logged in as 'admin'
// Changing our own admin flag should fail.
rv, err = modifyUser(db1, "admin", "admin3", 0)
if err != nil {
t.Fatal(err)
}
if rv != SQLITE_AUTH {
t.Fatal("Successfully changed admin flag while not allowed")
}
// Modify admin flag through (*SQLiteConn)
// Because we are current logged in as 'admin'
// Changing our own admin flag should fail.
err = c1.AuthUserChange("admin", "admin3", false)
if err != ErrAdminRequired {
t.Fatal("Successfully changed admin flag while not allowed")
}
// Add normal user
rv, err = addUser(db1, "user", "password", 0)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
rv, err = addUser(db1, "user2", "user2", 0)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
// Modify other user password and flag through SQL
rv, err = modifyUser(db1, "user", "pass", 1)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to modify password for user")
}
// Modify other user password and flag through *SQLiteConn
err = c1.AuthUserChange("user", "newpass", false)
if err != nil {
t.Fatal(err)
}
// Disconnect database for reconnect
db1.Close()
_, db2, c2, err := connect(t, f1, "user", "newpass")
if err != nil {
t.Fatal(err)
}
defer db2.Close()
// Modify other user password through SQL
rv, err = modifyUser(db2, "user2", "newpass", 0)
if err != nil {
t.Fatal(err)
}
if rv != SQLITE_AUTH {
t.Fatal("Password change successful while not allowed")
}
// Modify other user password and flag through *SQLiteConn
err = c2.AuthUserChange("user2", "invalid", false)
if err != ErrAdminRequired {
t.Fatal("Password change successful while not allowed")
}
}
func TestUserAuthDeleteUser(t *testing.T) {
f1, db1, c, err := connect(t, "", "admin", "admin")
if err != nil && c == nil && db == nil {
t.Fatal(err)
}
defer os.Remove(f1)
// Add Admin User 2
rv, err := addUser(db1, "admin2", "admin2", 1)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
rv, err = addUser(db1, "admin3", "admin3", 1)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
// Check if user was created
exists, err := userExists(db1, "admin2")
if err != nil {
t.Fatal(err)
}
if exists != 1 {
t.Fatal("UserAuth: 'admin2' does not exists")
}
exists, err = userExists(db1, "admin3")
if err != nil {
t.Fatal(err)
}
if exists != 1 {
t.Fatal("UserAuth: 'admin2' does not exists")
}
// Delete user through SQL
rv, err = deleteUser(db1, "admin2")
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to delete admin2")
}
// Verify user admin2 deleted
exists, err = userExists(db1, "admin2")
if err != nil {
t.Fatal(err)
}
if exists != 0 {
t.Fatal("UserAuth: 'admin2' still exists")
}
// Delete user through *SQLiteConn
rv, err = deleteUser(db1, "admin3")
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to delete admin3")
}
// Verify user admin3 deleted
exists, err = userExists(db1, "admin3")
if err != nil {
t.Fatal(err)
}
if exists != 0 {
t.Fatal("UserAuth: 'admin3' still exists")
}
// Add normal user for reconnect and privileges check
rv, err = addUser(db1, "reconnect", "reconnect", 0)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
// Add normal user for deletion through SQL
rv, err = addUser(db1, "user", "user", 0)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
rv, err = addUser(db1, "user2", "user2", 0)
if err != nil {
t.Fatal(err)
}
if rv != 0 {
t.Fatal("Failed to add user")
}
// Close database for reconnect
db1.Close()
// Reconnect as normal user
_, db2, c2, err := connect(t, f1, "reconnect", "reconnect")
if err != nil {
t.Fatal(err)
}
defer db2.Close()
// Delete user while logged in as normal user
// through SQL
rv, err = deleteUser(db2, "user")
if err != nil {
t.Fatal(err)
}
if rv != SQLITE_AUTH {
t.Fatal("Successfully deleted user wthout proper privileges")
}
// Delete user while logged in as normal user
// through *SQLiteConn
err = c2.AuthUserDelete("user2")
if err != ErrAdminRequired {
t.Fatal("Successfully deleted user wthout proper privileges")
}
}
func TestUserAuthEncoders(t *testing.T) {
cases := map[string]string{
"sha1": "",
"ssha1": "salted",
"sha256": "",
"ssha256": "salted",
"sha384": "",
"ssha384": "salted",
"sha512": "",
"ssha512": "salted",
}
for enc, salt := range cases {
f, err := createWithCrypt(t, "admin", "admin", enc, salt)
if err != nil {
t.Fatal(err)
}
defer os.Remove(f)
_, db, _, err := connectWithCrypt(t, f, "admin", "admin", enc, salt)
if err != nil {
t.Fatal(err)
}
defer db.Close()
if e, err := authEnabled(db); err != nil && !e {
t.Fatalf("UserAuth (%s) not enabled %s", enc, err)
}
}
}

15
sqlite3_opt_vacuum_full.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_vacuum_full
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_DEFAULT_AUTOVACUUM=1
#cgo LDFLAGS: -lm
*/
import "C"

15
sqlite3_opt_vacuum_incr.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_vacuum_incr
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_DEFAULT_AUTOVACUUM=2
#cgo LDFLAGS: -lm
*/
import "C"

10
sqlite3_vtable.go → sqlite3_opt_vtable.go
View File

@ -2,14 +2,18 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build vtable
// +build sqlite_vtable vtable
package sqlite3 package sqlite3
/* /*
#cgo CFLAGS: -std=gnu99 #cgo CFLAGS: -std=gnu99
#cgo CFLAGS: -DSQLITE_ENABLE_RTREE -DSQLITE_THREADSAFE #cgo CFLAGS: -DSQLITE_ENABLE_RTREE
#cgo CFLAGS: -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4_UNICODE61 #cgo CFLAGS: -DSQLITE_THREADSAFE
#cgo CFLAGS: -DSQLITE_ENABLE_FTS3
#cgo CFLAGS: -DSQLITE_ENABLE_FTS3_PARENTHESIS
#cgo CFLAGS: -DSQLITE_ENABLE_FTS4_UNICODE61
#cgo CFLAGS: -DSQLITE_TRACE_SIZE_LIMIT=15 #cgo CFLAGS: -DSQLITE_TRACE_SIZE_LIMIT=15
#cgo CFLAGS: -DSQLITE_ENABLE_COLUMN_METADATA=1 #cgo CFLAGS: -DSQLITE_ENABLE_COLUMN_METADATA=1
#cgo CFLAGS: -Wno-deprecated-declarations #cgo CFLAGS: -Wno-deprecated-declarations

3
sqlite3_vtable_test.go → sqlite3_opt_vtable_test.go
View File

@ -2,7 +2,8 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build vtable
// +build sqlite_vtable vtable
package sqlite3 package sqlite3

5
sqlite3_other.go
View File

@ -2,6 +2,7 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build !windows // +build !windows
package sqlite3 package sqlite3
@ -9,6 +10,8 @@ package sqlite3
/* /*
#cgo CFLAGS: -I. #cgo CFLAGS: -I.
#cgo linux LDFLAGS: -ldl #cgo linux LDFLAGS: -ldl
#cgo solaris LDFLAGS: -lc #cgo linux,ppc LDFLAGS: -lpthread
#cgo linux,ppc64 LDFLAGS: -lpthread
#cgo linux,ppc64le LDFLAGS: -lpthread
*/ */
import "C" import "C"

2
sqlite3_solaris.go
View File

@ -2,11 +2,13 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build solaris // +build solaris
package sqlite3 package sqlite3
/* /*
#cgo CFLAGS: -D__EXTENSIONS__=1 #cgo CFLAGS: -D__EXTENSIONS__=1
#cgo LDFLAGS: -lc
*/ */
import "C" import "C"

295
sqlite3_test.go
View File

@ -87,6 +87,64 @@ func TestOpen(t *testing.T) {
} }
} }
func TestOpenNoCreate(t *testing.T) {
filename := t.Name() + ".sqlite"
if err := os.Remove(filename); err != nil && !os.IsNotExist(err) {
t.Fatal(err)
}
defer os.Remove(filename)
// https://golang.org/pkg/database/sql/#Open
// "Open may just validate its arguments without creating a connection
// to the database. To verify that the data source name is valid, call Ping."
db, err := sql.Open("sqlite3", fmt.Sprintf("file:%s?mode=rw", filename))
if err == nil {
defer db.Close()
err = db.Ping()
if err == nil {
t.Fatal("expected error from Open or Ping")
}
}
sqlErr, ok := err.(Error)
if !ok {
t.Fatalf("expected sqlite3.Error, but got %T", err)
}
if sqlErr.Code != ErrCantOpen {
t.Fatalf("expected SQLITE_CANTOPEN, but got %v", sqlErr)
}
// make sure database file truly was not created
if _, err := os.Stat(filename); !os.IsNotExist(err) {
if err != nil {
t.Fatal(err)
}
t.Fatal("expected database file to not exist")
}
// verify that it works if the mode is "rwc" instead
db, err = sql.Open("sqlite3", fmt.Sprintf("file:%s?mode=rwc", filename))
if err != nil {
t.Fatal(err)
}
defer db.Close()
if err := db.Ping(); err != nil {
t.Fatal(err)
}
// make sure database file truly was created
if _, err := os.Stat(filename); err != nil {
if !os.IsNotExist(err) {
t.Fatal(err)
}
t.Fatal("expected database file to exist")
}
}
func TestReadonly(t *testing.T) { func TestReadonly(t *testing.T) {
tempFilename := TempFilename(t) tempFilename := TempFilename(t)
defer os.Remove(tempFilename) defer os.Remove(tempFilename)
@ -231,6 +289,55 @@ func TestInsert(t *testing.T) {
} }
} }
func TestUpsert(t *testing.T) {
_, n, _ := Version()
if !(n >= 3024000) {
t.Skip("UPSERT requires sqlite3 => 3.24.0")
}
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
db, err := sql.Open("sqlite3", tempFilename)
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
_, err = db.Exec("drop table foo")
_, err = db.Exec("create table foo (name string primary key, counter integer)")
if err != nil {
t.Fatal("Failed to create table:", err)
}
for i := 0; i < 10; i++ {
res, err := db.Exec("insert into foo(name, counter) values('key', 1) on conflict (name) do update set counter=counter+1")
if err != nil {
t.Fatal("Failed to upsert record:", err)
}
affected, _ := res.RowsAffected()
if affected != 1 {
t.Fatalf("Expected %d for affected rows, but %d:", 1, affected)
}
}
rows, err := db.Query("select name, counter from foo")
if err != nil {
t.Fatal("Failed to select records:", err)
}
defer rows.Close()
rows.Next()
var resultName string
var resultCounter int
rows.Scan(&resultName, &resultCounter)
if resultName != "key" {
t.Errorf("Expected %s for fetched result, but %s:", "key", resultName)
}
if resultCounter != 10 {
t.Errorf("Expected %d for fetched result, but %d:", 10, resultCounter)
}
}
func TestUpdate(t *testing.T) { func TestUpdate(t *testing.T) {
tempFilename := TempFilename(t) tempFilename := TempFilename(t)
defer os.Remove(tempFilename) defer os.Remove(tempFilename)
@ -1086,7 +1193,7 @@ func TestEncryptoDatabase(t *testing.T) {
defer os.Remove(tempFilename) defer os.Remove(tempFilename)
defer db.Close() defer db.Close()
_, err = db.Exec("PRAGMA key = password;") _, err = db.Exec("PRAGMA key = 'password';")
if err != nil { if err != nil {
t.Error("Failed to set encrypto key") t.Error("Failed to set encrypto key")
} }
@ -1118,27 +1225,21 @@ func TestEncryptoDatabase(t *testing.T) {
t.Error("Failed to db.QueryRow: not matched results") t.Error("Failed to db.QueryRow: not matched results")
} }
db.Close() db.Close()
db, err = sql.Open("sqlite3", tempFilename) db, err = sql.Open("sqlite3", tempFilename+"?_key=not_password")
if err != nil { if err != nil {
t.Fatal("Failed to open database:", err) t.Fatal("Failed to open database:", err)
} }
_, err = db.Exec("PRAGMA key = not_password;")
if err != nil {
t.Error("Failed to set encrypto key")
}
rows, err = db.Query("select id from foo") rows, err = db.Query("select id from foo")
if err == nil { if err == nil {
t.Error("Failed to encrypto database") t.Error("Failed to encrypto database")
} }
db.Close() db.Close()
db, err = sql.Open("sqlite3", tempFilename) db, err = sql.Open("sqlite3", tempFilename+"?_key=password")
if err != nil { if err != nil {
t.Fatal("Failed to open database:", err) t.Fatal("Failed to open database:", err)
} }
_, err = db.Exec("PRAGMA key = password;")
if err != nil {
t.Error("Failed to set encrypto key")
}
rows, err = db.Query("select id from foo") rows, err = db.Query("select id from foo")
if rows == nil || err != nil { if rows == nil || err != nil {
t.Error("Failed to call db.Query") t.Error("Failed to call db.Query")
@ -1365,10 +1466,7 @@ func TestAggregatorRegistration(t *testing.T) {
sql.Register("sqlite3_AggregatorRegistration", &SQLiteDriver{ sql.Register("sqlite3_AggregatorRegistration", &SQLiteDriver{
ConnectHook: func(conn *SQLiteConn) error { ConnectHook: func(conn *SQLiteConn) error {
if err := conn.RegisterAggregator("customSum", customSum, true); err != nil { return conn.RegisterAggregator("customSum", customSum, true)
return err
}
return nil
}, },
}) })
db, err := sql.Open("sqlite3_AggregatorRegistration", ":memory:") db, err := sql.Open("sqlite3_AggregatorRegistration", ":memory:")
@ -1640,6 +1738,67 @@ func TestUpdateAndTransactionHooks(t *testing.T) {
} }
} }
func TestAuthorizer(t *testing.T) {
var authorizerReturn = 0
sql.Register("sqlite3_Authorizer", &SQLiteDriver{
ConnectHook: func(conn *SQLiteConn) error {
conn.RegisterAuthorizer(func(op int, arg1, arg2, arg3 string) int {
return authorizerReturn
})
return nil
},
})
db, err := sql.Open("sqlite3_Authorizer", ":memory:")
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
statements := []string{
"create table foo (id integer primary key, name varchar)",
"insert into foo values (9, 'test9')",
"update foo set name = 'test99' where id = 9",
"select * from foo",
}
authorizerReturn = SQLITE_OK
for _, statement := range statements {
_, err = db.Exec(statement)
if err != nil {
t.Fatalf("No error expected [%v]: %v", statement, err)
}
}
authorizerReturn = SQLITE_DENY
for _, statement := range statements {
_, err = db.Exec(statement)
if err == nil {
t.Fatalf("Authorizer didn't worked - nil received, but error expected: [%v]", statement)
}
}
}
func TestNonColumnString(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
var x interface{}
if err := db.QueryRow("SELECT 'hello'").Scan(&x); err != nil {
t.Fatal(err)
}
s, ok := x.(string)
if !ok {
t.Fatalf("non-column string must return string but got %T", x)
}
if s != "hello" {
t.Fatalf("non-column string must return %q but got %q", "hello", s)
}
}
func TestNilAndEmptyBytes(t *testing.T) { func TestNilAndEmptyBytes(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:") db, err := sql.Open("sqlite3", ":memory:")
if err != nil { if err != nil {
@ -1695,6 +1854,25 @@ func TestNilAndEmptyBytes(t *testing.T) {
} }
} }
func TestInsertNilByteSlice(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
if _, err := db.Exec("create table blob_not_null (b blob not null)"); err != nil {
t.Fatal(err)
}
var nilSlice []byte
if _, err := db.Exec("insert into blob_not_null (b) values (?)", nilSlice); err == nil {
t.Fatal("didn't expect INSERT to 'not null' column with a nil []byte slice to work")
}
zeroLenSlice := []byte{}
if _, err := db.Exec("insert into blob_not_null (b) values (?)", zeroLenSlice); err != nil {
t.Fatal("failed to insert zero-length slice")
}
}
var customFunctionOnce sync.Once var customFunctionOnce sync.Once
func BenchmarkCustomFunctions(b *testing.B) { func BenchmarkCustomFunctions(b *testing.B) {
@ -1737,7 +1915,10 @@ func TestSuite(t *testing.T) {
defer d.Close() defer d.Close()
db = &TestDB{t, d, SQLITE, sync.Once{}} db = &TestDB{t, d, SQLITE, sync.Once{}}
testing.RunTests(func(string, string) (bool, error) { return true, nil }, tests) ok := testing.RunTests(func(string, string) (bool, error) { return true, nil }, tests)
if !ok {
t.Fatal("A subtest failed")
}
if !testing.Short() { if !testing.Short() {
for _, b := range benchmarks { for _, b := range benchmarks {
@ -1775,6 +1956,8 @@ var testTables = []string{"foo", "bar", "t", "bench"}
var tests = []testing.InternalTest{ var tests = []testing.InternalTest{
{Name: "TestResult", F: testResult}, {Name: "TestResult", F: testResult},
{Name: "TestBlobs", F: testBlobs}, {Name: "TestBlobs", F: testBlobs},
{Name: "TestMultiBlobs", F: testMultiBlobs},
{Name: "TestNullZeroLengthBlobs", F: testNullZeroLengthBlobs},
{Name: "TestManyQueryRow", F: testManyQueryRow}, {Name: "TestManyQueryRow", F: testManyQueryRow},
{Name: "TestTxQuery", F: testTxQuery}, {Name: "TestTxQuery", F: testTxQuery},
{Name: "TestPreparedStmt", F: testPreparedStmt}, {Name: "TestPreparedStmt", F: testPreparedStmt},
@ -1930,6 +2113,86 @@ func testBlobs(t *testing.T) {
} }
} }
func testMultiBlobs(t *testing.T) {
db.tearDown()
db.mustExec("create table foo (id integer primary key, bar " + db.blobType(16) + ")")
var blob0 = []byte{0, 1, 2, 3, 4, 5, 6, 7}
db.mustExec(db.q("insert into foo (id, bar) values(?,?)"), 0, blob0)
var blob1 = []byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
db.mustExec(db.q("insert into foo (id, bar) values(?,?)"), 1, blob1)
r, err := db.Query(db.q("select bar from foo order by id"))
if err != nil {
t.Fatal(err)
}
defer r.Close()
if !r.Next() {
if r.Err() != nil {
t.Fatal(err)
}
t.Fatal("expected one rows")
}
want0 := fmt.Sprintf("%x", blob0)
b0 := make([]byte, 8)
err = r.Scan(&b0)
if err != nil {
t.Fatal(err)
}
got0 := fmt.Sprintf("%x", b0)
if !r.Next() {
if r.Err() != nil {
t.Fatal(err)
}
t.Fatal("expected one rows")
}
want1 := fmt.Sprintf("%x", blob1)
b1 := make([]byte, 16)
err = r.Scan(&b1)
if err != nil {
t.Fatal(err)
}
got1 := fmt.Sprintf("%x", b1)
if got0 != want0 {
t.Errorf("for []byte, got %q; want %q", got0, want0)
}
if got1 != want1 {
t.Errorf("for []byte, got %q; want %q", got1, want1)
}
}
// testBlobs tests that we distinguish between null and zero-length blobs
func testNullZeroLengthBlobs(t *testing.T) {
db.tearDown()
db.mustExec("create table foo (id integer primary key, bar " + db.blobType(16) + ")")
db.mustExec(db.q("insert into foo (id, bar) values(?,?)"), 0, nil)
db.mustExec(db.q("insert into foo (id, bar) values(?,?)"), 1, []byte{})
r0 := db.QueryRow(db.q("select bar from foo where id=0"))
var b0 []byte
err := r0.Scan(&b0)
if err != nil {
t.Fatal(err)
}
if b0 != nil {
t.Errorf("for id=0, got %x; want nil", b0)
}
r1 := db.QueryRow(db.q("select bar from foo where id=1"))
var b1 []byte
err = r1.Scan(&b1)
if err != nil {
t.Fatal(err)
}
if b1 == nil {
t.Error("for id=1, got nil; want zero-length slice")
} else if len(b1) > 0 {
t.Errorf("for id=1, got %x; want zero-length slice", b1)
}
}
// testManyQueryRow is test for many query row // testManyQueryRow is test for many query row
func testManyQueryRow(t *testing.T) { func testManyQueryRow(t *testing.T) {
if testing.Short() { if testing.Short() {

3
sqlite3_trace.go
View File

@ -2,7 +2,8 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build trace
// +build sqlite_trace trace
package sqlite3 package sqlite3

39
sqlite3_usleep_windows.go Normal file
View File

@ -0,0 +1,39 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build cgo
package sqlite3
// usleep is a function available on *nix based systems.
// This function is not present in Windows.
// Windows has a sleep function but this works with seconds
// and not with microseconds as usleep.
//
// This code should improve performance on windows because
// without the presence of usleep SQLite waits 1 second.
//
// Source: https://stackoverflow.com/questions/5801813/c-usleep-is-obsolete-workarounds-for-windows-mingw?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
/*
#include <windows.h>
void usleep(__int64 usec)
{
HANDLE timer;
LARGE_INTEGER ft;
// Convert to 100 nanosecond interval, negative value indicates relative time
ft.QuadPart = -(10*usec);
timer = CreateWaitableTimer(NULL, TRUE, NULL);
SetWaitableTimer(timer, &ft, 0, NULL, NULL, 0);
WaitForSingleObject(timer, INFINITE);
CloseHandle(timer);
}
*/
import "C"
// EOF

8
sqlite3_windows.go
View File

@ -2,13 +2,17 @@
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build windows // +build windows
package sqlite3 package sqlite3
/* /*
#cgo CFLAGS: -I. -fno-stack-check -fno-stack-protector -mno-stack-arg-probe #cgo CFLAGS: -I.
#cgo windows,386 CFLAGS: -D_USE_32BIT_TIME_T #cgo CFLAGS: -fno-stack-check
#cgo CFLAGS: -fno-stack-protector
#cgo CFLAGS: -mno-stack-arg-probe
#cgo LDFLAGS: -lmingwex -lmingw32 #cgo LDFLAGS: -lmingwex -lmingw32
#cgo windows,386 CFLAGS: -D_USE_32BIT_TIME_T
*/ */
import "C" import "C"

66
sqlite3ext.h
View File

@ -135,7 +135,7 @@ struct sqlite3_api_routines {
int (*set_authorizer)(sqlite3*,int(*)(void*,int,const char*,const char*, int (*set_authorizer)(sqlite3*,int(*)(void*,int,const char*,const char*,
const char*,const char*),void*); const char*,const char*),void*);
void (*set_auxdata)(sqlite3_context*,int,void*,void (*)(void*)); void (*set_auxdata)(sqlite3_context*,int,void*,void (*)(void*));
char * (*snprintf)(int,char*,const char*,...); char * (*xsnprintf)(int,char*,const char*,...);
int (*step)(sqlite3_stmt*); int (*step)(sqlite3_stmt*);
int (*table_column_metadata)(sqlite3*,const char*,const char*,const char*, int (*table_column_metadata)(sqlite3*,const char*,const char*,const char*,
char const**,char const**,int*,int*,int*); char const**,char const**,int*,int*,int*);
@ -247,7 +247,7 @@ struct sqlite3_api_routines {
int (*uri_boolean)(const char*,const char*,int); int (*uri_boolean)(const char*,const char*,int);
sqlite3_int64 (*uri_int64)(const char*,const char*,sqlite3_int64); sqlite3_int64 (*uri_int64)(const char*,const char*,sqlite3_int64);
const char *(*uri_parameter)(const char*,const char*); const char *(*uri_parameter)(const char*,const char*);
char *(*vsnprintf)(int,char*,const char*,va_list); char *(*xvsnprintf)(int,char*,const char*,va_list);
int (*wal_checkpoint_v2)(sqlite3*,const char*,int,int*,int*); int (*wal_checkpoint_v2)(sqlite3*,const char*,int,int*,int*);
/* Version 3.8.7 and later */ /* Version 3.8.7 and later */
int (*auto_extension)(void(*)(void)); int (*auto_extension)(void(*)(void));
@ -293,6 +293,33 @@ struct sqlite3_api_routines {
int (*bind_pointer)(sqlite3_stmt*,int,void*,const char*,void(*)(void*)); int (*bind_pointer)(sqlite3_stmt*,int,void*,const char*,void(*)(void*));
void (*result_pointer)(sqlite3_context*,void*,const char*,void(*)(void*)); void (*result_pointer)(sqlite3_context*,void*,const char*,void(*)(void*));
void *(*value_pointer)(sqlite3_value*,const char*); void *(*value_pointer)(sqlite3_value*,const char*);
int (*vtab_nochange)(sqlite3_context*);
int (*value_nochange)(sqlite3_value*);
const char *(*vtab_collation)(sqlite3_index_info*,int);
/* Version 3.24.0 and later */
int (*keyword_count)(void);
int (*keyword_name)(int,const char**,int*);
int (*keyword_check)(const char*,int);
sqlite3_str *(*str_new)(sqlite3*);
char *(*str_finish)(sqlite3_str*);
void (*str_appendf)(sqlite3_str*, const char *zFormat, ...);
void (*str_vappendf)(sqlite3_str*, const char *zFormat, va_list);
void (*str_append)(sqlite3_str*, const char *zIn, int N);
void (*str_appendall)(sqlite3_str*, const char *zIn);
void (*str_appendchar)(sqlite3_str*, int N, char C);
void (*str_reset)(sqlite3_str*);
int (*str_errcode)(sqlite3_str*);
int (*str_length)(sqlite3_str*);
char *(*str_value)(sqlite3_str*);
/* Version 3.25.0 and later */
int (*create_window_function)(sqlite3*,const char*,int,int,void*,
void (*xStep)(sqlite3_context*,int,sqlite3_value**),
void (*xFinal)(sqlite3_context*),
void (*xValue)(sqlite3_context*),
void (*xInv)(sqlite3_context*,int,sqlite3_value**),
void(*xDestroy)(void*));
/* Version 3.26.0 and later */
const char *(*normalized_sql)(sqlite3_stmt*);
}; };
/* /*
@ -419,7 +446,7 @@ typedef int (*sqlite3_loadext_entry)(
#define sqlite3_rollback_hook sqlite3_api->rollback_hook #define sqlite3_rollback_hook sqlite3_api->rollback_hook
#define sqlite3_set_authorizer sqlite3_api->set_authorizer #define sqlite3_set_authorizer sqlite3_api->set_authorizer
#define sqlite3_set_auxdata sqlite3_api->set_auxdata #define sqlite3_set_auxdata sqlite3_api->set_auxdata
#define sqlite3_snprintf sqlite3_api->snprintf #define sqlite3_snprintf sqlite3_api->xsnprintf
#define sqlite3_step sqlite3_api->step #define sqlite3_step sqlite3_api->step
#define sqlite3_table_column_metadata sqlite3_api->table_column_metadata #define sqlite3_table_column_metadata sqlite3_api->table_column_metadata
#define sqlite3_thread_cleanup sqlite3_api->thread_cleanup #define sqlite3_thread_cleanup sqlite3_api->thread_cleanup
@ -443,7 +470,7 @@ typedef int (*sqlite3_loadext_entry)(
#define sqlite3_value_text16le sqlite3_api->value_text16le #define sqlite3_value_text16le sqlite3_api->value_text16le
#define sqlite3_value_type sqlite3_api->value_type #define sqlite3_value_type sqlite3_api->value_type
#define sqlite3_vmprintf sqlite3_api->vmprintf #define sqlite3_vmprintf sqlite3_api->vmprintf
#define sqlite3_vsnprintf sqlite3_api->vsnprintf #define sqlite3_vsnprintf sqlite3_api->xvsnprintf
#define sqlite3_overload_function sqlite3_api->overload_function #define sqlite3_overload_function sqlite3_api->overload_function
#define sqlite3_prepare_v2 sqlite3_api->prepare_v2 #define sqlite3_prepare_v2 sqlite3_api->prepare_v2
#define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2 #define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2
@ -519,7 +546,7 @@ typedef int (*sqlite3_loadext_entry)(
#define sqlite3_uri_boolean sqlite3_api->uri_boolean #define sqlite3_uri_boolean sqlite3_api->uri_boolean
#define sqlite3_uri_int64 sqlite3_api->uri_int64 #define sqlite3_uri_int64 sqlite3_api->uri_int64
#define sqlite3_uri_parameter sqlite3_api->uri_parameter #define sqlite3_uri_parameter sqlite3_api->uri_parameter
#define sqlite3_uri_vsnprintf sqlite3_api->vsnprintf #define sqlite3_uri_vsnprintf sqlite3_api->xvsnprintf
#define sqlite3_wal_checkpoint_v2 sqlite3_api->wal_checkpoint_v2 #define sqlite3_wal_checkpoint_v2 sqlite3_api->wal_checkpoint_v2
/* Version 3.8.7 and later */ /* Version 3.8.7 and later */
#define sqlite3_auto_extension sqlite3_api->auto_extension #define sqlite3_auto_extension sqlite3_api->auto_extension
@ -559,6 +586,29 @@ typedef int (*sqlite3_loadext_entry)(
#define sqlite3_bind_pointer sqlite3_api->bind_pointer #define sqlite3_bind_pointer sqlite3_api->bind_pointer
#define sqlite3_result_pointer sqlite3_api->result_pointer #define sqlite3_result_pointer sqlite3_api->result_pointer
#define sqlite3_value_pointer sqlite3_api->value_pointer #define sqlite3_value_pointer sqlite3_api->value_pointer
/* Version 3.22.0 and later */
#define sqlite3_vtab_nochange sqlite3_api->vtab_nochange
#define sqlite3_value_nochange sqlite3_api->value_nochange
#define sqlite3_vtab_collation sqlite3_api->vtab_collation
/* Version 3.24.0 and later */
#define sqlite3_keyword_count sqlite3_api->keyword_count
#define sqlite3_keyword_name sqlite3_api->keyword_name
#define sqlite3_keyword_check sqlite3_api->keyword_check
#define sqlite3_str_new sqlite3_api->str_new
#define sqlite3_str_finish sqlite3_api->str_finish
#define sqlite3_str_appendf sqlite3_api->str_appendf
#define sqlite3_str_vappendf sqlite3_api->str_vappendf
#define sqlite3_str_append sqlite3_api->str_append
#define sqlite3_str_appendall sqlite3_api->str_appendall
#define sqlite3_str_appendchar sqlite3_api->str_appendchar
#define sqlite3_str_reset sqlite3_api->str_reset
#define sqlite3_str_errcode sqlite3_api->str_errcode
#define sqlite3_str_length sqlite3_api->str_length
#define sqlite3_str_value sqlite3_api->str_value
/* Version 3.25.0 and later */
#define sqlite3_create_window_function sqlite3_api->create_window_function
/* Version 3.26.0 and later */
#define sqlite3_normalized_sql sqlite3_api->normalized_sql
#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */ #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */
#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
@ -577,7 +627,7 @@ typedef int (*sqlite3_loadext_entry)(
#endif #endif
#endif /* SQLITE3EXT_H */ #endif /* SQLITE3EXT_H */
#else // USE_LIBSQLITE3
// If users really want to link against the system sqlite3 we // If users really want to link against the system sqlite3 we
// need to make this file a noop. // need to make this file a noop.
#endif #endif

111
tool/upgrade.go
View File

@ -1,111 +0,0 @@
// +build ignore
package main
import (
"archive/zip"
"bufio"
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path"
"path/filepath"
"strings"
"github.com/PuerkitoBio/goquery"
)
func main() {
site := "https://www.sqlite.org/download.html"
fmt.Printf("scraping %v\n", site)
doc, err := goquery.NewDocument(site)
if err != nil {
log.Fatal(err)
}
var url string
doc.Find("a").Each(func(_ int, s *goquery.Selection) {
if url == "" && strings.HasPrefix(s.Text(), "sqlite-amalgamation-") {
url = "https://www.sqlite.org/2018/" + s.Text()
}
})
if url == "" {
return
}
fmt.Printf("downloading %v\n", url)
resp, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
resp.Body.Close()
log.Fatal(err)
}
fmt.Printf("extracting %v\n", path.Base(url))
r, err := zip.NewReader(bytes.NewReader(b), resp.ContentLength)
if err != nil {
resp.Body.Close()
log.Fatal(err)
}
resp.Body.Close()
for _, zf := range r.File {
var f *os.File
switch path.Base(zf.Name) {
case "sqlite3.c":
f, err = os.Create("sqlite3-binding.c")
case "sqlite3.h":
f, err = os.Create("sqlite3-binding.h")
case "sqlite3ext.h":
f, err = os.Create("sqlite3ext.h")
default:
continue
}
if err != nil {
log.Fatal(err)
}
zr, err := zf.Open()
if err != nil {
log.Fatal(err)
}
_, err = io.WriteString(f, "#ifndef USE_LIBSQLITE3\n")
if err != nil {
zr.Close()
f.Close()
log.Fatal(err)
}
scanner := bufio.NewScanner(zr)
for scanner.Scan() {
text := scanner.Text()
if text == `#include "sqlite3.h"` {
text = `#include "sqlite3-binding.h"`
}
_, err = fmt.Fprintln(f, text)
if err != nil {
break
}
}
err = scanner.Err()
if err != nil {
zr.Close()
f.Close()
log.Fatal(err)
}
_, err = io.WriteString(f, "#else // USE_LIBSQLITE3\n // If users really want to link against the system sqlite3 we\n// need to make this file a noop.\n #endif")
if err != nil {
zr.Close()
f.Close()
log.Fatal(err)
}
zr.Close()
f.Close()
fmt.Printf("extracted %v\n", filepath.Base(f.Name()))
}
}

5
upgrade/package.go Normal file
View File

@ -0,0 +1,5 @@
// Package upgrade is a dummy package to ensure package can be loaded
//
// This file is to avoid the following error:
// can't load package: package go-sqlite3/upgrade: build constraints exclude all Go files in go-sqlite3\upgrade
package upgrade

218
upgrade/upgrade.go Normal file
View File

@ -0,0 +1,218 @@
// +build !cgo
// +build upgrade
package main
import (
"archive/zip"
"bufio"
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path"
"path/filepath"
"strings"
"time"
"github.com/PuerkitoBio/goquery"
)
func download(prefix string) (url string, content []byte, err error) {
year := time.Now().Year()
site := "https://www.sqlite.org/download.html"
//fmt.Printf("scraping %v\n", site)
doc, err := goquery.NewDocument(site)
if err != nil {
log.Fatal(err)
}
doc.Find("a").Each(func(_ int, s *goquery.Selection) {
if strings.HasPrefix(s.Text(), prefix) {
url = fmt.Sprintf("https://www.sqlite.org/%d/", year) + s.Text()
}
})
if url == "" {
return "", nil, fmt.Errorf("Unable to find prefix '%s' on sqlite.org", prefix)
}
fmt.Printf("Downloading %v\n", url)
resp, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
// Ready Body Content
content, err = ioutil.ReadAll(resp.Body)
defer resp.Body.Close()
if err != nil {
return "", nil, err
}
return url, content, nil
}
func mergeFile(src string, dst string) error {
defer func() error {
fmt.Printf("Removing: %s\n", src)
err := os.Remove(src)
if err != nil {
return err
}
return nil
}()
// Open destination
fdst, err := os.OpenFile(dst, os.O_APPEND|os.O_WRONLY, 0666)
if err != nil {
return err
}
defer fdst.Close()
// Read source content
content, err := ioutil.ReadFile(src)
if err != nil {
return err
}
// Add Additional newline
if _, err := fdst.WriteString("\n"); err != nil {
return err
}
fmt.Printf("Merging: %s into %s\n", src, dst)
if _, err = fdst.Write(content); err != nil {
return err
}
return nil
}
func main() {
fmt.Println("Go-SQLite3 Upgrade Tool")
// Download Amalgamation
_, amalgamation, err := download("sqlite-amalgamation-")
if err != nil {
fmt.Println("Failed to download: sqlite-amalgamation; %s", err)
}
// Download Source
_, source, err := download("sqlite-src-")
if err != nil {
fmt.Println("Failed to download: sqlite-src; %s", err)
}
// Create Amalgamation Zip Reader
rAmalgamation, err := zip.NewReader(bytes.NewReader(amalgamation), int64(len(amalgamation)))
if err != nil {
log.Fatal(err)
}
// Create Source Zip Reader
rSource, err := zip.NewReader(bytes.NewReader(source), int64(len(source)))
if err != nil {
log.Fatal(err)
}
// Extract Amalgamation
for _, zf := range rAmalgamation.File {
var f *os.File
switch path.Base(zf.Name) {
case "sqlite3.c":
f, err = os.Create("sqlite3-binding.c")
case "sqlite3.h":
f, err = os.Create("sqlite3-binding.h")
case "sqlite3ext.h":
f, err = os.Create("sqlite3ext.h")
default:
continue
}
if err != nil {
log.Fatal(err)
}
zr, err := zf.Open()
if err != nil {
log.Fatal(err)
}
_, err = io.WriteString(f, "#ifndef USE_LIBSQLITE3\n")
if err != nil {
zr.Close()
f.Close()
log.Fatal(err)
}
scanner := bufio.NewScanner(zr)
for scanner.Scan() {
text := scanner.Text()
if text == `#include "sqlite3.h"` {
text = `#include "sqlite3-binding.h"`
}
_, err = fmt.Fprintln(f, text)
if err != nil {
break
}
}
err = scanner.Err()
if err != nil {
zr.Close()
f.Close()
log.Fatal(err)
}
_, err = io.WriteString(f, "#else // USE_LIBSQLITE3\n // If users really want to link against the system sqlite3 we\n// need to make this file a noop.\n #endif")
if err != nil {
zr.Close()
f.Close()
log.Fatal(err)
}
zr.Close()
f.Close()
fmt.Printf("Extracted: %v\n", filepath.Base(f.Name()))
}
//Extract Source
for _, zf := range rSource.File {
var f *os.File
switch path.Base(zf.Name) {
case "userauth.c":
f, err = os.Create("userauth.c")
case "sqlite3userauth.h":
f, err = os.Create("userauth.h")
default:
continue
}
if err != nil {
log.Fatal(err)
}
zr, err := zf.Open()
if err != nil {
log.Fatal(err)
}
_, err = io.Copy(f, zr)
if err != nil {
log.Fatal(err)
}
zr.Close()
f.Close()
fmt.Printf("extracted %v\n", filepath.Base(f.Name()))
}
// Merge SQLite User Authentication into amalgamation
if err := mergeFile("userauth.c", "sqlite3-binding.c"); err != nil {
log.Fatal(err)
}
if err := mergeFile("userauth.h", "sqlite3-binding.h"); err != nil {
log.Fatal(err)
}
os.Exit(0)
}