Compare commits

...

146 Commits

Author SHA1 Message Date
Charlie Vieth 7ce62b2ade
Replace namedValue with driver.NamedValue to avoid copying exec/query args (#1128) 2023-02-11 17:14:42 -05:00
Philip O'Toole 1603038a4d
Add Serialize and Deserialize support (#1089)
Add support for Serialize and Deserialize, which wrap sqlite3_serialize and sqlite3_deserialize.
2022-11-17 08:03:02 -05:00
Philip O'Toole bce3773726 Update expected test output
Broken in https://github.com/mattn/go-sqlite3/pull/1085
2022-10-26 22:03:24 +09:00
Yasuhiro Matsumoto 31c761827c Update amalgamation code 2022-10-26 22:03:24 +09:00
David Vassallo 4b8633cceb
Updating vtable example, "BestIndex" method (#1099)
As it was, the vtable example will give an error when adding any kind of SQL constraint in the SQL statement. Updating the BestIndex method will ensure that adding SQL constraints will not result in errors

Signed-off-by: David Vassallo <davevassallo@gmail.com>

Signed-off-by: David Vassallo <davevassallo@gmail.com>
2022-10-18 19:04:52 -04:00
David Vassallo 0b3708425e
Update README.md to include vtable feature (#1100)
Added documentation for sqlite_vtable build tag to the README.
2022-10-18 18:31:09 -04:00
Luca Guidi 90900be5db Cross Compiling for Mac OS via `musl-cross`
# Enhancement

Update `README.md` with new instructions for Mac OS cross compiling.

# Why?

The current suggested solution `xgo` is no longer maintained (GitHub archived repository).

# Credits

Credits go to Pieter Claerhout (@pieterclaerhout) and his blog post: https://www.yellowduck.be/posts/cross-compile-a-go-package-which-uses-sqlite3
2022-09-18 22:34:48 +09:00
kkqy be28dec3df Golang's linker add mingwex and mingw32 automatically,so we don't need add them manually. 2022-09-18 22:20:58 +09:00
Levi Gruspe 17f6553f94
Add support for sqlite_math_functions tag (#1059)
Add support for SQLITE_ENABLE_MATH_FUNCTIONS compile-time option via the sqlite_math_functions build tag.

Co-authored-by: Dominik Kraus <dominik.kraus@nktek.de>
2022-09-17 10:45:46 -04:00
KiYugadgeter 7476442ed6 こんにちわ is wrong Japanse. The correct word is こんにちは 2022-09-04 18:00:08 +09:00
RewardedIvan da62659c58
Fix "ennviroment" (#1077)
Fix typo in README
2022-09-01 22:47:55 -04:00
Joshua Hull 4ef63c9c0d
Rollback on constraint failure (#1071)
Always rollback on a commit error
2022-09-01 22:45:11 -04:00
Joshua Hull f92b6bb2a1 Fix TestQueryer test to use exec for multistatement insertion 2022-09-02 00:14:06 +09:00
Joshua Hull d5355d86f9 Fix TestQueryer test 2022-09-02 00:14:06 +09:00
Yoshiki Shibukawa c8a114388a Update README to fix reference URL
R.I.P. godoc.org
2022-09-01 11:56:18 +09:00
Oliver Giles f1eef49b3f TestQueryer: actually check Rows returned
Fixes a test which did not correctly exercise the multi-statement
Queryer functionality
2022-08-16 21:24:19 +09:00
Ben Johnson a2e94c9d58 Add build tag to enable OSTRACE() logging
This commit adds the `sqlite_os_trace` build tag which sets the
`SQLITE_FORCE_OS_TRACE` and `SQLITE_DEBUG_OS_TRACE` compilation
flags. This produces verbose debugging output of every operating
system call made by SQLite.
2022-08-16 21:21:55 +09:00
Yasuhiro Matsumoto d8e192b752 Update amalgamation code 2022-08-16 16:16:48 +09:00
Bryan C. Mills 595e13285d Retract +incompatible releases
(For #965.)

This retraction will take effect when this commit is included in the
latest v1 release (presumably v1.14.11).
2022-08-16 14:22:03 +09:00
Aoang fd616a2f47 Update supported Go version to Go 1.19 2022-08-15 15:12:08 +09:00
Phil Eaton 3ccccfb4c9
Support returning any from callbacks (#1046)
Support returning any from callbacks
2022-05-29 21:06:43 -04:00
Kristóf Havasi b819467576 Add error checking in simple example for tx.Commit
Based on https://golang.org/pkg/database/sql/#Tx.Commit this function returns an error type.
So why not check it.
2022-05-28 23:40:45 +09:00
mattn 43dcd3131f Update _example/simple/Dockerfile 2022-05-18 11:53:00 +09:00
KEINOS c122302862 feat: simple example of Dockerfile w/ multi-stage build 2022-05-18 11:53:00 +09:00
Yasuhiro Matsumoto aa1e904220 Update amalgamation code 2022-05-13 11:15:23 +09:00
mattn adb060d295 Update .github/workflows/go.yaml 2022-05-13 10:44:42 +09:00
Aoang a68a2b7fc5 Update supported Golang version to Go 1.18 2022-05-13 10:44:42 +09:00
Corey Butler bedc2985ef Update sqlite3_opt_preupdate_omit.go 2022-04-23 14:23:00 +09:00
Corey Butler c1379fcb35 Update sqlite3_opt_preupdate_hook.go 2022-04-23 14:23:00 +09:00
Phil Eaton 2df077b74c Update amalgamation 2022-02-28 09:00:54 +09:00
Ben Johnson ae2a61f847 Add sqlite3_file_control() support
This commit adds the SQLiteConn.FileControlInt() method which calls the
underlying sqlite3_file_control() function with an int argument. This can
be used for low-level operations on SQLite databases such as persisting
the WAL file after database close.
2022-01-29 01:58:27 +09:00
Yasuhiro Matsumoto 671e666c2e Add example using driverName 2022-01-10 23:30:33 +09:00
Yasuhiro Matsumoto c0fa5ea6d6 Add driverName to be possible change driver name 2022-01-10 23:30:33 +09:00
Yasuhiro Matsumoto 98c52198ca Temporary disable test for dropping vtable 2021-12-29 22:26:06 +09:00
Yasuhiro Matsumoto 7fbc50c941 Update amalgamation code 2021-12-29 22:26:06 +09:00
Yasuhiro Matsumoto 85436841b3 Fix GitHub workflows 2021-10-26 10:18:49 +09:00
mattn 4761e9cad1
Do no test with go-acc on Windows (#980)
Currently, no way to fix failing
2021-10-26 10:18:18 +09:00
mattn 48c6a56ee0
Add go.mod and go.sum for upgrade (#978)
* Add go.mod and go.sum for upgrade

* Fix upgrade tools to have to run on upgrade directory
2021-10-26 09:39:02 +09:00
mattn bb15a32a4f
Drop old versions (#979) 2021-10-26 09:38:41 +09:00
Catena cyber 9537be5eb3
Adds CIFuzz for fuzzing as continuous integration (#919) 2021-10-26 00:32:35 +09:00
Evan Jones a4fc68a6cb
sqlite3_test.go: Fix go test -run=...: Use standard sub-tests (#881)
Selecting only some tests with go test -run=... does not work, because
some of the tests are executed using testing.RunTests(). That function
is documented as "an internal function". This changes TestSuite to use
the testing subtests feature instead.

This has a behaviour change: the benchmarks now need to be
selected at the command line with the standard go test -bench=.
flag. This will also set up the test database twice when running
benchmarks, rather than once.
2021-10-26 00:24:46 +09:00
Hanzhen Yi 2b131e01c1
change angle bracket import to quotes (#868) 2021-10-26 00:23:19 +09:00
Auler 5671e01493
Update SQLite3_ The columntypescantype method of type (#909)
* sqlite3_type update

The main reason for this change is that the original reflected values are nil. I found that there was no good mapping when dealing with the code here

* Update sqlite3_type.go

Update 'ColumnTypeScanType' method,
Different types of mapping values

* Restore copyright

* Update go.mod

* Update go.mod
2021-10-26 00:19:41 +09:00
Patrick DeVivo 2b780b4a7f
fix idxStr freeing issue (#898)
uses snippet suggested by @rittneje https://github.com/mattn/go-sqlite3/issues/897#issuecomment-752162125
2021-10-26 00:13:24 +09:00
Michael Hofmann 98d34f9dc5
Use single-quotes around string literals. (#934) 2021-10-26 00:09:24 +09:00
Dan Peterson 3bb6941859
sqlite3.go: use PRAGMA to set busy_timeout (#910)
The busy_timeout pragma was added in sqlite 3.7.15 as an alternative
to calling sqlite3_busy_timeout directly:

https://sqlite.org/pragma.html#pragma_busy_timeout

While there's no functional change here, using the pragma does align
setting busy_timeout with other settings and removes the special case
for calling sqlite3_busy_timeout directly.
2021-10-26 00:08:40 +09:00
Yasuhiro Matsumoto ab653675c9
Test on 1.17 2021-10-26 00:06:23 +09:00
Harry 1cdbb70da1
run tests against Go 1.16 (#967) 2021-10-26 00:05:59 +09:00
Ikko Ashimine b3df4a5ff0
Fix typo in README.md (#939)
seperated -> separated
2021-10-26 00:05:45 +09:00
hackerman 4bc7a1fc15
Resolve windows CI issues (#941)
Do not use `-u` flag when fetching go-acc
2021-10-26 00:05:01 +09:00
Denis Fondras 1f85ebd7c4
Allow building on OpenBSD (#976) 2021-10-26 00:02:17 +09:00
Ross Smith II 628398eed2
chore: readme: Fix link, typos, copy editing (#974)
* chore: readme: Fix link, typos, copy editing

Also closes #914, #939.

* Update README.md
2021-10-22 16:35:44 +09:00
Aviv Klasquin Komissar 3900dc3187
return non-nil result when calling exec with empty query (#973)
fixes #963
2021-10-19 18:18:21 +09:00
Ichinose Shogo 3392062c72
bump codecov/codecov-action@v2 (#957) 2021-07-20 22:28:26 +09:00
mattn cba6eaee48
Update amalgamation code (#955)
* Update amalgamation code

* Apply realPy's patch
2021-07-15 22:31:14 +09:00
mattn 1157a4212d
Update amalgamation code (#940) 2021-04-15 00:44:23 +09:00
Jesse Rittner ab91e9342b make column metadata functionality opt-in 2021-02-18 13:34:41 -05:00
Philip O'Toole 95e88ca693
Export sqlite3_column_table_name (#900) 2021-02-18 11:58:21 +09:00
Catena cyber 323de98a4c
Go get go-acc with environment variable for go modules (#915)
* Go get go-acc with environment variable for go modules

* Go get with modules for windows as well
2021-02-16 22:01:58 +09:00
Catena cyber 16175c1389
Adds a fuzz target (#908)
* Adds a fuzz target

* Fixes memory leak
2021-02-15 22:57:26 +09:00
Martin Tournoij 3cbdae750e
Export sqlite3_stmt_readonly() via SQLiteStmt.Readonly() (#895)
This can be used like in the test; I wrote a little wrapper around
sql.DB which uses this, and allows concurrent reads but just one single
write. This is perhaps a better generic "table locked"-solution than
setting the connections to 1 and/or cache=shared (although even better
would be to design your app in such a way that this doesn't happpen in
the first place, but even then a little seat belt isn't a bad thing).

The parsing adds about 0.1ms to 0.2ms of overhead in the wrapper, which
isn't too bad (and it caches the results, so only needs to do this
once).

At any rate, I can't really access functions from sqlite3-binding.c from
my application, so expose it via SQLiteStmt.
2020-12-28 08:52:08 +09:00
mattn 52436d4074
Update amalgamation code (#896) 2020-12-26 23:18:41 +09:00
Patrick DeVivo 92d23714a8
add support for defining an "eponymous only" virtual table (#885)
* add support for defining an "eponymous only" virtual table

As suggested here: https://github.com/mattn/go-sqlite3/issues/846#issuecomment-736206222

* add an example of an eponymous only vtab module

* add a test case for an eponymous only vtab module
2020-12-26 23:11:17 +09:00
Jinzhu 66ff625f34
RowsColumnTypeNullable not implemented (#848) 2020-12-26 23:07:28 +09:00
Martin Tournoij 02ce7ec581
Add ?_cache_size=[..] to connection parameters (#894)
Add a shortcut for PRAGMA cache_size; this is a pretty useful setting:
the default of -2000 (2M) is not especially high, and a lot of people
will probably want to increase this.

For example, while running a bunch of fairy expensive queries in
parallel:

	With SetMaxOpenConns(1):
	 -2000:  5762ms
	-20000:  4714ms

	With SetMaxOpenConns(20):
	 -2000:  3067ms
	-20000:  2532ms

Which isn't a bad performance boost for changing a single number.
2020-12-26 23:05:20 +09:00
Brad Rydzewski e30206cd31
clarify usleep license (#893) 2020-12-26 23:04:03 +09:00
Yasuhiro Matsumoto 223d277c80
Fix ci 2020-11-18 00:34:04 +09:00
mattn ae4208e73a
Drop go1.11 (#852)
* Drop go1.11

* Use go1.12
2020-11-17 10:04:27 +09:00
Buk Bukowski 0709612629
README.md: use link in markdown (#859) 2020-11-17 10:04:13 +09:00
Evan Jones 943e8f860d
sqlite3.go: Remove -DSQLITE_ENABLE_FTS4_UNICODE61: not supported (#872)
This option was enabled by default in sqlite3 on 2014-07-03.
This setting does nothing. It can now be disabled with
SQLITE_DISABLE_FTS3_UNICODE. See the upstream commit:
https://sqlite.org/src/info/0cc0230ae9cfc976

I think this change was imported into this project with commit
ee9da4840d on 2015-06-12.
2020-11-17 02:00:32 +09:00
Evan Jones 8e02107ef7
sqlite3.go: remove -DSQLITE_DISABLE_INTRINSIC: better builds (#878)
This "disables the use of compiler-specific built-in functions such
as __builtin_bswap32()" (from the SQLite docs) so this change might
produce slightly better code. My primary motivation, however, is that
the "default" configuration for SQLite, which is widely tested, does
not set this preprocessor macro.

From looking at Github issues, it appears this was added to avoid a
build error on Mac OS X 10.11, in 2017:
https://github.com/mattn/go-sqlite3/issues/386

There have been a number of changes to sqlite3 since we tried this
last. I think it would be worth trying to remove this setting again.
I found a machine running Mac OS X 10.11.6. It was able to build and
run the tests in this package with this change.

Mac OS X 10.11 is has not been supported by Apple since 2018
(currently Apple is releasing updates for Mac OS 10.13 and newer; 11
is the current release). However, Go 1.14 is supported, and it
requires Mac OS X 10.11 or newer: https://golang.org/doc/go1.14
Go 1.15 only supports Mac OS 10.12 and newer:
https://golang.org/doc/go1.15
2020-11-17 01:59:22 +09:00
Evan Jones 6da13a7bd9
.github/workflows: stop using deprecated add-path command (#873)
Github is disabling the add-path command in workflows on 2020-11-16.
Switch to their new preferred way of doing this, by appending to the
file referred to with $GITHUB_PATH. See:

https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-commands-for-github-actions#adding-a-system-path

This should fix the following warning from the workflows:

.github#L1
The `add-path` command is deprecated and will be disabled on November
16th. Please upgrade to using Environment Files. For more information
see:
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
2020-11-17 01:56:45 +09:00
mattn 1fbedab173
Support vfs for Open (#877)
Closes #876
2020-11-17 01:54:21 +09:00
Evan Jones 3fb3c0de37
TestExecContextCancel: Reduce timeout to make less flaky (#879)
This test fails fairly often. On my system, I can trigger it with:

    go test . -run=TestExecContextCancel -count=10 -failfast -v

This makes the test less flaky by timing out the context after a
consistent 50 millisecond delay. This was enough time for the query
to start, then get cancelled with sqlite3_interrupt() in my tests.
This now passes the above check.

This is a modified version of the change suggested in:
https://github.com/mattn/go-sqlite3/pull/865
2020-11-17 01:52:56 +09:00
Evan Jones 70c77097f2
sqlite3_test.go: Move Go 1.13 test to sqlite3_go113_test.go (#883)
Commit 4f7abea96e added a test that uses Conn.Raw, which was added in
Go >= 1.13. The go-sqlite3 project runs tests with Go >= 1.11. Remove
the test from sqlite3_test.go, so it only runs with the correct
versions of Go.

Instead of adding a new test, modify the existing test that already
uses Conn.Raw() to check the type of driverConn.
2020-11-17 01:52:26 +09:00
Evan Jones 4f7abea96e
doc.go: you can use Conn.Raw to get *SQLiteConn (#882)
This can be easier that registering a new driver, in some cases.
Add a test to verify that this works.
2020-11-16 23:42:00 +09:00
Macaully James Muir 784c625194
Expand documentation for extension functions (#880)
This relates to #870; it's not immediately clear that you need to pass a
different driver name to sql.Open from the documentation.
2020-11-16 23:40:44 +09:00
Yunus Ayar 92f580b350
Fix #860 extenstion entry bug (#861)
Fix overshadowing of entrypoint variable.
2020-10-02 17:13:14 +09:00
mattn 4120733fec
Add build tag ignore for upgrade script (#851) 2020-09-11 16:06:34 +09:00
Andrii Zavorotnii 862b95943f
Fix "cannot start a transaction within a transaction" issue (#764) (#765)
* Fix "cannot start a transaction within a transaction" issue

[why]
If db.BeginTx(ctx, nil) context is cancelled too fast, "BEGIN" statement can be
completed inside DB, but we still try to cancel it with sqlite3_interrupt.
In such case we get context.Cancelled or context.DeadlineExceeded from exec(),
but operation really completed. Connection returned into pool, and returns "cannot
start a transaction within a transaction" error for next db.BeginTx() call.

[how]
Handle status code returned from cancelled operation.

[testing]
Added unit-test which reproduces issue.

* Reduce TestQueryRowContextCancelParallel concurrency

[why]
Tests times out in travis-ci when run with -race option.
2020-08-29 00:43:21 +09:00
mattn 6a8b30186d
Use go-pointer instead of uintptr hacks. (#814)
* Use go-pointer instead of uintptr hacks.

Fixes #791

* Do same of go-pointer

* Drop older verion of Go

* Fix build

* Fix build
2020-08-26 09:36:43 +09:00
rittneje 59d4145b7f
treat vtable C arrays as Go slices rather than giant Go arrays (#840) 2020-08-26 03:22:54 +09:00
raa0121 4b46e2b60d
coveralls to codecov (#845)
* coveralls to codecov

* README badge coveralls to codecov
2020-08-25 18:13:57 +09:00
Yasuhiro Matsumoto c14096f123 Fix go.yaml 2020-08-25 14:24:45 +09:00
Yasuhiro Matsumoto fd606f826a Update README.md 2020-08-25 13:59:58 +09:00
raa0121 09fcd35eaa
Add Github Actions yaml and set LDFLAGS on windows (#843)
and remove .travis.yml
2020-08-25 13:56:34 +09:00
mattn 2329c6ea75
Update amalgamation code (#842)
* Update amalgamation code

* Drop go1.9 and go1.10
2020-08-19 20:15:57 +09:00
Olivier Mengué 4c2df3cc46
Travis-CI: set go_import_path (#829)
Set go_import_path to tell Travis-CI where to checkout the code, so the
Travis build can also work on forks. This is important when building
without Go modules support.
2020-07-21 17:00:10 +09:00
Olivier Mengué 979ea59e5a
Travis-CI: add Go 1.14.x, change order of Go versions (#830)
Changes in Go versions for the Travis-CI builds:
- reverse order of Go versions so the latest ones are checked first
- add Go 1.14.x (latest stable) at the first position
- use 'tip' instead of 'master' (like https://tip.golang.org) and move
  it just after 1.14.x
2020-07-21 16:59:02 +09:00
Yasuhiro Matsumoto aa77c03e2f
Update README.md 2020-06-18 22:22:56 +09:00
mattn baaf8a9784
upgrade amalgamation code (#821) 2020-06-06 12:49:38 +09:00
Martin Tournoij 0cec2d7524
Add mock versions of SQLiteDriver and SQLiteConn for +build !cgo (#819)
My app can use PostgreSQL and – optionally – SQLite. I would like to be
able to compile the app without cgo when SQLite isn't used, as this
removes the need for a C compiler which makes builds easier and faster,
especially for end-users.

In the simple case, this is not a problem go-sqlite3 already provides a
simple non-cgo mock so it compiles and gives a runtime error if you try
to use it anyway.

However, now I'd like to register a function for my SQLite connection to
match a PostgreSQL function like so:

	sql.Register("sqlite3_custom", &sqlite3.SQLiteDriver{
		ConnectHook: func(conn *sqlite3.SQLiteConn) error {
			return conn.RegisterFunc("pow", pow, true); err != nil {
		},
	})

But this makes it quite hard to keep the same logic since it refers to
types that don't exist with CGO_ENABLED=0. I will need to create a db.go
with `+build !cgo` and db_cgo.go with `+buid cgo` which duplicates all
the logic but with the sqlite hooks. In my case, this actually affects
quite a lot; for example I have a helper function which connects and
runs migrations and whatnot which looks like:

	type ConnectOptions struct {
		Connect    string // Connect string.
		Schema     []byte // Database schema to create on startup.
		Migrate    *Migrate
		SQLiteHook func(*sqlite3.SQLiteConn) error
	}

And I'd have to have two versions of that, too. You could perhaps do
something with interfaces, but because the sql.Register() call above
references the `sqlite3.SQLiteDriver.ConnectHook` struct field that's
not so straightforward (and wrapping stuff in interfaces probably won't
do much for the general clarity either).

This simplifies all of that by providing some common types that may be
used when setting up a SQLite connectin. I renamed the
`SQLiteDriverMock` to `&SQLiteDriver` for this reason. As far as I can
tell in my testing, this has no real downsides (but perhaps I missed
something?)

---

Note: it might also be worth doing something similar for error.go, as I
already have two variants of the below function (one with cgo as below,
and one without cgo which checks just PostgreSQL):

	// ErrUnique reports if this error reports a UNIQUE constraint violation.
	//
	// This is the cgo version which works for PostgreSQL and SQLite.
	func ErrUnique(err error) bool {
		var sqlErr *sqlite3.Error
		if errors.As(err, &sqlErr) && sqlErr.ExtendedCode == sqlite3.ErrConstraintUnique {
			return true
		}
		var pqErr *pq.Error
		if errors.As(err, &pqErr) && pqErr.Code == "23505" {
			return true
		}
		return false
	}

This is a lot more manageable than the ConnectHook case, but it would be
nicer if it would work without the need for build tags.
2020-06-05 22:06:55 +09:00
turtlemaster19 f600c4bddb
Edited note in README (#817) 2020-06-01 22:39:02 +09:00
Vishnu Mohandas e77f7dd61e
Document requirements for cross compiling from OSX (#804)
* Document requirements for cross compiling from OSX 

Inspiration: https://github.com/mattn/go-sqlite3/issues/384#issuecomment-433584967

* Document cross compilation steps using xgo for MACOSX
2020-05-16 00:48:12 +09:00
gber db4c9426f8
Enable all prefixes for named parameters and allow for unused named parameters (#811)
* Allow unused named parameters

Try to bind all named parameters and ignore those not used.

* Allow "@" and "$" for named parameters

* Add tests for named parameters

Co-authored-by: Guido Berhoerster <guido+go-sqlite3@berhoerster.name>
2020-05-14 23:28:04 +09:00
Diego Becciolini 44b2a6394a
Fix #808: remove goconvey (#809)
* don't need smartystreets/goconvey any longer

* synchronise coveralls
2020-05-14 23:24:58 +09:00
ShanePerron 61ad8da9d6
Fix for early termination of returned rows (#805)
Once the regex encountered the first instance of a non-match, it would return without processing the rest of the rows in the statement. This change allows it to process the remaining, only setting the sqlite3_result_int to zero then continuing. This worked fine for the example as it only had one item to process.
2020-05-03 00:42:42 +09:00
rittneje 98a44bcf59
report actual error message if sqlite3_load_extension fails (#800)
* report actual error message if sqlite3_load_extension fails

* more fixes and test cases

Co-authored-by: Jesse Rittner <jrittner@lutron.com>
2020-04-16 14:45:59 +09:00
Paul Bergeron 58b2310c97
Add extension-functions.c info to README (#779) 2020-04-16 14:44:20 +09:00
Yasuhiro Matsumoto 77fdcc27bb
Update dependencies 2020-04-16 14:43:15 +09:00
Karthik K a23162b94b
feat: add go modules (#781)
* feat: add go modules

* feat: upgrade goquery dependency
2020-04-16 14:42:04 +09:00
Manfred Touron 67986a7832
chore: remove debug code (#788) 2020-02-27 17:28:46 +09:00
mattn 9bdaffc12b
upgrade amalgamation code (#777) 2020-01-28 19:25:19 +09:00
Yasuhiro Matsumoto ed406dd4e2
Update README.md 2020-01-23 15:57:42 +09:00
Sergey Bronnikov d51eaf3b34 Fix typo (#763) 2020-01-09 18:43:04 +09:00
rittneje 53cff3fceb fix typo in doc comment (#770) 2019-12-17 16:07:49 +09:00
rittneje b4f5cc77d1 add SystemErrno to Error (#740)
* adding SystemErrno to Error, and fixing error logic when open fails

* fix for old versions of libsqlite3 that do not have sqlite3_system_errno defined

* fixing pre-processor logic
2019-12-17 15:58:28 +09:00
mattn 590d44c02b
Merge pull request #744 from azavorotnii/ctx_cancel
Fix context cancellation racy handling
2019-11-19 01:19:53 +09:00
mattn 0cf797e1cd
Merge pull request #760 from mattn/sqlite-amalgamation-3300100
upgrade amalgamation code
2019-11-19 01:18:17 +09:00
Yasuhiro Matsumoto a1e7f13af0
upgrade amalgamation code 2019-11-18 22:50:50 +09:00
mattn 4d8693d6ee
Merge pull request #758 from mattn/noncgo
Add build constraints for non cgo
2019-11-18 19:50:39 +09:00
Yasuhiro Matsumoto fc06e55305
Add build constraints for non cgo 2019-11-18 18:03:31 +09:00
mattn 67c1376b46
Merge pull request #754 from codesoap/master
Improve readability of Backup()
2019-11-05 14:44:21 +09:00
codesoap 17bc44792c Improve readability of Backup() 2019-10-31 15:26:56 +01:00
mattn c64f703b0d
Merge pull request #753 from opencollective/opencollective
Activating Open Collective
2019-10-31 09:30:00 +09:00
Jess c74e550595 Added financial contributors to the README 2019-10-30 15:39:31 -07:00
mattn 3f45aefa8d
Update FUNDING.yml 2019-10-08 17:38:25 +09:00
mattn a26641dc74
Merge pull request #750 from tpounds/add-new-go-vers
Add Go 1.12.x/1.13.x to CI
2019-10-08 16:23:26 +09:00
Trevor Pounds 34018519d0 Add Go 1.12.x/1.13.x to CI. 2019-10-07 10:46:06 -04:00
mattn 4396a38886
Merge pull request #747 from azavorotnii/open_journal_mode
Fix Open() journal mode regression
2019-09-24 10:39:45 +09:00
Andrii Zavorotnii 27d3ed467c Fix typo in "_locking_mode" DSN handling 2019-09-23 14:57:24 -07:00
Andrii Zavorotnii c4a8658099 Fix Open() journal mode regression
[why]
see https://github.com/mattn/go-sqlite3/issues/607

SQLite default journal mode is DELETE, but forcing it on open causes "database is locked"
if other connection exists with WAL mode, for example.

[how]
Don't set DELETE mode if not set in DSN explicitly.

[testing]
Run tests in my project where WAL mode is used.
2019-09-23 14:56:04 -07:00
Andrii Zavorotnii 7e1a61dbcd Fix context cancellation racy handling
[why]
Context cancellation goroutine is not in sync with Next() method lifetime.
It leads to sql.ErrNoRows instead of context.Canceled often (easy to reproduce).
It leads to interruption of next query executed on same connection (harder to reproduce).

[how]
Do query in goroutine, wait when interruption done.

[testing]
Add unit test that reproduces error cases.
2019-09-06 12:15:53 -07:00
Gert-Jan Timmer d3c690956b
Fix memory leak in ExpandedSQL (#738)
Fixes: #733
2019-08-22 14:55:11 +02:00
G.J.R. Timmer e3726ad6eb Fixed operator 2019-08-22 14:53:27 +02:00
G.J.R. Timmer b22da71572 Fix _auth_* parameter check
Fixes: #724
2019-08-22 14:53:27 +02:00
G.J.R. Timmer 2ea5857c0e Closes #597 2019-08-22 10:59:46 +02:00
mattn e24345b069
Merge pull request #734 from gwd/out/fix-libsqlite3-darwin-build
darwin/libsqlite3: Also use the homebrew include path
2019-08-20 00:43:48 +09:00
mattn f4ce3be229
Merge pull request #735 from mahler/master
Issue #651: Fix of typo
2019-08-20 00:43:38 +09:00
= 85bf186e05 Issue #651: Fix of typo
https://github.com/mattn/go-sqlite3/issues/651
2019-08-19 15:53:09 +02:00
George Dunlap f9aab7dda8 darwin/libsqlite3: Also use the homebrew include path
When building on darwin with the `libsqlite3` tag, go-sqlite3 adds the
homebrew library path.  It does not, however, add the homebrew include
path, which means that the MacOS sqlite3 header is used instead.  On
my system, this results in build errors that look like this:

./sqlite3_load_extension.go:25:8: could not determine kind of name for C.sqlite3_enable_load_extension
./sqlite3_load_extension.go:33:8: could not determine kind of name for C.sqlite3_load_extension

Add the homebrew include path as well, so that he header matches the
libraries we're using.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
2019-08-19 11:27:31 +01:00
mattn d6f416f91c
Merge pull request #732 from mattn/sqlite-amalgamation-3290000
upgrade amalgamation code
2019-07-16 16:15:03 +09:00
Yasuhiro Matsumoto b612a2feea
upgrade amalgamation code 2019-07-16 14:56:09 +09:00
mattn fba062bdb6
Merge pull request #731 from Jason-Cooke/patch-1
docs: fix typo
2019-07-09 13:10:54 +09:00
Jason Cooke 9889b11544
docs: fix typo 2019-07-09 09:09:23 +12:00
mattn 5dd71670cc
Merge pull request #727 from MichaelS11/master
Updated Ping and doTestOpen. Added TestOpenContext and TestFileCopyTruncate
2019-06-20 10:01:18 +09:00
MichaelS11 57484d0899 Updated Ping to return ErrBadConn
Added TestOpenContext
Added TestFileCopyTruncate
Added ping to doTestOpen
2019-06-19 16:50:49 -07:00
mattn c25061c4b7
Merge pull request #725 from auxten/patch-1
Fix typo in readme
2019-06-17 16:04:10 +09:00
auxten 2e7be7d089
Fix typo in readme 2019-06-17 14:29:55 +08:00
mattn 070b17a2fd
Merge pull request #680 from rittneje/improve-faq-in-memory-shared-cache
improve FAQ re: in-memory databases
2019-05-29 17:43:13 +09:00
mattn afa0250ddf
Create FUNDING.yml 2019-05-24 00:19:27 +09:00
mattn 2a192bf782
Merge pull request #716 from mattn/fix-variadic
Fix type of variadic
2019-05-11 01:41:06 +09:00
Yasuhiro Matsumoto 81b9db8126
Fix type of variadic 2019-05-10 23:23:32 +09:00
mattn 5994cc52df
Merge pull request #712 from mattn/sqlite-amalgamation-3280000
upgrade amalgamation code
2019-04-24 18:37:27 +09:00
Jesse Rittner 062811fcbd more code formatting 2019-01-04 18:57:33 -05:00
Jesse Rittner 2ee07a473f adding additional info to FAQ about in-memory databases 2019-01-04 18:54:34 -05:00
85 changed files with 49644 additions and 26829 deletions

4
.codecov.yml Normal file
View File

@ -0,0 +1,4 @@
coverage:
status:
project: off
patch: off

8
.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1,8 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: mattn # Replace with a single Patreon username
open_collective: mattn # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
custom: # Replace with a single custom sponsorship URL

29
.github/workflows/cifuzz.yaml vendored Normal file
View File

@ -0,0 +1,29 @@
name: CIFuzz
on: [pull_request]
jobs:
Fuzzing:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address]
steps:
- name: Build Fuzzers (${{ matrix.sanitizer }})
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
with:
oss-fuzz-project-name: 'go-sqlite3'
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Run Fuzzers (${{ matrix.sanitizer }})
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'go-sqlite3'
fuzz-seconds: 600
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Upload Crash
uses: actions/upload-artifact@v1
if: failure()
with:
name: ${{ matrix.sanitizer }}-artifacts
path: ./out/artifacts

22
.github/workflows/docker.yaml vendored Normal file
View File

@ -0,0 +1,22 @@
name: dockerfile
on:
workflow_dispatch:
push:
tags:
- 'v*'
pull_request:
branches: [ master ]
jobs:
dockerfile:
name: Run Dockerfiles in examples
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Run example - simple
run: |
cd ./_example/simple
docker build -t simple .
docker run simple | grep 99\ こんにちは世界099

115
.github/workflows/go.yaml vendored Normal file
View File

@ -0,0 +1,115 @@
name: Go
on: [push, pull_request]
jobs:
test:
name: Test
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
go: ['1.17', '1.18', '1.19']
fail-fast: false
env:
OS: ${{ matrix.os }}
GO: ${{ matrix.go }}
steps:
- if: startsWith(matrix.os, 'macos')
run: brew update
- uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go }}
- name: Get Build Tools
run: |
GO111MODULE=on go install github.com/ory/go-acc@latest
- name: Add $GOPATH/bin to $PATH
run: |
echo "$(go env GOPATH)/bin" >> "$GITHUB_PATH"
- uses: actions/checkout@v2
- name: 'Tags: default'
run: go-acc . -- -race -v -tags ""
- name: 'Tags: libsqlite3'
run: go-acc . -- -race -v -tags "libsqlite3"
- name: 'Tags: full'
run: go-acc . -- -race -v -tags "sqlite_allow_uri_authority sqlite_app_armor sqlite_column_metadata sqlite_foreign_keys sqlite_fts5 sqlite_icu sqlite_introspect sqlite_json sqlite_math_functions sqlite_os_trace sqlite_preupdate_hook sqlite_secure_delete sqlite_see sqlite_stat4 sqlite_trace sqlite_unlock_notify sqlite_userauth sqlite_vacuum_incr sqlite_vtable"
- name: 'Tags: vacuum'
run: go-acc . -- -race -v -tags "sqlite_vacuum_full"
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
env_vars: OS,GO
file: coverage.txt
test-windows:
name: Test for Windows
runs-on: windows-latest
defaults:
run:
shell: bash
strategy:
matrix:
go: ['1.17', '1.18', '1.19']
fail-fast: false
env:
OS: windows-latest
GO: ${{ matrix.go }}
steps:
- uses: msys2/setup-msys2@v2
with:
update: true
install: mingw-w64-x86_64-toolchain mingw-w64-x86_64-sqlite3
msystem: MINGW64
path-type: inherit
- uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go }}
- name: Add $GOPATH/bin to $PATH
run: |
echo "$(go env GOPATH)/bin" >> "$GITHUB_PATH"
shell: msys2 {0}
- uses: actions/checkout@v2
- name: 'Tags: default'
run: go build -race -v -tags ""
shell: msys2 {0}
- name: 'Tags: libsqlite3'
run: go build -race -v -tags "libsqlite3"
shell: msys2 {0}
- name: 'Tags: full'
run: |
echo 'skip this test'
echo go build -race -v -tags "sqlite_allow_uri_authority sqlite_app_armor sqlite_column_metadata sqlite_foreign_keys sqlite_fts5 sqlite_icu sqlite_introspect sqlite_json sqlite_math_functions sqlite_preupdate_hook sqlite_secure_delete sqlite_see sqlite_stat4 sqlite_trace sqlite_unlock_notify sqlite_userauth sqlite_vacuum_incr sqlite_vtable"
shell: msys2 {0}
- name: 'Tags: vacuum'
run: go build -race -v -tags "sqlite_vacuum_full"
shell: msys2 {0}
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
env_vars: OS,GO
file: coverage.txt
# based on: github.com/koron-go/_skeleton/.github/workflows/go.yml

View File

@ -1,31 +0,0 @@
language: go
os:
- linux
- osx
addons:
apt:
update: true
go:
- 1.9.x
- 1.10.x
- 1.11.x
- master
before_install:
- |
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
brew update
fi
- go get github.com/smartystreets/goconvey
- go get github.com/mattn/goveralls
- go get golang.org/x/tools/cmd/cover
script:
- $HOME/gopath/bin/goveralls -repotoken 3qJVUE0iQwqnCbmNcDsjYu1nh4J4KIFXx
- go test -race -v . -tags ""
- go test -race -v . -tags "libsqlite3"
- go test -race -v . -tags "sqlite_allow_uri_authority sqlite_app_armor sqlite_foreign_keys sqlite_fts5 sqlite_icu sqlite_introspect sqlite_json sqlite_secure_delete sqlite_see sqlite_stat4 sqlite_trace sqlite_userauth sqlite_vacuum_incr sqlite_vtable sqlite_unlock_notify"
- go test -race -v . -tags "sqlite_vacuum_full"

209
README.md
View File

@ -1,30 +1,41 @@
go-sqlite3 go-sqlite3
========== ==========
[![GoDoc Reference](https://godoc.org/github.com/mattn/go-sqlite3?status.svg)](http://godoc.org/github.com/mattn/go-sqlite3) [![Go Reference](https://pkg.go.dev/badge/github.com/mattn/go-sqlite3.svg)](https://pkg.go.dev/github.com/mattn/go-sqlite3)
[![Build Status](https://travis-ci.org/mattn/go-sqlite3.svg?branch=master)](https://travis-ci.org/mattn/go-sqlite3) [![GitHub Actions](https://github.com/mattn/go-sqlite3/workflows/Go/badge.svg)](https://github.com/mattn/go-sqlite3/actions?query=workflow%3AGo)
[![Coverage Status](https://coveralls.io/repos/mattn/go-sqlite3/badge.svg?branch=master)](https://coveralls.io/r/mattn/go-sqlite3?branch=master) [![Financial Contributors on Open Collective](https://opencollective.com/mattn-go-sqlite3/all/badge.svg?label=financial+contributors)](https://opencollective.com/mattn-go-sqlite3)
[![codecov](https://codecov.io/gh/mattn/go-sqlite3/branch/master/graph/badge.svg)](https://codecov.io/gh/mattn/go-sqlite3)
[![Go Report Card](https://goreportcard.com/badge/github.com/mattn/go-sqlite3)](https://goreportcard.com/report/github.com/mattn/go-sqlite3) [![Go Report Card](https://goreportcard.com/badge/github.com/mattn/go-sqlite3)](https://goreportcard.com/report/github.com/mattn/go-sqlite3)
Latest stable version is v1.14 or later, not v2.
~~**NOTE:** The increase to v2 was an accident. There were no major changes or features.~~
# Description # Description
sqlite3 driver conforming to the built-in database/sql interface A sqlite3 driver that conforms to the built-in database/sql interface.
Supported Golang version: See .travis.yml Supported Golang version: See [.github/workflows/go.yaml](./.github/workflows/go.yaml).
[This package follows the official Golang Release Policy.](https://golang.org/doc/devel/release.html#policy) This package follows the official [Golang Release Policy](https://golang.org/doc/devel/release.html#policy).
### Overview ### Overview
- [go-sqlite3](#go-sqlite3)
- [Description](#description)
- [Overview](#overview)
- [Installation](#installation) - [Installation](#installation)
- [API Reference](#api-reference) - [API Reference](#api-reference)
- [Connection String](#connection-string) - [Connection String](#connection-string)
- [DSN Examples](#dsn-examples)
- [Features](#features) - [Features](#features)
- [Usage](#usage)
- [Feature / Extension List](#feature--extension-list)
- [Compilation](#compilation) - [Compilation](#compilation)
- [Android](#android) - [Android](#android)
- [ARM](#arm) - [ARM](#arm)
- [Cross Compile](#cross-compile) - [Cross Compile](#cross-compile)
- [Google Cloud Platform](#google-cloud-platform) - [Google Cloud Platform](#google-cloud-platform)
- [Linux](#linux) - [Linux](#linux)
- [Alpine](#alpine) - [Alpine](#alpine)
- [Fedora](#fedora) - [Fedora](#fedora)
@ -34,15 +45,26 @@ Supported Golang version: See .travis.yml
- [Errors](#errors) - [Errors](#errors)
- [User Authentication](#user-authentication) - [User Authentication](#user-authentication)
- [Compile](#compile) - [Compile](#compile)
- [Usage](#usage) - [Usage](#usage-1)
- [Create protected database](#create-protected-database)
- [Password Encoding](#password-encoding)
- [Available Encoders](#available-encoders)
- [Restrictions](#restrictions)
- [Support](#support)
- [User Management](#user-management)
- [SQL](#sql)
- [Examples](#examples)
- [*SQLiteConn](#sqliteconn)
- [Attached database](#attached-database)
- [Extensions](#extensions) - [Extensions](#extensions)
- [Spatialite](#spatialite) - [Spatialite](#spatialite)
- [FAQ](#faq) - [FAQ](#faq)
- [License](#license) - [License](#license)
- [Author](#author)
# Installation # Installation
This package can be installed with the go get command: This package can be installed with the `go get` command:
go get github.com/mattn/go-sqlite3 go get github.com/mattn/go-sqlite3
@ -50,28 +72,28 @@ _go-sqlite3_ is *cgo* package.
If you want to build your app using go-sqlite3, you need gcc. If you want to build your app using go-sqlite3, you need gcc.
However, after you have built and installed _go-sqlite3_ with `go install github.com/mattn/go-sqlite3` (which requires gcc), you can build your app without relying on gcc in future. However, after you have built and installed _go-sqlite3_ with `go install github.com/mattn/go-sqlite3` (which requires gcc), you can build your app without relying on gcc in future.
***Important: because this is a `CGO` enabled package you are required to set the environment variable `CGO_ENABLED=1` and have a `gcc` compile present within your path.*** ***Important: because this is a `CGO` enabled package, you are required to set the environment variable `CGO_ENABLED=1` and have a `gcc` compile present within your path.***
# API Reference # API Reference
API documentation can be found here: http://godoc.org/github.com/mattn/go-sqlite3 API documentation can be found [here](http://godoc.org/github.com/mattn/go-sqlite3).
Examples can be found under the [examples](./_example) directory Examples can be found under the [examples](./_example) directory.
# Connection String # Connection String
When creating a new SQLite database or connection to an existing one, with the file name additional options can be given. When creating a new SQLite database or connection to an existing one, with the file name additional options can be given.
This is also known as a DSN string. (Data Source Name). This is also known as a DSN (Data Source Name) string.
Options are append after the filename of the SQLite database. Options are append after the filename of the SQLite database.
The database filename and options are seperated by an `?` (Question Mark). The database filename and options are separated by an `?` (Question Mark).
Options should be URL-encoded (see [url.QueryEscape](https://golang.org/pkg/net/url/#QueryEscape)). Options should be URL-encoded (see [url.QueryEscape](https://golang.org/pkg/net/url/#QueryEscape)).
This also applies when using an in-memory database instead of a file. This also applies when using an in-memory database instead of a file.
Options can be given using the following format: `KEYWORD=VALUE` and multiple options can be combined with the `&` ampersand. Options can be given using the following format: `KEYWORD=VALUE` and multiple options can be combined with the `&` ampersand.
This library supports dsn options of SQLite itself and provides additional options. This library supports DSN options of SQLite itself and provides additional options.
Boolean values can be one of: Boolean values can be one of:
* `0` `no` `false` `off` * `0` `no` `false` `off`
@ -103,6 +125,8 @@ Boolean values can be one of:
| Time Zone Location | `_loc` | auto | Specify location of time format. | | Time Zone Location | `_loc` | auto | Specify location of time format. |
| Transaction Lock | `_txlock` | <ul><li>immediate</li><li>deferred</li><li>exclusive</li></ul> | Specify locking behavior for transactions. | | Transaction Lock | `_txlock` | <ul><li>immediate</li><li>deferred</li><li>exclusive</li></ul> | Specify locking behavior for transactions. |
| Writable Schema | `_writable_schema` | `Boolean` | When this pragma is on, the SQLITE_MASTER tables in which database can be changed using ordinary UPDATE, INSERT, and DELETE statements. Warning: misuse of this pragma can easily result in a corrupt database file. | | Writable Schema | `_writable_schema` | `Boolean` | When this pragma is on, the SQLITE_MASTER tables in which database can be changed using ordinary UPDATE, INSERT, and DELETE statements. Warning: misuse of this pragma can easily result in a corrupt database file. |
| Cache Size | `_cache_size` | `int` | Maximum cache size; default is 2000K (2M). See [PRAGMA cache_size](https://sqlite.org/pragma.html#pragma_cache_size) |
## DSN Examples ## DSN Examples
@ -114,19 +138,18 @@ file:test.db?cache=shared&mode=memory
This package allows additional configuration of features available within SQLite3 to be enabled or disabled by golang build constraints also known as build `tags`. This package allows additional configuration of features available within SQLite3 to be enabled or disabled by golang build constraints also known as build `tags`.
[Click here for more information about build tags / constraints.](https://golang.org/pkg/go/build/#hdr-Build_Constraints) Click [here](https://golang.org/pkg/go/build/#hdr-Build_Constraints) for more information about build tags / constraints.
### Usage ### Usage
If you wish to build this library with additional extensions / features. If you wish to build this library with additional extensions / features, use the following command:
Use the following command.
```bash ```bash
go build --tags "<FEATURE>" go build --tags "<FEATURE>"
``` ```
For available features see the extension list. For available features, see the extension list.
When using multiple build tags, all the different tags should be space delimted. When using multiple build tags, all the different tags should be space delimited.
Example: Example:
@ -142,6 +165,7 @@ go build --tags "icu json1 fts5 secure_delete"
| Allow URI Authority | sqlite_allow_uri_authority | URI filenames normally throws an error if the authority section is not either empty or "localhost".<br><br>However, if SQLite is compiled with the SQLITE_ALLOW_URI_AUTHORITY compile-time option, then the URI is converted into a Uniform Naming Convention (UNC) filename and passed down to the underlying operating system that way | | Allow URI Authority | sqlite_allow_uri_authority | URI filenames normally throws an error if the authority section is not either empty or "localhost".<br><br>However, if SQLite is compiled with the SQLITE_ALLOW_URI_AUTHORITY compile-time option, then the URI is converted into a Uniform Naming Convention (UNC) filename and passed down to the underlying operating system that way |
| App Armor | sqlite_app_armor | When defined, this C-preprocessor macro activates extra code that attempts to detect misuse of the SQLite API, such as passing in NULL pointers to required parameters or using objects after they have been destroyed. <br><br>App Armor is not available under `Windows`. | | App Armor | sqlite_app_armor | When defined, this C-preprocessor macro activates extra code that attempts to detect misuse of the SQLite API, such as passing in NULL pointers to required parameters or using objects after they have been destroyed. <br><br>App Armor is not available under `Windows`. |
| Disable Load Extensions | sqlite_omit_load_extension | Loading of external extensions is enabled by default.<br><br>To disable extension loading add the build tag `sqlite_omit_load_extension`. | | Disable Load Extensions | sqlite_omit_load_extension | Loading of external extensions is enabled by default.<br><br>To disable extension loading add the build tag `sqlite_omit_load_extension`. |
| Enable Serialization with `libsqlite3` | sqlite_serialize | Serialization and deserialization of a SQLite database is available by default, unless the build tag `libsqlite3` is set.<br><br>To enable this functionality even if `libsqlite3` is set, add the build tag `sqlite_serialize`. |
| Foreign Keys | sqlite_foreign_keys | This macro determines whether enforcement of foreign key constraints is enabled or disabled by default for new database connections.<br><br>Each database connection can always turn enforcement of foreign key constraints on and off and run-time using the foreign_keys pragma.<br><br>Enforcement of foreign key constraints is normally off by default, but if this compile-time parameter is set to 1, enforcement of foreign key constraints will be on by default | | Foreign Keys | sqlite_foreign_keys | This macro determines whether enforcement of foreign key constraints is enabled or disabled by default for new database connections.<br><br>Each database connection can always turn enforcement of foreign key constraints on and off and run-time using the foreign_keys pragma.<br><br>Enforcement of foreign key constraints is normally off by default, but if this compile-time parameter is set to 1, enforcement of foreign key constraints will be on by default |
| Full Auto Vacuum | sqlite_vacuum_full | Set the default auto vacuum to full | | Full Auto Vacuum | sqlite_vacuum_full | Set the default auto vacuum to full |
| Incremental Auto Vacuum | sqlite_vacuum_incr | Set the default auto vacuum to incremental | | Incremental Auto Vacuum | sqlite_vacuum_incr | Set the default auto vacuum to incremental |
@ -149,16 +173,20 @@ go build --tags "icu json1 fts5 secure_delete"
| International Components for Unicode | sqlite_icu | This option causes the International Components for Unicode or "ICU" extension to SQLite to be added to the build | | International Components for Unicode | sqlite_icu | This option causes the International Components for Unicode or "ICU" extension to SQLite to be added to the build |
| Introspect PRAGMAS | sqlite_introspect | This option adds some extra PRAGMA statements. <ul><li>PRAGMA function_list</li><li>PRAGMA module_list</li><li>PRAGMA pragma_list</li></ul> | | Introspect PRAGMAS | sqlite_introspect | This option adds some extra PRAGMA statements. <ul><li>PRAGMA function_list</li><li>PRAGMA module_list</li><li>PRAGMA pragma_list</li></ul> |
| JSON SQL Functions | sqlite_json | When this option is defined in the amalgamation, the JSON SQL functions are added to the build automatically | | JSON SQL Functions | sqlite_json | When this option is defined in the amalgamation, the JSON SQL functions are added to the build automatically |
| Math Functions | sqlite_math_functions | This compile-time option enables built-in scalar math functions. For more information see [Built-In Mathematical SQL Functions](https://www.sqlite.org/lang_mathfunc.html) |
| OS Trace | sqlite_os_trace | This option enables OSTRACE() debug logging. This can be verbose and should not be used in production. |
| Pre Update Hook | sqlite_preupdate_hook | Registers a callback function that is invoked prior to each INSERT, UPDATE, and DELETE operation on a database table. |
| Secure Delete | sqlite_secure_delete | This compile-time option changes the default setting of the secure_delete pragma.<br><br>When this option is not used, secure_delete defaults to off. When this option is present, secure_delete defaults to on.<br><br>The secure_delete setting causes deleted content to be overwritten with zeros. There is a small performance penalty since additional I/O must occur.<br><br>On the other hand, secure_delete can prevent fragments of sensitive information from lingering in unused parts of the database file after it has been deleted. See the documentation on the secure_delete pragma for additional information | | Secure Delete | sqlite_secure_delete | This compile-time option changes the default setting of the secure_delete pragma.<br><br>When this option is not used, secure_delete defaults to off. When this option is present, secure_delete defaults to on.<br><br>The secure_delete setting causes deleted content to be overwritten with zeros. There is a small performance penalty since additional I/O must occur.<br><br>On the other hand, secure_delete can prevent fragments of sensitive information from lingering in unused parts of the database file after it has been deleted. See the documentation on the secure_delete pragma for additional information |
| Secure Delete (FAST) | sqlite_secure_delete_fast | For more information see [PRAGMA secure_delete](https://www.sqlite.org/pragma.html#pragma_secure_delete) | | Secure Delete (FAST) | sqlite_secure_delete_fast | For more information see [PRAGMA secure_delete](https://www.sqlite.org/pragma.html#pragma_secure_delete) |
| Tracing / Debug | sqlite_trace | Activate trace functions | | Tracing / Debug | sqlite_trace | Activate trace functions |
| User Authentication | sqlite_userauth | SQLite User Authentication see [User Authentication](#user-authentication) for more information. | | User Authentication | sqlite_userauth | SQLite User Authentication see [User Authentication](#user-authentication) for more information. |
| Virtual Tables | sqlite_vtable | SQLite Virtual Tables see [SQLite Official VTABLE Documentation](https://www.sqlite.org/vtab.html) for more information, and a [full example here](https://github.com/mattn/go-sqlite3/tree/master/_example/vtable) |
# Compilation # Compilation
This package requires `CGO_ENABLED=1` ennvironment variable if not set by default, and the presence of the `gcc` compiler. This package requires the `CGO_ENABLED=1` environment variable if not set by default, and the presence of the `gcc` compiler.
If you need to add additional CFLAGS or LDFLAGS to the build command, and do not want to modify this package. Then this can be achieved by using the `CGO_CFLAGS` and `CGO_LDFLAGS` environment variables. If you need to add additional CFLAGS or LDFLAGS to the build command, and do not want to modify this package, then this can be achieved by using the `CGO_CFLAGS` and `CGO_LDFLAGS` environment variables.
## Android ## Android
@ -173,7 +201,7 @@ For more information see [#201](https://github.com/mattn/go-sqlite3/issues/201)
# ARM # ARM
To compile for `ARM` use the following environment. To compile for `ARM` use the following environment:
```bash ```bash
env CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ \ env CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ \
@ -191,9 +219,14 @@ This library can be cross-compiled.
In some cases you are required to the `CC` environment variable with the cross compiler. In some cases you are required to the `CC` environment variable with the cross compiler.
Additional information: ## Cross Compiling from MAC OSX
- [#491](https://github.com/mattn/go-sqlite3/issues/491) The simplest way to cross compile from OSX is to use [musl-cross](https://github.com/FiloSottile/homebrew-musl-cross).
- [#560](https://github.com/mattn/go-sqlite3/issues/560)
Steps:
- Install [musl-cross](https://github.com/FiloSottile/homebrew-musl-cross) (`brew install FiloSottile/musl-cross/musl-cross`).
- Run `CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static"`.
Please refer to the project's [README](https://github.com/FiloSottile/homebrew-musl-cross#readme) for further information.
# Google Cloud Platform # Google Cloud Platform
@ -203,7 +236,7 @@ Please work only with compiled final binaries.
## Linux ## Linux
To compile this package on Linux you must install the development tools for your linux distribution. To compile this package on Linux, you must install the development tools for your linux distribution.
To compile under linux use the build tag `linux`. To compile under linux use the build tag `linux`.
@ -219,7 +252,7 @@ go build --tags "libsqlite3 linux"
### Alpine ### Alpine
When building in an `alpine` container run the following command before building. When building in an `alpine` container run the following command before building:
``` ```
apk add --update gcc musl-dev apk add --update gcc musl-dev
@ -239,29 +272,29 @@ sudo apt-get install build-essential
## Mac OSX ## Mac OSX
OSX should have all the tools present to compile this package, if not install XCode this will add all the developers tools. OSX should have all the tools present to compile this package. If not, install XCode to add all the developers tools.
Required dependency Required dependency:
```bash ```bash
brew install sqlite3 brew install sqlite3
``` ```
For OSX there is an additional package install which is required if you whish to build the `icu` extension. For OSX, there is an additional package to install which is required if you wish to build the `icu` extension.
This additional package can be installed with `homebrew`. This additional package can be installed with `homebrew`:
```bash ```bash
brew upgrade icu4c brew upgrade icu4c
``` ```
To compile for Mac OSX. To compile for Mac OSX:
```bash ```bash
go build --tags "darwin" go build --tags "darwin"
``` ```
If you wish to link directly to libsqlite3 then you can use the `libsqlite3` build tag. If you wish to link directly to libsqlite3, use the `libsqlite3` build tag:
``` ```
go build --tags "libsqlite3 darwin" go build --tags "libsqlite3 darwin"
@ -273,14 +306,14 @@ Additional information:
## Windows ## Windows
To compile this package on Windows OS you must have the `gcc` compiler installed. To compile this package on Windows, you must have the `gcc` compiler installed.
1) Install a Windows `gcc` toolchain. 1) Install a Windows `gcc` toolchain.
2) Add the `bin` folders to the Windows path if the installer did not do this by default. 2) Add the `bin` folder to the Windows path, if the installer did not do this by default.
3) Open a terminal for the TDM-GCC toolchain, can be found in the Windows Start menu. 3) Open a terminal for the TDM-GCC toolchain, which can be found in the Windows Start menu.
4) Navigate to your project folder and run the `go build ...` command for this package. 4) Navigate to your project folder and run the `go build ...` command for this package.
For example the TDM-GCC Toolchain can be found [here](https://sourceforge.net/projects/tdm-gcc/). For example the TDM-GCC Toolchain can be found [here](https://jmeubank.github.io/tdm-gcc/).
## Errors ## Errors
@ -318,28 +351,28 @@ This package supports the SQLite User Authentication module.
## Compile ## Compile
To use the User authentication module the package has to be compiled with the tag `sqlite_userauth`. See [Features](#features). To use the User authentication module, the package has to be compiled with the tag `sqlite_userauth`. See [Features](#features).
## Usage ## Usage
### Create protected database ### Create protected database
To create a database protected by user authentication provide the following argument to the connection string `_auth`. To create a database protected by user authentication, provide the following argument to the connection string `_auth`.
This will enable user authentication within the database. This option however requires two additional arguments: This will enable user authentication within the database. This option however requires two additional arguments:
- `_auth_user` - `_auth_user`
- `_auth_pass` - `_auth_pass`
When `_auth` is present on the connection string user authentication will be enabled and the provided user will be created When `_auth` is present in the connection string user authentication will be enabled and the provided user will be created
as an `admin` user. After initial creation, the parameter `_auth` has no effect anymore and can be omitted from the connection string. as an `admin` user. After initial creation, the parameter `_auth` has no effect anymore and can be omitted from the connection string.
Example connection string: Example connection strings:
Create an user authentication database with user `admin` and password `admin`. Create an user authentication database with user `admin` and password `admin`:
`file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin` `file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin`
Create an user authentication database with user `admin` and password `admin` and use `SHA1` for the password encoding. Create an user authentication database with user `admin` and password `admin` and use `SHA1` for the password encoding:
`file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin&_auth_crypt=sha1` `file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin&_auth_crypt=sha1`
@ -365,11 +398,11 @@ salt this can be configured with `_auth_salt`.
### Restrictions ### Restrictions
Operations on the database regarding to user management can only be preformed by an administrator user. Operations on the database regarding user management can only be preformed by an administrator user.
### Support ### Support
The user authentication supports two kinds of users The user authentication supports two kinds of users:
- administrators - administrators
- regular users - regular users
@ -380,7 +413,7 @@ User management can be done by directly using the `*SQLiteConn` or by SQL.
#### SQL #### SQL
The following sql functions are available for user management. The following sql functions are available for user management:
| Function | Arguments | Description | | Function | Arguments | Description |
|----------|-----------|-------------| |----------|-----------|-------------|
@ -389,7 +422,7 @@ The following sql functions are available for user management.
| `auth_user_change` | username `string`, password `string`, admin `int` | Function to modify an user. Users can change their own password, but only an administrator can change the administrator flag. | | `auth_user_change` | username `string`, password `string`, admin `int` | Function to modify an user. Users can change their own password, but only an administrator can change the administrator flag. |
| `authUserDelete` | username `string` | Delete an user from the database. Can only be used by an administrator. The current logged in administrator cannot be deleted. This is to make sure their is always an administrator remaining. | | `authUserDelete` | username `string` | Delete an user from the database. Can only be used by an administrator. The current logged in administrator cannot be deleted. This is to make sure their is always an administrator remaining. |
These functions will return an integer. These functions will return an integer:
- 0 (SQLITE_OK) - 0 (SQLITE_OK)
- 23 (SQLITE_AUTH) Failed to perform due to authentication or insufficient privileges - 23 (SQLITE_AUTH) Failed to perform due to authentication or insufficient privileges
@ -410,7 +443,7 @@ SELECT user_delete('user');
#### *SQLiteConn #### *SQLiteConn
The following functions are available for User authentication from the `*SQLiteConn`. The following functions are available for User authentication from the `*SQLiteConn`:
| Function | Description | | Function | Description |
|----------|-------------| |----------|-------------|
@ -421,16 +454,26 @@ The following functions are available for User authentication from the `*SQLiteC
### Attached database ### Attached database
When using attached databases. SQLite will use the authentication from the `main` database for the attached database(s). When using attached databases, SQLite will use the authentication from the `main` database for the attached database(s).
# Extensions # Extensions
If you want your own extension to be listed here or you want to add a reference to an extension; please submit an Issue for this. If you want your own extension to be listed here, or you want to add a reference to an extension; please submit an Issue for this.
## Spatialite ## Spatialite
Spatialite is available as an extension to SQLite, and can be used in combination with this repository. Spatialite is available as an extension to SQLite, and can be used in combination with this repository.
For an example see [shaxbee/go-spatialite](https://github.com/shaxbee/go-spatialite). For an example, see [shaxbee/go-spatialite](https://github.com/shaxbee/go-spatialite).
## extension-functions.c from SQLite3 Contrib
extension-functions.c is available as an extension to SQLite, and provides the following functions:
- Math: acos, asin, atan, atn2, atan2, acosh, asinh, atanh, difference, degrees, radians, cos, sin, tan, cot, cosh, sinh, tanh, coth, exp, log, log10, power, sign, sqrt, square, ceil, floor, pi.
- String: replicate, charindex, leftstr, rightstr, ltrim, rtrim, trim, replace, reverse, proper, padl, padr, padc, strfilter.
- Aggregate: stdev, variance, mode, median, lower_quartile, upper_quartile
For an example, see [dinedal/go-sqlite3-extension-functions](https://github.com/dinedal/go-sqlite3-extension-functions).
# FAQ # FAQ
@ -450,40 +493,44 @@ For an example see [shaxbee/go-spatialite](https://github.com/shaxbee/go-spatial
- Can I use this in multiple routines concurrently? - Can I use this in multiple routines concurrently?
Yes for readonly. But, No for writable. See [#50](https://github.com/mattn/go-sqlite3/issues/50), [#51](https://github.com/mattn/go-sqlite3/issues/51), [#209](https://github.com/mattn/go-sqlite3/issues/209), [#274](https://github.com/mattn/go-sqlite3/issues/274). Yes for readonly. But not for writable. See [#50](https://github.com/mattn/go-sqlite3/issues/50), [#51](https://github.com/mattn/go-sqlite3/issues/51), [#209](https://github.com/mattn/go-sqlite3/issues/209), [#274](https://github.com/mattn/go-sqlite3/issues/274).
- Why I'm getting `no such table` error? - Why I'm getting `no such table` error?
Why is it racy if I use a `sql.Open("sqlite3", ":memory:")` database? Why is it racy if I use a `sql.Open("sqlite3", ":memory:")` database?
Each connection to :memory: opens a brand new in-memory sql database, so if Each connection to `":memory:"` opens a brand new in-memory sql database, so if
the stdlib's sql engine happens to open another connection and you've only the stdlib's sql engine happens to open another connection and you've only
specified ":memory:", that connection will see a brand new database. A specified `":memory:"`, that connection will see a brand new database. A
workaround is to use "file::memory:?mode=memory&cache=shared". Every workaround is to use `"file::memory:?cache=shared"` (or `"file:foobar?mode=memory&cache=shared"`). Every
connection to this string will point to the same in-memory database. connection to this string will point to the same in-memory database.
For more information see Note that if the last database connection in the pool closes, the in-memory database is deleted. Make sure the [max idle connection limit](https://golang.org/pkg/database/sql/#DB.SetMaxIdleConns) is > 0, and the [connection lifetime](https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime) is infinite.
For more information see:
* [#204](https://github.com/mattn/go-sqlite3/issues/204) * [#204](https://github.com/mattn/go-sqlite3/issues/204)
* [#511](https://github.com/mattn/go-sqlite3/issues/511) * [#511](https://github.com/mattn/go-sqlite3/issues/511)
* https://www.sqlite.org/sharedcache.html#shared_cache_and_in_memory_databases
* https://www.sqlite.org/inmemorydb.html#sharedmemdb
- Reading from database with large amount of goroutines fails on OSX. - Reading from database with large amount of goroutines fails on OSX.
OS X limits OS-wide to not have more than 1000 files open simultaneously by default. OS X limits OS-wide to not have more than 1000 files open simultaneously by default.
For more information see [#289](https://github.com/mattn/go-sqlite3/issues/289) For more information, see [#289](https://github.com/mattn/go-sqlite3/issues/289)
- Trying to execute a `.` (dot) command throws an error. - Trying to execute a `.` (dot) command throws an error.
Error: `Error: near ".": syntax error` Error: `Error: near ".": syntax error`
Dot command are part of SQLite3 CLI not of this library. Dot command are part of SQLite3 CLI, not of this library.
You need to implement the feature or call the sqlite3 cli. You need to implement the feature or call the sqlite3 cli.
More infomation see [#305](https://github.com/mattn/go-sqlite3/issues/305) More information see [#305](https://github.com/mattn/go-sqlite3/issues/305).
- Error: `database is locked` - Error: `database is locked`
When you get a database is locked. Please use the following options. When you get a database is locked, please use the following options.
Add to DSN: `cache=shared` Add to DSN: `cache=shared`
@ -492,13 +539,43 @@ For an example see [shaxbee/go-spatialite](https://github.com/shaxbee/go-spatial
db, err := sql.Open("sqlite3", "file:locked.sqlite?cache=shared") db, err := sql.Open("sqlite3", "file:locked.sqlite?cache=shared")
``` ```
Second please set the database connections of the SQL package to 1. Next, please set the database connections of the SQL package to 1:
```go ```go
db.SetMaxOpenConns(1) db.SetMaxOpenConns(1)
``` ```
More information see [#209](https://github.com/mattn/go-sqlite3/issues/209) For more information, see [#209](https://github.com/mattn/go-sqlite3/issues/209).
## Contributors
### Code Contributors
This project exists thanks to all the people who [[contribute](CONTRIBUTING.md)].
<a href="https://github.com/mattn/go-sqlite3/graphs/contributors"><img src="https://opencollective.com/mattn-go-sqlite3/contributors.svg?width=890&button=false" /></a>
### Financial Contributors
Become a financial contributor and help us sustain our community. [[Contribute here](https://opencollective.com/mattn-go-sqlite3/contribute)].
#### Individuals
<a href="https://opencollective.com/mattn-go-sqlite3"><img src="https://opencollective.com/mattn-go-sqlite3/individuals.svg?width=890"></a>
#### Organizations
Support this project with your organization. Your logo will show up here with a link to your website. [[Contribute](https://opencollective.com/mattn-go-sqlite3/contribute)]
<a href="https://opencollective.com/mattn-go-sqlite3/organization/0/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/0/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/1/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/1/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/2/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/2/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/3/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/3/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/4/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/4/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/5/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/5/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/6/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/6/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/7/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/7/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/8/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/8/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/9/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/9/avatar.svg"></a>
# License # License

View File

@ -0,0 +1,12 @@
TARGET = custom_driver_name
ifeq ($(OS),Windows_NT)
TARGET := $(TARGET).exe
endif
all : $(TARGET)
$(TARGET) : main.go
go build -ldflags="-X 'github.com/mattn/go-sqlite3.driverName=my-sqlite3'"
clean :
rm -f $(TARGET)

View File

@ -0,0 +1,13 @@
package main
import (
"database/sql"
_ "github.com/mattn/go-sqlite3"
)
func main() {
for _, driver := range sql.Drivers() {
println(driver)
}
}

View File

@ -0,0 +1,30 @@
package sqlite3_fuzz
import (
"bytes"
"database/sql"
"io/ioutil"
_ "github.com/mattn/go-sqlite3"
)
func FuzzOpenExec(data []byte) int {
sep := bytes.IndexByte(data, 0)
if sep <= 0 {
return 0
}
err := ioutil.WriteFile("/tmp/fuzz.db", data[sep+1:], 0644)
if err != nil {
return 0
}
db, err := sql.Open("sqlite3", "/tmp/fuzz.db")
if err != nil {
return 0
}
defer db.Close()
_, err = db.Exec(string(data[:sep-1]))
if err != nil {
return 0
}
return 1
}

View File

@ -17,7 +17,7 @@ func createBulkInsertQuery(n int, start int) (query string, args []interface{})
for i := 0; i < n; i++ { for i := 0; i < n; i++ {
values[i] = "(?, ?)" values[i] = "(?, ?)"
args[pos] = start + i args[pos] = start + i
args[pos+1] = fmt.Sprintf("こんにち世界%03d", i) args[pos+1] = fmt.Sprintf("こんにち世界%03d", i)
pos += 2 pos += 2
} }
query = fmt.Sprintf( query = fmt.Sprintf(
@ -27,7 +27,7 @@ func createBulkInsertQuery(n int, start int) (query string, args []interface{})
return return
} }
func bukInsert(db *sql.DB, query string, args []interface{}) (err error) { func bulkInsert(db *sql.DB, query string, args []interface{}) (err error) {
stmt, err := db.Prepare(query) stmt, err := db.Prepare(query)
if err != nil { if err != nil {
return return
@ -76,7 +76,7 @@ func main() {
num := 400 num := 400
query, args := createBulkInsertQuery(num, 0) query, args := createBulkInsertQuery(num, 0)
err = bukInsert(db, query, args) err = bulkInsert(db, query, args)
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)
} }
@ -88,7 +88,7 @@ func main() {
log.Printf("updated SQLITE_LIMIT_VARIABLE_NUMBER: %d", limitVariableNumber) log.Printf("updated SQLITE_LIMIT_VARIABLE_NUMBER: %d", limitVariableNumber)
query, args = createBulkInsertQuery(num, num) query, args = createBulkInsertQuery(num, num)
err = bukInsert(db, query, args) err = bulkInsert(db, query, args)
if err != nil { if err != nil {
if err != nil { if err != nil {
log.Printf("expect failed since SQLITE_LIMIT_VARIABLE_NUMBER is too small: %v", err) log.Printf("expect failed since SQLITE_LIMIT_VARIABLE_NUMBER is too small: %v", err)
@ -102,7 +102,7 @@ func main() {
log.Printf("updated SQLITE_LIMIT_VARIABLE_NUMBER: %d", limitVariableNumber) log.Printf("updated SQLITE_LIMIT_VARIABLE_NUMBER: %d", limitVariableNumber)
query, args = createBulkInsertQuery(500, num+num) query, args = createBulkInsertQuery(500, num+num)
err = bukInsert(db, query, args) err = bulkInsert(db, query, args)
if err != nil { if err != nil {
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)

View File

@ -1,22 +1,27 @@
ifeq ($(OS),Windows_NT) ifeq ($(OS),Windows_NT)
EXE=extension.exe EXE=extension.exe
EXT=sqlite3_mod_regexp.dll LIB_EXT=dll
RM=cmd /c del RM=cmd /c del
LDFLAG= LDFLAG=
else else
EXE=extension EXE=extension
EXT=sqlite3_mod_regexp.so ifeq ($(shell uname -s),Darwin)
RM=rm LIB_EXT=dylib
else
LIB_EXT=so
endif
RM=rm -f
LDFLAG=-fPIC LDFLAG=-fPIC
endif endif
LIB=sqlite3_mod_regexp.$(LIB_EXT)
all : $(EXE) $(EXT) all : $(EXE) $(LIB)
$(EXE) : extension.go $(EXE) : extension.go
go build $< go build $<
$(EXT) : sqlite3_mod_regexp.c $(LIB) : sqlite3_mod_regexp.c
gcc $(LDFLAG) -shared -o $@ $< -lsqlite3 -lpcre gcc $(LDFLAG) -shared -o $@ $< -lsqlite3 -lpcre
clean : clean :
@-$(RM) $(EXE) $(EXT) @-$(RM) $(EXE) $(LIB)

View File

@ -13,9 +13,13 @@ static void regexp_func(sqlite3_context *context, int argc, sqlite3_value **argv
int vec[500]; int vec[500];
int n, rc; int n, rc;
pcre* re = pcre_compile(pattern, 0, &errstr, &erroff, NULL); pcre* re = pcre_compile(pattern, 0, &errstr, &erroff, NULL);
if (!re) {
sqlite3_result_error(context, errstr, 0);
return;
}
rc = pcre_exec(re, NULL, target, strlen(target), 0, 0, vec, 500); rc = pcre_exec(re, NULL, target, strlen(target), 0, 0, vec, 500);
if (rc <= 0) { if (rc <= 0) {
sqlite3_result_error(context, errstr, 0); sqlite3_result_int(context, 0);
return; return;
} }
sqlite3_result_int(context, 1); sqlite3_result_int(context, 1);

View File

@ -1,24 +1,29 @@
ifeq ($(OS),Windows_NT) ifeq ($(OS),Windows_NT)
EXE=extension.exe EXE=extension.exe
EXT=sqlite3_mod_vtable.dll LIB_EXT=dll
RM=cmd /c del RM=cmd /c del
LIBCURL=-lcurldll LIBCURL=-lcurldll
LDFLAG= LDFLAG=
else else
EXE=extension EXE=extension
EXT=sqlite3_mod_vtable.so ifeq ($(shell uname -s),Darwin)
RM=rm LIB_EXT=dylib
else
LIB_EXT=so
endif
RM=rm -f
LDFLAG=-fPIC LDFLAG=-fPIC
LIBCURL=-lcurl LIBCURL=-lcurl
endif endif
LIB=sqlite3_mod_vtable.$(LIB_EXT)
all : $(EXE) $(EXT) all : $(EXE) $(LIB)
$(EXE) : extension.go $(EXE) : extension.go
go build $< go build $<
$(EXT) : sqlite3_mod_vtable.cc $(LIB) : sqlite3_mod_vtable.cc
g++ $(LDFLAG) -shared -o $@ $< -lsqlite3 $(LIBCURL) g++ $(LDFLAG) -shared -o $@ $< -lsqlite3 $(LIBCURL)
clean : clean :
@-$(RM) $(EXE) $(EXT) @-$(RM) $(EXE) $(LIB)

View File

@ -1,6 +1,6 @@
#include <string> #include <string>
#include <sstream> #include <sstream>
#include <sqlite3-binding.h> #include <sqlite3.h>
#include <sqlite3ext.h> #include <sqlite3ext.h>
#include <curl/curl.h> #include <curl/curl.h>
#include "picojson.h" #include "picojson.h"

View File

@ -0,0 +1,45 @@
# =============================================================================
# Multi-stage Dockerfile Example
# =============================================================================
# This is a simple Dockerfile that will build an image of scratch-base image.
# Usage:
# docker build -t simple:local . && docker run --rm simple:local
# =============================================================================
# -----------------------------------------------------------------------------
# Build Stage
# -----------------------------------------------------------------------------
FROM golang:alpine AS build
# Important:
# Because this is a CGO enabled package, you are required to set it as 1.
ENV CGO_ENABLED=1
RUN apk add --no-cache \
# Important: required for go-sqlite3
gcc \
# Required for Alpine
musl-dev
WORKDIR /workspace
COPY . /workspace/
RUN \
go mod init github.com/mattn/sample && \
go mod tidy && \
go install -ldflags='-s -w -extldflags "-static"' ./simple.go
RUN \
# Smoke test
set -o pipefail; \
/go/bin/simple | grep 99\ こんにちは世界099
# -----------------------------------------------------------------------------
# Main Stage
# -----------------------------------------------------------------------------
FROM scratch
COPY --from=build /go/bin/simple /usr/local/bin/simple
ENTRYPOINT [ "/usr/local/bin/simple" ]

View File

@ -37,12 +37,15 @@ func main() {
} }
defer stmt.Close() defer stmt.Close()
for i := 0; i < 100; i++ { for i := 0; i < 100; i++ {
_, err = stmt.Exec(i, fmt.Sprintf("こんにち世界%03d", i)) _, err = stmt.Exec(i, fmt.Sprintf("こんにち世界%03d", i))
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)
} }
} }
tx.Commit() err = tx.Commit()
if err != nil {
log.Fatal(err)
}
rows, err := db.Query("select id, name from foo") rows, err := db.Query("select id, name from foo")
if err != nil { if err != nil {

View File

@ -63,7 +63,12 @@ func (v *ghRepoTable) Open() (sqlite3.VTabCursor, error) {
} }
func (v *ghRepoTable) BestIndex(cst []sqlite3.InfoConstraint, ob []sqlite3.InfoOrderBy) (*sqlite3.IndexResult, error) { func (v *ghRepoTable) BestIndex(cst []sqlite3.InfoConstraint, ob []sqlite3.InfoOrderBy) (*sqlite3.IndexResult, error) {
return &sqlite3.IndexResult{}, nil used := make([]bool, len(csts))
return &sqlite3.IndexResult{
IdxNum: 0,
IdxStr: "default",
Used: used,
}, nil
} }
func (v *ghRepoTable) Disconnect() error { return nil } func (v *ghRepoTable) Disconnect() error { return nil }

View File

@ -0,0 +1,33 @@
package main
import (
"database/sql"
"fmt"
"log"
"github.com/mattn/go-sqlite3"
)
func main() {
sql.Register("sqlite3_with_extensions", &sqlite3.SQLiteDriver{
ConnectHook: func(conn *sqlite3.SQLiteConn) error {
return conn.CreateModule("series", &seriesModule{})
},
})
db, err := sql.Open("sqlite3_with_extensions", ":memory:")
if err != nil {
log.Fatal(err)
}
defer db.Close()
rows, err := db.Query("select * from series")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var value int
rows.Scan(&value)
fmt.Printf("value: %d\n", value)
}
}

View File

@ -0,0 +1,118 @@
package main
import (
"fmt"
"github.com/mattn/go-sqlite3"
)
type seriesModule struct{}
func (m *seriesModule) EponymousOnlyModule() {}
func (m *seriesModule) Create(c *sqlite3.SQLiteConn, args []string) (sqlite3.VTab, error) {
err := c.DeclareVTab(fmt.Sprintf(`
CREATE TABLE %s (
value INT,
start HIDDEN,
stop HIDDEN,
step HIDDEN
)`, args[0]))
if err != nil {
return nil, err
}
return &seriesTable{0, 0, 1}, nil
}
func (m *seriesModule) Connect(c *sqlite3.SQLiteConn, args []string) (sqlite3.VTab, error) {
return m.Create(c, args)
}
func (m *seriesModule) DestroyModule() {}
type seriesTable struct {
start int64
stop int64
step int64
}
func (v *seriesTable) Open() (sqlite3.VTabCursor, error) {
return &seriesCursor{v, 0}, nil
}
func (v *seriesTable) BestIndex(csts []sqlite3.InfoConstraint, ob []sqlite3.InfoOrderBy) (*sqlite3.IndexResult, error) {
used := make([]bool, len(csts))
for c, cst := range csts {
if cst.Usable && cst.Op == sqlite3.OpEQ {
used[c] = true
}
}
return &sqlite3.IndexResult{
IdxNum: 0,
IdxStr: "default",
Used: used,
}, nil
}
func (v *seriesTable) Disconnect() error { return nil }
func (v *seriesTable) Destroy() error { return nil }
type seriesCursor struct {
*seriesTable
value int64
}
func (vc *seriesCursor) Column(c *sqlite3.SQLiteContext, col int) error {
switch col {
case 0:
c.ResultInt64(vc.value)
case 1:
c.ResultInt64(vc.seriesTable.start)
case 2:
c.ResultInt64(vc.seriesTable.stop)
case 3:
c.ResultInt64(vc.seriesTable.step)
}
return nil
}
func (vc *seriesCursor) Filter(idxNum int, idxStr string, vals []interface{}) error {
switch {
case len(vals) < 1:
vc.seriesTable.start = 0
vc.seriesTable.stop = 1000
vc.value = vc.seriesTable.start
case len(vals) < 2:
vc.seriesTable.start = vals[0].(int64)
vc.seriesTable.stop = 1000
vc.value = vc.seriesTable.start
case len(vals) < 3:
vc.seriesTable.start = vals[0].(int64)
vc.seriesTable.stop = vals[1].(int64)
vc.value = vc.seriesTable.start
case len(vals) < 4:
vc.seriesTable.start = vals[0].(int64)
vc.seriesTable.stop = vals[1].(int64)
vc.seriesTable.step = vals[2].(int64)
}
return nil
}
func (vc *seriesCursor) Next() error {
vc.value += vc.step
return nil
}
func (vc *seriesCursor) EOF() bool {
return vc.value > vc.stop
}
func (vc *seriesCursor) Rowid() (int64, error) {
return int64(vc.value), nil
}
func (vc *seriesCursor) Close() error {
return nil
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -7,7 +7,7 @@ package sqlite3
/* /*
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
@ -25,18 +25,18 @@ type SQLiteBackup struct {
} }
// Backup make backup from src to dest. // Backup make backup from src to dest.
func (c *SQLiteConn) Backup(dest string, conn *SQLiteConn, src string) (*SQLiteBackup, error) { func (destConn *SQLiteConn) Backup(dest string, srcConn *SQLiteConn, src string) (*SQLiteBackup, error) {
destptr := C.CString(dest) destptr := C.CString(dest)
defer C.free(unsafe.Pointer(destptr)) defer C.free(unsafe.Pointer(destptr))
srcptr := C.CString(src) srcptr := C.CString(src)
defer C.free(unsafe.Pointer(srcptr)) defer C.free(unsafe.Pointer(srcptr))
if b := C.sqlite3_backup_init(c.db, destptr, conn.db, srcptr); b != nil { if b := C.sqlite3_backup_init(destConn.db, destptr, srcConn.db, srcptr); b != nil {
bb := &SQLiteBackup{b: b} bb := &SQLiteBackup{b: b}
runtime.SetFinalizer(bb, (*SQLiteBackup).Finish) runtime.SetFinalizer(bb, (*SQLiteBackup).Finish)
return bb, nil return bb, nil
} }
return nil, c.lastError() return nil, destConn.lastError()
} }
// Step to backs up for one step. Calls the underlying `sqlite3_backup_step` // Step to backs up for one step. Calls the underlying `sqlite3_backup_step`

View File

@ -1,6 +1,10 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build cgo
package sqlite3 package sqlite3
import ( import (

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -12,7 +12,7 @@ package sqlite3
/* /*
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
@ -35,86 +35,97 @@ import (
//export callbackTrampoline //export callbackTrampoline
func callbackTrampoline(ctx *C.sqlite3_context, argc int, argv **C.sqlite3_value) { func callbackTrampoline(ctx *C.sqlite3_context, argc int, argv **C.sqlite3_value) {
args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:argc:argc] args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:argc:argc]
fi := lookupHandle(uintptr(C.sqlite3_user_data(ctx))).(*functionInfo) fi := lookupHandle(C.sqlite3_user_data(ctx)).(*functionInfo)
fi.Call(ctx, args) fi.Call(ctx, args)
} }
//export stepTrampoline //export stepTrampoline
func stepTrampoline(ctx *C.sqlite3_context, argc C.int, argv **C.sqlite3_value) { func stepTrampoline(ctx *C.sqlite3_context, argc C.int, argv **C.sqlite3_value) {
args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:int(argc):int(argc)] args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:int(argc):int(argc)]
ai := lookupHandle(uintptr(C.sqlite3_user_data(ctx))).(*aggInfo) ai := lookupHandle(C.sqlite3_user_data(ctx)).(*aggInfo)
ai.Step(ctx, args) ai.Step(ctx, args)
} }
//export doneTrampoline //export doneTrampoline
func doneTrampoline(ctx *C.sqlite3_context) { func doneTrampoline(ctx *C.sqlite3_context) {
handle := uintptr(C.sqlite3_user_data(ctx)) ai := lookupHandle(C.sqlite3_user_data(ctx)).(*aggInfo)
ai := lookupHandle(handle).(*aggInfo)
ai.Done(ctx) ai.Done(ctx)
} }
//export compareTrampoline //export compareTrampoline
func compareTrampoline(handlePtr uintptr, la C.int, a *C.char, lb C.int, b *C.char) C.int { func compareTrampoline(handlePtr unsafe.Pointer, la C.int, a *C.char, lb C.int, b *C.char) C.int {
cmp := lookupHandle(handlePtr).(func(string, string) int) cmp := lookupHandle(handlePtr).(func(string, string) int)
return C.int(cmp(C.GoStringN(a, la), C.GoStringN(b, lb))) return C.int(cmp(C.GoStringN(a, la), C.GoStringN(b, lb)))
} }
//export commitHookTrampoline //export commitHookTrampoline
func commitHookTrampoline(handle uintptr) int { func commitHookTrampoline(handle unsafe.Pointer) int {
callback := lookupHandle(handle).(func() int) callback := lookupHandle(handle).(func() int)
return callback() return callback()
} }
//export rollbackHookTrampoline //export rollbackHookTrampoline
func rollbackHookTrampoline(handle uintptr) { func rollbackHookTrampoline(handle unsafe.Pointer) {
callback := lookupHandle(handle).(func()) callback := lookupHandle(handle).(func())
callback() callback()
} }
//export updateHookTrampoline //export updateHookTrampoline
func updateHookTrampoline(handle uintptr, op int, db *C.char, table *C.char, rowid int64) { func updateHookTrampoline(handle unsafe.Pointer, op int, db *C.char, table *C.char, rowid int64) {
callback := lookupHandle(handle).(func(int, string, string, int64)) callback := lookupHandle(handle).(func(int, string, string, int64))
callback(op, C.GoString(db), C.GoString(table), rowid) callback(op, C.GoString(db), C.GoString(table), rowid)
} }
//export authorizerTrampoline //export authorizerTrampoline
func authorizerTrampoline(handle uintptr, op int, arg1 *C.char, arg2 *C.char, arg3 *C.char) int { func authorizerTrampoline(handle unsafe.Pointer, op int, arg1 *C.char, arg2 *C.char, arg3 *C.char) int {
callback := lookupHandle(handle).(func(int, string, string, string) int) callback := lookupHandle(handle).(func(int, string, string, string) int)
return callback(op, C.GoString(arg1), C.GoString(arg2), C.GoString(arg3)) return callback(op, C.GoString(arg1), C.GoString(arg2), C.GoString(arg3))
} }
// Use handles to avoid passing Go pointers to C. //export preUpdateHookTrampoline
func preUpdateHookTrampoline(handle unsafe.Pointer, dbHandle uintptr, op int, db *C.char, table *C.char, oldrowid int64, newrowid int64) {
hval := lookupHandleVal(handle)
data := SQLitePreUpdateData{
Conn: hval.db,
Op: op,
DatabaseName: C.GoString(db),
TableName: C.GoString(table),
OldRowID: oldrowid,
NewRowID: newrowid,
}
callback := hval.val.(func(SQLitePreUpdateData))
callback(data)
}
// Use handles to avoid passing Go pointers to C.
type handleVal struct { type handleVal struct {
db *SQLiteConn db *SQLiteConn
val interface{} val interface{}
} }
var handleLock sync.Mutex var handleLock sync.Mutex
var handleVals = make(map[uintptr]handleVal) var handleVals = make(map[unsafe.Pointer]handleVal)
var handleIndex uintptr = 100
func newHandle(db *SQLiteConn, v interface{}) uintptr { func newHandle(db *SQLiteConn, v interface{}) unsafe.Pointer {
handleLock.Lock() handleLock.Lock()
defer handleLock.Unlock() defer handleLock.Unlock()
i := handleIndex val := handleVal{db: db, val: v}
handleIndex++ var p unsafe.Pointer = C.malloc(C.size_t(1))
handleVals[i] = handleVal{db, v} if p == nil {
return i panic("can't allocate 'cgo-pointer hack index pointer': ptr == nil")
}
handleVals[p] = val
return p
} }
func lookupHandle(handle uintptr) interface{} { func lookupHandleVal(handle unsafe.Pointer) handleVal {
handleLock.Lock() handleLock.Lock()
defer handleLock.Unlock() defer handleLock.Unlock()
r, ok := handleVals[handle] return handleVals[handle]
if !ok { }
if handle >= 100 && handle < handleIndex {
panic("deleted handle") func lookupHandle(handle unsafe.Pointer) interface{} {
} else { return lookupHandleVal(handle).val
panic("invalid handle")
}
}
return r.val
} }
func deleteHandles(db *SQLiteConn) { func deleteHandles(db *SQLiteConn) {
@ -123,6 +134,7 @@ func deleteHandles(db *SQLiteConn) {
for handle, val := range handleVals { for handle, val := range handleVals {
if val.db == db { if val.db == db {
delete(handleVals, handle) delete(handleVals, handle)
C.free(handle)
} }
} }
} }
@ -341,6 +353,20 @@ func callbackRetNil(ctx *C.sqlite3_context, v reflect.Value) error {
return nil return nil
} }
func callbackRetGeneric(ctx *C.sqlite3_context, v reflect.Value) error {
if v.IsNil() {
C.sqlite3_result_null(ctx)
return nil
}
cb, err := callbackRet(v.Elem().Type())
if err != nil {
return err
}
return cb(ctx, v.Elem())
}
func callbackRet(typ reflect.Type) (callbackRetConverter, error) { func callbackRet(typ reflect.Type) (callbackRetConverter, error) {
switch typ.Kind() { switch typ.Kind() {
case reflect.Interface: case reflect.Interface:
@ -348,6 +374,11 @@ func callbackRet(typ reflect.Type) (callbackRetConverter, error) {
if typ.Implements(errorInterface) { if typ.Implements(errorInterface) {
return callbackRetNil, nil return callbackRetNil, nil
} }
if typ.NumMethod() == 0 {
return callbackRetGeneric, nil
}
fallthrough fallthrough
case reflect.Slice: case reflect.Slice:
if typ.Elem().Kind() != reflect.Uint8 { if typ.Elem().Kind() != reflect.Uint8 {

View File

@ -1,3 +1,10 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build cgo
package sqlite3 package sqlite3
import ( import (
@ -95,3 +102,15 @@ func TestCallbackConverters(t *testing.T) {
} }
} }
} }
func TestCallbackReturnAny(t *testing.T) {
udf := func() interface{} {
return 1
}
typ := reflect.TypeOf(udf)
_, err := callbackRet(typ.Out(0))
if err != nil {
t.Errorf("Expected valid callback for any return type, got: %s", err)
}
}

299
convert.go Normal file
View File

@ -0,0 +1,299 @@
// Extracted from Go database/sql source code
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Type conversions for Scan.
package sqlite3
import (
"database/sql"
"database/sql/driver"
"errors"
"fmt"
"reflect"
"strconv"
"time"
)
var errNilPtr = errors.New("destination pointer is nil") // embedded in descriptive error
// convertAssign copies to dest the value in src, converting it if possible.
// An error is returned if the copy would result in loss of information.
// dest should be a pointer type.
func convertAssign(dest, src interface{}) error {
// Common cases, without reflect.
switch s := src.(type) {
case string:
switch d := dest.(type) {
case *string:
if d == nil {
return errNilPtr
}
*d = s
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = []byte(s)
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = append((*d)[:0], s...)
return nil
}
case []byte:
switch d := dest.(type) {
case *string:
if d == nil {
return errNilPtr
}
*d = string(s)
return nil
case *interface{}:
if d == nil {
return errNilPtr
}
*d = cloneBytes(s)
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = cloneBytes(s)
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = s
return nil
}
case time.Time:
switch d := dest.(type) {
case *time.Time:
*d = s
return nil
case *string:
*d = s.Format(time.RFC3339Nano)
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = []byte(s.Format(time.RFC3339Nano))
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = s.AppendFormat((*d)[:0], time.RFC3339Nano)
return nil
}
case nil:
switch d := dest.(type) {
case *interface{}:
if d == nil {
return errNilPtr
}
*d = nil
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = nil
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = nil
return nil
}
}
var sv reflect.Value
switch d := dest.(type) {
case *string:
sv = reflect.ValueOf(src)
switch sv.Kind() {
case reflect.Bool,
reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64,
reflect.Float32, reflect.Float64:
*d = asString(src)
return nil
}
case *[]byte:
sv = reflect.ValueOf(src)
if b, ok := asBytes(nil, sv); ok {
*d = b
return nil
}
case *sql.RawBytes:
sv = reflect.ValueOf(src)
if b, ok := asBytes([]byte(*d)[:0], sv); ok {
*d = sql.RawBytes(b)
return nil
}
case *bool:
bv, err := driver.Bool.ConvertValue(src)
if err == nil {
*d = bv.(bool)
}
return err
case *interface{}:
*d = src
return nil
}
if scanner, ok := dest.(sql.Scanner); ok {
return scanner.Scan(src)
}
dpv := reflect.ValueOf(dest)
if dpv.Kind() != reflect.Ptr {
return errors.New("destination not a pointer")
}
if dpv.IsNil() {
return errNilPtr
}
if !sv.IsValid() {
sv = reflect.ValueOf(src)
}
dv := reflect.Indirect(dpv)
if sv.IsValid() && sv.Type().AssignableTo(dv.Type()) {
switch b := src.(type) {
case []byte:
dv.Set(reflect.ValueOf(cloneBytes(b)))
default:
dv.Set(sv)
}
return nil
}
if dv.Kind() == sv.Kind() && sv.Type().ConvertibleTo(dv.Type()) {
dv.Set(sv.Convert(dv.Type()))
return nil
}
// The following conversions use a string value as an intermediate representation
// to convert between various numeric types.
//
// This also allows scanning into user defined types such as "type Int int64".
// For symmetry, also check for string destination types.
switch dv.Kind() {
case reflect.Ptr:
if src == nil {
dv.Set(reflect.Zero(dv.Type()))
return nil
}
dv.Set(reflect.New(dv.Type().Elem()))
return convertAssign(dv.Interface(), src)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
s := asString(src)
i64, err := strconv.ParseInt(s, 10, dv.Type().Bits())
if err != nil {
err = strconvErr(err)
return fmt.Errorf("converting driver.Value type %T (%q) to a %s: %v", src, s, dv.Kind(), err)
}
dv.SetInt(i64)
return nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
s := asString(src)
u64, err := strconv.ParseUint(s, 10, dv.Type().Bits())
if err != nil {
err = strconvErr(err)
return fmt.Errorf("converting driver.Value type %T (%q) to a %s: %v", src, s, dv.Kind(), err)
}
dv.SetUint(u64)
return nil
case reflect.Float32, reflect.Float64:
s := asString(src)
f64, err := strconv.ParseFloat(s, dv.Type().Bits())
if err != nil {
err = strconvErr(err)
return fmt.Errorf("converting driver.Value type %T (%q) to a %s: %v", src, s, dv.Kind(), err)
}
dv.SetFloat(f64)
return nil
case reflect.String:
switch v := src.(type) {
case string:
dv.SetString(v)
return nil
case []byte:
dv.SetString(string(v))
return nil
}
}
return fmt.Errorf("unsupported Scan, storing driver.Value type %T into type %T", src, dest)
}
func strconvErr(err error) error {
if ne, ok := err.(*strconv.NumError); ok {
return ne.Err
}
return err
}
func cloneBytes(b []byte) []byte {
if b == nil {
return nil
}
c := make([]byte, len(b))
copy(c, b)
return c
}
func asString(src interface{}) string {
switch v := src.(type) {
case string:
return v
case []byte:
return string(v)
}
rv := reflect.ValueOf(src)
switch rv.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return strconv.FormatInt(rv.Int(), 10)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return strconv.FormatUint(rv.Uint(), 10)
case reflect.Float64:
return strconv.FormatFloat(rv.Float(), 'g', -1, 64)
case reflect.Float32:
return strconv.FormatFloat(rv.Float(), 'g', -1, 32)
case reflect.Bool:
return strconv.FormatBool(rv.Bool())
}
return fmt.Sprintf("%v", src)
}
func asBytes(buf []byte, rv reflect.Value) (b []byte, ok bool) {
switch rv.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return strconv.AppendInt(buf, rv.Int(), 10), true
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return strconv.AppendUint(buf, rv.Uint(), 10), true
case reflect.Float32:
return strconv.AppendFloat(buf, rv.Float(), 'g', -1, 32), true
case reflect.Float64:
return strconv.AppendFloat(buf, rv.Float(), 'g', -1, 64), true
case reflect.Bool:
return strconv.AppendBool(buf, rv.Bool()), true
case reflect.String:
s := rv.String()
return append(buf, s...), true
}
return
}

35
doc.go
View File

@ -79,9 +79,8 @@ Then, you can use this extension.
Connection Hook Connection Hook
You can hook and inject your code when the connection is established. database/sql You can hook and inject your code when the connection is established by setting
doesn't provide a way to get native go-sqlite3 interfaces. So if you want, ConnectHook to get the SQLiteConn.
you need to set ConnectHook and get the SQLiteConn.
sql.Register("sqlite3_with_hook_example", sql.Register("sqlite3_with_hook_example",
&sqlite3.SQLiteDriver{ &sqlite3.SQLiteDriver{
@ -91,21 +90,45 @@ you need to set ConnectHook and get the SQLiteConn.
}, },
}) })
You can also use database/sql.Conn.Raw (Go >= 1.13):
conn, err := db.Conn(context.Background())
// if err != nil { ... }
defer conn.Close()
err = conn.Raw(func (driverConn interface{}) error {
sqliteConn := driverConn.(*sqlite3.SQLiteConn)
// ... use sqliteConn
})
// if err != nil { ... }
Go SQlite3 Extensions Go SQlite3 Extensions
If you want to register Go functions as SQLite extension functions, If you want to register Go functions as SQLite extension functions
call RegisterFunction from ConnectHook. you can make a custom driver by calling RegisterFunction from
ConnectHook.
regex = func(re, s string) (bool, error) { regex = func(re, s string) (bool, error) {
return regexp.MatchString(re, s) return regexp.MatchString(re, s)
} }
sql.Register("sqlite3_with_go_func", sql.Register("sqlite3_extended",
&sqlite3.SQLiteDriver{ &sqlite3.SQLiteDriver{
ConnectHook: func(conn *sqlite3.SQLiteConn) error { ConnectHook: func(conn *sqlite3.SQLiteConn) error {
return conn.RegisterFunc("regexp", regex, true) return conn.RegisterFunc("regexp", regex, true)
}, },
}) })
You can then use the custom driver by passing its name to sql.Open.
var i int
conn, err := sql.Open("sqlite3_extended", "./foo.db")
if err != nil {
panic(err)
}
err = db.QueryRow(`SELECT regexp("foo.*", "seafood")`).Scan(&i)
if err != nil {
panic(err)
}
See the documentation of RegisterFunc for more details. See the documentation of RegisterFunc for more details.
*/ */

View File

@ -1,11 +1,19 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
package sqlite3 package sqlite3
/*
#ifndef USE_LIBSQLITE3
#include "sqlite3-binding.h"
#else
#include <sqlite3.h>
#endif
*/
import "C" import "C"
import "syscall"
// ErrNo inherit errno. // ErrNo inherit errno.
type ErrNo int type ErrNo int
@ -20,6 +28,7 @@ type ErrNoExtended int
type Error struct { type Error struct {
Code ErrNo /* The error code returned by SQLite */ Code ErrNo /* The error code returned by SQLite */
ExtendedCode ErrNoExtended /* The extended error code returned by SQLite */ ExtendedCode ErrNoExtended /* The extended error code returned by SQLite */
SystemErrno syscall.Errno /* The system errno returned by the OS through SQLite, if applicable */
err string /* The error string returned by sqlite3_errmsg(), err string /* The error string returned by sqlite3_errmsg(),
this usually contains more specific details. */ this usually contains more specific details. */
} }
@ -72,10 +81,16 @@ func (err ErrNoExtended) Error() string {
} }
func (err Error) Error() string { func (err Error) Error() string {
var str string
if err.err != "" { if err.err != "" {
return err.err str = err.err
} else {
str = C.GoString(C.sqlite3_errstr(C.int(err.Code)))
} }
return errorString(err) if err.SystemErrno != 0 {
str += ": " + err.SystemErrno.Error()
}
return str
} }
// result codes from http://www.sqlite.org/c3ref/c_abort_rollback.html // result codes from http://www.sqlite.org/c3ref/c_abort_rollback.html

View File

@ -1,8 +1,10 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build cgo
package sqlite3 package sqlite3
import ( import (
@ -238,5 +240,36 @@ func TestExtendedErrorCodes_Unique(t *testing.T) {
extended, expected) extended, expected)
} }
} }
}
func TestError_SystemErrno(t *testing.T) {
_, n, _ := Version()
if n < 3012000 {
t.Skip("sqlite3_system_errno requires sqlite3 >= 3.12.0")
}
// open a non-existent database in read-only mode so we get an IO error.
db, err := sql.Open("sqlite3", "file:nonexistent.db?mode=ro")
if err != nil {
t.Fatal(err)
}
defer db.Close()
err = db.Ping()
if err == nil {
t.Fatal("expected error pinging read-only non-existent database, but got nil")
}
serr, ok := err.(Error)
if !ok {
t.Fatalf("expected error to be of type Error, but got %[1]T %[1]v", err)
}
if serr.SystemErrno == 0 {
t.Fatal("expected SystemErrno to be set")
}
if !os.IsNotExist(serr.SystemErrno) {
t.Errorf("expected SystemErrno to be a not exists error, but got %v", serr.SystemErrno)
}
} }

7
go.mod Normal file
View File

@ -0,0 +1,7 @@
module github.com/mattn/go-sqlite3
go 1.16
retract (
[v2.0.0+incompatible, v2.0.6+incompatible] // Accidental; no major changes or features.
)

0
go.sum Normal file
View File

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -1,9 +1,10 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
//go:build cgo
// +build cgo // +build cgo
package sqlite3 package sqlite3
@ -15,16 +16,16 @@ package sqlite3
#cgo CFLAGS: -DHAVE_USLEEP=1 #cgo CFLAGS: -DHAVE_USLEEP=1
#cgo CFLAGS: -DSQLITE_ENABLE_FTS3 #cgo CFLAGS: -DSQLITE_ENABLE_FTS3
#cgo CFLAGS: -DSQLITE_ENABLE_FTS3_PARENTHESIS #cgo CFLAGS: -DSQLITE_ENABLE_FTS3_PARENTHESIS
#cgo CFLAGS: -DSQLITE_ENABLE_FTS4_UNICODE61
#cgo CFLAGS: -DSQLITE_TRACE_SIZE_LIMIT=15 #cgo CFLAGS: -DSQLITE_TRACE_SIZE_LIMIT=15
#cgo CFLAGS: -DSQLITE_OMIT_DEPRECATED #cgo CFLAGS: -DSQLITE_OMIT_DEPRECATED
#cgo CFLAGS: -DSQLITE_DISABLE_INTRINSIC
#cgo CFLAGS: -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1 #cgo CFLAGS: -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1
#cgo CFLAGS: -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT #cgo CFLAGS: -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT
#cgo CFLAGS: -Wno-deprecated-declarations #cgo CFLAGS: -Wno-deprecated-declarations
#cgo linux,!android CFLAGS: -DHAVE_PREAD64=1 -DHAVE_PWRITE64=1 #cgo linux,!android CFLAGS: -DHAVE_PREAD64=1 -DHAVE_PWRITE64=1
#cgo openbsd CFLAGS: -I/usr/local/include
#cgo openbsd LDFLAGS: -L/usr/local/lib
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
@ -183,6 +184,12 @@ static int _sqlite3_limit(sqlite3* db, int limitId, int newLimit) {
return sqlite3_limit(db, limitId, newLimit); return sqlite3_limit(db, limitId, newLimit);
#endif #endif
} }
#if SQLITE_VERSION_NUMBER < 3012000
static int sqlite3_system_errno(sqlite3 *db) {
return 0;
}
#endif
*/ */
import "C" import "C"
import ( import (
@ -198,6 +205,7 @@ import (
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
"syscall"
"time" "time"
"unsafe" "unsafe"
) )
@ -226,8 +234,14 @@ const (
columnTimestamp string = "timestamp" columnTimestamp string = "timestamp"
) )
// This variable can be replaced with -ldflags like below:
// go build -ldflags="-X 'github.com/mattn/go-sqlite3.driverName=my-sqlite3'"
var driverName = "sqlite3"
func init() { func init() {
sql.Register("sqlite3", &SQLiteDriver{}) if driverName != "" {
sql.Register(driverName, &SQLiteDriver{})
}
} }
// Version returns SQLite library version information. // Version returns SQLite library version information.
@ -283,6 +297,51 @@ const (
/*SQLITE_RECURSIVE = C.SQLITE_RECURSIVE*/ /*SQLITE_RECURSIVE = C.SQLITE_RECURSIVE*/
) )
// Standard File Control Opcodes
// See: https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html
const (
SQLITE_FCNTL_LOCKSTATE = int(1)
SQLITE_FCNTL_GET_LOCKPROXYFILE = int(2)
SQLITE_FCNTL_SET_LOCKPROXYFILE = int(3)
SQLITE_FCNTL_LAST_ERRNO = int(4)
SQLITE_FCNTL_SIZE_HINT = int(5)
SQLITE_FCNTL_CHUNK_SIZE = int(6)
SQLITE_FCNTL_FILE_POINTER = int(7)
SQLITE_FCNTL_SYNC_OMITTED = int(8)
SQLITE_FCNTL_WIN32_AV_RETRY = int(9)
SQLITE_FCNTL_PERSIST_WAL = int(10)
SQLITE_FCNTL_OVERWRITE = int(11)
SQLITE_FCNTL_VFSNAME = int(12)
SQLITE_FCNTL_POWERSAFE_OVERWRITE = int(13)
SQLITE_FCNTL_PRAGMA = int(14)
SQLITE_FCNTL_BUSYHANDLER = int(15)
SQLITE_FCNTL_TEMPFILENAME = int(16)
SQLITE_FCNTL_MMAP_SIZE = int(18)
SQLITE_FCNTL_TRACE = int(19)
SQLITE_FCNTL_HAS_MOVED = int(20)
SQLITE_FCNTL_SYNC = int(21)
SQLITE_FCNTL_COMMIT_PHASETWO = int(22)
SQLITE_FCNTL_WIN32_SET_HANDLE = int(23)
SQLITE_FCNTL_WAL_BLOCK = int(24)
SQLITE_FCNTL_ZIPVFS = int(25)
SQLITE_FCNTL_RBU = int(26)
SQLITE_FCNTL_VFS_POINTER = int(27)
SQLITE_FCNTL_JOURNAL_POINTER = int(28)
SQLITE_FCNTL_WIN32_GET_HANDLE = int(29)
SQLITE_FCNTL_PDB = int(30)
SQLITE_FCNTL_BEGIN_ATOMIC_WRITE = int(31)
SQLITE_FCNTL_COMMIT_ATOMIC_WRITE = int(32)
SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE = int(33)
SQLITE_FCNTL_LOCK_TIMEOUT = int(34)
SQLITE_FCNTL_DATA_VERSION = int(35)
SQLITE_FCNTL_SIZE_LIMIT = int(36)
SQLITE_FCNTL_CKPT_DONE = int(37)
SQLITE_FCNTL_RESERVE_BYTES = int(38)
SQLITE_FCNTL_CKPT_START = int(39)
SQLITE_FCNTL_EXTERNAL_READER = int(40)
SQLITE_FCNTL_CKSM_FILE = int(41)
)
// SQLiteDriver implements driver.Driver. // SQLiteDriver implements driver.Driver.
type SQLiteDriver struct { type SQLiteDriver struct {
Extensions []string Extensions []string
@ -328,7 +387,7 @@ type SQLiteRows struct {
decltype []string decltype []string
cls bool cls bool
closed bool closed bool
done chan struct{} ctx context.Context // no better alternative to pass context into Next() method
} }
type functionInfo struct { type functionInfo struct {
@ -435,10 +494,12 @@ func (ai *aggInfo) Done(ctx *C.sqlite3_context) {
// Commit transaction. // Commit transaction.
func (tx *SQLiteTx) Commit() error { func (tx *SQLiteTx) Commit() error {
_, err := tx.c.exec(context.Background(), "COMMIT", nil) _, err := tx.c.exec(context.Background(), "COMMIT", nil)
if err != nil && err.(Error).Code == C.SQLITE_BUSY { if err != nil {
// sqlite3 will leave the transaction open in this scenario. // sqlite3 may leave the transaction open in this scenario.
// However, database/sql considers the transaction complete once we // However, database/sql considers the transaction complete once we
// return from Commit() - we must clean up to honour its semantics. // return from Commit() - we must clean up to honour its semantics.
// We don't know if the ROLLBACK is strictly necessary, but according
// to sqlite's docs, there is no harm in calling ROLLBACK unnecessarily.
tx.c.exec(context.Background(), "ROLLBACK", nil) tx.c.exec(context.Background(), "ROLLBACK", nil)
} }
return err return err
@ -466,7 +527,7 @@ func (c *SQLiteConn) RegisterCollation(name string, cmp func(string, string) int
handle := newHandle(c, cmp) handle := newHandle(c, cmp)
cname := C.CString(name) cname := C.CString(name)
defer C.free(unsafe.Pointer(cname)) defer C.free(unsafe.Pointer(cname))
rv := C.sqlite3_create_collation(c.db, cname, C.SQLITE_UTF8, unsafe.Pointer(handle), (*[0]byte)(unsafe.Pointer(C.compareTrampoline))) rv := C.sqlite3_create_collation(c.db, cname, C.SQLITE_UTF8, handle, (*[0]byte)(unsafe.Pointer(C.compareTrampoline)))
if rv != C.SQLITE_OK { if rv != C.SQLITE_OK {
return c.lastError() return c.lastError()
} }
@ -484,7 +545,7 @@ func (c *SQLiteConn) RegisterCommitHook(callback func() int) {
if callback == nil { if callback == nil {
C.sqlite3_commit_hook(c.db, nil, nil) C.sqlite3_commit_hook(c.db, nil, nil)
} else { } else {
C.sqlite3_commit_hook(c.db, (*[0]byte)(C.commitHookTrampoline), unsafe.Pointer(newHandle(c, callback))) C.sqlite3_commit_hook(c.db, (*[0]byte)(C.commitHookTrampoline), newHandle(c, callback))
} }
} }
@ -497,7 +558,7 @@ func (c *SQLiteConn) RegisterRollbackHook(callback func()) {
if callback == nil { if callback == nil {
C.sqlite3_rollback_hook(c.db, nil, nil) C.sqlite3_rollback_hook(c.db, nil, nil)
} else { } else {
C.sqlite3_rollback_hook(c.db, (*[0]byte)(C.rollbackHookTrampoline), unsafe.Pointer(newHandle(c, callback))) C.sqlite3_rollback_hook(c.db, (*[0]byte)(C.rollbackHookTrampoline), newHandle(c, callback))
} }
} }
@ -514,7 +575,7 @@ func (c *SQLiteConn) RegisterUpdateHook(callback func(int, string, string, int64
if callback == nil { if callback == nil {
C.sqlite3_update_hook(c.db, nil, nil) C.sqlite3_update_hook(c.db, nil, nil)
} else { } else {
C.sqlite3_update_hook(c.db, (*[0]byte)(C.updateHookTrampoline), unsafe.Pointer(newHandle(c, callback))) C.sqlite3_update_hook(c.db, (*[0]byte)(C.updateHookTrampoline), newHandle(c, callback))
} }
} }
@ -528,7 +589,7 @@ func (c *SQLiteConn) RegisterAuthorizer(callback func(int, string, string, strin
if callback == nil { if callback == nil {
C.sqlite3_set_authorizer(c.db, nil, nil) C.sqlite3_set_authorizer(c.db, nil, nil)
} else { } else {
C.sqlite3_set_authorizer(c.db, (*[0]byte)(C.authorizerTrampoline), unsafe.Pointer(newHandle(c, callback))) C.sqlite3_set_authorizer(c.db, (*[0]byte)(C.authorizerTrampoline), newHandle(c, callback))
} }
} }
@ -609,8 +670,8 @@ func (c *SQLiteConn) RegisterFunc(name string, impl interface{}, pure bool) erro
return nil return nil
} }
func sqlite3CreateFunction(db *C.sqlite3, zFunctionName *C.char, nArg C.int, eTextRep C.int, pApp uintptr, xFunc unsafe.Pointer, xStep unsafe.Pointer, xFinal unsafe.Pointer) C.int { func sqlite3CreateFunction(db *C.sqlite3, zFunctionName *C.char, nArg C.int, eTextRep C.int, pApp unsafe.Pointer, xFunc unsafe.Pointer, xStep unsafe.Pointer, xFinal unsafe.Pointer) C.int {
return C._sqlite3_create_function(db, zFunctionName, nArg, eTextRep, C.uintptr_t(pApp), (*[0]byte)(xFunc), (*[0]byte)(xStep), (*[0]byte)(xFinal)) return C._sqlite3_create_function(db, zFunctionName, nArg, eTextRep, C.uintptr_t(uintptr(pApp)), (*[0]byte)(xFunc), (*[0]byte)(xStep), (*[0]byte)(xFinal))
} }
// RegisterAggregator makes a Go type available as a SQLite aggregation function. // RegisterAggregator makes a Go type available as a SQLite aggregation function.
@ -683,7 +744,7 @@ func (c *SQLiteConn) RegisterAggregator(name string, impl interface{}, pure bool
ai.stepArgConverters = append(ai.stepArgConverters, conv) ai.stepArgConverters = append(ai.stepArgConverters, conv)
} }
if step.IsVariadic() { if step.IsVariadic() {
conv, err := callbackArg(t.In(start + stepNArgs).Elem()) conv, err := callbackArg(step.In(start + stepNArgs).Elem())
if err != nil { if err != nil {
return err return err
} }
@ -749,23 +810,36 @@ func (c *SQLiteConn) lastError() error {
return lastError(c.db) return lastError(c.db)
} }
// Note: may be called with db == nil
func lastError(db *C.sqlite3) error { func lastError(db *C.sqlite3) error {
rv := C.sqlite3_errcode(db) rv := C.sqlite3_errcode(db) // returns SQLITE_NOMEM if db == nil
if rv == C.SQLITE_OK { if rv == C.SQLITE_OK {
return nil return nil
} }
extrv := C.sqlite3_extended_errcode(db) // returns SQLITE_NOMEM if db == nil
errStr := C.GoString(C.sqlite3_errmsg(db)) // returns "out of memory" if db == nil
// https://www.sqlite.org/c3ref/system_errno.html
// sqlite3_system_errno is only meaningful if the error code was SQLITE_CANTOPEN,
// or it was SQLITE_IOERR and the extended code was not SQLITE_IOERR_NOMEM
var systemErrno syscall.Errno
if rv == C.SQLITE_CANTOPEN || (rv == C.SQLITE_IOERR && extrv != C.SQLITE_IOERR_NOMEM) {
systemErrno = syscall.Errno(C.sqlite3_system_errno(db))
}
return Error{ return Error{
Code: ErrNo(rv), Code: ErrNo(rv),
ExtendedCode: ErrNoExtended(C.sqlite3_extended_errcode(db)), ExtendedCode: ErrNoExtended(extrv),
err: C.GoString(C.sqlite3_errmsg(db)), SystemErrno: systemErrno,
err: errStr,
} }
} }
// Exec implements Execer. // Exec implements Execer.
func (c *SQLiteConn) Exec(query string, args []driver.Value) (driver.Result, error) { func (c *SQLiteConn) Exec(query string, args []driver.Value) (driver.Result, error) {
list := make([]namedValue, len(args)) list := make([]driver.NamedValue, len(args))
for i, v := range args { for i, v := range args {
list[i] = namedValue{ list[i] = driver.NamedValue{
Ordinal: i + 1, Ordinal: i + 1,
Value: v, Value: v,
} }
@ -773,7 +847,7 @@ func (c *SQLiteConn) Exec(query string, args []driver.Value) (driver.Result, err
return c.exec(context.Background(), query, list) return c.exec(context.Background(), query, list)
} }
func (c *SQLiteConn) exec(ctx context.Context, query string, args []namedValue) (driver.Result, error) { func (c *SQLiteConn) exec(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {
start := 0 start := 0
for { for {
s, err := c.prepare(ctx, query) s, err := c.prepare(ctx, query)
@ -782,42 +856,49 @@ func (c *SQLiteConn) exec(ctx context.Context, query string, args []namedValue)
} }
var res driver.Result var res driver.Result
if s.(*SQLiteStmt).s != nil { if s.(*SQLiteStmt).s != nil {
stmtArgs := make([]driver.NamedValue, 0, len(args))
na := s.NumInput() na := s.NumInput()
if len(args) < na { if len(args)-start < na {
s.Close() s.Close()
return nil, fmt.Errorf("not enough args to execute query: want %d got %d", na, len(args)) return nil, fmt.Errorf("not enough args to execute query: want %d got %d", na, len(args))
} }
for i := 0; i < na; i++ { // consume the number of arguments used in the current
args[i].Ordinal -= start // statement and append all named arguments not
// contained therein
stmtArgs = append(stmtArgs, args[start:start+na]...)
for i := range args {
if (i < start || i >= na) && args[i].Name != "" {
stmtArgs = append(stmtArgs, args[i])
}
} }
res, err = s.(*SQLiteStmt).exec(ctx, args[:na]) for i := range stmtArgs {
stmtArgs[i].Ordinal = i + 1
}
res, err = s.(*SQLiteStmt).exec(ctx, stmtArgs)
if err != nil && err != driver.ErrSkip { if err != nil && err != driver.ErrSkip {
s.Close() s.Close()
return nil, err return nil, err
} }
args = args[na:]
start += na start += na
} }
tail := s.(*SQLiteStmt).t tail := s.(*SQLiteStmt).t
s.Close() s.Close()
if tail == "" { if tail == "" {
if res == nil {
// https://github.com/mattn/go-sqlite3/issues/963
res = &SQLiteResult{0, 0}
}
return res, nil return res, nil
} }
query = tail query = tail
} }
} }
type namedValue struct {
Name string
Ordinal int
Value driver.Value
}
// Query implements Queryer. // Query implements Queryer.
func (c *SQLiteConn) Query(query string, args []driver.Value) (driver.Rows, error) { func (c *SQLiteConn) Query(query string, args []driver.Value) (driver.Rows, error) {
list := make([]namedValue, len(args)) list := make([]driver.NamedValue, len(args))
for i, v := range args { for i, v := range args {
list[i] = namedValue{ list[i] = driver.NamedValue{
Ordinal: i + 1, Ordinal: i + 1,
Value: v, Value: v,
} }
@ -825,27 +906,36 @@ func (c *SQLiteConn) Query(query string, args []driver.Value) (driver.Rows, erro
return c.query(context.Background(), query, list) return c.query(context.Background(), query, list)
} }
func (c *SQLiteConn) query(ctx context.Context, query string, args []namedValue) (driver.Rows, error) { func (c *SQLiteConn) query(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {
start := 0 start := 0
for { for {
stmtArgs := make([]driver.NamedValue, 0, len(args))
s, err := c.prepare(ctx, query) s, err := c.prepare(ctx, query)
if err != nil { if err != nil {
return nil, err return nil, err
} }
s.(*SQLiteStmt).cls = true s.(*SQLiteStmt).cls = true
na := s.NumInput() na := s.NumInput()
if len(args) < na { if len(args)-start < na {
return nil, fmt.Errorf("not enough args to execute query: want %d got %d", na, len(args)) return nil, fmt.Errorf("not enough args to execute query: want %d got %d", na, len(args)-start)
} }
for i := 0; i < na; i++ { // consume the number of arguments used in the current
args[i].Ordinal -= start // statement and append all named arguments not contained
// therein
stmtArgs = append(stmtArgs, args[start:start+na]...)
for i := range args {
if (i < start || i >= na) && args[i].Name != "" {
stmtArgs = append(stmtArgs, args[i])
}
} }
rows, err := s.(*SQLiteStmt).query(ctx, args[:na]) for i := range stmtArgs {
stmtArgs[i].Ordinal = i + 1
}
rows, err := s.(*SQLiteStmt).query(ctx, stmtArgs)
if err != nil && err != driver.ErrSkip { if err != nil && err != driver.ErrSkip {
s.Close() s.Close()
return rows, err return rows, err
} }
args = args[na:]
start += na start += na
tail := s.(*SQLiteStmt).t tail := s.(*SQLiteStmt).t
if tail == "" { if tail == "" {
@ -869,10 +959,6 @@ func (c *SQLiteConn) begin(ctx context.Context) (driver.Tx, error) {
return &SQLiteTx{c}, nil return &SQLiteTx{c}, nil
} }
func errorString(err Error) string {
return C.GoString(C.sqlite3_errstr(C.int(err.Code)))
}
// Open database and return a new connection. // Open database and return a new connection.
// //
// A pragma can take either zero or one argument. // A pragma can take either zero or one argument.
@ -897,7 +983,7 @@ func errorString(err Error) string {
// - rwc // - rwc
// - memory // - memory
// //
// shared // cache
// SQLite Shared-Cache Mode // SQLite Shared-Cache Mode
// https://www.sqlite.org/sharedcache.html // https://www.sqlite.org/sharedcache.html
// Values: // Values:
@ -1000,13 +1086,15 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
deferForeignKeys := -1 deferForeignKeys := -1
foreignKeys := -1 foreignKeys := -1
ignoreCheckConstraints := -1 ignoreCheckConstraints := -1
journalMode := "DELETE" var journalMode string
lockingMode := "NORMAL" lockingMode := "NORMAL"
queryOnly := -1 queryOnly := -1
recursiveTriggers := -1 recursiveTriggers := -1
secureDelete := "DEFAULT" secureDelete := "DEFAULT"
synchronousMode := "NORMAL" synchronousMode := "NORMAL"
writableSchema := -1 writableSchema := -1
vfsName := ""
var cacheSize *int64
pos := strings.IndexRune(dsn, '?') pos := strings.IndexRune(dsn, '?')
if pos >= 1 { if pos >= 1 {
@ -1232,7 +1320,7 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
if _, ok := params["_locking"]; ok { if _, ok := params["_locking"]; ok {
pkey = "_locking" pkey = "_locking"
} }
if val := params.Get("_locking"); val != "" { if val := params.Get(pkey); val != "" {
switch strings.ToUpper(val) { switch strings.ToUpper(val) {
case "NORMAL", "EXCLUSIVE": case "NORMAL", "EXCLUSIVE":
lockingMode = strings.ToUpper(val) lockingMode = strings.ToUpper(val)
@ -1330,6 +1418,22 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
} }
} }
// Cache size (_cache_size)
//
// https://sqlite.org/pragma.html#pragma_cache_size
//
if val := params.Get("_cache_size"); val != "" {
iv, err := strconv.ParseInt(val, 10, 64)
if err != nil {
return nil, fmt.Errorf("Invalid _cache_size: %v: %v", val, err)
}
cacheSize = &iv
}
if val := params.Get("vfs"); val != "" {
vfsName = val
}
if !strings.HasPrefix(dsn, "file:") { if !strings.HasPrefix(dsn, "file:") {
dsn = dsn[:pos] dsn = dsn[:pos]
} }
@ -1338,25 +1442,27 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
var db *C.sqlite3 var db *C.sqlite3
name := C.CString(dsn) name := C.CString(dsn)
defer C.free(unsafe.Pointer(name)) defer C.free(unsafe.Pointer(name))
var vfs *C.char
if vfsName != "" {
vfs = C.CString(vfsName)
defer C.free(unsafe.Pointer(vfs))
}
rv := C._sqlite3_open_v2(name, &db, rv := C._sqlite3_open_v2(name, &db,
mutex|C.SQLITE_OPEN_READWRITE|C.SQLITE_OPEN_CREATE, mutex|C.SQLITE_OPEN_READWRITE|C.SQLITE_OPEN_CREATE,
nil) vfs)
if rv != 0 { if rv != 0 {
// Save off the error _before_ closing the database.
// This is safe even if db is nil.
err := lastError(db)
if db != nil { if db != nil {
C.sqlite3_close_v2(db) C.sqlite3_close_v2(db)
} }
return nil, Error{Code: ErrNo(rv)} return nil, err
} }
if db == nil { if db == nil {
return nil, errors.New("sqlite succeeded without returning a database") return nil, errors.New("sqlite succeeded without returning a database")
} }
rv = C.sqlite3_busy_timeout(db, C.int(busyTimeout))
if rv != C.SQLITE_OK {
C.sqlite3_close_v2(db)
return nil, Error{Code: ErrNo(rv)}
}
exec := func(s string) error { exec := func(s string) error {
cs := C.CString(s) cs := C.CString(s)
rv := C.sqlite3_exec(db, cs, nil, nil, nil) rv := C.sqlite3_exec(db, cs, nil, nil, nil)
@ -1367,6 +1473,12 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
return nil return nil
} }
// Busy timeout
if err := exec(fmt.Sprintf("PRAGMA busy_timeout = %d;", busyTimeout)); err != nil {
C.sqlite3_close_v2(db)
return nil, err
}
// USER AUTHENTICATION // USER AUTHENTICATION
// //
// User Authentication is always performed even when // User Authentication is always performed even when
@ -1522,10 +1634,10 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
// Before going any further, we need to check that the user // Before going any further, we need to check that the user
// has provided an username and password within the DSN. // has provided an username and password within the DSN.
// We are not allowed to continue. // We are not allowed to continue.
if len(authUser) < 0 { if len(authUser) == 0 {
return nil, fmt.Errorf("Missing '_auth_user' while user authentication was requested with '_auth'") return nil, fmt.Errorf("Missing '_auth_user' while user authentication was requested with '_auth'")
} }
if len(authPass) < 0 { if len(authPass) == 0 {
return nil, fmt.Errorf("Missing '_auth_pass' while user authentication was requested with '_auth'") return nil, fmt.Errorf("Missing '_auth_pass' while user authentication was requested with '_auth'")
} }
@ -1571,10 +1683,11 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
} }
// Journal Mode // Journal Mode
// Because default Journal Mode is DELETE this PRAGMA can always be executed. if journalMode != "" {
if err := exec(fmt.Sprintf("PRAGMA journal_mode = %s;", journalMode)); err != nil { if err := exec(fmt.Sprintf("PRAGMA journal_mode = %s;", journalMode)); err != nil {
C.sqlite3_close_v2(db) C.sqlite3_close_v2(db)
return nil, err return nil, err
}
} }
// Locking Mode // Locking Mode
@ -1617,7 +1730,7 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
// //
// Because default is NORMAL this statement is always executed // Because default is NORMAL this statement is always executed
if err := exec(fmt.Sprintf("PRAGMA synchronous = %s;", synchronousMode)); err != nil { if err := exec(fmt.Sprintf("PRAGMA synchronous = %s;", synchronousMode)); err != nil {
C.sqlite3_close_v2(db) conn.Close()
return nil, err return nil, err
} }
@ -1629,6 +1742,14 @@ func (d *SQLiteDriver) Open(dsn string) (driver.Conn, error) {
} }
} }
// Cache Size
if cacheSize != nil {
if err := exec(fmt.Sprintf("PRAGMA cache_size = %d;", *cacheSize)); err != nil {
C.sqlite3_close_v2(db)
return nil, err
}
}
if len(d.Extensions) > 0 { if len(d.Extensions) > 0 {
if err := conn.loadExtensions(d.Extensions); err != nil { if err := conn.loadExtensions(d.Extensions); err != nil {
conn.Close() conn.Close()
@ -1733,6 +1854,31 @@ func (c *SQLiteConn) SetLimit(id int, newVal int) int {
return int(C._sqlite3_limit(c.db, C.int(id), C.int(newVal))) return int(C._sqlite3_limit(c.db, C.int(id), C.int(newVal)))
} }
// SetFileControlInt invokes the xFileControl method on a given database. The
// dbName is the name of the database. It will default to "main" if left blank.
// The op is one of the opcodes prefixed by "SQLITE_FCNTL_". The arg argument
// and return code are both opcode-specific. Please see the SQLite documentation.
//
// This method is not thread-safe as the returned error code can be changed by
// another call if invoked concurrently.
//
// See: sqlite3_file_control, https://www.sqlite.org/c3ref/file_control.html
func (c *SQLiteConn) SetFileControlInt(dbName string, op int, arg int) error {
if dbName == "" {
dbName = "main"
}
cDBName := C.CString(dbName)
defer C.free(unsafe.Pointer(cDBName))
cArg := C.int(arg)
rv := C.sqlite3_file_control(c.db, cDBName, C.int(op), unsafe.Pointer(&cArg))
if rv != C.SQLITE_OK {
return c.lastError()
}
return nil
}
// Close the statement. // Close the statement.
func (s *SQLiteStmt) Close() error { func (s *SQLiteStmt) Close() error {
s.mu.Lock() s.mu.Lock()
@ -1758,65 +1904,71 @@ func (s *SQLiteStmt) NumInput() int {
return int(C.sqlite3_bind_parameter_count(s.s)) return int(C.sqlite3_bind_parameter_count(s.s))
} }
type bindArg struct {
n int
v driver.Value
}
var placeHolder = []byte{0} var placeHolder = []byte{0}
func (s *SQLiteStmt) bind(args []namedValue) error { func (s *SQLiteStmt) bind(args []driver.NamedValue) error {
rv := C.sqlite3_reset(s.s) rv := C.sqlite3_reset(s.s)
if rv != C.SQLITE_ROW && rv != C.SQLITE_OK && rv != C.SQLITE_DONE { if rv != C.SQLITE_ROW && rv != C.SQLITE_OK && rv != C.SQLITE_DONE {
return s.c.lastError() return s.c.lastError()
} }
bindIndices := make([][3]int, len(args))
prefixes := []string{":", "@", "$"}
for i, v := range args { for i, v := range args {
bindIndices[i][0] = args[i].Ordinal
if v.Name != "" { if v.Name != "" {
cname := C.CString(":" + v.Name) for j := range prefixes {
args[i].Ordinal = int(C.sqlite3_bind_parameter_index(s.s, cname)) cname := C.CString(prefixes[j] + v.Name)
C.free(unsafe.Pointer(cname)) bindIndices[i][j] = int(C.sqlite3_bind_parameter_index(s.s, cname))
C.free(unsafe.Pointer(cname))
}
args[i].Ordinal = bindIndices[i][0]
} }
} }
for _, arg := range args { for i, arg := range args {
n := C.int(arg.Ordinal) for j := range bindIndices[i] {
switch v := arg.Value.(type) { if bindIndices[i][j] == 0 {
case nil: continue
rv = C.sqlite3_bind_null(s.s, n) }
case string: n := C.int(bindIndices[i][j])
if len(v) == 0 { switch v := arg.Value.(type) {
rv = C._sqlite3_bind_text(s.s, n, (*C.char)(unsafe.Pointer(&placeHolder[0])), C.int(0)) case nil:
} else { rv = C.sqlite3_bind_null(s.s, n)
b := []byte(v) case string:
if len(v) == 0 {
rv = C._sqlite3_bind_text(s.s, n, (*C.char)(unsafe.Pointer(&placeHolder[0])), C.int(0))
} else {
b := []byte(v)
rv = C._sqlite3_bind_text(s.s, n, (*C.char)(unsafe.Pointer(&b[0])), C.int(len(b)))
}
case int64:
rv = C.sqlite3_bind_int64(s.s, n, C.sqlite3_int64(v))
case bool:
if v {
rv = C.sqlite3_bind_int(s.s, n, 1)
} else {
rv = C.sqlite3_bind_int(s.s, n, 0)
}
case float64:
rv = C.sqlite3_bind_double(s.s, n, C.double(v))
case []byte:
if v == nil {
rv = C.sqlite3_bind_null(s.s, n)
} else {
ln := len(v)
if ln == 0 {
v = placeHolder
}
rv = C._sqlite3_bind_blob(s.s, n, unsafe.Pointer(&v[0]), C.int(ln))
}
case time.Time:
b := []byte(v.Format(SQLiteTimestampFormats[0]))
rv = C._sqlite3_bind_text(s.s, n, (*C.char)(unsafe.Pointer(&b[0])), C.int(len(b))) rv = C._sqlite3_bind_text(s.s, n, (*C.char)(unsafe.Pointer(&b[0])), C.int(len(b)))
} }
case int64: if rv != C.SQLITE_OK {
rv = C.sqlite3_bind_int64(s.s, n, C.sqlite3_int64(v)) return s.c.lastError()
case bool:
if v {
rv = C.sqlite3_bind_int(s.s, n, 1)
} else {
rv = C.sqlite3_bind_int(s.s, n, 0)
} }
case float64:
rv = C.sqlite3_bind_double(s.s, n, C.double(v))
case []byte:
if v == nil {
rv = C.sqlite3_bind_null(s.s, n)
} else {
ln := len(v)
if ln == 0 {
v = placeHolder
}
rv = C._sqlite3_bind_blob(s.s, n, unsafe.Pointer(&v[0]), C.int(ln))
}
case time.Time:
b := []byte(v.Format(SQLiteTimestampFormats[0]))
rv = C._sqlite3_bind_text(s.s, n, (*C.char)(unsafe.Pointer(&b[0])), C.int(len(b)))
}
if rv != C.SQLITE_OK {
return s.c.lastError()
} }
} }
return nil return nil
@ -1824,9 +1976,9 @@ func (s *SQLiteStmt) bind(args []namedValue) error {
// Query the statement with arguments. Return records. // Query the statement with arguments. Return records.
func (s *SQLiteStmt) Query(args []driver.Value) (driver.Rows, error) { func (s *SQLiteStmt) Query(args []driver.Value) (driver.Rows, error) {
list := make([]namedValue, len(args)) list := make([]driver.NamedValue, len(args))
for i, v := range args { for i, v := range args {
list[i] = namedValue{ list[i] = driver.NamedValue{
Ordinal: i + 1, Ordinal: i + 1,
Value: v, Value: v,
} }
@ -1834,7 +1986,7 @@ func (s *SQLiteStmt) Query(args []driver.Value) (driver.Rows, error) {
return s.query(context.Background(), list) return s.query(context.Background(), list)
} }
func (s *SQLiteStmt) query(ctx context.Context, args []namedValue) (driver.Rows, error) { func (s *SQLiteStmt) query(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
if err := s.bind(args); err != nil { if err := s.bind(args); err != nil {
return nil, err return nil, err
} }
@ -1846,28 +1998,13 @@ func (s *SQLiteStmt) query(ctx context.Context, args []namedValue) (driver.Rows,
decltype: nil, decltype: nil,
cls: s.cls, cls: s.cls,
closed: false, closed: false,
done: make(chan struct{}), ctx: ctx,
}
if ctxdone := ctx.Done(); ctxdone != nil {
go func(db *C.sqlite3) {
select {
case <-ctxdone:
select {
case <-rows.done:
default:
C.sqlite3_interrupt(db)
rows.Close()
}
case <-rows.done:
}
}(s.c.db)
} }
return rows, nil return rows, nil
} }
// LastInsertId teturn last inserted ID. // LastInsertId return last inserted ID.
func (r *SQLiteResult) LastInsertId() (int64, error) { func (r *SQLiteResult) LastInsertId() (int64, error) {
return r.id, nil return r.id, nil
} }
@ -1879,9 +2016,9 @@ func (r *SQLiteResult) RowsAffected() (int64, error) {
// Exec execute the statement with arguments. Return result object. // Exec execute the statement with arguments. Return result object.
func (s *SQLiteStmt) Exec(args []driver.Value) (driver.Result, error) { func (s *SQLiteStmt) Exec(args []driver.Value) (driver.Result, error) {
list := make([]namedValue, len(args)) list := make([]driver.NamedValue, len(args))
for i, v := range args { for i, v := range args {
list[i] = namedValue{ list[i] = driver.NamedValue{
Ordinal: i + 1, Ordinal: i + 1,
Value: v, Value: v,
} }
@ -1889,29 +2026,54 @@ func (s *SQLiteStmt) Exec(args []driver.Value) (driver.Result, error) {
return s.exec(context.Background(), list) return s.exec(context.Background(), list)
} }
func (s *SQLiteStmt) exec(ctx context.Context, args []namedValue) (driver.Result, error) { func isInterruptErr(err error) bool {
sqliteErr, ok := err.(Error)
if ok {
return sqliteErr.Code == ErrInterrupt
}
return false
}
// exec executes a query that doesn't return rows. Attempts to honor context timeout.
func (s *SQLiteStmt) exec(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
if ctx.Done() == nil {
return s.execSync(args)
}
type result struct {
r driver.Result
err error
}
resultCh := make(chan result)
go func() {
r, err := s.execSync(args)
resultCh <- result{r, err}
}()
var rv result
select {
case rv = <-resultCh:
case <-ctx.Done():
select {
case rv = <-resultCh: // no need to interrupt, operation completed in db
default:
// this is still racy and can be no-op if executed between sqlite3_* calls in execSync.
C.sqlite3_interrupt(s.c.db)
rv = <-resultCh // wait for goroutine completed
if isInterruptErr(rv.err) {
return nil, ctx.Err()
}
}
}
return rv.r, rv.err
}
func (s *SQLiteStmt) execSync(args []driver.NamedValue) (driver.Result, error) {
if err := s.bind(args); err != nil { if err := s.bind(args); err != nil {
C.sqlite3_reset(s.s) C.sqlite3_reset(s.s)
C.sqlite3_clear_bindings(s.s) C.sqlite3_clear_bindings(s.s)
return nil, err return nil, err
} }
if ctxdone := ctx.Done(); ctxdone != nil {
done := make(chan struct{})
defer close(done)
go func(db *C.sqlite3) {
select {
case <-done:
case <-ctxdone:
select {
case <-done:
default:
C.sqlite3_interrupt(db)
}
}
}(s.c.db)
}
var rowid, changes C.longlong var rowid, changes C.longlong
rv := C._sqlite3_step_row_internal(s.s, &rowid, &changes) rv := C._sqlite3_step_row_internal(s.s, &rowid, &changes)
if rv != C.SQLITE_ROW && rv != C.SQLITE_OK && rv != C.SQLITE_DONE { if rv != C.SQLITE_ROW && rv != C.SQLITE_OK && rv != C.SQLITE_DONE {
@ -1924,6 +2086,13 @@ func (s *SQLiteStmt) exec(ctx context.Context, args []namedValue) (driver.Result
return &SQLiteResult{id: int64(rowid), changes: int64(changes)}, nil return &SQLiteResult{id: int64(rowid), changes: int64(changes)}, nil
} }
// Readonly reports if this statement is considered readonly by SQLite.
//
// See: https://sqlite.org/c3ref/stmt_readonly.html
func (s *SQLiteStmt) Readonly() bool {
return C.sqlite3_stmt_readonly(s.s) == 1
}
// Close the rows. // Close the rows.
func (rc *SQLiteRows) Close() error { func (rc *SQLiteRows) Close() error {
rc.s.mu.Lock() rc.s.mu.Lock()
@ -1932,9 +2101,6 @@ func (rc *SQLiteRows) Close() error {
return nil return nil
} }
rc.closed = true rc.closed = true
if rc.done != nil {
close(rc.done)
}
if rc.cls { if rc.cls {
rc.s.mu.Unlock() rc.s.mu.Unlock()
return rc.s.Close() return rc.s.Close()
@ -1978,13 +2144,39 @@ func (rc *SQLiteRows) DeclTypes() []string {
return rc.declTypes() return rc.declTypes()
} }
// Next move cursor to next. // Next move cursor to next. Attempts to honor context timeout from QueryContext call.
func (rc *SQLiteRows) Next(dest []driver.Value) error { func (rc *SQLiteRows) Next(dest []driver.Value) error {
rc.s.mu.Lock() rc.s.mu.Lock()
defer rc.s.mu.Unlock() defer rc.s.mu.Unlock()
if rc.s.closed { if rc.s.closed {
return io.EOF return io.EOF
} }
if rc.ctx.Done() == nil {
return rc.nextSyncLocked(dest)
}
resultCh := make(chan error)
go func() {
resultCh <- rc.nextSyncLocked(dest)
}()
select {
case err := <-resultCh:
return err
case <-rc.ctx.Done():
select {
case <-resultCh: // no need to interrupt
default:
// this is still racy and can be no-op if executed between sqlite3_* calls in nextSyncLocked.
C.sqlite3_interrupt(rc.s.c.db)
<-resultCh // ensure goroutine completed
}
return rc.ctx.Err()
}
}
// nextSyncLocked moves cursor to next; must be called with locked mutex.
func (rc *SQLiteRows) nextSyncLocked(dest []driver.Value) error {
rv := C._sqlite3_step_internal(rc.s.s) rv := C._sqlite3_step_internal(rc.s.s)
if rv == C.SQLITE_DONE { if rv == C.SQLITE_DONE {
return io.EOF return io.EOF

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -8,7 +8,7 @@ package sqlite3
/* /*
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif

119
sqlite3_go113_test.go Normal file
View File

@ -0,0 +1,119 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build go1.13,cgo
package sqlite3
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"os"
"testing"
)
func TestBeginTxCancel(t *testing.T) {
srcTempFilename := TempFilename(t)
defer os.Remove(srcTempFilename)
db, err := sql.Open("sqlite3", srcTempFilename)
if err != nil {
t.Fatal(err)
}
db.SetMaxOpenConns(10)
db.SetMaxIdleConns(5)
defer db.Close()
initDatabase(t, db, 100)
// create several go-routines to expose racy issue
for i := 0; i < 1000; i++ {
func() {
ctx, cancel := context.WithCancel(context.Background())
conn, err := db.Conn(ctx)
if err != nil {
t.Fatal(err)
}
defer func() {
if err := conn.Close(); err != nil {
t.Error(err)
}
}()
err = conn.Raw(func(driverConn interface{}) error {
d, ok := driverConn.(driver.ConnBeginTx)
if !ok {
t.Fatal("unexpected: wrong type")
}
// checks that conn.Raw can be used to get *SQLiteConn
if _, ok = driverConn.(*SQLiteConn); !ok {
t.Fatalf("conn.Raw() driverConn type=%T, expected *SQLiteConn", driverConn)
}
go cancel() // make it cancel concurrently with exec("BEGIN");
tx, err := d.BeginTx(ctx, driver.TxOptions{})
switch err {
case nil:
switch err := tx.Rollback(); err {
case nil, sql.ErrTxDone:
default:
return err
}
case context.Canceled:
default:
// must not fail with "cannot start a transaction within a transaction"
return err
}
return nil
})
if err != nil {
t.Fatal(err)
}
}()
}
}
func TestStmtReadonly(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatal(err)
}
_, err = db.Exec("CREATE TABLE t (count INT)")
if err != nil {
t.Fatal(err)
}
isRO := func(query string) bool {
c, err := db.Conn(context.Background())
if err != nil {
return false
}
var ro bool
c.Raw(func(dc interface{}) error {
stmt, err := dc.(*SQLiteConn).Prepare(query)
if err != nil {
return err
}
if stmt == nil {
return errors.New("stmt is nil")
}
ro = stmt.(*SQLiteStmt).Readonly()
return nil
})
return ro // On errors ro will remain false.
}
if !isRO(`select * from t`) {
t.Error("select not seen as read-only")
}
if isRO(`insert into t values (1), (2)`) {
t.Error("insert seen as read-only")
}
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -10,7 +10,6 @@ package sqlite3
import ( import (
"database/sql/driver" "database/sql/driver"
"errors"
"context" "context"
) )
@ -18,27 +17,20 @@ import (
// Ping implement Pinger. // Ping implement Pinger.
func (c *SQLiteConn) Ping(ctx context.Context) error { func (c *SQLiteConn) Ping(ctx context.Context) error {
if c.db == nil { if c.db == nil {
return errors.New("Connection was closed") // must be ErrBadConn for sql to close the database
return driver.ErrBadConn
} }
return nil return nil
} }
// QueryContext implement QueryerContext. // QueryContext implement QueryerContext.
func (c *SQLiteConn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) { func (c *SQLiteConn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {
list := make([]namedValue, len(args)) return c.query(ctx, query, args)
for i, nv := range args {
list[i] = namedValue(nv)
}
return c.query(ctx, query, list)
} }
// ExecContext implement ExecerContext. // ExecContext implement ExecerContext.
func (c *SQLiteConn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) { func (c *SQLiteConn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {
list := make([]namedValue, len(args)) return c.exec(ctx, query, args)
for i, nv := range args {
list[i] = namedValue(nv)
}
return c.exec(ctx, query, list)
} }
// PrepareContext implement ConnPrepareContext. // PrepareContext implement ConnPrepareContext.
@ -53,18 +45,10 @@ func (c *SQLiteConn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver
// QueryContext implement QueryerContext. // QueryContext implement QueryerContext.
func (s *SQLiteStmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) { func (s *SQLiteStmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
list := make([]namedValue, len(args)) return s.query(ctx, args)
for i, nv := range args {
list[i] = namedValue(nv)
}
return s.query(ctx, list)
} }
// ExecContext implement ExecerContext. // ExecContext implement ExecerContext.
func (s *SQLiteStmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) { func (s *SQLiteStmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
list := make([]namedValue, len(args)) return s.exec(ctx, args)
for i, nv := range args {
list[i] = namedValue(nv)
}
return s.exec(ctx, list)
} }

View File

@ -1,9 +1,9 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build go1.8 // +build go1.8,cgo
package sqlite3 package sqlite3
@ -11,8 +11,10 @@ import (
"context" "context"
"database/sql" "database/sql"
"fmt" "fmt"
"io/ioutil"
"math/rand" "math/rand"
"os" "os"
"sync"
"testing" "testing"
"time" "time"
) )
@ -134,6 +136,137 @@ func TestShortTimeout(t *testing.T) {
} }
} }
func TestExecContextCancel(t *testing.T) {
srcTempFilename := TempFilename(t)
defer os.Remove(srcTempFilename)
db, err := sql.Open("sqlite3", srcTempFilename)
if err != nil {
t.Fatal(err)
}
defer db.Close()
ts := time.Now()
initDatabase(t, db, 1000)
spent := time.Since(ts)
const minTestTime = 100 * time.Millisecond
if spent < minTestTime {
t.Skipf("test will be too racy (spent=%s < min=%s) as ExecContext below will be too fast.",
spent.String(), minTestTime.String(),
)
}
// expected to be extremely slow query
q := `
INSERT INTO test_table (key1, key_id, key2, key3, key4, key5, key6, data)
SELECT t1.key1 || t2.key1, t1.key_id || t2.key_id, t1.key2 || t2.key2, t1.key3 || t2.key3, t1.key4 || t2.key4, t1.key5 || t2.key5, t1.key6 || t2.key6, t1.data || t2.data
FROM test_table t1 LEFT OUTER JOIN test_table t2`
// expect query above take ~ same time as setup above
// This is racy: the context must be valid so sql/db.ExecContext calls the sqlite3 driver.
// It starts the query, the context expires, then calls sqlite3_interrupt
ctx, cancel := context.WithTimeout(context.Background(), minTestTime/2)
defer cancel()
ts = time.Now()
r, err := db.ExecContext(ctx, q)
// racy check
if r != nil {
n, err := r.RowsAffected()
t.Logf("query should not have succeeded: rows=%d; err=%v; duration=%s",
n, err, time.Since(ts).String())
}
if err != context.DeadlineExceeded {
t.Fatal(err, ctx.Err())
}
}
func TestQueryRowContextCancel(t *testing.T) {
srcTempFilename := TempFilename(t)
defer os.Remove(srcTempFilename)
db, err := sql.Open("sqlite3", srcTempFilename)
if err != nil {
t.Fatal(err)
}
defer db.Close()
initDatabase(t, db, 100)
const query = `SELECT key_id FROM test_table ORDER BY key2 ASC`
var keyID string
unexpectedErrors := make(map[string]int)
for i := 0; i < 10000; i++ {
ctx, cancel := context.WithCancel(context.Background())
row := db.QueryRowContext(ctx, query)
cancel()
// it is fine to get "nil" as context cancellation can be handled with delay
if err := row.Scan(&keyID); err != nil && err != context.Canceled {
if err.Error() == "sql: Rows are closed" {
// see https://github.com/golang/go/issues/24431
// fixed in 1.11.1 to properly return context error
continue
}
unexpectedErrors[err.Error()]++
}
}
for errText, count := range unexpectedErrors {
t.Error(errText, count)
}
}
func TestQueryRowContextCancelParallel(t *testing.T) {
srcTempFilename := TempFilename(t)
defer os.Remove(srcTempFilename)
db, err := sql.Open("sqlite3", srcTempFilename)
if err != nil {
t.Fatal(err)
}
db.SetMaxOpenConns(10)
db.SetMaxIdleConns(5)
defer db.Close()
initDatabase(t, db, 100)
const query = `SELECT key_id FROM test_table ORDER BY key2 ASC`
wg := sync.WaitGroup{}
defer wg.Wait()
testCtx, cancel := context.WithCancel(context.Background())
defer cancel()
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
var keyID string
for {
select {
case <-testCtx.Done():
return
default:
}
ctx, cancel := context.WithCancel(context.Background())
row := db.QueryRowContext(ctx, query)
cancel()
_ = row.Scan(&keyID) // see TestQueryRowContextCancel
}
}()
}
var keyID string
for i := 0; i < 10000; i++ {
// note that testCtx is not cancelled during query execution
row := db.QueryRowContext(testCtx, query)
if err := row.Scan(&keyID); err != nil {
t.Fatal(i, err)
}
}
}
func TestExecCancel(t *testing.T) { func TestExecCancel(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:") db, err := sql.Open("sqlite3", ":memory:")
if err != nil { if err != nil {
@ -154,3 +287,217 @@ func TestExecCancel(t *testing.T) {
} }
} }
} }
func doTestOpenContext(t *testing.T, option string) (string, error) {
tempFilename := TempFilename(t)
url := tempFilename + option
defer func() {
err := os.Remove(tempFilename)
if err != nil {
t.Error("temp file remove error:", err)
}
}()
db, err := sql.Open("sqlite3", url)
if err != nil {
return "Failed to open database:", err
}
defer func() {
err = db.Close()
if err != nil {
t.Error("db close error:", err)
}
}()
ctx, cancel := context.WithTimeout(context.Background(), 55*time.Second)
err = db.PingContext(ctx)
cancel()
if err != nil {
return "ping error:", err
}
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "drop table foo")
cancel()
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "create table foo (id integer)")
cancel()
if err != nil {
return "Failed to create table:", err
}
if stat, err := os.Stat(tempFilename); err != nil || stat.IsDir() {
return "Failed to create ./foo.db", nil
}
return "", nil
}
func TestOpenContext(t *testing.T) {
cases := map[string]bool{
"": true,
"?_txlock=immediate": true,
"?_txlock=deferred": true,
"?_txlock=exclusive": true,
"?_txlock=bogus": false,
}
for option, expectedPass := range cases {
result, err := doTestOpenContext(t, option)
if result == "" {
if !expectedPass {
errmsg := fmt.Sprintf("_txlock error not caught at dbOpen with option: %s", option)
t.Fatal(errmsg)
}
} else if expectedPass {
if err == nil {
t.Fatal(result)
} else {
t.Fatal(result, err)
}
}
}
}
func TestFileCopyTruncate(t *testing.T) {
var err error
tempFilename := TempFilename(t)
defer func() {
err = os.Remove(tempFilename)
if err != nil {
t.Error("temp file remove error:", err)
}
}()
db, err := sql.Open("sqlite3", tempFilename)
if err != nil {
t.Fatal("open error:", err)
}
defer func() {
err = db.Close()
if err != nil {
t.Error("db close error:", err)
}
}()
ctx, cancel := context.WithTimeout(context.Background(), 55*time.Second)
err = db.PingContext(ctx)
cancel()
if err != nil {
t.Fatal("ping error:", err)
}
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "drop table foo")
cancel()
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "create table foo (id integer)")
cancel()
if err != nil {
t.Fatal("create table error:", err)
}
// copy db to new file
var data []byte
data, err = ioutil.ReadFile(tempFilename)
if err != nil {
t.Fatal("read file error:", err)
}
var f *os.File
f, err = os.Create(tempFilename + "-db-copy")
if err != nil {
t.Fatal("create file error:", err)
}
defer func() {
err = os.Remove(tempFilename + "-db-copy")
if err != nil {
t.Error("temp file moved remove error:", err)
}
}()
_, err = f.Write(data)
if err != nil {
f.Close()
t.Fatal("write file error:", err)
}
err = f.Close()
if err != nil {
t.Fatal("close file error:", err)
}
// truncate current db file
f, err = os.OpenFile(tempFilename, os.O_WRONLY|os.O_TRUNC, 0666)
if err != nil {
t.Fatal("open file error:", err)
}
err = f.Close()
if err != nil {
t.Fatal("close file error:", err)
}
// test db after file truncate
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
err = db.PingContext(ctx)
cancel()
if err != nil {
t.Fatal("ping error:", err)
}
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "drop table foo")
cancel()
if err == nil {
t.Fatal("drop table no error")
}
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "create table foo (id integer)")
cancel()
if err != nil {
t.Fatal("create table error:", err)
}
err = db.Close()
if err != nil {
t.Error("db close error:", err)
}
// test copied file
db, err = sql.Open("sqlite3", tempFilename+"-db-copy")
if err != nil {
t.Fatal("open error:", err)
}
defer func() {
err = db.Close()
if err != nil {
t.Error("db close error:", err)
}
}()
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
err = db.PingContext(ctx)
cancel()
if err != nil {
t.Fatal("ping error:", err)
}
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "drop table foo")
cancel()
if err != nil {
t.Fatal("drop table error:", err)
}
ctx, cancel = context.WithTimeout(context.Background(), 55*time.Second)
_, err = db.ExecContext(ctx, "create table foo (id integer)")
cancel()
if err != nil {
t.Fatal("create table error:", err)
}
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -11,7 +11,9 @@ package sqlite3
#cgo CFLAGS: -DUSE_LIBSQLITE3 #cgo CFLAGS: -DUSE_LIBSQLITE3
#cgo linux LDFLAGS: -lsqlite3 #cgo linux LDFLAGS: -lsqlite3
#cgo darwin LDFLAGS: -L/usr/local/opt/sqlite/lib -lsqlite3 #cgo darwin LDFLAGS: -L/usr/local/opt/sqlite/lib -lsqlite3
#cgo darwin CFLAGS: -I/usr/local/opt/sqlite/include
#cgo openbsd LDFLAGS: -lsqlite3 #cgo openbsd LDFLAGS: -lsqlite3
#cgo solaris LDFLAGS: -lsqlite3 #cgo solaris LDFLAGS: -lsqlite3
#cgo windows LDFLAGS: -lsqlite3
*/ */
import "C" import "C"

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -9,7 +9,7 @@ package sqlite3
/* /*
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
@ -28,12 +28,9 @@ func (c *SQLiteConn) loadExtensions(extensions []string) error {
} }
for _, extension := range extensions { for _, extension := range extensions {
cext := C.CString(extension) if err := c.loadExtension(extension, nil); err != nil {
defer C.free(unsafe.Pointer(cext))
rv = C.sqlite3_load_extension(c.db, cext, nil, nil)
if rv != C.SQLITE_OK {
C.sqlite3_enable_load_extension(c.db, 0) C.sqlite3_enable_load_extension(c.db, 0)
return errors.New(C.GoString(C.sqlite3_errmsg(c.db))) return err
} }
} }
@ -41,6 +38,7 @@ func (c *SQLiteConn) loadExtensions(extensions []string) error {
if rv != C.SQLITE_OK { if rv != C.SQLITE_OK {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db))) return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
} }
return nil return nil
} }
@ -51,14 +49,9 @@ func (c *SQLiteConn) LoadExtension(lib string, entry string) error {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db))) return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
} }
clib := C.CString(lib) if err := c.loadExtension(lib, &entry); err != nil {
defer C.free(unsafe.Pointer(clib)) C.sqlite3_enable_load_extension(c.db, 0)
centry := C.CString(entry) return err
defer C.free(unsafe.Pointer(centry))
rv = C.sqlite3_load_extension(c.db, clib, centry, nil)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
} }
rv = C.sqlite3_enable_load_extension(c.db, 0) rv = C.sqlite3_enable_load_extension(c.db, 0)
@ -68,3 +61,24 @@ func (c *SQLiteConn) LoadExtension(lib string, entry string) error {
return nil return nil
} }
func (c *SQLiteConn) loadExtension(lib string, entry *string) error {
clib := C.CString(lib)
defer C.free(unsafe.Pointer(clib))
var centry *C.char
if entry != nil {
centry = C.CString(*entry)
defer C.free(unsafe.Pointer(centry))
}
var errMsg *C.char
defer C.sqlite3_free(unsafe.Pointer(errMsg))
rv := C.sqlite3_load_extension(c.db, clib, centry, &errMsg)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(errMsg))
}
return nil
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -0,0 +1,63 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !sqlite_omit_load_extension
package sqlite3
import (
"database/sql"
"testing"
)
func TestExtensionsError(t *testing.T) {
sql.Register("sqlite3_TestExtensionsError",
&SQLiteDriver{
Extensions: []string{
"foobar",
},
},
)
db, err := sql.Open("sqlite3_TestExtensionsError", ":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
err = db.Ping()
if err == nil {
t.Fatal("expected error loading non-existent extension")
}
if err.Error() == "not an error" {
t.Fatal("expected error from sqlite3_enable_load_extension to be returned")
}
}
func TestLoadExtensionError(t *testing.T) {
sql.Register("sqlite3_TestLoadExtensionError",
&SQLiteDriver{
ConnectHook: func(c *SQLiteConn) error {
return c.LoadExtension("foobar", "")
},
},
)
db, err := sql.Open("sqlite3_TestLoadExtensionError", ":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
err = db.Ping()
if err == nil {
t.Fatal("expected error loading non-existent extension")
}
if err.Error() == "not an error" {
t.Fatal("expected error from sqlite3_enable_load_extension to be returned")
}
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -1,6 +1,6 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -0,0 +1,21 @@
// +build sqlite_column_metadata
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#cgo CFLAGS: -DSQLITE_ENABLE_COLUMN_METADATA
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
*/
import "C"
// ColumnTableName returns the table that is the origin of a particular result
// column in a SELECT statement.
//
// See https://www.sqlite.org/c3ref/column_database_name.html
func (s *SQLiteStmt) ColumnTableName(n int) string {
return C.GoString(C.sqlite3_column_table_name(s.s, C.int(n)))
}

View File

@ -0,0 +1,39 @@
// +build sqlite_column_metadata
package sqlite3
import "testing"
func TestColumnTableName(t *testing.T) {
d := SQLiteDriver{}
conn, err := d.Open(":memory:")
if err != nil {
t.Fatal("failed to get database connection:", err)
}
defer conn.Close()
sqlite3conn := conn.(*SQLiteConn)
_, err = sqlite3conn.Exec(`CREATE TABLE foo (name string)`, nil)
if err != nil {
t.Fatal("Failed to create table:", err)
}
_, err = sqlite3conn.Exec(`CREATE TABLE bar (name string)`, nil)
if err != nil {
t.Fatal("Failed to create table:", err)
}
stmt, err := sqlite3conn.Prepare(`SELECT * FROM foo JOIN bar ON foo.name = bar.name`)
if err != nil {
t.Fatal(err)
}
if exp, got := "foo", stmt.(*SQLiteStmt).ColumnTableName(0); exp != got {
t.Fatalf("Incorrect table name returned expected: %s, got: %s", exp, got)
}
if exp, got := "bar", stmt.(*SQLiteStmt).ColumnTableName(1); exp != got {
t.Fatalf("Incorrect table name returned expected: %s, got: %s", exp, got)
}
if exp, got := "", stmt.(*SQLiteStmt).ColumnTableName(2); exp != got {
t.Fatalf("Incorrect table name returned expected: %s, got: %s", exp, got)
}
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -1,8 +1,10 @@
// Copyright (C) 2015 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
// +build cgo
package sqlite3 package sqlite3
import ( import (

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -1,13 +0,0 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_json sqlite_json1 json1
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_JSON1
*/
import "C"

View File

@ -0,0 +1,14 @@
// Copyright (C) 2022 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_math_functions
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_MATH_FUNCTIONS
#cgo LDFLAGS: -lm
*/
import "C"

View File

@ -0,0 +1,29 @@
// +build sqlite_math_functions
package sqlite3
import (
"database/sql"
"testing"
)
func TestMathFunctions(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
queries := []string{
`SELECT acos(1)`,
`SELECT log(10, 100)`,
`SELECT power(2, 2)`,
}
for _, query := range queries {
var result float64
if err := db.QueryRow(query).Scan(&result); err != nil {
t.Errorf("invoking math function query %q: %v", query, err)
}
}
}

15
sqlite3_opt_os_trace.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2022 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
//go:build sqlite_os_trace
// +build sqlite_os_trace
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_FORCE_OS_TRACE=1
#cgo CFLAGS: -DSQLITE_DEBUG_OS_TRACE=1
*/
import "C"

20
sqlite3_opt_preupdate.go Normal file
View File

@ -0,0 +1,20 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build cgo
package sqlite3
// SQLitePreUpdateData represents all of the data available during a
// pre-update hook call.
type SQLitePreUpdateData struct {
Conn *SQLiteConn
Op int
DatabaseName string
TableName string
OldRowID int64
NewRowID int64
}

View File

@ -0,0 +1,112 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_preupdate_hook
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_PREUPDATE_HOOK
#cgo LDFLAGS: -lm
#ifndef USE_LIBSQLITE3
#include "sqlite3-binding.h"
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
#include <string.h>
void preUpdateHookTrampoline(void*, sqlite3 *, int, char *, char *, sqlite3_int64, sqlite3_int64);
*/
import "C"
import (
"errors"
"unsafe"
)
// RegisterPreUpdateHook sets the pre-update hook for a connection.
//
// The callback is passed a SQLitePreUpdateData struct with the data for
// the update, as well as methods for fetching copies of impacted data.
//
// If there is an existing preupdate hook for this connection, it will be
// removed. If callback is nil the existing hook (if any) will be removed
// without creating a new one.
func (c *SQLiteConn) RegisterPreUpdateHook(callback func(SQLitePreUpdateData)) {
if callback == nil {
C.sqlite3_preupdate_hook(c.db, nil, nil)
} else {
C.sqlite3_preupdate_hook(c.db, (*[0]byte)(unsafe.Pointer(C.preUpdateHookTrampoline)), unsafe.Pointer(newHandle(c, callback)))
}
}
// Depth returns the source path of the write, see sqlite3_preupdate_depth()
func (d *SQLitePreUpdateData) Depth() int {
return int(C.sqlite3_preupdate_depth(d.Conn.db))
}
// Count returns the number of columns in the row
func (d *SQLitePreUpdateData) Count() int {
return int(C.sqlite3_preupdate_count(d.Conn.db))
}
func (d *SQLitePreUpdateData) row(dest []interface{}, new bool) error {
for i := 0; i < d.Count() && i < len(dest); i++ {
var val *C.sqlite3_value
var src interface{}
// Initially I tried making this just a function pointer argument, but
// it's absurdly complicated to pass C function pointers.
if new {
C.sqlite3_preupdate_new(d.Conn.db, C.int(i), &val)
} else {
C.sqlite3_preupdate_old(d.Conn.db, C.int(i), &val)
}
switch C.sqlite3_value_type(val) {
case C.SQLITE_INTEGER:
src = int64(C.sqlite3_value_int64(val))
case C.SQLITE_FLOAT:
src = float64(C.sqlite3_value_double(val))
case C.SQLITE_BLOB:
len := C.sqlite3_value_bytes(val)
blobptr := C.sqlite3_value_blob(val)
src = C.GoBytes(blobptr, len)
case C.SQLITE_TEXT:
len := C.sqlite3_value_bytes(val)
cstrptr := unsafe.Pointer(C.sqlite3_value_text(val))
src = C.GoBytes(cstrptr, len)
case C.SQLITE_NULL:
src = nil
}
err := convertAssign(&dest[i], src)
if err != nil {
return err
}
}
return nil
}
// Old populates dest with the row data to be replaced. This works similar to
// database/sql's Rows.Scan()
func (d *SQLitePreUpdateData) Old(dest ...interface{}) error {
if d.Op == SQLITE_INSERT {
return errors.New("There is no old row for INSERT operations")
}
return d.row(dest, false)
}
// New populates dest with the replacement row data. This works similar to
// database/sql's Rows.Scan()
func (d *SQLitePreUpdateData) New(dest ...interface{}) error {
if d.Op == SQLITE_DELETE {
return errors.New("There is no new row for DELETE operations")
}
return d.row(dest, true)
}

View File

@ -0,0 +1,128 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_preupdate_hook
package sqlite3
import (
"database/sql"
"testing"
)
type preUpdateHookDataForTest struct {
databaseName string
tableName string
count int
op int
oldRow []interface{}
newRow []interface{}
}
func TestPreUpdateHook(t *testing.T) {
var events []preUpdateHookDataForTest
sql.Register("sqlite3_PreUpdateHook", &SQLiteDriver{
ConnectHook: func(conn *SQLiteConn) error {
conn.RegisterPreUpdateHook(func(data SQLitePreUpdateData) {
eval := -1
oldRow := []interface{}{eval}
if data.Op != SQLITE_INSERT {
err := data.Old(oldRow...)
if err != nil {
t.Fatalf("Unexpected error calling SQLitePreUpdateData.Old: %v", err)
}
}
eval2 := -1
newRow := []interface{}{eval2}
if data.Op != SQLITE_DELETE {
err := data.New(newRow...)
if err != nil {
t.Fatalf("Unexpected error calling SQLitePreUpdateData.New: %v", err)
}
}
// tests dest bound checks in loop
var tooSmallRow []interface{}
if data.Op != SQLITE_INSERT {
err := data.Old(tooSmallRow...)
if err != nil {
t.Fatalf("Unexpected error calling SQLitePreUpdateData.Old: %v", err)
}
if len(tooSmallRow) != 0 {
t.Errorf("Expected tooSmallRow to be empty, got: %v", tooSmallRow)
}
}
events = append(events, preUpdateHookDataForTest{
databaseName: data.DatabaseName,
tableName: data.TableName,
count: data.Count(),
op: data.Op,
oldRow: oldRow,
newRow: newRow,
})
})
return nil
},
})
db, err := sql.Open("sqlite3_PreUpdateHook", ":memory:")
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
statements := []string{
"create table foo (id integer primary key)",
"insert into foo values (9)",
"update foo set id = 99 where id = 9",
"delete from foo where id = 99",
}
for _, statement := range statements {
_, err = db.Exec(statement)
if err != nil {
t.Fatalf("Unable to prepare test data [%v]: %v", statement, err)
}
}
if len(events) != 3 {
t.Errorf("Events should be 3 entries, got: %d", len(events))
}
if events[0].op != SQLITE_INSERT {
t.Errorf("Op isn't as expected: %v", events[0].op)
}
if events[1].op != SQLITE_UPDATE {
t.Errorf("Op isn't as expected: %v", events[1].op)
}
if events[1].count != 1 {
t.Errorf("Expected event row 1 to have 1 column, had: %v", events[1].count)
}
newRow_0_0 := events[0].newRow[0].(int64)
if newRow_0_0 != 9 {
t.Errorf("Expected event row 0 new column 0 to be == 9, got: %v", newRow_0_0)
}
oldRow_1_0 := events[1].oldRow[0].(int64)
if oldRow_1_0 != 9 {
t.Errorf("Expected event row 1 old column 0 to be == 9, got: %v", oldRow_1_0)
}
newRow_1_0 := events[1].newRow[0].(int64)
if newRow_1_0 != 99 {
t.Errorf("Expected event row 1 new column 0 to be == 99, got: %v", newRow_1_0)
}
oldRow_2_0 := events[2].oldRow[0].(int64)
if oldRow_2_0 != 99 {
t.Errorf("Expected event row 1 new column 0 to be == 99, got: %v", oldRow_2_0)
}
}

View File

@ -0,0 +1,21 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !sqlite_preupdate_hook,cgo
package sqlite3
// RegisterPreUpdateHook sets the pre-update hook for a connection.
//
// The callback is passed a SQLitePreUpdateData struct with the data for
// the update, as well as methods for fetching copies of impacted data.
//
// If there is an existing preupdate hook for this connection, it will be
// removed. If callback is nil the existing hook (if any) will be removed
// without creating a new one.
func (c *SQLiteConn) RegisterPreUpdateHook(callback func(SQLitePreUpdateData)) {
// NOOP
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

82
sqlite3_opt_serialize.go Normal file
View File

@ -0,0 +1,82 @@
// +build !libsqlite3 sqlite_serialize
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
#include <stdint.h>
*/
import "C"
import (
"fmt"
"math"
"reflect"
"unsafe"
)
// Serialize returns a byte slice that is a serialization of the database.
//
// See https://www.sqlite.org/c3ref/serialize.html
func (c *SQLiteConn) Serialize(schema string) ([]byte, error) {
if schema == "" {
schema = "main"
}
var zSchema *C.char
zSchema = C.CString(schema)
defer C.free(unsafe.Pointer(zSchema))
var sz C.sqlite3_int64
ptr := C.sqlite3_serialize(c.db, zSchema, &sz, 0)
if ptr == nil {
return nil, fmt.Errorf("serialize failed")
}
defer C.sqlite3_free(unsafe.Pointer(ptr))
if sz > C.sqlite3_int64(math.MaxInt) {
return nil, fmt.Errorf("serialized database is too large (%d bytes)", sz)
}
cBuf := *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
Data: uintptr(unsafe.Pointer(ptr)),
Len: int(sz),
Cap: int(sz),
}))
res := make([]byte, int(sz))
copy(res, cBuf)
return res, nil
}
// Deserialize causes the connection to disconnect from the current database and
// then re-open as an in-memory database based on the contents of the byte slice.
//
// See https://www.sqlite.org/c3ref/deserialize.html
func (c *SQLiteConn) Deserialize(b []byte, schema string) error {
if schema == "" {
schema = "main"
}
var zSchema *C.char
zSchema = C.CString(schema)
defer C.free(unsafe.Pointer(zSchema))
tmpBuf := (*C.uchar)(C.sqlite3_malloc64(C.sqlite3_uint64(len(b))))
cBuf := *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
Data: uintptr(unsafe.Pointer(tmpBuf)),
Len: len(b),
Cap: len(b),
}))
copy(cBuf, b)
rc := C.sqlite3_deserialize(c.db, zSchema, tmpBuf, C.sqlite3_int64(len(b)),
C.sqlite3_int64(len(b)), C.SQLITE_DESERIALIZE_FREEONCLOSE)
if rc != C.SQLITE_OK {
return fmt.Errorf("deserialize failed with return %v", rc)
}
return nil
}

View File

@ -0,0 +1,20 @@
// +build libsqlite3,!sqlite_serialize
package sqlite3
import (
"errors"
)
/*
#cgo CFLAGS: -DSQLITE_OMIT_DESERIALIZE
*/
import "C"
func (c *SQLiteConn) Serialize(schema string) ([]byte, error) {
return nil, errors.New("sqlite3: Serialize requires the sqlite_serialize build tag when using the libsqlite3 build tag")
}
func (c *SQLiteConn) Deserialize(b []byte, schema string) error {
return errors.New("sqlite3: Deserialize requires the sqlite_serialize build tag when using the libsqlite3 build tag")
}

View File

@ -0,0 +1,99 @@
// +build !libsqlite3 sqlite_serialize
package sqlite3
import (
"context"
"database/sql"
"os"
"testing"
)
func TestSerializeDeserialize(t *testing.T) {
// Connect to the source database.
srcTempFilename := TempFilename(t)
defer os.Remove(srcTempFilename)
srcDb, err := sql.Open(driverName, srcTempFilename)
if err != nil {
t.Fatal("Failed to open the source database:", err)
}
defer srcDb.Close()
err = srcDb.Ping()
if err != nil {
t.Fatal("Failed to connect to the source database:", err)
}
// Connect to the destination database.
destTempFilename := TempFilename(t)
defer os.Remove(destTempFilename)
destDb, err := sql.Open(driverName, destTempFilename)
if err != nil {
t.Fatal("Failed to open the destination database:", err)
}
defer destDb.Close()
err = destDb.Ping()
if err != nil {
t.Fatal("Failed to connect to the destination database:", err)
}
// Write data to source database.
_, err = srcDb.Exec(`CREATE TABLE foo (name string)`)
if err != nil {
t.Fatal("Failed to create table in source database:", err)
}
_, err = srcDb.Exec(`INSERT INTO foo(name) VALUES("alice")`)
if err != nil {
t.Fatal("Failed to insert data into source database", err)
}
// Serialize the source database
srcConn, err := srcDb.Conn(context.Background())
if err != nil {
t.Fatal("Failed to get connection to source database:", err)
}
defer srcConn.Close()
var serialized []byte
if err := srcConn.Raw(func(raw interface{}) error {
var err error
serialized, err = raw.(*SQLiteConn).Serialize("")
return err
}); err != nil {
t.Fatal("Failed to serialize source database:", err)
}
srcConn.Close()
// Confirm that the destination database is initially empty.
var destTableCount int
err = destDb.QueryRow("SELECT COUNT(*) FROM sqlite_master WHERE type = 'table'").Scan(&destTableCount)
if err != nil {
t.Fatal("Failed to check the destination table count:", err)
}
if destTableCount != 0 {
t.Fatalf("The destination database is not empty; %v table(s) found.", destTableCount)
}
// Deserialize to destination database
destConn, err := destDb.Conn(context.Background())
if err != nil {
t.Fatal("Failed to get connection to destination database:", err)
}
defer destConn.Close()
if err := destConn.Raw(func(raw interface{}) error {
return raw.(*SQLiteConn).Deserialize(serialized, "")
}); err != nil {
t.Fatal("Failed to deserialize source database:", err)
}
destConn.Close()
// Confirm that destination database has been loaded correctly.
var destRowCount int
err = destDb.QueryRow(`SELECT COUNT(*) FROM foo`).Scan(&destRowCount)
if err != nil {
t.Fatal("Failed to count rows in destination database table", err)
}
if destRowCount != 1 {
t.Fatalf("Destination table does not have the expected records")
}
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -5,7 +5,7 @@
#ifdef SQLITE_ENABLE_UNLOCK_NOTIFY #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY
#include <stdio.h> #include <stdio.h>
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
extern int unlock_notify_wait(sqlite3 *db); extern int unlock_notify_wait(sqlite3 *db);

View File

@ -1,4 +1,4 @@
// Copyright (C) 2018 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -12,7 +12,7 @@ package sqlite3
#cgo CFLAGS: -DSQLITE_ENABLE_UNLOCK_NOTIFY #cgo CFLAGS: -DSQLITE_ENABLE_UNLOCK_NOTIFY
#include <stdlib.h> #include <stdlib.h>
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
extern void unlock_notify_callback(void *arg, int argc); extern void unlock_notify_callback(void *arg, int argc);
*/ */

View File

@ -1,4 +1,4 @@
// Copyright (C) 2018 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -11,7 +11,7 @@ package sqlite3
#cgo CFLAGS: -DSQLITE_USER_AUTHENTICATION #cgo CFLAGS: -DSQLITE_USER_AUTHENTICATION
#cgo LDFLAGS: -lm #cgo LDFLAGS: -lm
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif

View File

@ -60,7 +60,14 @@ func init() {
file = TempFilename(t) file = TempFilename(t)
} }
db, err = sql.Open("sqlite3_with_conn", "file:"+file+fmt.Sprintf("?_auth&_auth_user=%s&_auth_pass=%s", username, password)) params := "?_auth"
if len(username) > 0 {
params = fmt.Sprintf("%s&_auth_user=%s", params, username)
}
if len(password) > 0 {
params = fmt.Sprintf("%s&_auth_pass=%s", params, password)
}
db, err = sql.Open("sqlite3_with_conn", "file:"+file+params)
if err != nil { if err != nil {
defer os.Remove(file) defer os.Remove(file)
return file, nil, nil, err return file, nil, nil, err
@ -164,6 +171,23 @@ func TestUserAuthCreateDatabase(t *testing.T) {
} }
} }
func TestUserAuthCreateDatabaseWithoutArgs(t *testing.T) {
_, db, c, err := connect(t, "", "", "")
if err == nil && c == nil && db == nil {
t.Fatal("Should have failed due to missing _auth_* parameters")
}
_, db, c, err = connect(t, "", "", "admin")
if err == nil && c == nil && db == nil {
t.Fatal("Should have failed due to missing _auth_user parameter")
}
_, db, c, err = connect(t, "", "admin", "")
if err == nil && c == nil && db == nil {
t.Fatal("Should have failed due to missing _auth_pass parameter")
}
}
func TestUserAuthLogin(t *testing.T) { func TestUserAuthLogin(t *testing.T) {
f1, err := create(t, "admin", "admin") f1, err := create(t, "admin", "admin")
if err != nil { if err != nil {

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>. // Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -19,7 +19,7 @@ package sqlite3
#cgo CFLAGS: -Wno-deprecated-declarations #cgo CFLAGS: -Wno-deprecated-declarations
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
@ -226,11 +226,43 @@ static sqlite3_module goModule = {
0 // xRollbackTo 0 // xRollbackTo
}; };
// See https://sqlite.org/vtab.html#eponymous_only_virtual_tables
static sqlite3_module goModuleEponymousOnly = {
0, // iVersion
0, // xCreate - create a table, which here is null
cXConnect, // xConnect - connect to an existing table
cXBestIndex, // xBestIndex - Determine search strategy
cXDisconnect, // xDisconnect - Disconnect from a table
cXDestroy, // xDestroy - Drop a table
cXOpen, // xOpen - open a cursor
cXClose, // xClose - close a cursor
cXFilter, // xFilter - configure scan constraints
cXNext, // xNext - advance a cursor
cXEof, // xEof
cXColumn, // xColumn - read data
cXRowid, // xRowid - read data
cXUpdate, // xUpdate - write data
// Not implemented
0, // xBegin - begin transaction
0, // xSync - sync transaction
0, // xCommit - commit transaction
0, // xRollback - rollback transaction
0, // xFindFunction - function overloading
0, // xRename - rename the table
0, // xSavepoint
0, // xRelease
0 // xRollbackTo
};
void goMDestroy(void*); void goMDestroy(void*);
static int _sqlite3_create_module(sqlite3 *db, const char *zName, uintptr_t pClientData) { static int _sqlite3_create_module(sqlite3 *db, const char *zName, uintptr_t pClientData) {
return sqlite3_create_module_v2(db, zName, &goModule, (void*) pClientData, goMDestroy); return sqlite3_create_module_v2(db, zName, &goModule, (void*) pClientData, goMDestroy);
} }
static int _sqlite3_create_module_eponymous_only(sqlite3 *db, const char *zName, uintptr_t pClientData) {
return sqlite3_create_module_v2(db, zName, &goModuleEponymousOnly, (void*) pClientData, goMDestroy);
}
*/ */
import "C" import "C"
@ -288,10 +320,13 @@ type InfoOrderBy struct {
} }
func constraints(info *C.sqlite3_index_info) []InfoConstraint { func constraints(info *C.sqlite3_index_info) []InfoConstraint {
l := info.nConstraint slice := *(*[]C.struct_sqlite3_index_constraint)(unsafe.Pointer(&reflect.SliceHeader{
slice := (*[1 << 30]C.struct_sqlite3_index_constraint)(unsafe.Pointer(info.aConstraint))[:l:l] Data: uintptr(unsafe.Pointer(info.aConstraint)),
Len: int(info.nConstraint),
Cap: int(info.nConstraint),
}))
cst := make([]InfoConstraint, 0, l) cst := make([]InfoConstraint, 0, len(slice))
for _, c := range slice { for _, c := range slice {
var usable bool var usable bool
if c.usable > 0 { if c.usable > 0 {
@ -307,10 +342,13 @@ func constraints(info *C.sqlite3_index_info) []InfoConstraint {
} }
func orderBys(info *C.sqlite3_index_info) []InfoOrderBy { func orderBys(info *C.sqlite3_index_info) []InfoOrderBy {
l := info.nOrderBy slice := *(*[]C.struct_sqlite3_index_orderby)(unsafe.Pointer(&reflect.SliceHeader{
slice := (*[1 << 30]C.struct_sqlite3_index_orderby)(unsafe.Pointer(info.aOrderBy))[:l:l] Data: uintptr(unsafe.Pointer(info.aOrderBy)),
Len: int(info.nOrderBy),
Cap: int(info.nOrderBy),
}))
ob := make([]InfoOrderBy, 0, l) ob := make([]InfoOrderBy, 0, len(slice))
for _, c := range slice { for _, c := range slice {
var desc bool var desc bool
if c.desc > 0 { if c.desc > 0 {
@ -347,7 +385,7 @@ func mPrintf(format, arg string) *C.char {
//export goMInit //export goMInit
func goMInit(db, pClientData unsafe.Pointer, argc C.int, argv **C.char, pzErr **C.char, isCreate C.int) C.uintptr_t { func goMInit(db, pClientData unsafe.Pointer, argc C.int, argv **C.char, pzErr **C.char, isCreate C.int) C.uintptr_t {
m := lookupHandle(uintptr(pClientData)).(*sqliteModule) m := lookupHandle(pClientData).(*sqliteModule)
if m.c.db != (*C.sqlite3)(db) { if m.c.db != (*C.sqlite3)(db) {
*pzErr = mPrintf("%s", "Inconsistent db handles") *pzErr = mPrintf("%s", "Inconsistent db handles")
return 0 return 0
@ -373,12 +411,12 @@ func goMInit(db, pClientData unsafe.Pointer, argc C.int, argv **C.char, pzErr **
} }
vt := sqliteVTab{m, vTab} vt := sqliteVTab{m, vTab}
*pzErr = nil *pzErr = nil
return C.uintptr_t(newHandle(m.c, &vt)) return C.uintptr_t(uintptr(newHandle(m.c, &vt)))
} }
//export goVRelease //export goVRelease
func goVRelease(pVTab unsafe.Pointer, isDestroy C.int) *C.char { func goVRelease(pVTab unsafe.Pointer, isDestroy C.int) *C.char {
vt := lookupHandle(uintptr(pVTab)).(*sqliteVTab) vt := lookupHandle(pVTab).(*sqliteVTab)
var err error var err error
if isDestroy == 1 { if isDestroy == 1 {
err = vt.vTab.Destroy() err = vt.vTab.Destroy()
@ -393,7 +431,7 @@ func goVRelease(pVTab unsafe.Pointer, isDestroy C.int) *C.char {
//export goVOpen //export goVOpen
func goVOpen(pVTab unsafe.Pointer, pzErr **C.char) C.uintptr_t { func goVOpen(pVTab unsafe.Pointer, pzErr **C.char) C.uintptr_t {
vt := lookupHandle(uintptr(pVTab)).(*sqliteVTab) vt := lookupHandle(pVTab).(*sqliteVTab)
vTabCursor, err := vt.vTab.Open() vTabCursor, err := vt.vTab.Open()
if err != nil { if err != nil {
*pzErr = mPrintf("%s", err.Error()) *pzErr = mPrintf("%s", err.Error())
@ -401,12 +439,12 @@ func goVOpen(pVTab unsafe.Pointer, pzErr **C.char) C.uintptr_t {
} }
vtc := sqliteVTabCursor{vt, vTabCursor} vtc := sqliteVTabCursor{vt, vTabCursor}
*pzErr = nil *pzErr = nil
return C.uintptr_t(newHandle(vt.module.c, &vtc)) return C.uintptr_t(uintptr(newHandle(vt.module.c, &vtc)))
} }
//export goVBestIndex //export goVBestIndex
func goVBestIndex(pVTab unsafe.Pointer, icp unsafe.Pointer) *C.char { func goVBestIndex(pVTab unsafe.Pointer, icp unsafe.Pointer) *C.char {
vt := lookupHandle(uintptr(pVTab)).(*sqliteVTab) vt := lookupHandle(pVTab).(*sqliteVTab)
info := (*C.sqlite3_index_info)(icp) info := (*C.sqlite3_index_info)(icp)
csts := constraints(info) csts := constraints(info)
res, err := vt.vTab.BestIndex(csts, orderBys(info)) res, err := vt.vTab.BestIndex(csts, orderBys(info))
@ -418,22 +456,37 @@ func goVBestIndex(pVTab unsafe.Pointer, icp unsafe.Pointer) *C.char {
} }
// Get a pointer to constraint_usage struct so we can update in place. // Get a pointer to constraint_usage struct so we can update in place.
l := info.nConstraint
s := (*[1 << 30]C.struct_sqlite3_index_constraint_usage)(unsafe.Pointer(info.aConstraintUsage))[:l:l] slice := *(*[]C.struct_sqlite3_index_constraint_usage)(unsafe.Pointer(&reflect.SliceHeader{
Data: uintptr(unsafe.Pointer(info.aConstraintUsage)),
Len: int(info.nConstraint),
Cap: int(info.nConstraint),
}))
index := 1 index := 1
for i := C.int(0); i < info.nConstraint; i++ { for i := range slice {
if res.Used[i] { if res.Used[i] {
s[i].argvIndex = C.int(index) slice[i].argvIndex = C.int(index)
s[i].omit = C.uchar(1) slice[i].omit = C.uchar(1)
index++ index++
} }
} }
info.idxNum = C.int(res.IdxNum) info.idxNum = C.int(res.IdxNum)
idxStr := C.CString(res.IdxStr) info.idxStr = (*C.char)(C.sqlite3_malloc(C.int(len(res.IdxStr) + 1)))
defer C.free(unsafe.Pointer(idxStr)) if info.idxStr == nil {
info.idxStr = idxStr // C.malloc and C.CString ordinarily do this for you. See https://golang.org/cmd/cgo/
info.needToFreeIdxStr = C.int(0) panic("out of memory")
}
info.needToFreeIdxStr = C.int(1)
idxStr := *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
Data: uintptr(unsafe.Pointer(info.idxStr)),
Len: len(res.IdxStr) + 1,
Cap: len(res.IdxStr) + 1,
}))
copy(idxStr, res.IdxStr)
idxStr[len(idxStr)-1] = 0 // null-terminated string
if res.AlreadyOrdered { if res.AlreadyOrdered {
info.orderByConsumed = C.int(1) info.orderByConsumed = C.int(1)
} }
@ -445,7 +498,7 @@ func goVBestIndex(pVTab unsafe.Pointer, icp unsafe.Pointer) *C.char {
//export goVClose //export goVClose
func goVClose(pCursor unsafe.Pointer) *C.char { func goVClose(pCursor unsafe.Pointer) *C.char {
vtc := lookupHandle(uintptr(pCursor)).(*sqliteVTabCursor) vtc := lookupHandle(pCursor).(*sqliteVTabCursor)
err := vtc.vTabCursor.Close() err := vtc.vTabCursor.Close()
if err != nil { if err != nil {
return mPrintf("%s", err.Error()) return mPrintf("%s", err.Error())
@ -455,13 +508,13 @@ func goVClose(pCursor unsafe.Pointer) *C.char {
//export goMDestroy //export goMDestroy
func goMDestroy(pClientData unsafe.Pointer) { func goMDestroy(pClientData unsafe.Pointer) {
m := lookupHandle(uintptr(pClientData)).(*sqliteModule) m := lookupHandle(pClientData).(*sqliteModule)
m.module.DestroyModule() m.module.DestroyModule()
} }
//export goVFilter //export goVFilter
func goVFilter(pCursor unsafe.Pointer, idxNum C.int, idxName *C.char, argc C.int, argv **C.sqlite3_value) *C.char { func goVFilter(pCursor unsafe.Pointer, idxNum C.int, idxName *C.char, argc C.int, argv **C.sqlite3_value) *C.char {
vtc := lookupHandle(uintptr(pCursor)).(*sqliteVTabCursor) vtc := lookupHandle(pCursor).(*sqliteVTabCursor)
args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:argc:argc] args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:argc:argc]
vals := make([]interface{}, 0, argc) vals := make([]interface{}, 0, argc)
for _, v := range args { for _, v := range args {
@ -480,7 +533,7 @@ func goVFilter(pCursor unsafe.Pointer, idxNum C.int, idxName *C.char, argc C.int
//export goVNext //export goVNext
func goVNext(pCursor unsafe.Pointer) *C.char { func goVNext(pCursor unsafe.Pointer) *C.char {
vtc := lookupHandle(uintptr(pCursor)).(*sqliteVTabCursor) vtc := lookupHandle(pCursor).(*sqliteVTabCursor)
err := vtc.vTabCursor.Next() err := vtc.vTabCursor.Next()
if err != nil { if err != nil {
return mPrintf("%s", err.Error()) return mPrintf("%s", err.Error())
@ -490,7 +543,7 @@ func goVNext(pCursor unsafe.Pointer) *C.char {
//export goVEof //export goVEof
func goVEof(pCursor unsafe.Pointer) C.int { func goVEof(pCursor unsafe.Pointer) C.int {
vtc := lookupHandle(uintptr(pCursor)).(*sqliteVTabCursor) vtc := lookupHandle(pCursor).(*sqliteVTabCursor)
err := vtc.vTabCursor.EOF() err := vtc.vTabCursor.EOF()
if err { if err {
return 1 return 1
@ -500,7 +553,7 @@ func goVEof(pCursor unsafe.Pointer) C.int {
//export goVColumn //export goVColumn
func goVColumn(pCursor, cp unsafe.Pointer, col C.int) *C.char { func goVColumn(pCursor, cp unsafe.Pointer, col C.int) *C.char {
vtc := lookupHandle(uintptr(pCursor)).(*sqliteVTabCursor) vtc := lookupHandle(pCursor).(*sqliteVTabCursor)
c := (*SQLiteContext)(cp) c := (*SQLiteContext)(cp)
err := vtc.vTabCursor.Column(c, int(col)) err := vtc.vTabCursor.Column(c, int(col))
if err != nil { if err != nil {
@ -511,7 +564,7 @@ func goVColumn(pCursor, cp unsafe.Pointer, col C.int) *C.char {
//export goVRowid //export goVRowid
func goVRowid(pCursor unsafe.Pointer, pRowid *C.sqlite3_int64) *C.char { func goVRowid(pCursor unsafe.Pointer, pRowid *C.sqlite3_int64) *C.char {
vtc := lookupHandle(uintptr(pCursor)).(*sqliteVTabCursor) vtc := lookupHandle(pCursor).(*sqliteVTabCursor)
rowid, err := vtc.vTabCursor.Rowid() rowid, err := vtc.vTabCursor.Rowid()
if err != nil { if err != nil {
return mPrintf("%s", err.Error()) return mPrintf("%s", err.Error())
@ -522,7 +575,7 @@ func goVRowid(pCursor unsafe.Pointer, pRowid *C.sqlite3_int64) *C.char {
//export goVUpdate //export goVUpdate
func goVUpdate(pVTab unsafe.Pointer, argc C.int, argv **C.sqlite3_value, pRowid *C.sqlite3_int64) *C.char { func goVUpdate(pVTab unsafe.Pointer, argc C.int, argv **C.sqlite3_value, pRowid *C.sqlite3_int64) *C.char {
vt := lookupHandle(uintptr(pVTab)).(*sqliteVTab) vt := lookupHandle(pVTab).(*sqliteVTab)
var tname string var tname string
if n, ok := vt.vTab.(interface { if n, ok := vt.vTab.(interface {
@ -585,6 +638,13 @@ type Module interface {
DestroyModule() DestroyModule()
} }
// EponymousOnlyModule is a "virtual table module" (as above), but
// for defining "eponymous only" virtual tables See: https://sqlite.org/vtab.html#eponymous_only_virtual_tables
type EponymousOnlyModule interface {
Module
EponymousOnlyModule()
}
// VTab describes a particular instance of the virtual table. // VTab describes a particular instance of the virtual table.
// See: http://sqlite.org/c3ref/vtab.html // See: http://sqlite.org/c3ref/vtab.html
type VTab interface { type VTab interface {
@ -642,9 +702,19 @@ func (c *SQLiteConn) CreateModule(moduleName string, module Module) error {
mname := C.CString(moduleName) mname := C.CString(moduleName)
defer C.free(unsafe.Pointer(mname)) defer C.free(unsafe.Pointer(mname))
udm := sqliteModule{c, moduleName, module} udm := sqliteModule{c, moduleName, module}
rv := C._sqlite3_create_module(c.db, mname, C.uintptr_t(newHandle(c, &udm))) switch module.(type) {
if rv != C.SQLITE_OK { case EponymousOnlyModule:
return c.lastError() rv := C._sqlite3_create_module_eponymous_only(c.db, mname, C.uintptr_t(uintptr(newHandle(c, &udm))))
if rv != C.SQLITE_OK {
return c.lastError()
}
return nil
case Module:
rv := C._sqlite3_create_module(c.db, mname, C.uintptr_t(uintptr(newHandle(c, &udm))))
if rv != C.SQLITE_OK {
return c.lastError()
}
return nil
} }
return nil return nil
} }

View File

@ -1,8 +1,9 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
//go:build sqlite_vtable || vtable
// +build sqlite_vtable vtable // +build sqlite_vtable vtable
package sqlite3 package sqlite3
@ -484,3 +485,125 @@ func (c *vtabUpdateCursor) Rowid() (int64, error) {
func (c *vtabUpdateCursor) Close() error { func (c *vtabUpdateCursor) Close() error {
return nil return nil
} }
type testModuleEponymousOnly struct {
t *testing.T
intarray []int
}
type testVTabEponymousOnly struct {
intarray []int
}
type testVTabCursorEponymousOnly struct {
vTab *testVTabEponymousOnly
index int
}
func (m testModuleEponymousOnly) EponymousOnlyModule() {}
func (m testModuleEponymousOnly) Create(c *SQLiteConn, args []string) (VTab, error) {
err := c.DeclareVTab("CREATE TABLE x(test INT)")
if err != nil {
return nil, err
}
return &testVTabEponymousOnly{m.intarray}, nil
}
func (m testModuleEponymousOnly) Connect(c *SQLiteConn, args []string) (VTab, error) {
return m.Create(c, args)
}
func (m testModuleEponymousOnly) DestroyModule() {}
func (v *testVTabEponymousOnly) BestIndex(cst []InfoConstraint, ob []InfoOrderBy) (*IndexResult, error) {
used := make([]bool, 0, len(cst))
for range cst {
used = append(used, false)
}
return &IndexResult{
Used: used,
IdxNum: 0,
IdxStr: "test-index",
AlreadyOrdered: true,
EstimatedCost: 100,
EstimatedRows: 200,
}, nil
}
func (v *testVTabEponymousOnly) Disconnect() error {
return nil
}
func (v *testVTabEponymousOnly) Destroy() error {
return nil
}
func (v *testVTabEponymousOnly) Open() (VTabCursor, error) {
return &testVTabCursorEponymousOnly{v, 0}, nil
}
func (vc *testVTabCursorEponymousOnly) Close() error {
return nil
}
func (vc *testVTabCursorEponymousOnly) Filter(idxNum int, idxStr string, vals []interface{}) error {
vc.index = 0
return nil
}
func (vc *testVTabCursorEponymousOnly) Next() error {
vc.index++
return nil
}
func (vc *testVTabCursorEponymousOnly) EOF() bool {
return vc.index >= len(vc.vTab.intarray)
}
func (vc *testVTabCursorEponymousOnly) Column(c *SQLiteContext, col int) error {
if col != 0 {
return fmt.Errorf("column index out of bounds: %d", col)
}
c.ResultInt(vc.vTab.intarray[vc.index])
return nil
}
func (vc *testVTabCursorEponymousOnly) Rowid() (int64, error) {
return int64(vc.index), nil
}
func TestCreateModuleEponymousOnly(t *testing.T) {
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
intarray := []int{1, 2, 3}
sql.Register("sqlite3_TestCreateModuleEponymousOnly", &SQLiteDriver{
ConnectHook: func(conn *SQLiteConn) error {
return conn.CreateModule("test", testModuleEponymousOnly{t, intarray})
},
})
db, err := sql.Open("sqlite3_TestCreateModuleEponymousOnly", tempFilename)
if err != nil {
t.Fatalf("could not open db: %v", err)
}
var i, value int
rows, err := db.Query("SELECT rowid, * FROM test")
if err != nil {
t.Fatalf("couldn't select from virtual table: %v", err)
}
for rows.Next() {
err := rows.Scan(&i, &value)
if err != nil {
t.Fatal(err)
}
if intarray[i] != value {
t.Fatalf("want %v but %v", intarray[i], value)
}
}
_, err = db.Exec("DROP TABLE test")
if err != nil {
t.Logf("couldn't drop virtual table: %v", err)
}
}

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -1,4 +1,4 @@
// Copyright (C) 2018 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.

View File

@ -1,8 +1,11 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
//go:build cgo
// +build cgo
package sqlite3 package sqlite3
import ( import (
@ -17,6 +20,7 @@ import (
"os" "os"
"reflect" "reflect"
"regexp" "regexp"
"runtime"
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
@ -24,7 +28,7 @@ import (
"time" "time"
) )
func TempFilename(t *testing.T) string { func TempFilename(t testing.TB) string {
f, err := ioutil.TempFile("", "go-sqlite3-test-") f, err := ioutil.TempFile("", "go-sqlite3-test-")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@ -34,20 +38,32 @@ func TempFilename(t *testing.T) string {
} }
func doTestOpen(t *testing.T, option string) (string, error) { func doTestOpen(t *testing.T, option string) (string, error) {
var url string
tempFilename := TempFilename(t) tempFilename := TempFilename(t)
defer os.Remove(tempFilename) url := tempFilename + option
if option != "" {
url = tempFilename + option defer func() {
} else { err := os.Remove(tempFilename)
url = tempFilename if err != nil {
} t.Error("temp file remove error:", err)
}
}()
db, err := sql.Open("sqlite3", url) db, err := sql.Open("sqlite3", url)
if err != nil { if err != nil {
return "Failed to open database:", err return "Failed to open database:", err
} }
defer os.Remove(tempFilename)
defer db.Close() defer func() {
err = db.Close()
if err != nil {
t.Error("db close error:", err)
}
}()
err = db.Ping()
if err != nil {
return "ping error:", err
}
_, err = db.Exec("drop table foo") _, err = db.Exec("drop table foo")
_, err = db.Exec("create table foo (id integer)") _, err = db.Exec("create table foo (id integer)")
@ -87,6 +103,43 @@ func TestOpen(t *testing.T) {
} }
} }
func TestOpenWithVFS(t *testing.T) {
filename := t.Name() + ".sqlite"
if err := os.Remove(filename); err != nil && !os.IsNotExist(err) {
t.Fatal(err)
}
defer os.Remove(filename)
db, err := sql.Open("sqlite3", fmt.Sprintf("file:%s?vfs=hello", filename))
if err != nil {
t.Fatal("Failed to open", err)
}
err = db.Ping()
if err == nil {
t.Fatal("Failed to open", err)
}
db.Close()
defer os.Remove(filename)
var vfs string
if runtime.GOOS == "windows" {
vfs = "win32-none"
} else {
vfs = "unix-none"
}
db, err = sql.Open("sqlite3", fmt.Sprintf("file:%s?vfs=%s", filename, vfs))
if err != nil {
t.Fatal("Failed to open", err)
}
err = db.Ping()
if err != nil {
t.Fatal("Failed to ping", err)
}
db.Close()
}
func TestOpenNoCreate(t *testing.T) { func TestOpenNoCreate(t *testing.T) {
filename := t.Name() + ".sqlite" filename := t.Name() + ".sqlite"
@ -195,6 +248,43 @@ func TestForeignKeys(t *testing.T) {
} }
} }
func TestDeferredForeignKey(t *testing.T) {
fname := TempFilename(t)
uri := "file:" + fname + "?_foreign_keys=1"
db, err := sql.Open("sqlite3", uri)
if err != nil {
os.Remove(fname)
t.Errorf("sql.Open(\"sqlite3\", %q): %v", uri, err)
}
_, err = db.Exec("CREATE TABLE bar (id INTEGER PRIMARY KEY)")
if err != nil {
t.Errorf("failed creating tables: %v", err)
}
_, err = db.Exec("CREATE TABLE foo (bar_id INTEGER, FOREIGN KEY(bar_id) REFERENCES bar(id) DEFERRABLE INITIALLY DEFERRED)")
if err != nil {
t.Errorf("failed creating tables: %v", err)
}
tx, err := db.Begin()
if err != nil {
t.Errorf("Failed to begin transaction: %v", err)
}
_, err = tx.Exec("INSERT INTO foo (bar_id) VALUES (123)")
if err != nil {
t.Errorf("Failed to insert row: %v", err)
}
err = tx.Commit()
if err == nil {
t.Errorf("Expected an error: %v", err)
}
_, err = db.Begin()
if err != nil {
t.Errorf("Failed to begin transaction: %v", err)
}
db.Close()
os.Remove(fname)
}
func TestRecursiveTriggers(t *testing.T) { func TestRecursiveTriggers(t *testing.T) {
cases := map[string]bool{ cases := map[string]bool{
"?_recursive_triggers=1": true, "?_recursive_triggers=1": true,
@ -291,8 +381,8 @@ func TestInsert(t *testing.T) {
func TestUpsert(t *testing.T) { func TestUpsert(t *testing.T) {
_, n, _ := Version() _, n, _ := Version()
if !(n >= 3024000) { if n < 3024000 {
t.Skip("UPSERT requires sqlite3 => 3.24.0") t.Skip("UPSERT requires sqlite3 >= 3.24.0")
} }
tempFilename := TempFilename(t) tempFilename := TempFilename(t)
defer os.Remove(tempFilename) defer os.Remove(tempFilename)
@ -1010,34 +1100,44 @@ func TestQueryer(t *testing.T) {
defer db.Close() defer db.Close()
_, err = db.Exec(` _, err = db.Exec(`
create table foo (id integer); create table foo (id integer);
`) `)
if err != nil { if err != nil {
t.Error("Failed to call db.Query:", err) t.Error("Failed to call db.Query:", err)
} }
rows, err := db.Query(` _, err = db.Exec(`
insert into foo(id) values(?); insert into foo(id) values(?);
insert into foo(id) values(?); insert into foo(id) values(?);
insert into foo(id) values(?); insert into foo(id) values(?);
select id from foo order by id;
`, 3, 2, 1) `, 3, 2, 1)
if err != nil {
t.Error("Failed to call db.Exec:", err)
}
rows, err := db.Query(`
select id from foo order by id;
`)
if err != nil { if err != nil {
t.Error("Failed to call db.Query:", err) t.Error("Failed to call db.Query:", err)
} }
defer rows.Close() defer rows.Close()
n := 1 n := 0
if rows != nil { for rows.Next() {
for rows.Next() { var id int
var id int err = rows.Scan(&id)
err = rows.Scan(&id) if err != nil {
if err != nil { t.Error("Failed to db.Query:", err)
t.Error("Failed to db.Query:", err)
}
if id != n {
t.Error("Failed to db.Query: not matched results")
}
} }
if id != n + 1 {
t.Error("Failed to db.Query: not matched results")
}
n = n + 1
}
if err := rows.Err(); err != nil {
t.Errorf("Post-scan failed: %v\n", err)
}
if n != 3 {
t.Errorf("Expected 3 rows but retrieved %v", n)
} }
} }
@ -1316,16 +1416,16 @@ func TestFunctionRegistration(t *testing.T) {
{"SELECT addf32_64(1.5,1.5)", float64(3)}, {"SELECT addf32_64(1.5,1.5)", float64(3)},
{"SELECT not(1)", false}, {"SELECT not(1)", false},
{"SELECT not(0)", true}, {"SELECT not(0)", true},
{`SELECT regex("^foo.*", "foobar")`, true}, {`SELECT regex('^foo.*', 'foobar')`, true},
{`SELECT regex("^foo.*", "barfoobar")`, false}, {`SELECT regex('^foo.*', 'barfoobar')`, false},
{"SELECT generic(1)", int64(1)}, {"SELECT generic(1)", int64(1)},
{"SELECT generic(1.1)", int64(2)}, {"SELECT generic(1.1)", int64(2)},
{`SELECT generic(NULL)`, int64(3)}, {`SELECT generic(NULL)`, int64(3)},
{`SELECT generic("foo")`, int64(4)}, {`SELECT generic('foo')`, int64(4)},
{"SELECT variadic(1,2)", int64(3)}, {"SELECT variadic(1,2)", int64(3)},
{"SELECT variadic(1,2,3,4)", int64(10)}, {"SELECT variadic(1,2,3,4)", int64(10)},
{"SELECT variadic(1,1,1,1,1,1,1,1,1,1)", int64(10)}, {"SELECT variadic(1,1,1,1,1,1,1,1,1,1)", int64(10)},
{`SELECT variadicGeneric(1,"foo",2.3, NULL)`, int64(4)}, {`SELECT variadicGeneric(1,'foo',2.3, NULL)`, int64(4)},
} }
for _, op := range ops { for _, op := range ops {
@ -1396,6 +1496,63 @@ func TestAggregatorRegistration(t *testing.T) {
} }
} }
type mode struct {
counts map[interface{}]int
top interface{}
topCount int
}
func newMode() *mode {
return &mode{
counts: map[interface{}]int{},
}
}
func (m *mode) Step(x interface{}) {
m.counts[x]++
c := m.counts[x]
if c > m.topCount {
m.top = x
m.topCount = c
}
}
func (m *mode) Done() interface{} {
return m.top
}
func TestAggregatorRegistration_GenericReturn(t *testing.T) {
sql.Register("sqlite3_AggregatorRegistration_GenericReturn", &SQLiteDriver{
ConnectHook: func(conn *SQLiteConn) error {
return conn.RegisterAggregator("mode", newMode, true)
},
})
db, err := sql.Open("sqlite3_AggregatorRegistration_GenericReturn", ":memory:")
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
_, err = db.Exec("create table foo (department integer, profits integer)")
if err != nil {
t.Fatal("Failed to create table:", err)
}
_, err = db.Exec("insert into foo values (1, 10), (1, 20), (1, 45), (2, 42), (2, 115), (2, 20)")
if err != nil {
t.Fatal("Failed to insert records:", err)
}
var mode int
err = db.QueryRow("select mode(profits) from foo").Scan(&mode)
if err != nil {
t.Fatal("MODE query error:", err)
}
if mode != 20 {
t.Fatal("Got incorrect mode. Wanted 20, got: ", mode)
}
}
func rot13(r rune) rune { func rot13(r rune) rune {
switch { switch {
case r >= 'A' && r <= 'Z': case r >= 'A' && r <= 'Z':
@ -1435,11 +1592,11 @@ func TestCollationRegistration(t *testing.T) {
populate := []string{ populate := []string{
`CREATE TABLE test (s TEXT)`, `CREATE TABLE test (s TEXT)`,
`INSERT INTO test VALUES ("aaaa")`, `INSERT INTO test VALUES ('aaaa')`,
`INSERT INTO test VALUES ("ffff")`, `INSERT INTO test VALUES ('ffff')`,
`INSERT INTO test VALUES ("qqqq")`, `INSERT INTO test VALUES ('qqqq')`,
`INSERT INTO test VALUES ("tttt")`, `INSERT INTO test VALUES ('tttt')`,
`INSERT INTO test VALUES ("zzzz")`, `INSERT INTO test VALUES ('zzzz')`,
} }
for _, stmt := range populate { for _, stmt := range populate {
if _, err := db.Exec(stmt); err != nil { if _, err := db.Exec(stmt); err != nil {
@ -1534,7 +1691,7 @@ func TestDeclTypes(t *testing.T) {
t.Fatal("Failed to create table:", err) t.Fatal("Failed to create table:", err)
} }
_, err = sqlite3conn.Exec("insert into foo(name) values(\"bar\")", nil) _, err = sqlite3conn.Exec("insert into foo(name) values('bar')", nil)
if err != nil { if err != nil {
t.Fatal("Failed to insert:", err) t.Fatal("Failed to insert:", err)
} }
@ -1670,6 +1827,43 @@ func TestAuthorizer(t *testing.T) {
} }
} }
func TestSetFileControlInt(t *testing.T) {
t.Run("PERSIST_WAL", func(t *testing.T) {
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
sql.Register("sqlite3_FCNTL_PERSIST_WAL", &SQLiteDriver{
ConnectHook: func(conn *SQLiteConn) error {
if err := conn.SetFileControlInt("", SQLITE_FCNTL_PERSIST_WAL, 1); err != nil {
return fmt.Errorf("Unexpected error from SetFileControlInt(): %w", err)
}
return nil
},
})
db, err := sql.Open("sqlite3_FCNTL_PERSIST_WAL", tempFilename)
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
// Set to WAL mode & write a page.
if _, err := db.Exec(`PRAGMA journal_mode = wal`); err != nil {
t.Fatal("Failed to set journal mode:", err)
} else if _, err := db.Exec(`CREATE TABLE t (x)`); err != nil {
t.Fatal("Failed to create table:", err)
}
if err := db.Close(); err != nil {
t.Fatal("Failed to close database", err)
}
// Ensure WAL file persists after close.
if _, err := os.Stat(tempFilename + "-wal"); err != nil {
t.Fatal("Expected WAL file to be persisted after close", err)
}
})
}
func TestNonColumnString(t *testing.T) { func TestNonColumnString(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:") db, err := sql.Open("sqlite3", ":memory:")
if err != nil { if err != nil {
@ -1764,6 +1958,45 @@ func TestInsertNilByteSlice(t *testing.T) {
} }
} }
func TestNamedParam(t *testing.T) {
tempFilename := TempFilename(t)
defer os.Remove(tempFilename)
db, err := sql.Open("sqlite3", tempFilename)
if err != nil {
t.Fatal("Failed to open database:", err)
}
defer db.Close()
_, err = db.Exec("drop table foo")
_, err = db.Exec("create table foo (id integer, name text, amount integer)")
if err != nil {
t.Fatal("Failed to create table:", err)
}
_, err = db.Exec("insert into foo(id, name, amount) values(:id, @name, $amount)",
sql.Named("bar", 42), sql.Named("baz", "quux"),
sql.Named("amount", 123), sql.Named("corge", "waldo"),
sql.Named("id", 2), sql.Named("name", "grault"))
if err != nil {
t.Fatal("Failed to insert record with named parameters:", err)
}
rows, err := db.Query("select id, name, amount from foo")
if err != nil {
t.Fatal("Failed to select records:", err)
}
defer rows.Close()
rows.Next()
var id, amount int
var name string
rows.Scan(&id, &name, &amount)
if id != 2 || name != "grault" || amount != 123 {
t.Errorf("Expected %d, %q, %d for fetched result, but got %d, %q, %d:", 2, "grault", 123, id, name, amount)
}
}
var customFunctionOnce sync.Once var customFunctionOnce sync.Once
func BenchmarkCustomFunctions(b *testing.B) { func BenchmarkCustomFunctions(b *testing.B) {
@ -1797,28 +2030,21 @@ func BenchmarkCustomFunctions(b *testing.B) {
} }
func TestSuite(t *testing.T) { func TestSuite(t *testing.T) {
tempFilename := TempFilename(t) initializeTestDB(t)
defer os.Remove(tempFilename) defer freeTestDB()
d, err := sql.Open("sqlite3", tempFilename+"?_busy_timeout=99999")
if err != nil {
t.Fatal(err)
}
defer d.Close()
db = &TestDB{t, d, SQLITE, sync.Once{}} for _, test := range tests {
ok := testing.RunTests(func(string, string) (bool, error) { return true, nil }, tests) t.Run(test.Name, test.F)
if !ok {
t.Fatal("A subtest failed")
} }
}
if !testing.Short() { func BenchmarkSuite(b *testing.B) {
for _, b := range benchmarks { initializeTestDB(b)
fmt.Printf("%-20s", b.Name) defer freeTestDB()
r := testing.Benchmark(b.F)
fmt.Printf("%10d %10.0f req/s\n", r.N, float64(r.N)/r.T.Seconds()) for _, benchmark := range benchmarks {
} b.Run(benchmark.Name, benchmark.F)
} }
db.tearDown()
} }
// Dialect is a type of dialect of databases. // Dialect is a type of dialect of databases.
@ -1833,14 +2059,37 @@ const (
// DB provide context for the tests // DB provide context for the tests
type TestDB struct { type TestDB struct {
*testing.T testing.TB
*sql.DB *sql.DB
dialect Dialect dialect Dialect
once sync.Once once sync.Once
tempFilename string
} }
var db *TestDB var db *TestDB
func initializeTestDB(t testing.TB) {
tempFilename := TempFilename(t)
d, err := sql.Open("sqlite3", tempFilename+"?_busy_timeout=99999")
if err != nil {
os.Remove(tempFilename)
t.Fatal(err)
}
db = &TestDB{t, d, SQLITE, sync.Once{}, tempFilename}
}
func freeTestDB() {
err := db.DB.Close()
if err != nil {
panic(err)
}
err = os.Remove(db.tempFilename)
if err != nil {
panic(err)
}
}
// the following tables will be created and dropped during the test // the following tables will be created and dropped during the test
var testTables = []string{"foo", "bar", "t", "bench"} var testTables = []string{"foo", "bar", "t", "bench"}
@ -1852,6 +2101,7 @@ var tests = []testing.InternalTest{
{Name: "TestManyQueryRow", F: testManyQueryRow}, {Name: "TestManyQueryRow", F: testManyQueryRow},
{Name: "TestTxQuery", F: testTxQuery}, {Name: "TestTxQuery", F: testTxQuery},
{Name: "TestPreparedStmt", F: testPreparedStmt}, {Name: "TestPreparedStmt", F: testPreparedStmt},
{Name: "TestExecEmptyQuery", F: testExecEmptyQuery},
} }
var benchmarks = []testing.InternalBenchmark{ var benchmarks = []testing.InternalBenchmark{
@ -2182,6 +2432,25 @@ func testPreparedStmt(t *testing.T) {
wg.Wait() wg.Wait()
} }
// testEmptyQuery is test for validating the API in case of empty query
func testExecEmptyQuery(t *testing.T) {
db.tearDown()
res, err := db.Exec(" -- this is just a comment ")
if err != nil {
t.Fatalf("empty query err: %v", err)
}
_, err = res.LastInsertId()
if err != nil {
t.Fatalf("LastInsertId returned an error: %v", err)
}
_, err = res.RowsAffected()
if err != nil {
t.Fatalf("RowsAffected returned an error: %v", err)
}
}
// Benchmarks need to use panic() since b.Error errors are lost when // Benchmarks need to use panic() since b.Error errors are lost when
// running via testing.Benchmark() I would like to run these via go // running via testing.Benchmark() I would like to run these via go
// test -bench but calling Benchmark() from a benchmark test // test -bench but calling Benchmark() from a benchmark test

View File

@ -1,4 +1,4 @@
// Copyright (C) 2016 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -9,7 +9,7 @@ package sqlite3
/* /*
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
@ -89,6 +89,7 @@ func fillExpandedSQL(info *TraceInfo, db *C.sqlite3, pStmt unsafe.Pointer) {
} }
expSQLiteCStr := C.sqlite3_expanded_sql((*C.sqlite3_stmt)(pStmt)) expSQLiteCStr := C.sqlite3_expanded_sql((*C.sqlite3_stmt)(pStmt))
defer C.sqlite3_free(unsafe.Pointer(expSQLiteCStr))
if expSQLiteCStr == nil { if expSQLiteCStr == nil {
fillDBError(&info.DBError, db) fillDBError(&info.DBError, db)
return return
@ -214,7 +215,6 @@ func addTraceMapping(connHandle uintptr, traceConf TraceConfig) {
traceConf, connHandle, oldEntryCopy.config)) traceConf, connHandle, oldEntryCopy.config))
} }
traceMap[connHandle] = traceMapEntry{config: traceConf} traceMap[connHandle] = traceMapEntry{config: traceConf}
fmt.Printf("Added trace config %v: handle 0x%x.\n", traceConf, connHandle)
} }
func lookupTraceMapping(connHandle uintptr) (TraceConfig, bool) { func lookupTraceMapping(connHandle uintptr) (TraceConfig, bool) {
@ -233,7 +233,6 @@ func popTraceMapping(connHandle uintptr) (TraceConfig, bool) {
entryCopy, found := traceMap[connHandle] entryCopy, found := traceMap[connHandle]
if found { if found {
delete(traceMap, connHandle) delete(traceMap, connHandle)
fmt.Printf("Pop handle 0x%x: deleted trace config %v.\n", connHandle, entryCopy.config)
} }
return entryCopy.config, found return entryCopy.config, found
} }

View File

@ -1,16 +1,21 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3 package sqlite3
/* /*
#ifndef USE_LIBSQLITE3 #ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h> #include "sqlite3-binding.h"
#else #else
#include <sqlite3.h> #include <sqlite3.h>
#endif #endif
*/ */
import "C" import "C"
import ( import (
"database/sql"
"reflect" "reflect"
"time" "strings"
) )
// ColumnTypeDatabaseTypeName implement RowsColumnTypeDatabaseTypeName. // ColumnTypeDatabaseTypeName implement RowsColumnTypeDatabaseTypeName.
@ -35,23 +40,69 @@ func (rc *SQLiteRows) ColumnTypeNullable(i int) (nullable, ok bool) {
// ColumnTypeScanType implement RowsColumnTypeScanType. // ColumnTypeScanType implement RowsColumnTypeScanType.
func (rc *SQLiteRows) ColumnTypeScanType(i int) reflect.Type { func (rc *SQLiteRows) ColumnTypeScanType(i int) reflect.Type {
switch C.sqlite3_column_type(rc.s.s, C.int(i)) { //ct := C.sqlite3_column_type(rc.s.s, C.int(i)) // Always returns 5
case C.SQLITE_INTEGER: return scanType(C.GoString(C.sqlite3_column_decltype(rc.s.s, C.int(i))))
switch C.GoString(C.sqlite3_column_decltype(rc.s.s, C.int(i))) { }
case "timestamp", "datetime", "date":
return reflect.TypeOf(time.Time{}) const (
case "boolean": SQLITE_INTEGER = iota
return reflect.TypeOf(false) SQLITE_TEXT
} SQLITE_BLOB
return reflect.TypeOf(int64(0)) SQLITE_REAL
case C.SQLITE_FLOAT: SQLITE_NUMERIC
return reflect.TypeOf(float64(0)) SQLITE_TIME
case C.SQLITE_BLOB: SQLITE_BOOL
return reflect.SliceOf(reflect.TypeOf(byte(0))) SQLITE_NULL
case C.SQLITE_NULL: )
return reflect.TypeOf(nil)
case C.SQLITE_TEXT: func scanType(cdt string) reflect.Type {
return reflect.TypeOf("") t := strings.ToUpper(cdt)
} i := databaseTypeConvSqlite(t)
return reflect.SliceOf(reflect.TypeOf(byte(0))) switch i {
case SQLITE_INTEGER:
return reflect.TypeOf(sql.NullInt64{})
case SQLITE_TEXT:
return reflect.TypeOf(sql.NullString{})
case SQLITE_BLOB:
return reflect.TypeOf(sql.RawBytes{})
case SQLITE_REAL:
return reflect.TypeOf(sql.NullFloat64{})
case SQLITE_NUMERIC:
return reflect.TypeOf(sql.NullFloat64{})
case SQLITE_BOOL:
return reflect.TypeOf(sql.NullBool{})
case SQLITE_TIME:
return reflect.TypeOf(sql.NullTime{})
}
return reflect.TypeOf(new(interface{}))
}
func databaseTypeConvSqlite(t string) int {
if strings.Contains(t, "INT") {
return SQLITE_INTEGER
}
if t == "CLOB" || t == "TEXT" ||
strings.Contains(t, "CHAR") {
return SQLITE_TEXT
}
if t == "BLOB" {
return SQLITE_BLOB
}
if t == "REAL" || t == "FLOAT" ||
strings.Contains(t, "DOUBLE") {
return SQLITE_REAL
}
if t == "DATE" || t == "DATETIME" ||
t == "TIMESTAMP" {
return SQLITE_TIME
}
if t == "NUMERIC" ||
strings.Contains(t, "DECIMAL") {
return SQLITE_NUMERIC
}
if t == "BOOLEAN" {
return SQLITE_BOOL
}
return SQLITE_NULL
} }

View File

@ -15,7 +15,9 @@ package sqlite3
// This code should improve performance on windows because // This code should improve performance on windows because
// without the presence of usleep SQLite waits 1 second. // without the presence of usleep SQLite waits 1 second.
// //
// Source: https://stackoverflow.com/questions/5801813/c-usleep-is-obsolete-workarounds-for-windows-mingw?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa // Source: https://github.com/php/php-src/blob/PHP-5.0/win32/time.c
// License: https://github.com/php/php-src/blob/PHP-5.0/LICENSE
// Details: https://stackoverflow.com/questions/5801813/c-usleep-is-obsolete-workarounds-for-windows-mingw?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
/* /*
#include <windows.h> #include <windows.h>

View File

@ -1,4 +1,4 @@
// Copyright (C) 2014 Yasuhiro Matsumoto <mattn.jp@gmail.com>. // Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// //
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
@ -12,7 +12,6 @@ package sqlite3
#cgo CFLAGS: -fno-stack-check #cgo CFLAGS: -fno-stack-check
#cgo CFLAGS: -fno-stack-protector #cgo CFLAGS: -fno-stack-protector
#cgo CFLAGS: -mno-stack-arg-probe #cgo CFLAGS: -mno-stack-arg-probe
#cgo LDFLAGS: -lmingwex -lmingw32
#cgo windows,386 CFLAGS: -D_USE_32BIT_TIME_T #cgo windows,386 CFLAGS: -D_USE_32BIT_TIME_T
*/ */
import "C" import "C"

View File

@ -19,6 +19,10 @@
#ifndef SQLITE3EXT_H #ifndef SQLITE3EXT_H
#define SQLITE3EXT_H #define SQLITE3EXT_H
#include "sqlite3-binding.h" #include "sqlite3-binding.h"
#ifdef __clang__
#define assert(condition) ((void)0)
#endif
/* /*
** The following structure holds pointers to all of the SQLite API ** The following structure holds pointers to all of the SQLite API
@ -323,6 +327,41 @@ struct sqlite3_api_routines {
/* Version 3.28.0 and later */ /* Version 3.28.0 and later */
int (*stmt_isexplain)(sqlite3_stmt*); int (*stmt_isexplain)(sqlite3_stmt*);
int (*value_frombind)(sqlite3_value*); int (*value_frombind)(sqlite3_value*);
/* Version 3.30.0 and later */
int (*drop_modules)(sqlite3*,const char**);
/* Version 3.31.0 and later */
sqlite3_int64 (*hard_heap_limit64)(sqlite3_int64);
const char *(*uri_key)(const char*,int);
const char *(*filename_database)(const char*);
const char *(*filename_journal)(const char*);
const char *(*filename_wal)(const char*);
/* Version 3.32.0 and later */
char *(*create_filename)(const char*,const char*,const char*,
int,const char**);
void (*free_filename)(char*);
sqlite3_file *(*database_file_object)(const char*);
/* Version 3.34.0 and later */
int (*txn_state)(sqlite3*,const char*);
/* Version 3.36.1 and later */
sqlite3_int64 (*changes64)(sqlite3*);
sqlite3_int64 (*total_changes64)(sqlite3*);
/* Version 3.37.0 and later */
int (*autovacuum_pages)(sqlite3*,
unsigned int(*)(void*,const char*,unsigned int,unsigned int,unsigned int),
void*, void(*)(void*));
/* Version 3.38.0 and later */
int (*error_offset)(sqlite3*);
int (*vtab_rhs_value)(sqlite3_index_info*,int,sqlite3_value**);
int (*vtab_distinct)(sqlite3_index_info*);
int (*vtab_in)(sqlite3_index_info*,int,int);
int (*vtab_in_first)(sqlite3_value*,sqlite3_value**);
int (*vtab_in_next)(sqlite3_value*,sqlite3_value**);
/* Version 3.39.0 and later */
int (*deserialize)(sqlite3*,const char*,unsigned char*,
sqlite3_int64,sqlite3_int64,unsigned);
unsigned char *(*serialize)(sqlite3*,const char *,sqlite3_int64*,
unsigned int);
const char *(*db_name)(sqlite3*,int);
}; };
/* /*
@ -613,8 +652,40 @@ typedef int (*sqlite3_loadext_entry)(
/* Version 3.26.0 and later */ /* Version 3.26.0 and later */
#define sqlite3_normalized_sql sqlite3_api->normalized_sql #define sqlite3_normalized_sql sqlite3_api->normalized_sql
/* Version 3.28.0 and later */ /* Version 3.28.0 and later */
#define sqlite3_stmt_isexplain sqlite3_api->isexplain #define sqlite3_stmt_isexplain sqlite3_api->stmt_isexplain
#define sqlite3_value_frombind sqlite3_api->frombind #define sqlite3_value_frombind sqlite3_api->value_frombind
/* Version 3.30.0 and later */
#define sqlite3_drop_modules sqlite3_api->drop_modules
/* Version 3.31.0 and later */
#define sqlite3_hard_heap_limit64 sqlite3_api->hard_heap_limit64
#define sqlite3_uri_key sqlite3_api->uri_key
#define sqlite3_filename_database sqlite3_api->filename_database
#define sqlite3_filename_journal sqlite3_api->filename_journal
#define sqlite3_filename_wal sqlite3_api->filename_wal
/* Version 3.32.0 and later */
#define sqlite3_create_filename sqlite3_api->create_filename
#define sqlite3_free_filename sqlite3_api->free_filename
#define sqlite3_database_file_object sqlite3_api->database_file_object
/* Version 3.34.0 and later */
#define sqlite3_txn_state sqlite3_api->txn_state
/* Version 3.36.1 and later */
#define sqlite3_changes64 sqlite3_api->changes64
#define sqlite3_total_changes64 sqlite3_api->total_changes64
/* Version 3.37.0 and later */
#define sqlite3_autovacuum_pages sqlite3_api->autovacuum_pages
/* Version 3.38.0 and later */
#define sqlite3_error_offset sqlite3_api->error_offset
#define sqlite3_vtab_rhs_value sqlite3_api->vtab_rhs_value
#define sqlite3_vtab_distinct sqlite3_api->vtab_distinct
#define sqlite3_vtab_in sqlite3_api->vtab_in
#define sqlite3_vtab_in_first sqlite3_api->vtab_in_first
#define sqlite3_vtab_in_next sqlite3_api->vtab_in_next
/* Version 3.39.0 and later */
#ifndef SQLITE_OMIT_DESERIALIZE
#define sqlite3_deserialize sqlite3_api->deserialize
#define sqlite3_serialize sqlite3_api->serialize
#endif
#define sqlite3_db_name sqlite3_api->db_name
#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */ #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */
#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)

View File

@ -1,3 +1,8 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !cgo // +build !cgo
package sqlite3 package sqlite3
@ -8,14 +13,25 @@ import (
"errors" "errors"
) )
func init() {
sql.Register("sqlite3", &SQLiteDriverMock{})
}
type SQLiteDriverMock struct{}
var errorMsg = errors.New("Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub") var errorMsg = errors.New("Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub")
func (SQLiteDriverMock) Open(s string) (driver.Conn, error) { func init() {
return nil, errorMsg sql.Register("sqlite3", &SQLiteDriver{})
} }
type (
SQLiteDriver struct {
Extensions []string
ConnectHook func(*SQLiteConn) error
}
SQLiteConn struct{}
)
func (SQLiteDriver) Open(s string) (driver.Conn, error) { return nil, errorMsg }
func (c *SQLiteConn) RegisterAggregator(string, interface{}, bool) error { return errorMsg }
func (c *SQLiteConn) RegisterAuthorizer(func(int, string, string, string) int) {}
func (c *SQLiteConn) RegisterCollation(string, func(string, string) int) error { return errorMsg }
func (c *SQLiteConn) RegisterCommitHook(func() int) {}
func (c *SQLiteConn) RegisterFunc(string, interface{}, bool) error { return errorMsg }
func (c *SQLiteConn) RegisterRollbackHook(func()) {}
func (c *SQLiteConn) RegisterUpdateHook(func(int, string, string, int64)) {}

5
upgrade/go.mod Normal file
View File

@ -0,0 +1,5 @@
module github.com/mattn/go-sqlite3/upgrade
go 1.16
require github.com/PuerkitoBio/goquery v1.7.1 // indirect

12
upgrade/go.sum Normal file
View File

@ -0,0 +1,12 @@
github.com/PuerkitoBio/goquery v1.7.1 h1:oE+T06D+1T7LNrn91B4aERsRIeCLJ/oPSa6xB9FPnz4=
github.com/PuerkitoBio/goquery v1.7.1/go.mod h1:XY0pP4kfraEmmV1O7Uf6XyjoslwsneBbgeDjLYuN8xY=
github.com/andybalholm/cascadia v1.2.0 h1:vuRCkM5Ozh/BfmsaTm26kbjm0mIOM3yS5Ek/F5h18aE=
github.com/andybalholm/cascadia v1.2.0/go.mod h1:YCyR8vOZT9aZ1CHEd8ap0gMVm2aFgxBp0T0eFw1RUQY=
golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=

View File

@ -1,5 +1,5 @@
// +build !cgo //go:build !cgo && upgrade && ignore
// +build upgrade // +build !cgo,upgrade,ignore
package main package main
@ -98,16 +98,25 @@ func mergeFile(src string, dst string) error {
func main() { func main() {
fmt.Println("Go-SQLite3 Upgrade Tool") fmt.Println("Go-SQLite3 Upgrade Tool")
wd, err := os.Getwd()
if err != nil {
log.Fatal(err)
}
if filepath.Base(wd) != "upgrade" {
log.Printf("Current directory is %q but should run in upgrade directory", wd)
os.Exit(1)
}
// Download Amalgamation // Download Amalgamation
_, amalgamation, err := download("sqlite-amalgamation-") _, amalgamation, err := download("sqlite-amalgamation-")
if err != nil { if err != nil {
fmt.Println("Failed to download: sqlite-amalgamation; %s", err) log.Fatalf("Failed to download: sqlite-amalgamation; %s", err)
} }
// Download Source // Download Source
_, source, err := download("sqlite-src-") _, source, err := download("sqlite-src-")
if err != nil { if err != nil {
fmt.Println("Failed to download: sqlite-src; %s", err) log.Fatalf("Failed to download: sqlite-src; %s", err)
} }
// Create Amalgamation Zip Reader // Create Amalgamation Zip Reader
@ -127,11 +136,11 @@ func main() {
var f *os.File var f *os.File
switch path.Base(zf.Name) { switch path.Base(zf.Name) {
case "sqlite3.c": case "sqlite3.c":
f, err = os.Create("sqlite3-binding.c") f, err = os.Create("../sqlite3-binding.c")
case "sqlite3.h": case "sqlite3.h":
f, err = os.Create("sqlite3-binding.h") f, err = os.Create("../sqlite3-binding.h")
case "sqlite3ext.h": case "sqlite3ext.h":
f, err = os.Create("sqlite3ext.h") f, err = os.Create("../sqlite3ext.h")
default: default:
continue continue
} }
@ -153,7 +162,11 @@ func main() {
for scanner.Scan() { for scanner.Scan() {
text := scanner.Text() text := scanner.Text()
if text == `#include "sqlite3.h"` { if text == `#include "sqlite3.h"` {
text = `#include "sqlite3-binding.h"` text = `#include "sqlite3-binding.h"
#ifdef __clang__
#define assert(condition) ((void)0)
#endif
`
} }
_, err = fmt.Fprintln(f, text) _, err = fmt.Fprintln(f, text)
if err != nil { if err != nil {
@ -182,9 +195,9 @@ func main() {
var f *os.File var f *os.File
switch path.Base(zf.Name) { switch path.Base(zf.Name) {
case "userauth.c": case "userauth.c":
f, err = os.Create("userauth.c") f, err = os.Create("../userauth.c")
case "sqlite3userauth.h": case "sqlite3userauth.h":
f, err = os.Create("userauth.h") f, err = os.Create("../userauth.h")
default: default:
continue continue
} }
@ -207,10 +220,10 @@ func main() {
} }
// Merge SQLite User Authentication into amalgamation // Merge SQLite User Authentication into amalgamation
if err := mergeFile("userauth.c", "sqlite3-binding.c"); err != nil { if err := mergeFile("../userauth.c", "../sqlite3-binding.c"); err != nil {
log.Fatal(err) log.Fatal(err)
} }
if err := mergeFile("userauth.h", "sqlite3-binding.h"); err != nil { if err := mergeFile("../userauth.h", "../sqlite3-binding.h"); err != nil {
log.Fatal(err) log.Fatal(err)
} }