mirror
/
tile38
mirror of https://github.com/tidwall/tile38.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,115 Commits 14 Branches 101 Tags 54 MiB
Commit Graph

80 Commits

Author SHA1 Message Date
tidwall 9e68703841 Update expiration logic 
This commit changes the logic for managing the expiration of
objects in the database.

Before: There was a server-wide hashmap that stored the
collection key, id, and expiration timestamp for all objects
that had a TTL. The hashmap was occasionally probed at 20
random positions, looking for objects that have expired. Those
expired objects were immediately deleted, and if there was 5
or more objects deleted, then the probe happened again, with
no delay. If the number of objects was less than 5 then the
there was a 1/10th of a second delay before the next probe.

Now: Rather than a server-wide hashmap, each collection has
its own ordered priority queue that stores objects with TTLs.
Rather than probing, there is a background routine that
executes every 1/10th of a second, which pops the expired
objects from the collection queues, and deletes them.

The collection/queue method is a more stable approach than
the hashmap/probing method. With probing, we can run into
major cache misses for some cases where there is wide
TTL duration, such as in the hours or days. This may cause
the system to occasionally fall behind, leaving should-be
expired objects in memory. Using a queue, there is no
cache misses, all objects that should be expired will be
right away, regardless of the TTL durations.

Fixes #616
 2021-07-12 13:37:50 -07:00
tidwall 3c699183e0 Additional KNN test 2021-07-11 14:49:23 -07:00
tidwall 579a41abae Merge branch 'housecanary-fix-knn' 2021-07-11 10:02:59 -07:00
tidwall 0d83b1ca53 Merge branch 'rshura-optimize-field-match' 2021-07-10 17:59:22 -07:00
Josh 1467cba769
Merge pull request #552 from rshura/clip-by 
Add CLIPBY subcommand to INTERSECTS/WITHIN
 2021-07-10 09:24:57 -07:00
Oliver 56a0ad70c6 fix tests 2021-05-13 21:14:06 -04:00
Oliver de055384f9 add tests 2021-05-13 21:14:06 -04:00
Oliver 6b0282d960 add more metrics, add metrics index page 2021-05-13 21:14:05 -04:00
Oliver 7ec4d58392 cleanup / fix tests 2021-05-13 21:14:05 -04:00
Oliver c78415e26e go fmt 2021-05-13 21:12:49 -04:00
Benjamin Ramser 2a8b98778b fix: distance if point and object have the same coordinates 2021-03-19 09:42:38 +01:00
tidwall 4f4954020f enable fence tests 2020-11-03 14:56:31 -07:00
tidwall 4504ea75ac Updated Github Action files 
Disabled some fence tests due to them hanging and
timing out in the Github Actions workflow, which is
something that did not happen with Travis CI.

I suspect that there's something up with Go routines
and or num of CPUs in the GH virtual environment.

Tests pass locally though
 2020-11-03 14:36:07 -07:00
tidwall 100be7be3c Added tests 2020-10-23 09:50:51 -07:00
tidwall b1dc463c41 Updated dependencies 2020-09-22 16:43:58 -07:00
Alex Roitman 25579a052c Fix a bug in WHEREIN -- 0 values would always match, incorrectly. 2020-04-12 16:06:10 -07:00
Mike Poindexter fe6e3863ba Add bench tests for knn 2020-04-08 11:36:37 -07:00
Alex Roitman 34cb2affdc Add clipby subcommand to INTERSECTS/WITHIN 2020-04-03 16:49:06 -07:00
tidwall ff48054d3d Fixed a missing faraway event for roaming geofences 
This commit fixes a case where a roaming geofence will not fire
a "faraway" event when it's supposed to.

The fix required rewriting the nearby/faraway detection logic. It
is now much more accurate and takes overall less memory, but it's
also a little slower per operation because each object proximity
is checked twice per update. Once to compare the old object's
surrounding, and once to evaulated the new object. The two lists
are then used to generate accurate "nearby" and "faraway" results.
 2020-03-22 11:54:56 -07:00
Josh Baker 90c2474e3d
Merge pull request #501 from JordanArmstrong/strict-jset-numbers 
Strictly check if values passed to JSET are numbers
 2019-10-29 15:28:40 -07:00
Josh Baker df477bf3f4
Merge pull request #464 from rshura/area-expression 
Add area expressions.
 2019-10-28 13:45:10 -07:00
Jordan Armstrong 41fb410e2e Strictly check if values to JSET are numbers 
Fixes #493
 2019-10-26 23:37:48 -03:00
tidwall f7888c1edf Fixed malformed json for chans command 
Mentioned by ds2xor on Slack
 2019-08-03 10:10:28 -07:00
Alex Roitman dd09ffbe13 Fix parser for multiple negations. 2019-06-14 10:02:26 -07:00
Alex Roitman 2d83e18934 Add expression errors test. Make parser stricter. 2019-06-13 13:10:47 -07:00
Alex Roitman 0c3a5d02ca Fixes 2019-06-13 12:04:04 -07:00
Alex Roitman 496ace25d3 Refactoring/cleanup/fixes. 2019-06-11 17:13:33 -07:00
Alex Roitman d0ca579f5c Add area expressions. 
Add parser.
Hook up to test command.
Add tests for expressions in test command.
 2019-06-10 14:47:42 -07:00
Melissa Baker bedddcbe13 Added RESP output fields test for NEARBY/WITHIN 2019-06-04 10:43:32 -07:00
Melissa Baker 4ba237b10e Added test for correct json fields output 2019-05-28 12:28:56 -07:00
tidwall b86d2287f4 Removed elapsed member for json testing 2019-05-28 09:22:15 -07:00
Alex Roitman cbf10b7472 Rework timeouts to allow prepending any command with the TIMEOUT seconds 2019-04-25 14:15:53 -07:00
Alex Roitman 2020f537dc Add tests for script timeouts. 2019-04-24 17:17:49 -07:00
Alex Roitman 31525487c3 Add timeout tests. Fix a bug. 2019-04-24 13:20:57 -07:00
tidwall fc3e8b4359 Fix nearby with match query invalid results 
closes #421
 2019-03-01 06:55:26 -07:00
tidwall 30f903bd51 Require properties member for geojson features 2019-02-12 06:49:13 -07:00
tidwall fb7259b10b Changed clip errors and json result type 2019-02-12 05:33:20 -07:00
Alex Roitman 1cffdcc31f Add tests for the TEST comand 2019-02-08 13:58:04 -08:00
Steven Wolfe ffd8ca915c Testing for valid INFO and CLIENT Json output 2019-01-15 11:08:19 -07:00
Steven Wolfe 9c8639497f Adding test for 'not' operator 2019-01-14 11:37:11 -07:00
Steven Wolfe 4392fb44bd Fixing KEYS command and tests 2019-01-14 11:06:12 -07:00
Steven Wolfe 087975abd1 Roaming geofence tests 2019-01-10 11:28:29 -07:00
Steven Wolfe a5fbcf5f15 Ensuring channel publish order 2019-01-09 00:23:53 -07:00
Alex Roitman 5e215c247d Add distance_to function to the tile38 namespace in lua. 
distance_to(lat_a, lon_a, lat_b, lon_b) returns distance in meters between points A and B.
 2019-01-04 14:57:00 -08:00
Alex Roitman e1c82e17f7 Refactor to be more consistent with redis responses. 2018-12-28 14:46:54 -08:00
Alex Roitman 01a7dda2a1 Add RENAME and RENAMENX commands. 2018-12-27 17:15:53 -08:00
Alex Roitman 0933c541f4 Refactor cursor/paging. 2018-10-31 22:01:37 -07:00
tidwall 60b054d58c Fixed test 2018-10-29 08:16:04 -07:00
tidwall e577f60481 Updated redigo imports 2018-10-29 05:00:54 -07:00
tidwall 555e47036c Replaced net package with evio 
- Added threads startup flag
- Replaced net package with evio
- Refactored controller into server
 2018-10-28 15:51:47 -07:00