mirror
/
tile38
mirror of https://github.com/tidwall/tile38.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,214 Commits 14 Branches 101 Tags 54 MiB
Commit Graph

1214 Commits

Author SHA1 Message Date
tidwall 9e68703841 Update expiration logic 
This commit changes the logic for managing the expiration of
objects in the database.

Before: There was a server-wide hashmap that stored the
collection key, id, and expiration timestamp for all objects
that had a TTL. The hashmap was occasionally probed at 20
random positions, looking for objects that have expired. Those
expired objects were immediately deleted, and if there was 5
or more objects deleted, then the probe happened again, with
no delay. If the number of objects was less than 5 then the
there was a 1/10th of a second delay before the next probe.

Now: Rather than a server-wide hashmap, each collection has
its own ordered priority queue that stores objects with TTLs.
Rather than probing, there is a background routine that
executes every 1/10th of a second, which pops the expired
objects from the collection queues, and deletes them.

The collection/queue method is a more stable approach than
the hashmap/probing method. With probing, we can run into
major cache misses for some cases where there is wide
TTL duration, such as in the hours or days. This may cause
the system to occasionally fall behind, leaving should-be
expired objects in memory. Using a queue, there is no
cache misses, all objects that should be expired will be
right away, regardless of the TTL durations.

Fixes #616
 2021-07-12 13:37:50 -07:00
tidwall 3c699183e0 Additional KNN test 2021-07-11 14:49:23 -07:00
tidwall dd4d31ae1b Fix last merge 2021-07-11 10:09:51 -07:00
tidwall 579a41abae Merge branch 'housecanary-fix-knn' 2021-07-11 10:02:59 -07:00
tidwall 991963268a Fix last merge 2021-07-10 19:32:21 -07:00
tidwall f2bbf10c36 Merge branch 'mpoindexter-optimize-field-value-access' 2021-07-10 19:26:29 -07:00
tidwall 0d83b1ca53 Merge branch 'rshura-optimize-field-match' 2021-07-10 17:59:22 -07:00
Josh 1467cba769
Merge pull request #552 from rshura/clip-by 
Add CLIPBY subcommand to INTERSECTS/WITHIN
 2021-07-10 09:24:57 -07:00
Josh 43e2722574
Merge pull request #555 from housecanary/fix-clip-test 
Fix clip test after geoOptions change.
 2021-07-10 09:23:50 -07:00
tidwall d2f747933f clear linter messages 2021-07-10 03:59:27 -07:00
Josh d3d4694691
Merge pull request #504 from tidwall/nats-tls 
Added TLS support for Nats webhook provider
 2021-07-08 07:42:54 -07:00
tidwall 6c6e9b087b Merge branch 'mscno-feat/gcp-pubsub' 2021-07-08 07:37:36 -07:00
tidwall e60cbac7cf Merge distance updates 2021-07-08 07:03:36 -07:00
tidwall aea7d77de5 Fix Memory Leak in Kafka Producer 
This commit addresses an issue where the sarama kafka library
leaks memory when a connection closes unless the metrics
configuration that was passed to new connection is also closed.

Fixes #613
 2021-07-08 06:46:58 -07:00
tidwall 1497663b6d Close follower files before finishing aofshrink 
fixes #449
 2021-07-08 06:46:58 -07:00
Benjamin Ramser bb2253204c Add option to select sha function 2021-07-08 06:46:57 -07:00
Benjamin Ramser 6dfcf896d4 Leave ClientID default untouched 2021-07-08 06:46:57 -07:00
Benjamin Ramser e4b03ca174 Add endpoint option for sasl 2021-07-08 06:46:57 -07:00
Mathieu 55e503c378 Add sasl 2021-07-08 06:46:57 -07:00
tidwall a087852d47 1.24.3 2021-07-08 06:46:57 -07:00
tidwall f067e3ba67 Fix invalid healthz output 2021-07-08 06:46:57 -07:00
tidwall 3b3177963d 1.24.2 2021-07-08 06:46:57 -07:00
tidwall f7a6850edb Update Go version to 1.16 2021-07-08 06:46:57 -07:00
tidwall 4d0abbb8a4 1.24.1 2021-07-08 06:46:57 -07:00
tidwall 87702b3e29 Always OK for leader 2021-07-08 06:46:57 -07:00
tidwall 5a37198602 Added HEALTHZ command 
Returns 'ok' if the server is the leader or a follower with
a 'caught up' log.

This is mainly for HTTP connections that are using an
orchestration environment like kubernetes, but will work as a
general RESP command.

For HTTP a '200 OK' for 'caught up' and
'500 Internal Server Error' otherwise.

See #608
 2021-07-08 06:46:57 -07:00
tidwall d307d93c89 1.24.0 2021-07-08 06:46:57 -07:00
Oliver caaeff346f add quick blurb to README 2021-07-08 06:46:57 -07:00
Oliver fdc5de3e56 fix tests 2021-07-08 06:46:57 -07:00
Oliver bfb9145849 add tests 2021-07-08 06:46:57 -07:00
Oliver fc0ff42297 add more metrics, add metrics index page 2021-07-08 06:46:57 -07:00
Oliver f448599e72 cleanup / fix tests 2021-07-08 06:46:57 -07:00
Oliver a233aaa66e don't rely on stats code for new metrics if we can avoid it 2021-07-08 06:46:57 -07:00
tidwall 5b3aeb5db7 add prometheus metrics endpoint 2021-07-08 06:46:52 -07:00
Oliver d9fec8699a go fmt 2021-07-08 06:46:33 -07:00
tidwall 22c4a05f03 vendor dir 2021-07-08 06:46:28 -07:00
Sepehr Rafiee 2b6e7159d1 add deprecation warning for depricated threads flag 2021-07-08 06:46:13 -07:00
Sepehr Rafiee 552236f9f9 remove deprecated threads flag 2021-07-08 06:46:13 -07:00
tidwall 0b8254e7c2 Change 255 to 256 2021-07-08 06:46:13 -07:00
tidwall 5cf6c4e9f4 Add comment 2021-07-08 06:46:13 -07:00
tidwall 48b46f2752 Update btree 2021-07-08 06:46:13 -07:00
Josh Baker c01cb2a5eb Delete file 2021-07-08 06:46:13 -07:00
Josh Baker 7652d37c63 Update README.md 2021-07-08 06:46:13 -07:00
Josh Baker 45a5132b8c Update README.md 2021-07-08 06:46:13 -07:00
Josh Baker 8eefe46b71 Update README.md 2021-07-08 06:46:13 -07:00
tidwall c0d1fdf1ee Update the client libraries 2021-07-08 06:46:13 -07:00
Mike 1867841406 Update README.md 2021-07-08 06:46:13 -07:00
tidwall af4a56d534 1.23.0 2021-07-08 06:46:13 -07:00
tidwall b5691f7dd5 Allow for all command types for roaming event 
Prior to this commit roaming geofences only registered changes to
the SET command. Now it will work for SET/DEL/FSET, just like
non-roaming geofences.

To opt out of these events, explicitly choose which event you
would like to register when you create the geofences.

For example:

  NEARBY fleet FENCE COMMANDS set,del ROAM fleet * 5000

Will only fire off events from SET and DEL command.

Closes #597
 2021-07-08 06:46:13 -07:00
tidwall 189de6496a Update BuntDB 2021-07-08 06:46:13 -07:00