Commit Graph

51 Commits

Author SHA1 Message Date
Alexander Yastrebov 6c05a9f6b1 fix: reduce `SetAddrs` shards lock contention
Introduces a new lock to make `SetAddrs` calls exclusive.
This allows release of shards lock for the duration of potentially long `newRingShards` call.

`TestRingSetAddrsContention` observes number of pings increased from <1000 to ~40_000.

See https://github.com/go-redis/redis/pull/2190#discussion_r953040289

Updates #2077
2022-11-22 23:55:13 +01:00
Alexander Yastrebov a31c1d6ff0 test: add TestRingSetAddrsContention
See https://github.com/go-redis/redis/pull/2190#discussion_r953040289

Updates #2077
2022-11-21 16:06:22 +01:00
Vladimir Mihailenco 0dff3d1461 feat: add OpenTelemetry metrics instrumentation 2022-10-12 11:09:41 +03:00
Vladimir Mihailenco b0231c659e chore: always retry write timeouts 2022-10-06 10:06:02 +03:00
Vladimir Mihailenco 20ba6d59e6 chore: cleanup 2022-08-17 13:50:03 +03:00
Vladimir Mihailenco 59798f9dba chore: cleanup 2022-08-17 13:18:58 +03:00
Sandor Szücs 6f7f800107
feat: ring.SetAddrs to add and remove shards by the ring client and reuse old connections
test: ring scale-in and scale-out

rewrite as suggested by @AlexanderYastrebov
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
2022-08-01 22:06:09 +02:00
Max Riveiro a80b84f01f fix: handle panic in ringShards Hash function when Ring got closed
Fixes #2126
2022-07-13 18:09:42 +03:00
Vladimir Mihailenco a18fad5bd3 chore: v9 2022-06-04 17:42:06 +03:00
Monkey 335d946cd6
chore: sync master (#2051)
* Upgrade redis-server version (#1833)

* Upgrade redis-server version

Signed-off-by: monkey <golang@88.com>

* XAutoClaim changed the return value

Signed-off-by: monkey <golang@88.com>

* add cmd: geosearch, geosearchstore (#1836)

* add cmd: geosearch, geosearchstore

Signed-off-by: monkey92t <golang@88.com>

* GeoSearchQuery and GeoSearchLocationQuery changed to pointer passing

Signed-off-by: monkey92t <golang@88.com>

* Added missing method XInfoStreamFull to Cmdable interface

* Run go mod tidy in redisotel

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* Revert "ConnPool check  fd for bad conns (#1824)" (#1849)

This reverts commit 346bfafddd.

* Automate release process (#1852)

* Bump github.com/onsi/gomega from 1.10.5 to 1.14.0 (#1832)

* Bump github.com/onsi/gomega from 1.10.5 to 1.14.0

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.10.5 to 1.14.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.10.5...v1.14.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Upgrade gomega to v1.15.0

Signed-off-by: monkey92t <golang@88.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: monkey92t <golang@88.com>

* Add version.go

* Fix otel example

* Fix package name in release script

* More fixes for otel example

* And more

* Fix release.sh

* Release v8.11.3 (release.sh)

* Create an annotated tag to give release.yml chance to run

* Tweak tag.sh

* Add Cmd.Slice helper to cast to []interface{} (#1859)

* after the connection pool is closed, no new connections should be added (#1863)

* after the connection pool is closed, no new connections should be added

Signed-off-by: monkey92t <golang@88.com>

* remove runGoroutine

Signed-off-by: monkey92t <golang@88.com>

* pool.popIdle add p.closed check

Signed-off-by: monkey92t <golang@88.com>

* upgrade golangci-lint v1.42.0

Signed-off-by: monkey92t <golang@88.com>

* Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#1865)

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add go 1.17 to the build matrix

* Remove go 1.15 from build matrix

* Add scan struct example (#1870)

* Replace release job

* Bump github.com/cespare/xxhash/v2 from 2.1.1 to 2.1.2 (#1872)

Bumps [github.com/cespare/xxhash/v2](https://github.com/cespare/xxhash) from 2.1.1 to 2.1.2.
- [Release notes](https://github.com/cespare/xxhash/releases)
- [Commits](https://github.com/cespare/xxhash/compare/v2.1.1...v2.1.2)

---
updated-dependencies:
- dependency-name: github.com/cespare/xxhash/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix tag script to push tag by tag

* Fix releasing.md

* Fix/pubsub ping mutex (#1878)

* Fix PubSub.Ping to hold the lock

* Fix PubSub.Ping to hold the lock

* add write cmd data-race test

Signed-off-by: monkey92t <golang@88.com>

Co-authored-by: monkey92t <golang@88.com>

* chore: cleanup OpenTelemetry example

* chore: gofmt all code

* Refactor TestParseURL

This is in preparation for supporting query parameters
in ParseURL:

- use an expected *Options instance to execute assertions on
- extract assertions into helper function
- enable parallel testing
- condense test table

* Add query parameter parsing to ParseURL()

Before this change, ParseURL would only accept a very restricted
set of URLs (it returned an error, if it encountered any parameter).

This commit introduces the ability to process URLs like

	redis://localhost/1?dial_timeout=10s

and similar.

Go programs which were providing a configuration tunable (e.g.
CLI flag, config entry or environment variable) to configure
the Redis connection now don't need to perform this task
themselves.

* chore: add links to readme

* chore: fix discussions link

* empty hooks.withContext removed

* chore: gofmt

* chore: use conventional commits and auto-generate changelog

* feat: add acl auth support for sentinels

* chore: swap to acl auth at the test-level

* Add support for BLMove command

* chore: update dependencies

* chore: update link

* feat: add SetVal method for each command

* feat: add Cmd.{String,Int,Float,Bool}Slice helpers and an example

* chore: tweak GH actions to run all jobs

* chore: add Lua scripting example

* Fix Redis Cluster issue during roll outs of new nodes with same addr (#1914)

* fix: recycle connections in some Redis Cluster scenarios

This issue was surfaced in a Cloud Provider solution that used for
rolling out new nodes using the same address (hostname) of the nodes
that will be replaced in a Redis Cluster, while the former ones once
depromoted as Slaves would continue in service during some mintues
for redirecting traffic.

The solution basically identifies when the connection could be stale
since a MOVED response will be returned using the same address (hostname)
that is being used by the connection. At that moment we consider the
connection as no longer usable forcing to recycle the connection.

* chore: lazy reload when moved or ask

* chore: use conv commit message

* chore: release v8.11.4 (release.sh)

* fix: add whitespace for avoid unlikely colisions

* fix: format

* chore: fix links

* chore: use ctx parameter in cmdInfo

* Bump github.com/onsi/ginkgo from 1.16.4 to 1.16.5 (#1925)

Bumps [github.com/onsi/ginkgo](https://github.com/onsi/ginkgo) from 1.16.4 to 1.16.5.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v1.16.4...v1.16.5)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: add support for time.Duration write and scan

* test: add test case for setting and scanning durations

* chore: fix linter

* fix(extra/redisotel): set span.kind attribute to client

According to the opentelemetry specification this should always be set to client for database client
libraries.

I've also removed the SetAttributes call and instead set the attributes during creation of the span.
This is what the library SHOULD be doing according to the opentelemetry api specification.

* chore: update otel example

* fix: update some argument counts in pre-allocs

In some cases number of pre-allocated places in
argument array is missing 1 or 2 elements,
which results in re-allocation of twice as large array

* chore: add example how to delete keys without a ttl

* chore: don't enable all lints

* chore(deps): bump github.com/onsi/gomega from 1.16.0 to 1.17.0

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat: Add redis v7's NX, XX, GT, LT expire variants

* chore: add missing readme

* chore: tweak feature links

* chore: remove Discord

* fix: set timeout for WAIT command. Fixes #1963

* build: update `go` directive in `go.mod` to 1.17

This commit enables support for module graph pruning and lazy module
loading for projects that are at Go 1.17 or higher.

Reference: https://go.dev/ref/mod#go-mod-file-go
Reference: https://go.dev/ref/mod#graph-pruning
Reference: https://go.dev/ref/mod#lazy-loading
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* chore: update link

* chore: export cmder.SetFirstKeyPos to support build module commands

* feat(redisotel): ability to override TracerProvider (#1998)

* fix: add missing Expire methods to Cmdable

This is a followup to https://github.com/go-redis/redis/pull/1928

* chore(deps): bump github.com/onsi/gomega from 1.17.0 to 1.18.1

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.17.0 to 1.18.1.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.17.0...v1.18.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update README.md (#2011)

chore: add fmt library in example code

* chore: instrumentation name and version (#2012)

* fix: invalid type assert in stringArg

* chore: cleanup

* fix: example/otel compile error (#2028)

* fix: rename Golang to Go (#2030)

https://go.dev/doc/faq#go_or_golang

* feat: add support for passing extra attributes added to spans

* feat: set net.peer.name and net.peer.port in otel example

* chore: tweak Uptrace copy

* feat: add support for COPY command (#2016)

* feat: add support for acl sentinel auth in universal client

* chore(deps): bump actions/checkout from 2 to 3

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: add hll example

* chore: tweak release script

* chore: release v8.11.5 (release.sh)

* chore: add discord back

Co-authored-by: Eugene Ponizovsky <ponizovsky@gmail.com>
Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Vladimir Mihailenco <vladimir.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kishan B <kishancs46@gmail.com>
Co-authored-by: Dominik Menke <dom@digineo.de>
Co-authored-by: Gökhan Özeloğlu <gozeloglu@gmail.com>
Co-authored-by: Justin Sievenpiper <justin@sievenpiper.co>
Co-authored-by: Алексей Романовский <aromanovsky@epiphan.com>
Co-authored-by: Stavros Panakakakis <stavrospanakakis@gmail.com>
Co-authored-by: Pau Freixes <pfreixes@gmail.com>
Co-authored-by: Ethan Hur <ethan0311@gmail.com>
Co-authored-by: Jackie <18378976+Pyrodash@users.noreply.github.com>
Co-authored-by: Kristinn Björgvin Árdal <kristinnardalsecondary@gmail.com>
Co-authored-by: ffenix113 <razerer@bigmir.net>
Co-authored-by: Bastien Penavayre <bastienPenava@gmail.com>
Co-authored-by: James3 Li(李麒傑) <james3_li@asus.com>
Co-authored-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: gzjiangtao2014 <gzjiangtao2014@corp.netease.com>
Co-authored-by: Nelz <nelz9999@users.noreply.github.com>
Co-authored-by: Daniel Richter <Nexyz9@gmail.com>
Co-authored-by: Seyed Ali Ghaffari <ali.ghaffari@outlook.com>
Co-authored-by: lintanghui <lintanghui@bilibili.com>
Co-authored-by: hidu <duv123+github@gmail.com>
Co-authored-by: Jonas Lergell <jonas.lergell@volvocars.com>
Co-authored-by: Alex Kahn <alexanderkahn@gmail.com>
2022-03-19 12:40:31 +08:00
monkey92t 38d1749d56
adjust the code (#1842)
* Upgrade redis-server version (#1833)

* Upgrade redis-server version

Signed-off-by: monkey <golang@88.com>

* XAutoClaim changed the return value

Signed-off-by: monkey <golang@88.com>

* add cmd: geosearch, geosearchstore (#1836)

* add cmd: geosearch, geosearchstore

Signed-off-by: monkey92t <golang@88.com>

* GeoSearchQuery and GeoSearchLocationQuery changed to pointer passing

Signed-off-by: monkey92t <golang@88.com>

* adjust the code, and fix #1553, #1676

Signed-off-by: monkey92t <golang@88.com>
2021-08-02 19:01:01 +08:00
monkey92t 8ad01240a4
Add support for resp3 protocol (#1739)
* support resp3 protocol

Signed-off-by: monkey <golang@88.com>

* Upgrade mod version limit go1.14

https://github.com/go-redis/redis/issues/1715#issuecomment-820685614

Signed-off-by: monkey <golang@88.com>

* Remove the redundant check of ReadReply

Signed-off-by: monkey <golang@88.com>

* fix the problem

Signed-off-by: monkey <golang@88.com>

* workflows add v9

Signed-off-by: monkey <golang@88.com>

* update StringStringMapCmd to MapStringStringCmd

Signed-off-by: monkey <golang@88.com>
2021-04-27 10:04:46 +03:00
monkey 4a7d03359b add miss Pipeline and TxPipeline
Signed-off-by: monkey <golang@88.com>
2021-03-07 17:34:40 +08:00
monkey 783725f3f7 fix ring test `supports Process hook` data race
Signed-off-by: monkey <golang@88.com>
2021-03-07 17:10:19 +08:00
monkey f4ae0ba02b solve possible unit test errors in ring_test.go
/redis/ring_test.go:560, Expected: <int>: 10, to equal: <int>: 9
/redis/ring_test.go:560, Expected: <int>: 1, to equal: <int>: 0

Signed-off-by: monkey <golang@88.com>
2021-02-25 11:43:41 +08:00
Vladimir Mihailenco 694e518a8c Use Rendezvous in Ring. Thanks @rafaeleyng for initial idea and implementation 2020-06-08 13:22:37 +03:00
Vladimir Mihailenco 073d805708 Fix build 2020-06-05 09:55:37 +03:00
Vladimir Mihailenco 558263e5eb Remove deprecated Ring options 2020-05-19 08:52:38 +03:00
Vladimir Mihailenco 3dcb844106 Bump major version 2020-05-19 08:52:38 +03:00
Vladimir Mihailenco f5593121e0 Add ctx as first arg 2020-05-19 08:52:38 +03:00
Dirkjan Bussink 9a49a4d91d
Add client instantiation callback to allow more flexible configuration (#1281)
* Add NewClient to Ring options
2020-03-27 15:48:18 +02:00
Vladimir Mihailenco 218b17f0fc Include multi & exec in pipeline hook 2020-02-14 15:55:16 +02:00
Vladimir Mihailenco 49a0c8c319 Add test for ring and cluster hooks 2020-02-14 14:30:40 +02:00
Vladimir Mihailenco dd4ef4e9cb Change Tx.Pipeline and Tx.TxPipeline meaning 2020-01-12 14:19:21 +02:00
Vladimir Mihailenco 7779accbea Use semantic import versioning 2019-08-08 14:29:44 +03:00
Michael Benford 9169633580 Add support for individual passwords for ring shards 2019-07-17 11:31:42 -03:00
Vladimir Mihailenco 2cbb5194fb Fix WithContext and add tests 2019-07-04 11:57:54 +03:00
Vladimir Mihailenco 8cc316035b Add GinkgoRecover 2019-06-16 14:47:24 +03:00
Vladimir Mihailenco 8476dfea4a Replace Wrap* with hooks that support context 2019-05-31 16:36:57 +03:00
Andrea Spacca 97e6ed8178 Ring instrumentantions (#1017)
* Ring instrumentantions
2019-04-24 09:33:36 +03:00
Andrea Spacca be4c4f3f38 Ring Watch (#1015)
* Ring Watch
2019-04-22 12:48:06 +03:00
Vladimir Mihailenco ea9da7c2e8 Rework ReceiveMessage 2018-07-23 17:10:58 +03:00
Vladimir Mihailenco f1ed2ad288 Add FlushDBAsync and FlushAllAsync 2017-06-17 12:53:16 +03:00
Felipe Cavalcanti eeb4d09198 rename Pipelineable to Pipeliner 2017-05-02 12:00:53 -03:00
Felipe Cavalcanti 6fca4d5ad0 pipeline now has its own interface "Pipelineable" 2017-05-01 12:42:58 -03:00
Vladimir Mihailenco 8040d63c4f Remove gopkg.in 2017-02-19 09:49:21 +02:00
Vladimir Mihailenco 639950777c More cluster tests. 2016-10-09 11:27:38 +00:00
Vladimir Mihailenco f5245efa73 Prepare v5 release. 2016-10-09 10:56:53 +00:00
Artem Chernyshev 03da66c18a Add new parameter to the RingOptions to override CommandInfo first key
There is problem with `eval` and `evalsha` commands.
`COMMAND INFO eval` returns first key position equals `0`.
After that, redis ring chooses `eval` as a value for sharding.
They, if you try to delete created value, ring may choose another shard
and delete won't work.

Eval command should be parsed, to be sharded properly, according to
redis specs: http://redis.io/commands/command .

I've introduced a new flag in the `RingOptions`, which will enable new
behavior: `EnableKeyLocationParsing`.

If it is enabled, `cmdFirstKey` will try to get key position using
`cmd.getFirstKeyPos()`. This function is defined for `eval` and
`evalsha` commands.
If it has parameters, it will return `3`, otherwise it will return `0`.
2016-10-04 17:09:04 +03:00
Vladimir Mihailenco 7cbee9d337 Enable reaper on ClusterClient and add tests. 2016-10-02 12:44:01 +00:00
Vladimir Mihailenco 66f2eb1584 ring: reduce HeartbeatFrequency. 2016-08-09 13:54:04 +00:00
Vladimir Mihailenco 38d30a4bab Use redis.v4 that is in alpha/beta state. 2016-04-09 13:27:16 +03:00
Vladimir Mihailenco 3b051d2374 Reuse single Pipeline type in Client, ClusterClient and Ring. 2016-04-09 10:47:15 +03:00
Vladimir Mihailenco 36487d8462 Stabilize build. 2015-12-22 12:08:35 +02:00
Vladimir Mihailenco b6b689904a Fix test. 2015-11-22 17:53:44 +02:00
Vladimir Mihailenco 98414ea72a Increase test timeout. 2015-11-21 10:20:01 +02:00
Vladimir Mihailenco 54dcf59909 ring: return an error in pipeline when all shards are down. 2015-07-13 12:56:16 +03:00
Vladimir Mihailenco 1608a33e55 ring: fix key hashing in Ring pipeline. 2015-06-24 15:37:41 +03:00
Vladimir Mihailenco fee949ecbf ring: improve pipelining tests. 2015-06-24 13:41:50 +03:00
Vladimir Mihailenco 0cf1b73698 Add RingPipeline. 2015-06-04 11:50:24 +03:00