Commit Graph

56 Commits

Author SHA1 Message Date
Jason Kwon d24fbd02da Adds the ability to use SQS FIFO queues.
All SQS FIFO queues have the `.fifo` suffix at the end of their URL - this is how we determine if we are using a FIFO queue or not. The `MessageGroupId` (A required field for FIFO) is set using the key & id of the event's message.

Note: This commit does not add the the `MessageDeduplicationID` so the FIFO queue must have content-based deduplication enabled.
2024-12-05 11:57:50 +01:00
oftenoccur c3b9b40ee8 chore: fix some typos in comments
Signed-off-by: oftenoccur <ezc5@sina.com>
2024-04-11 11:16:44 +08:00
Josh Baker ee61cd11c1
Merge pull request #702 from LLluma/bug_sqs_url_china_region
[BUG] Fix AWS SQS base domain parsing for China region
2023-11-20 11:27:32 -07:00
Chris Rice c113180d0b Add support for 'none' authentication for kafka while still allowing SSL 2023-09-08 08:58:23 -07:00
Yaroslav Shumlianskyi 8ac31c6234 [BUG] Fix AWS SQS base domain parsing for China region
For AWS China region base domain is slightly different than for
the rest world. It is `amazon.com.cn`.
Currently AWS domain is hard coded in the `sqs.go`. Would be nice if
that will be configured parameter.

But this is a minimal posible change which is not breaking exist code
and make possible to use it for AWS China region.
2023-08-22 21:56:20 +03:00
tidwall fd72c5e539 Update dependencies
Update golang.org/x/text and xdg-go/scram

Fixes #671
2023-05-11 03:46:33 -07:00
tidwall 9ea0803ce0 Merge branch 'master' of https://github.com/pacaj2am/tile38 into pacaj2am-master 2023-05-06 06:56:00 -07:00
uwer c63468a704 disabled AMQP queue and binding for type topic 2023-05-03 16:22:37 +10:00
tidwall 9c471e3f9c Replace out old style atomics 2022-11-03 10:07:17 -07:00
tidwall 3cb8e0509a Thread safe log and support for concurrent tile38 instances 2022-09-26 10:02:02 -07:00
tidwall 906824323b More graceful Tile38 shutdown 2022-09-25 06:28:17 -07:00
tidwall dd11eded5c Cleanup code for Go 1.19 2022-09-12 17:06:27 -07:00
Benjamin Ramser 09ca96717b fix: allow host ca sets for SASL and TLS connections
?ssl=true previously would require the user to provide a cacertfile
stripping the option to use the hosts ca set.

bumping sarama to version 1.36.0
bumping alpine to 3.16.2

fix: tls path
2022-09-07 11:10:10 +02:00
Simon Tuohy 4454995e06 Azure EventHub hook support 2021-10-15 15:22:41 +01:00
Benjamin Ramser 9ee91ca5c3 refactor: kafka auth 2021-08-03 14:46:40 +02:00
tidwall d2f747933f clear linter messages 2021-07-10 03:59:27 -07:00
Josh d3d4694691
Merge pull request #504 from tidwall/nats-tls
Added TLS support for Nats webhook provider
2021-07-08 07:42:54 -07:00
tidwall aea7d77de5 Fix Memory Leak in Kafka Producer
This commit addresses an issue where the sarama kafka library
leaks memory when a connection closes unless the metrics
configuration that was passed to new connection is also closed.

Fixes #613
2021-07-08 06:46:58 -07:00
Benjamin Ramser bb2253204c Add option to select sha function 2021-07-08 06:46:57 -07:00
Benjamin Ramser 6dfcf896d4 Leave ClientID default untouched 2021-07-08 06:46:57 -07:00
Benjamin Ramser e4b03ca174 Add endpoint option for sasl 2021-07-08 06:46:57 -07:00
Mathieu 55e503c378 Add sasl 2021-07-08 06:46:57 -07:00
tidwall 6bb9f8f54c Remove deprecated function
Go recommends not calling BuildNameToCertificate and to allow
for the names to be selected automatically.
2021-07-08 06:46:12 -07:00
tidwall 58221adccb Code cleanup
- Removed unused functions and variables
- Wrapped client formatted errors
- Updated deprecated packages
- Changed suggested code patterns
2021-07-08 06:46:08 -07:00
Benjamin Ramser 8234f6dc6d refactor: dont set client id 2021-07-08 06:36:10 -07:00
tidwall b64c87004f Expose log output writer 2021-07-08 06:36:10 -07:00
Benjamin Ramser 248c3d8b72 add kafka tls config
fix endpoint, add logging to tlsconfig creation

add logging if log.Level > 2
2021-07-08 06:36:10 -07:00
tidwall 094d35757c Various updates
- Updated all dependencies
- Updated geoindex Box api
2021-07-08 06:36:05 -07:00
tidwall 1477026721 Updated Kafka version 2021-07-08 06:34:15 -07:00
Jan Pacák 71fa16a19e amqp: removal of queue and binding creation 2021-05-13 00:44:27 +02:00
tidwall 5e440b621b Remove deprecated function
Go recommends not calling BuildNameToCertificate and to allow
for the names to be selected automatically.
2021-03-31 08:15:52 -07:00
tidwall 6b08f7fa9e Code cleanup
- Removed unused functions and variables
- Wrapped client formatted errors
- Updated deprecated packages
- Changed suggested code patterns
2021-03-31 08:13:44 -07:00
tidwall 0d69799653 Merge branch 'kafka-tls' of https://github.com/iwpnd/tile38 into iwpnd-kafka-tls 2021-02-18 15:34:09 -07:00
tidwall 9ac25647ad Expose log output writer 2021-02-18 15:34:01 -07:00
Benjamin Ramser d7c0c5b855 refactor: dont set client id 2021-02-18 22:35:49 +01:00
Benjamin Ramser 734d33365a add kafka tls config
fix endpoint, add logging to tlsconfig creation

add logging if log.Level > 2
2021-02-17 14:35:42 +01:00
tidwall 3ed048242e Various updates
- Updated all dependencies
- Updated geoindex Box api
2021-02-03 14:30:55 -07:00
tidwall 68e2b6d736 Updated Kafka version 2020-07-03 11:57:15 -07:00
Mads Schou-Andreasen cf757802a3 fixed default credentials 2020-05-30 10:26:25 +02:00
Mads Schou-Andreasen bff11ad0ae minor documentation fix 2020-05-19 17:24:06 +02:00
Mads Schou-Andreasen 37224791f4 added geofence webhook for GCP Pubsub 2020-05-19 17:11:31 +02:00
tidwall 3718cd766b Added priority option for AMQP endpoints 2020-05-16 14:31:52 -07:00
tidwall c084aeedc2 Code cleanup
This commit cleans up various Go code in the internal directory.
- Ensures comments on exported functions
- Changes all *Server receiver in all files to be "s", instead
  of mixed "c", "s", "server", etc.
- Silenced Go warnings for if/else with returns.
- Cleaned up import ordering.
2019-10-30 10:17:59 -07:00
tidwall feba00040e Added TLS support for Nats webhook provider
Use the `tls=1` and the set the the `tlscert` and `tlskey` query
string params. The cert and key files must be on the tile38
server and the Nats server must be started using the same files.

nats://54.12.34.121:4222/fleet?tls=1&tlscert=cert.crt&tlskey=cert.key
2019-10-30 08:17:40 -07:00
tidwall b092cea0d2 Use WaitTimeout for MQTT 2019-10-08 11:13:18 -07:00
Plamen Todorov 6b82fd94eb randomize mqtt client id with math/rand
Cryptographic randomizer is not required for mqtt clientIds. They
should be unique only among currently selected clients.
2019-10-08 20:24:31 +03:00
Plamen Todorov c3b9a689bb Use uuid as mqtt clientId
Use crypto/random to generate unique mqtt client id. The tile38 prefix
makes the connections easily identifiable in the mqtt broker logs.
2019-10-08 09:34:31 +03:00
Plamen Todorov fb2aef2ce6 MQTT clientId should be unique
Each mqtt hook establishes separate connection to the MQTT broker. If
their clientIds are all equal the MQTT broker will disconnect the clients - the
protocol does not allow 2 connected clients with the same name
2019-10-06 22:15:06 +03:00
tidwall 7e006af713 Add verbose messaging for aws/sqs 2019-03-17 18:40:34 -07:00
tidwall 5335aec942 Allow for standard SQS URLs
Both now work:

https://sqs.us-east-1.amazonaws.com/349840735605/TestTile38Queue
sqs://us-east-1:349840735605/TestTile38Queue
2019-03-13 15:41:49 -07:00