Commit Graph

1294 Commits

Author SHA1 Message Date
Bartlomiej Plotka 11ee9add27
gocollector: Reverted client_golang v1.12 addition of runtime/metrics metrics by default. (#1033)
Fixes https://github.com/prometheus/client_golang/issues/967

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-04-13 20:43:29 +02:00
Sourik Ghosh cd90f33be8
smart diff to testutil.GatherAndCompare (#998)
* added smart diff to testutil.GatherAndCompare

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* v2 testuitls added for better diff

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* replaced CollectAndCompare to CollectAndCompareV2 in testutil_test

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* renamed methods from v2 to withT

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* replaced testify with custom diff func

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* difflib GetUnifiedDiffString added with test

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* license meta data added to file

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>

* moved difflib to internal

Signed-off-by: Sourik Ghosh <sourikghosh31@gmail.com>
2022-04-13 18:38:05 +01:00
Manuel Rüger 0c691ed35f
go.mod: Exclude prometheus/client_golang v1.12.1 (#1027)
See also https://github.com/prometheus/client_golang/issues/1012

As suggested in https://github.com/prometheus/client_golang/issues/1012#issuecomment-1090482644

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-04-13 18:35:14 +01:00
Manuel Rüger 46d3dd4e6c
Bump minimum required Go version to 1.16 (#1032)
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-04-13 16:21:42 +01:00
Bartlomiej Plotka 24172847e3
gocollector: Added options to Go Collector for changing the (#1031)
* Renamed files.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* gocollector: Added options to Go Collector for diffetent collections.

Fixes https://github.com/prometheus/client_golang/issues/983

Also:

* fixed TestMemStatsEquivalence, it was noop before (:
* Removed gc_cpu_fraction metric completely, since it's not working completely for Go1.17+

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-04-13 09:55:22 +01:00
Tomas Dohnalek cc7991d977
Make Query requests idempotent (#1022)
* Make Query requests idempotent

Address #1020.

Signed-off-by: Tomáš Dohnálek <dohnto@gmail.com>

* Use empty header

Signed-off-by: Tomáš Dohnálek <dohnto@gmail.com>

* Document issue with original documentation

Signed-off-by: Tomáš Dohnálek <dohnto@gmail.com>
2022-04-13 07:21:50 +02:00
Tatsuhiro Tsujikawa 0bab4fda94
push: Add PushContext and AddContext to Pusher (#1028)
Add PushContext and AddContext to Pusher, which are context-aware
version of Push and Add respectively.  They give a caller the ability
to cancel an HTTP request.

Signed-off-by: Tatsuhiro Tsujikawa <ttsujika@zlab.co.jp>
2022-04-12 16:25:43 +02:00
Bartlomiej Plotka 06b641214c
Added info about our slack channel. (#1029) 2022-04-12 16:24:39 +02:00
Kemal Akkoyun 130da3b8ec
Merge pull request #1021 from dohnto/dohnto/line-of-sight
Refactor apiClientImpl.DoGetFallback
2022-04-12 12:15:45 +01:00
Kemal Akkoyun 3e9269d7d3
Merge pull request #1019 from prometheus/repo_sync
Synchronize common files from prometheus/prometheus
2022-04-11 13:50:41 +01:00
Tomáš Dohnálek 40e54a75a6 Refactor apiClientImpl.DoGetFallback
Make apiClientImpl.DoGetFallback more idiomatic and efficient:
 * Save result of args.Encode() operation as it might be used 2 times in
   the function and due to looping and sorting it might be heavy.
 * Follow line-of-sight practise and therefore simplify the code.

Signed-off-by: Tomáš Dohnálek <dohnto@gmail.com>
2022-04-07 09:18:32 +02:00
prombot e2504f86bb Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
2022-03-31 19:50:35 +00:00
Manuel Rüger 29e8191aff
go.mod: Update dependencies (#1018)
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-03-29 14:15:42 +01:00
Mitsuo Heijo 8dfa334295
Remove workaround for pre go1.15 (#1010)
Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2022-03-27 19:11:56 +01:00
PrometheusBot 3bc8f2c651
Update common Prometheus files (#1009)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
2022-03-18 11:00:13 +00:00
Kemal Akkoyun 6559749c87
Add statebot config (#1001)
* Add statebot config

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Disable for milestones

Remove references to remind bot

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-03-17 15:44:57 +00:00
zhijian 36b47eb0ab
When prefix is empty, no more dots should be written (#1005)
Signed-off-by: zhijian <djx@juicedata.io>
2022-03-17 15:43:50 +00:00
Kemal Akkoyun 5d78aaad41
.circleci: Add config to test against go1.18 (#1006)
* Add config to test against go1.18

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Try to fix circleci

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-03-17 15:34:12 +00:00
Kemal Akkoyun 157170dd1a
Merge pull request #1007 from prometheus/fix-lint
Fixed lint warning.
2022-03-17 14:26:26 +01:00
Bartlomiej Plotka 9894406186 Fixed lint warning.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-03-17 11:31:49 +00:00
William Perron 66837e3298
Add exemplar support for const histogram and const metric (#986)
* Add support for exemplars on constHistogram

Co-authored-by: William Perron <william.perron@shopify.com>
Signed-off-by: William Perron <william.perron@shopify.com>

* remove GetExemplars function

Signed-off-by: William Perron <william.perron@shopify.com>

* fixed linting warnings

reduce repetition in constHistogram w/ exemplar

Signed-off-by: William Perron <william.perron@shopify.com>

* Add values to correct bucket

Signed-off-by: William Perron <william.perron@shopify.com>

* Misc fixes

Co-authored-by: Francis Bogsanyi <francis.bogsanyi@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* avoid panic when there are fewer buckets than exemplars

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* Added MustNewMetricWithExemplars that wraps metrics with exemplar (#3)

Changes:
* Make sure to not "leak" dto.Metric
* Reused upper bounds we already have for histogram
* Common code for all types.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-03-17 11:30:41 +00:00
Kemal Akkoyun fe8d1e13cd
Merge pull request #1000 from alissa-tung/patch-1
Fix deprecated `NewBuildInfoCollector` API
2022-03-17 08:36:01 +01:00
Kemal Akkoyun 6c18569eab
Merge pull request #1003 from prometheus/repo_sync
Synchronize common files from prometheus/prometheus
2022-03-17 07:58:06 +01:00
prombot 0291563b9f Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
2022-03-16 19:50:22 +00:00
alissa-tung ffd6362a06 Update Dockerfile example
Signed-off-by: alissa-tung <alissa-tung@outlook.com>

Co-authored-by: Cruise_Hua <zlh429375251@qq.com>
2022-03-16 18:14:51 +08:00
alissa-tung b05177a553 Fix deprecated `NewBuildInfoCollector` API
Update `examples/random/main.go`:
  `prometheus.NewBuildInfoCollector` is deprecated. Use `collectors.NewBuildInfoCollector` instead.

Signed-off-by: alissa-tung <alissa-tung@outlook.com>
2022-03-16 17:46:48 +08:00
PrometheusBot 868ec2137f
Update common Prometheus files (#992)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
2022-03-16 08:26:54 +00:00
Bartlomiej Plotka 1f81b3e913
Added Transactional Gatherer allowed cached solutions (#989)
* Added cached collector.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

update.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Attempt 2

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added blocking registry, with raw collector and transactional handler.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added fast path to normal (empty) registry to save 8 allocs and 3K5B per Gather.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Simplified API, added tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Fix.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Simplified implementation.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added benchmark.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Optimized.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Optimization attempt.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Revert "Optimization attempt."

This reverts commit 2fcaf51be9.

Optimization was not worth it:

 benchstat v1.txt v2.txt
name                                                           old time/op    new time/op    delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12    2.64µs ± 0%    4.05µs ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12       701ms ± 0%     358ms ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                  535µs ± 0%  703934µs ± 0%   ~     (p=1.000 n=1+1)

name                                                           old alloc/op   new alloc/op   delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      208B ± 0%      208B ± 0%   ~     (all equal)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12      40.2MB ± 0%    41.1MB ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                 48.6kB ± 0%    84.3kB ± 0%   ~     (p=1.000 n=1+1)

name                                                           old allocs/op  new allocs/op  delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      3.00 ± 0%      3.00 ± 0%   ~     (all equal)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12        6.00 ± 0%   4003.00 ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                  1.00k ± 0%     2.01k ± 0%   ~     (p=1.000 n=1+1)

* nit.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Another optimization attempt.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* rename and further optimization.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Hopefully final optimization.

benchstat -delta-test=none v6.txt v9.txt
name                                                           old time/op    new time/op    delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12    13.1ms ± 0%     0.0ms ± 0%  -99.81%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12       309ms ± 0%     282ms ± 0%   -8.77%
CachedTGatherer_Update/Gather-12                                  422ms ± 0%       0ms ± 0%  -99.95%

name                                                           old alloc/op   new alloc/op   delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      208B ± 0%      208B ± 0%    0.00%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12      2.47kB ± 0%    1.67kB ± 0%  -32.56%
CachedTGatherer_Update/Gather-12                                 52.8kB ± 0%    24.6kB ± 0%  -53.34%

name                                                           old allocs/op  new allocs/op  delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      3.00 ± 0%      3.00 ± 0%    0.00%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12        0.00           0.00         0.00%
CachedTGatherer_Update/Gather-12                                  1.00k ± 0%     0.00k ± 0%  -99.60%

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed obsolete comment

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Re-add cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-02-23 11:22:52 +00:00
Kemal Akkoyun f3021b0b81
Create codeql-analysis.yml (#982)
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-02-15 10:09:48 +00:00
Kemal Akkoyun 5ac1e9208b
Merge pull request #980 from mrueg/min-go-1.15
go.mod: Set minimal support version of go to 1.15
2022-02-03 09:34:06 +01:00
Manuel Rüger 5678ca5ed1 go.mod: Set minimal support version of go to 1.15
As requested in https://github.com/prometheus/common/pull/353#discussion_r793113705

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-01-31 11:03:13 +01:00
Kemal Akkoyun 2e1c4818cc
Cut v1.12.1 (#978)
* Cut v1.12.1

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Apply review suggestions

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-01-29 15:48:34 +01:00
Michael Knyszek 77626d64fa
Reduce granularity of histogram buckets for Go 1.17 collector (#974)
The Go runtime/metrics package currently exports extremely granular
histograms. Exponentially bucket any histogram with unit "seconds"
or "bytes" instead to dramatically reduce the number of buckets, and
thus the number of metrics.

This change also adds a test to check for expected cardinality to
prevent cardinality surprises in the future.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-28 05:46:45 +01:00
Bryan Boreham 4dd3cbb4ab
API client: make http reads more efficient (#976)
Replace `io.ReadAll` with `bytes.Buffer.ReadFrom`.
Both need to resize a buffer until they have finished reading;
the former increases by 1.25x each time while the latter uses 2x.

Also added a benchmark to demonstrate the benefit:
name             old time/op    new time/op    delta
Client/4KB-8       35.9µs ± 4%    35.3µs ± 3%     ~     (p=0.310 n=5+5)
Client/50KB-8      83.1µs ± 8%    69.5µs ± 1%  -16.37%  (p=0.008 n=5+5)
Client/1000KB-8     891µs ± 6%     750µs ± 0%  -15.83%  (p=0.016 n=5+4)
Client/2000KB-8    1.74ms ± 2%    1.35ms ± 1%  -22.72%  (p=0.008 n=5+5)

name             old alloc/op   new alloc/op   delta
Client/4KB-8       20.2kB ± 0%    20.4kB ± 0%   +1.26%  (p=0.008 n=5+5)
Client/50KB-8       218kB ± 0%     136kB ± 0%  -37.65%  (p=0.008 n=5+5)
Client/1000KB-8    5.88MB ± 0%    2.11MB ± 0%  -64.10%  (p=0.008 n=5+5)
Client/2000KB-8    11.7MB ± 0%     4.2MB ± 0%  -63.93%  (p=0.008 n=5+5)

name             old allocs/op  new allocs/op  delta
Client/4KB-8         75.0 ± 0%      72.0 ± 0%   -4.00%  (p=0.008 n=5+5)
Client/50KB-8         109 ± 0%        98 ± 0%  -10.09%  (p=0.079 n=4+5)
Client/1000KB-8       617 ± 0%       593 ± 0%   -3.89%  (p=0.008 n=5+5)
Client/2000KB-8     1.13k ± 0%     1.09k ± 0%   -3.27%  (p=0.008 n=5+5)

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-01-25 11:16:10 +01:00
Michael Knyszek 85206714ae
Use simpler locking in the Go 1.17 collector (#975)
A previous PR made it so that the Go 1.17 collector locked only around
uses of rmSampleBuf, but really that means that Metric values may be
sent over the channel containing some values from future metrics.Read
calls. While generally-speaking this isn't a problem, we lose any
consistency guarantees provided by the runtime/metrics package.

Also, that optimization to not just lock around all of Collect was
premature. Truthfully, Collect is called relatively infrequently, and
its critical path is fairly fast (10s of µs). To prove it, this change
also adds a benchmark.

name            old time/op  new time/op  delta
GoCollector-16  43.7µs ± 2%  43.2µs ± 2%   ~     (p=0.190 n=9+9)

Note that because the benchmark is single-threaded it actually looks
like it might be getting *slightly* faster, because all those Collect
calls for the Metrics are direct calls instead of interface calls.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-25 08:43:45 +01:00
Michael Knyszek f63e219e6b
Make the Go 1.17 collector thread-safe (#969) 2022-01-21 08:34:45 +01:00
Kemal Akkoyun 01087964d0
Cut v1.12.0 (#966)
* Cut v1.12.0

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Bump the day

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-01-19 07:59:25 +00:00
Kemal Akkoyun 5bd9ee52dc
go.*: Update dependencies (#965)
* Update dependencies

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Update deprecated APIs

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Add missing dependency explicitly

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-01-18 20:32:44 +01:00
Kemal Akkoyun 9075cdf616
promhttp: Check validity of method and code label values (#962)
* Check validity of method and code label values

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Use more flexibly functional option pattern for configuration

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Update documentation

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Simplify

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Fix inconsistent method naming

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-01-18 10:19:28 +01:00
Michael Knyszek 22da9497b8
Use the runtime/metrics package for the Go collector for 1.17+ (#955)
This change introduces use of the runtime/metrics package in place of
runtime.MemStats for Go 1.17 or later. The runtime/metrics package was
introduced in Go 1.16, but not all the old metrics were accounted for
until 1.17.

The runtime/metrics package offers several advantages over using
runtime.MemStats:
* The list of metrics and their descriptions are machine-readable,
  allowing new metrics to get added without any additional work.
* Detailed histogram-based metrics are now available, offering much
  deeper insights into the Go runtime.
* The runtime/metrics API is significantly more efficient than
  runtime.MemStats, even with the additional metrics added, because
  it does not require any stop-the-world events.

That being said, integrating the package comes with some caveats, some
of which were discussed in #842. Namely:
* The old MemStats-based metrics need to continue working, so they're
  exported under their old names backed by equivalent runtime/metrics
  metrics.
* Earlier versions of Go need to continue working, so the old code
  remains, but behind a build tag.

Finally, a few notes about the implementation:
* This change includes a whole bunch of refactoring to avoid significant
  code duplication.
* This change adds a new histogram metric type specifically optimized
  for runtime/metrics histograms. This type's methods also include
  additional logic to deal with differences in bounds conventions.
* This change makes a whole bunch of decisions about how runtime/metrics
  names are translated.
* This change adds a `go generate` script to generate a list of expected
  runtime/metrics names for a given Go version for auditing. Users of
  new versions of Go will transparently be allowed to use new metrics,
  however.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-16 16:41:56 +00:00
Łukasz Mierzwa dc1559e8ef
Update /api/v1/status/tsdb to include headStats (#925)
Current structs for v1 HTTP API assume that there is chunkCount and timeSeriesCount on /api/v1/status/runtimeinfo, which seems to be gone for at least a few releases. /api/v1/status/tsdb now holds an extra headStats block with chunkCount, numSeries and a few other fields.
Update API models and tests to reflect this change.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2022-01-05 10:09:29 +01:00
Kemal Akkoyun e6e54e8082
Merge pull request #944 from yeya24/add-wal-replay-status
API: support wal replay status api
2022-01-05 10:04:10 +01:00
Kemal Akkoyun 98fbd996b0
Merge pull request #952 from prometheus/repo_sync
Synchronize common files from prometheus/prometheus
2021-12-21 08:41:18 +00:00
prombot 70a41d58f8 Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
2021-12-20 20:51:20 +00:00
Kemal Akkoyun dbf420e7ce
Merge pull request #928 from prometheus/repo_sync
Synchronize common files from prometheus/prometheus
2021-12-20 11:53:34 +00:00
Kemal Akkoyun 1d09783cb9
Merge pull request #950 from mrueg/go-1.17
Add support for go 1.17
2021-12-20 11:51:53 +00:00
Manuel Rüger 8535b62021 Add new go:build lines for go 1.17
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2021-12-17 23:38:35 +01:00
José Carlos Chávez d28a147d3c chore: adds support for go 1.17 on CI.
Signed-off-by: José Carlos Chávez <jcchavezs@gmail.com>
2021-12-17 23:06:20 +01:00
Ben Ye 440c09d3ec support wal replay status api
Signed-off-by: Ben Ye <ben.ye@bytedance.com>
2021-12-04 22:19:58 -08:00
Björn Rabenstein 6d5cf25fcc
Merge pull request #939 from gozeloglu/fix-typo
Fix typo
2021-11-29 13:18:11 +01:00