Commit Graph

501 Commits

Author SHA1 Message Date
Bartlomiej Plotka 1f81b3e913
Added Transactional Gatherer allowed cached solutions (#989)
* Added cached collector.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

update.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Attempt 2

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added blocking registry, with raw collector and transactional handler.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added fast path to normal (empty) registry to save 8 allocs and 3K5B per Gather.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Simplified API, added tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Fix.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Simplified implementation.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added benchmark.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Optimized.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Optimization attempt.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Revert "Optimization attempt."

This reverts commit 2fcaf51be9.

Optimization was not worth it:

 benchstat v1.txt v2.txt
name                                                           old time/op    new time/op    delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12    2.64µs ± 0%    4.05µs ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12       701ms ± 0%     358ms ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                  535µs ± 0%  703934µs ± 0%   ~     (p=1.000 n=1+1)

name                                                           old alloc/op   new alloc/op   delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      208B ± 0%      208B ± 0%   ~     (all equal)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12      40.2MB ± 0%    41.1MB ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                 48.6kB ± 0%    84.3kB ± 0%   ~     (p=1.000 n=1+1)

name                                                           old allocs/op  new allocs/op  delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      3.00 ± 0%      3.00 ± 0%   ~     (all equal)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12        6.00 ± 0%   4003.00 ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                  1.00k ± 0%     2.01k ± 0%   ~     (p=1.000 n=1+1)

* nit.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Another optimization attempt.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* rename and further optimization.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Hopefully final optimization.

benchstat -delta-test=none v6.txt v9.txt
name                                                           old time/op    new time/op    delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12    13.1ms ± 0%     0.0ms ± 0%  -99.81%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12       309ms ± 0%     282ms ± 0%   -8.77%
CachedTGatherer_Update/Gather-12                                  422ms ± 0%       0ms ± 0%  -99.95%

name                                                           old alloc/op   new alloc/op   delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      208B ± 0%      208B ± 0%    0.00%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12      2.47kB ± 0%    1.67kB ± 0%  -32.56%
CachedTGatherer_Update/Gather-12                                 52.8kB ± 0%    24.6kB ± 0%  -53.34%

name                                                           old allocs/op  new allocs/op  delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      3.00 ± 0%      3.00 ± 0%    0.00%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12        0.00           0.00         0.00%
CachedTGatherer_Update/Gather-12                                  1.00k ± 0%     0.00k ± 0%  -99.60%

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed obsolete comment

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Re-add cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-02-23 11:22:52 +00:00
Michael Knyszek 77626d64fa
Reduce granularity of histogram buckets for Go 1.17 collector (#974)
The Go runtime/metrics package currently exports extremely granular
histograms. Exponentially bucket any histogram with unit "seconds"
or "bytes" instead to dramatically reduce the number of buckets, and
thus the number of metrics.

This change also adds a test to check for expected cardinality to
prevent cardinality surprises in the future.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-28 05:46:45 +01:00
Michael Knyszek 85206714ae
Use simpler locking in the Go 1.17 collector (#975)
A previous PR made it so that the Go 1.17 collector locked only around
uses of rmSampleBuf, but really that means that Metric values may be
sent over the channel containing some values from future metrics.Read
calls. While generally-speaking this isn't a problem, we lose any
consistency guarantees provided by the runtime/metrics package.

Also, that optimization to not just lock around all of Collect was
premature. Truthfully, Collect is called relatively infrequently, and
its critical path is fairly fast (10s of µs). To prove it, this change
also adds a benchmark.

name            old time/op  new time/op  delta
GoCollector-16  43.7µs ± 2%  43.2µs ± 2%   ~     (p=0.190 n=9+9)

Note that because the benchmark is single-threaded it actually looks
like it might be getting *slightly* faster, because all those Collect
calls for the Metrics are direct calls instead of interface calls.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-25 08:43:45 +01:00
Michael Knyszek f63e219e6b
Make the Go 1.17 collector thread-safe (#969) 2022-01-21 08:34:45 +01:00
Kemal Akkoyun 5bd9ee52dc
go.*: Update dependencies (#965)
* Update dependencies

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Update deprecated APIs

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Add missing dependency explicitly

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-01-18 20:32:44 +01:00
Kemal Akkoyun 9075cdf616
promhttp: Check validity of method and code label values (#962)
* Check validity of method and code label values

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Use more flexibly functional option pattern for configuration

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Update documentation

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Simplify

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Fix inconsistent method naming

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2022-01-18 10:19:28 +01:00
Michael Knyszek 22da9497b8
Use the runtime/metrics package for the Go collector for 1.17+ (#955)
This change introduces use of the runtime/metrics package in place of
runtime.MemStats for Go 1.17 or later. The runtime/metrics package was
introduced in Go 1.16, but not all the old metrics were accounted for
until 1.17.

The runtime/metrics package offers several advantages over using
runtime.MemStats:
* The list of metrics and their descriptions are machine-readable,
  allowing new metrics to get added without any additional work.
* Detailed histogram-based metrics are now available, offering much
  deeper insights into the Go runtime.
* The runtime/metrics API is significantly more efficient than
  runtime.MemStats, even with the additional metrics added, because
  it does not require any stop-the-world events.

That being said, integrating the package comes with some caveats, some
of which were discussed in #842. Namely:
* The old MemStats-based metrics need to continue working, so they're
  exported under their old names backed by equivalent runtime/metrics
  metrics.
* Earlier versions of Go need to continue working, so the old code
  remains, but behind a build tag.

Finally, a few notes about the implementation:
* This change includes a whole bunch of refactoring to avoid significant
  code duplication.
* This change adds a new histogram metric type specifically optimized
  for runtime/metrics histograms. This type's methods also include
  additional logic to deal with differences in bounds conventions.
* This change makes a whole bunch of decisions about how runtime/metrics
  names are translated.
* This change adds a `go generate` script to generate a list of expected
  runtime/metrics names for a given Go version for auditing. Users of
  new versions of Go will transparently be allowed to use new metrics,
  however.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-16 16:41:56 +00:00
Manuel Rüger 8535b62021 Add new go:build lines for go 1.17
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2021-12-17 23:38:35 +01:00
Gökhan Özeloğlu 55320aa8b0 Fix typo
Signed-off-by: Gökhan Özeloğlu <gozeloglu@gmail.com>
2021-11-25 11:21:10 +03:00
Seth Bunce 2261d5cda1
add ExponentialBucketsRange function (#899)
This function calculates exponential buckets with different arguments
than the existing ExponentialBuckets function. Instead of specifying the
start and factor, the user can specify the min and max bucket value. We
have been doing it this way internally at my company for some time.

Signed-off-by: Seth Bunce <seth.bunce@getcruise.com>
2021-08-12 16:56:44 +01:00
SuperQ 6d946b5526
Update status badgets
Update to new locations
* Switch to CircleCI build badge.
* Use new pkg.go.dev for docs.

Signed-off-by: SuperQ <superq@gmail.com>
2021-06-23 10:12:43 +02:00
beorn7 f34145a85e Document implications of negative observations
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-26 23:41:30 +02:00
beorn7 81a9556c8b Make dbStatsCollector more DRY
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-20 22:31:53 +02:00
Mitsuo Heijo a66da1df4a
Add collector for database/sql#DBStats (#866)
* Add collector for database/sql#DBStats

Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2021-05-20 22:22:17 +02:00
Mitsuo Heijo e07445a773
Add new collectors package
Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2021-05-08 10:43:31 +09:00
Dean Coakley 3998a677f9
Improve description of MaxAge in summary docs (#864)
* Improve description of MaxAge in summary docs

Signed-off-by: Dean Coakley <dean.s.coakley@gmail.com>
2021-05-04 15:58:41 +02:00
beorn7 e92283d644 Fix linter ignores
Signed-off-by: beorn7 <beorn@grafana.com>
2021-03-16 17:19:03 +01:00
beorn7 6635a8f35b Bump minumim required Go version to 1.13
Since 1.16 is out, we still support the last four minor releases.

The bump was required by the prometheus/procfs package using the new
`%w` printf directives. However, it also allows us to remove some
special casing about build info.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-03-11 20:45:00 +01:00
debiandebiandebian 9443f4449b Fix typo in comments
Signed-off-by: debiandebiandebian <debiandebiandebiandebian@gmail.com>
2021-02-08 08:58:51 +09:00
beorn7 98eb6cbf7c promhttp: Correctly detect invalid metric and label names
Without this fix, the `InstrumentHandler...` middlewares get locked in
an endless loop in case of an invalid Collector, eating all the memory.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-12-09 17:58:53 +01:00
beorn7 34ca120377 Be more explicit about the multi-line properties of MultiError
Signed-off-by: beorn7 <beorn@grafana.com>
2020-12-02 19:53:38 +01:00
Michael Vetter cf6dc82780 Correct spelling: possibilites -> possibilities
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
2020-12-01 21:20:26 +01:00
sbookworm 469ec2747b add the NewPidFileFn to helper
Signed-off-by: sbookworm <lisong.cruise@gmail.com>
2020-10-23 09:23:59 +08:00
beorn7 85aa957f63 Export MetricVec (again)
MetricVec was already exported in early versions of this library, but
nobody really used it to implement vectors of custom Metric
implementations. Now #796 has shown up with a fairly special use case
for which I'd prefer a custom implementation of a special
"auto-sampling histogram" outside of this library. Therefore, I'd like
to reinstate support for creating vectors of custom Metric
implementations.

I played around for quite some while with the option of a separate
package providing the tools one would need to create vectors of custom
Metric implementations. However, with the current structure of the
prometheus/client_golang/prometheus package, this leads to a lot of
complications with circular dependencies. (The new package would need
the primitives from the prometheus package, while the existing metric
vectors like GaugeVec need to import the new vector package to not
duplicate the implementation. Separating vector types from the main
prometheus package is out of the question at this point because that
would be a breaking change.)

Signed-off-by: beorn7 <beorn@grafana.com>
2020-10-15 14:32:54 +02:00
beorn7 b54b73c7b1 Remove spurious commas from links to the docs site
I assume older Nanoc versions rendered the anchors with commas, but
the current doesn't.

Also, this adds the same link to another doc comment where it is also
relevant.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-09-07 22:31:29 +02:00
Björn Rabenstein c650ae9fa1
Merge pull request #784 from hummerd/master
Remove reference to Metric after reslicing
2020-07-25 00:01:35 +02:00
Björn Rabenstein 77b35d0377
Merge pull request #783 from roidelapluie/gc15
Update collector comment about GC stop-the-world
2020-07-24 23:48:32 +02:00
Dima Kozlov 0c48254b62 remove reference to metrics after reslicing
Signed-off-by: Dima Kozlov <hummerd@mail.ru>
2020-07-22 23:12:23 +03:00
Julien Pivotto 4759649876 Update collector comment about GC stop-the-world
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-22 01:42:37 +02:00
Mitsuo Heijo 916659fee0 fix tests warning about string(int) type conversions
See https://github.com/golang/go/issues/32479

Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2020-07-06 21:19:31 +09:00
Mark Hansen 7687ac2137 histogram.go: fix copy/paste typo
This method calls NewConstHistogram.

Signed-off-by: Mark Hansen <mark@markhansen.co.nz>
2020-06-21 13:47:29 +10:00
Tom Wilkie 9c8ba1f945 Review feedback: add comment and tests for WrapRegistererWith.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2020-06-04 11:51:51 +01:00
Tom Wilkie 614377c550 Review feedback: use one line.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2020-06-04 10:57:40 +01:00
Tom Wilkie 8961609f91 Ensure that nil registers are treat as a no-op, even when wrapping.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2020-06-04 10:30:51 +01:00
beorn7 3ba240a80f Improve doc comments in promauto
In particular, point out that `With(nil)` is valid.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-06-03 13:56:49 +02:00
Matt Layher 3afa5ab1e1
testutil/promlint: allow Kelvin as a base unit for color temperature
Signed-off-by: Matt Layher <mdlayher@gmail.com>
2020-05-28 11:54:54 -04:00
Björn Rabenstein bff02dd561
Merge pull request #754 from prometheus/beorn7/deprecation
Add lint:ignore for protobuf deprecation
2020-05-18 15:31:02 +02:00
Björn Rabenstein d8b51d4981
Merge pull request #753 from prometheus/beorn7/test
Improve CollectAndCount
2020-05-15 13:19:43 +02:00
beorn7 dba1478b8a Add lint:ignore for protobuf deprecation
`github.com/golang/protobuf/proto` is deprecated in lieu of
`google.golang.org/protobuf/proto`. However, we cannot simply
migrate. Types from the proto package are exposed to users of packages
in this repo. If we migrate here, users have to migrate to. Thus, we
could only migrate with a major version bump.

In different news, with all the inline lint:ignore comments, including
the existing ones, there is no need to repeat the exception in the
Makefile.

A current version of `staticcheck` is happy with the code after this
commit. golangci-lint is broken at the moment, however, and ignores
the lint:ignore comments in the code as well as those via envvar.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-05-14 20:11:22 +02:00
beorn7 0577ef6c57 Improve CollectAndCount
Now that we have also added CollectAndLint and GatherAndLint, I
thought we should bring CollectAndCount in line. So:

- Add GatherAndCount.
- Add filtering by metric name.
- Add tests.

Minor wart: CollectAndCount should now return `(int, error)`, but that
would be a breaking change as the current version just returns
`int`. I decided to let the new version panic when it should return an
error. An error is anyway very unlikely, so the biggest annoyance here
is really just the inconsistency.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-05-14 00:17:45 +02:00
beorn7 389d3c3b9a Properly handle empty job and label values
An empty job name was always an error, but it was previously only
detected when pushing to the PGW and receiving an error. With this
commit, the error is detected before pushing.

An empty label value should have been OK but was encoded in a way that
couldn't be pushed to the
PGW. Cf. https://github.com/prometheus/pushgateway/issues/344 . This
commit changes the creation of the path in the URL so that it works
with empty label values.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-05-13 23:35:31 +02:00
Michael Cristina be019d1f24 Fix typo in godoc help text
Signed-off-by: Michael Cristina <mcristina444@gmail.com>
2020-05-02 10:44:25 -05:00
beorn7 dc79bd6093 Improve various comments
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-25 15:59:53 +02:00
beorn7 39dbb24d13 Add helper functions for linting to testutil
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-24 23:44:59 +02:00
beorn7 6433bcf819 Add the capability to lint MetricFamilies directly
Also, change all the `dto.MetricFamily` arguments to pointers to be
more consistent with what we do in client_golang in general.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-24 23:42:49 +02:00
Björn Rabenstein a3a59237d1
Merge pull request #739 from RainbowMango/pr_porting_promlint
Porting promlint from prometheus/prometheus
2020-04-24 16:06:34 +02:00
Björn Rabenstein a200f1930c
Merge pull request #738 from colega/colega/gauge-func-const-labels-godoc
GaugeFunc: ConstLabels godoc example
2020-04-24 14:29:25 +02:00
Oleg Zaytsev 38045061c3
GaugeFunc: ConstLabels godoc example
Document the example usage of ConstLabels to register several GaugeFuncs
on the same metric name. Also reference it from the NewCounterFunc
documentation as it's similar.

Ref: https://github.com/prometheus/client_golang/pull/736

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2020-04-24 09:33:20 +02:00
RainbowMango 50cda505d1 fix static check warnings by Goland.
Signed-off-by: RainbowMango <renhongcai@huawei.com>
2020-04-24 10:15:27 +08:00
RainbowMango af6ade7230 Fix unit test failing with Go1.14+
Signed-off-by: RainbowMango <renhongcai@huawei.com>
2020-04-23 15:14:34 +08:00