Commit Graph

106 Commits

Author SHA1 Message Date
re 1919ef1b31 repo fix 2022-12-13 11:48:40 +03:00
Robert Fratto 83d56b1144
Extend prometheus.Registry to implement Collector (#1103)
* prometheus: implement Collector interface for Registry

This change allows Registries to be used as Collectors.

This enables new instances of Registry to be passed to ephemeral
subroutines for collecting metrics from subroutines which are still
running:

```go
package main

import (
  "fmt"

  "github.com/prometheus/client_golang/prometheus"
)

func main() {
  globalReg := prometheus.NewRegistry()

  for i := 0; i < 100; i++ {
    workerReg := prometheus.WrapRegistererWith(prometheus.Labels{
      // Add an ID label so registered metrics from workers don't
      // collide.
      "worker_id": fmt.Sprintf("%d", i),
    }, prometheus.NewRegistry()

    globalReg.MustRegister(workerReg)

    go func(i int) {
      runWorker(workerReg)

      // Unregister any metrics the worker may have created.
      globalReg.Unregister(workerReg)
    }(i)
  }
}

// runWorker runs a worker, registering worker-specific metrics.
func runWorker(reg *prometheus.Registry) {
  // ... register metrics ...
  // ... do work ...
}
```

This change makes it easier to avoid leaking metrics from subroutines
which do not consistently properly unregister metrics.

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* fix grammar in doc comment

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* document why Registry implements Collector with example

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

Signed-off-by: Robert Fratto <robertfratto@gmail.com>
2022-08-23 11:09:29 +02:00
Christoph Mewes 618194de6a
fix assorted oddities found by golangci-lint (#1040)
* fix assorted oddities found by golangci-lint

Signed-off-by: Christoph Mewes <christoph@kubermatic.com>

* permanently enable the linters

Signed-off-by: Christoph Mewes <christoph@kubermatic.com>

* post-rebase blues

Signed-off-by: Christoph Mewes <christoph@kubermatic.com>
2022-08-03 06:30:51 +02:00
inosato 44c2c4de85
Remove ioutil (#1096)
Signed-off-by: inosato <si17_21@yahoo.co.jp>
2022-08-02 10:27:49 +02:00
Soroosh Azary Marhabi 2cfd1eb960
Enable same linters as the Prometheus repo itself (#1056)
* Add gofumpt to github workflow & fix all files for it

Signed-off-by: sazary <soroosh@azary.ir>

* Add goimports to golangci & fix it's issues

Signed-off-by: sazary <soroosh@azary.ir>

* Add revive to golangci & fix it's issues

Signed-off-by: sazary <soroosh@azary.ir>

* Add errcheck & misspell to golangci and fix their issues

Signed-off-by: sazary <soroosh@azary.ir>

* Add govet & gosimple to golangci and fix their issues

Signed-off-by: sazary <soroosh@azary.ir>

* Enable all default linters of golangci

Signed-off-by: sazary <soroosh@azary.ir>
2022-06-17 09:04:06 +02:00
Bartlomiej Plotka 1f81b3e913
Added Transactional Gatherer allowed cached solutions (#989)
* Added cached collector.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

update.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Attempt 2

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added blocking registry, with raw collector and transactional handler.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added fast path to normal (empty) registry to save 8 allocs and 3K5B per Gather.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Simplified API, added tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Fix.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Simplified implementation.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added benchmark.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Optimized.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Optimization attempt.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Revert "Optimization attempt."

This reverts commit 2fcaf51be9.

Optimization was not worth it:

 benchstat v1.txt v2.txt
name                                                           old time/op    new time/op    delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12    2.64µs ± 0%    4.05µs ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12       701ms ± 0%     358ms ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                  535µs ± 0%  703934µs ± 0%   ~     (p=1.000 n=1+1)

name                                                           old alloc/op   new alloc/op   delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      208B ± 0%      208B ± 0%   ~     (all equal)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12      40.2MB ± 0%    41.1MB ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                 48.6kB ± 0%    84.3kB ± 0%   ~     (p=1.000 n=1+1)

name                                                           old allocs/op  new allocs/op  delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      3.00 ± 0%      3.00 ± 0%   ~     (all equal)
CachedTGatherer_Update/Update_of_all_elements_with_reset-12        6.00 ± 0%   4003.00 ± 0%   ~     (p=1.000 n=1+1)
CachedTGatherer_Update/Gather-12                                  1.00k ± 0%     2.01k ± 0%   ~     (p=1.000 n=1+1)

* nit.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Another optimization attempt.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* rename and further optimization.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Hopefully final optimization.

benchstat -delta-test=none v6.txt v9.txt
name                                                           old time/op    new time/op    delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12    13.1ms ± 0%     0.0ms ± 0%  -99.81%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12       309ms ± 0%     282ms ± 0%   -8.77%
CachedTGatherer_Update/Gather-12                                  422ms ± 0%       0ms ± 0%  -99.95%

name                                                           old alloc/op   new alloc/op   delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      208B ± 0%      208B ± 0%    0.00%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12      2.47kB ± 0%    1.67kB ± 0%  -32.56%
CachedTGatherer_Update/Gather-12                                 52.8kB ± 0%    24.6kB ± 0%  -53.34%

name                                                           old allocs/op  new allocs/op  delta
CachedTGatherer_Update/Update_of_one_element_without_reset-12      3.00 ± 0%      3.00 ± 0%    0.00%
CachedTGatherer_Update/Update_of_all_elements_with_reset-12        0.00           0.00         0.00%
CachedTGatherer_Update/Gather-12                                  1.00k ± 0%     0.00k ± 0%  -99.60%

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed obsolete comment

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Re-add cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed cache.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-02-23 11:22:52 +00:00
beorn7 e92283d644 Fix linter ignores
Signed-off-by: beorn7 <beorn@grafana.com>
2021-03-16 17:19:03 +01:00
beorn7 34ca120377 Be more explicit about the multi-line properties of MultiError
Signed-off-by: beorn7 <beorn@grafana.com>
2020-12-02 19:53:38 +01:00
beorn7 dba1478b8a Add lint:ignore for protobuf deprecation
`github.com/golang/protobuf/proto` is deprecated in lieu of
`google.golang.org/protobuf/proto`. However, we cannot simply
migrate. Types from the proto package are exposed to users of packages
in this repo. If we migrate here, users have to migrate to. Thus, we
could only migrate with a major version bump.

In different news, with all the inline lint:ignore comments, including
the existing ones, there is no need to repeat the exception in the
Makefile.

A current version of `staticcheck` is happy with the code after this
commit. golangci-lint is broken at the moment, however, and ignores
the lint:ignore comments in the code as well as those via envvar.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-05-14 20:11:22 +02:00
Leo Antunes 4be0ab45ec ensure same collectorID calculated on reg and unreg
Signed-off-by: Leo Antunes <leo@costela.net>
2019-10-17 14:16:32 +02:00
beorn7 ee1078a03c Move registry hashing to xxhash
This is a much stronger hash function than fnv64a and comparably fast
(with super-fast assembly implementation for amd64).

Performance is not critical here anyway.

The old fnv64a is kept for vectors, where collision detection is in
place and the weakness of the hashing doesn't matter that much. I
implemented a vector version with xxhash and found that xxhash is
slower in all cases except very very high cardinality (where it is
only slightly faster). Also, ``xxhash.New`` comes with an allocation
of 80 bytes. Thus, to keep vectors alloc-free, we needed to add a
`sync.Pool`, which would have an additional performance overhead.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-10-14 21:18:38 +02:00
beorn7 c2e3855f3b Minimal “fix” for hash collisions
This makes the collisions a bit less likely by XOR'ing descIDs rather
than adding them up for the collectorID.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-10-14 20:14:43 +02:00
Andrey Yurchenkov b0bcec8f2e
Fix typo in documentation
Signed-off-by: Andrey Yurchenkov <painhardcore@gmail.com>
2019-08-13 13:43:39 +03:00
beorn7 2f3a0f8f2e Make the AlreadyRegisteredError useful for wrapped registries
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-14 17:55:35 +02:00
beorn7 000ceb266b Fix doc comment typo
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-12-18 16:30:34 +01:00
beorn7 619eb595ba Simplify an `append` to `copy`
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-12-06 11:37:34 +01:00
beorn7 fae889635c Fix #512
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-12-06 11:35:30 +01:00
Sevag Hanssian 88f4223778
Fix permissions of tempfile
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-11-02 09:59:03 -07:00
Sevag Hanssian 924d5919f3
Improve WriteToTextfile doc
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-11-02 09:31:41 -07:00
Sevag Hanssian 42e6616334
Use code that already existed
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-11-02 08:37:26 -07:00
Sevag Hanssian 1d54dabd43
Add WriteToTextfile test
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-10-31 23:34:50 -07:00
Sevag Hanssian c5bdd15ac3
Remove typo
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-10-31 15:31:42 -07:00
Sevag Hanssian e6fe89ce22
Add support for histograms and summaries
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-10-31 15:27:11 -07:00
Sevag Hanssian 9416ff209b
First commit - WriteToTextfile
Signed-off-by: Sevag Hanssian <sevag.hanssian@gmail.com>
2018-10-31 14:22:11 -07:00
beorn7 edb489a1ef Add check for duplicated label names
Fixes #471

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-30 16:25:48 +02:00
beorn7 ff0177896a Be more precise about concurrency requirements
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-29 10:47:28 +02:00
beorn7 7eb5e8a08a Drain the desc channel after registering
This avoids leaking goroutines.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-29 10:44:19 +02:00
beorn7 84d7aa0cd9 Add wrapping of Registerers with labels and prefix
Essentially middleware for Registerers!

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-12 15:34:07 +02:00
beorn7 773f502723 Rework process collector
This unifies both constructors in one with an options argument.

The options argument allows to switch on error reporting, as discussed
in #219.

The change of the contructor signature is breaking, but only mildly
so. Plus, the process collector is rarely used explicitly. I used
Sourcegraph to search for public usages, with the following results:

- 2 occurrences of NewProcessCollectorPIDFn, once in @discordianfish's
  glimpse, once in @fabxc's etcd_exporter (deprecated anyway). Both
  are Prom veterans and will simply do the one line change if needed.
- 8 occurrences of NewProcessCollector, of which 7 are of the form
    NewProcessCollector(os.Getpid(), "")
  Thus, it's a very easy change, which I even hinted at in the doc
  comment.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-07 12:09:26 +02:00
beorn7 da330f4281 Un-export prometheus.LabelPairSorter
The only known external usage of it was in prometheus/pushgateway,
where it was removed by
https://github.com/prometheus/pushgateway/pull/200 .

Originally, the expectation was that users would implement the Metric
interface now and then. As we know now, neither it is happening, nor
would it make a lot of sense. (Users implement the Collector interface
instead.) By now, LabelPairSorter is essentially noise in the already
quite cluttered namespace in the prometheus package.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-03 16:20:55 +02:00
beorn7 7be86f93c1 Create an internal package
This is for types we don't want to export but which are used in
different packages within client_golang.

Currently, that's only NormalizeMetricFamilies (used in the prometheus
package and in the testutil package). More to be added as needed.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-03 00:18:11 +02:00
beorn7 4572e24546 Add suffix collision checks during gathering
So far, if a gauge was named `xxx_count`, and a summary or histogram
`xxx`, this would have led to a legal protobuf exposition but would
have created a name collision on `xxx_count` in the text format and
within the Prometheus server.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-07-13 16:29:17 +02:00
beorn7 767a0218df Add more label checksn during gathering
Including check for an invalid "quantile" label in summaries.

Also, improve error messages.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-07-13 14:14:39 +02:00
beorn7 ad1b9f7754 Introduce unchecked Collectors
Fixes #47 . See there for more detailed discussion.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-07-09 14:33:18 +02:00
beorn7 f6b1978ed4 Use stricter wording for Gather returning a non-nil error
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-06-07 14:35:06 +02:00
beorn7 c06fb788be Relax consistency checks during gathering
Also, clarify in the doc comment.

Previously, the assumption was that inconsistent label dimensions are
violating the exposition format spec. However, especially with the
knowledge that OpenMetrics will explicitly allow inconsistent label
dimensions in expositions, we should allow it in client_golang, too.

Note that registration with proper Descs provided will still check for
consistont label dimensions. However, you can "cheat" with custom
Collectors as you can collect metrics that don't follew the provided
Desc (this will be made more explicit and less cheaty once #47 is
fixed). You can also create expositions with inconsistent label
dimensions by merging Gatherers with the Gatherers slice type. (The
latter is used in the Pushgateway.)

Effectively, normal direct instrumentation will always have consistent
label dimensions in this way, but you can cover special use cases with
custom collectors or merging of different Gatherers.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-06-06 19:30:53 +02:00
beorn7 1b56b5c497 Be more robust about nil pointers in protobuf
While not strictly correct, it can easily happen that proto messages
are created that use nil pointers instead of pointers in empty strings
to denote an empty string.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-05-29 17:57:33 +02:00
beorn7 d66ac8f863 Document the stop-the-world implications of the Go collector
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-05-19 21:14:04 +02:00
Karsten Weiss 958ea82988 Fix typos
Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:23:52 +02:00
beorn7 154bb450e4 Document that the process collector only works on Linux 2018-02-08 14:17:54 +01:00
beorn7 4957f7bba4 Add a safety goroutine budget
This makes sure we don't spin up a possibly infinite number of
goroutines in `Gather`, which could theoretically happen with unlucky
scheduling.
2018-01-31 14:49:15 +01:00
beorn7 e04451f4be Create goroutines adaptively during metrics gathering 2018-01-26 19:58:07 +01:00
Brian Brazil 9f5d03c01f Fix typo in comment 2017-10-04 17:44:49 +01:00
Marco Jantke 0b8aef084e implement review feedback 2017-08-25 14:51:19 +02:00
Marco Jantke 685a3c90d4 fail Gather'ing when label value is not utf8 2017-08-20 00:10:32 +02:00
Maxime Song 90494ea7b1 fix typo in comments
Signed-off-by: Maxime Song <me@cppdo.com>
2017-07-07 23:28:24 +08:00
Julius Volz 69bb387064 Fix typo 2017-02-23 10:28:41 +01:00
beorn7 2fee50beaa Remove deprecated features that are esay to replace
That's the "soft" part of the deprecation: Everything that has been
marked deprecated in v0.8 or earlier and is straight-forward to
replace by a non-deprecated way, is removed here.

Sadly, this does not include the HTTP part. We first need to provide a
replacement for HTTP instrumentation (as planned for v0.8) to then
remove the deprecated parts in v0.9.
2016-10-25 18:28:15 +02:00
beorn7 c9325a4a67 Add goreport card and remove warnings where feasible 2016-09-16 19:59:04 +02:00
beorn7 a6321dd0b1 Create a "merge gatherer"
This allows to finally get rid of the infamous injection hook in the
interface. The old SetMetricFamilyInjectionHook still exist as a
deprecated function but is now implemented with the new plumbing under
the hood.

Now that we have multiple Gatherer implementation, I renamed
push.Registry to push.FromGatherer.

This commit also improves the consistency checks, which happened as a
byproduct of the refactoring to allow checking in both the "merge
gatherer" Gatherers as well as in the normal Registry.
2016-08-12 21:34:17 +02:00