Commit Graph

41 Commits

Author SHA1 Message Date
Fabian Stäber 10b0550932
Fix race condition with Exemplar in Counter (#1146)
* Fix race condition with Exemplar in Counter

Potential fix for #1145.

Signed-off-by: Fabian Stäber <fabian@fstab.de>

* Fix race condition with Exemplar in Counter

Signed-off-by: Fabian Stäber <fabian@fstab.de>

Signed-off-by: Fabian Stäber <fabian@fstab.de>
2022-10-17 20:50:50 +02:00
Fredrik Enestad a528affed9
Update documentation for exemplar label limit (#1095)
Signed-off-by: Fredrik Enestad <fredrik@enestad.com>
2022-07-27 17:45:49 +02:00
Michael Knyszek 22da9497b8
Use the runtime/metrics package for the Go collector for 1.17+ (#955)
This change introduces use of the runtime/metrics package in place of
runtime.MemStats for Go 1.17 or later. The runtime/metrics package was
introduced in Go 1.16, but not all the old metrics were accounted for
until 1.17.

The runtime/metrics package offers several advantages over using
runtime.MemStats:
* The list of metrics and their descriptions are machine-readable,
  allowing new metrics to get added without any additional work.
* Detailed histogram-based metrics are now available, offering much
  deeper insights into the Go runtime.
* The runtime/metrics API is significantly more efficient than
  runtime.MemStats, even with the additional metrics added, because
  it does not require any stop-the-world events.

That being said, integrating the package comes with some caveats, some
of which were discussed in #842. Namely:
* The old MemStats-based metrics need to continue working, so they're
  exported under their old names backed by equivalent runtime/metrics
  metrics.
* Earlier versions of Go need to continue working, so the old code
  remains, but behind a build tag.

Finally, a few notes about the implementation:
* This change includes a whole bunch of refactoring to avoid significant
  code duplication.
* This change adds a new histogram metric type specifically optimized
  for runtime/metrics histograms. This type's methods also include
  additional logic to deal with differences in bounds conventions.
* This change makes a whole bunch of decisions about how runtime/metrics
  names are translated.
* This change adds a `go generate` script to generate a list of expected
  runtime/metrics names for a given Go version for auditing. Users of
  new versions of Go will transparently be allowed to use new metrics,
  however.

Signed-off-by: Michael Anthony Knyszek <mknyszek@google.com>
2022-01-16 16:41:56 +00:00
beorn7 85aa957f63 Export MetricVec (again)
MetricVec was already exported in early versions of this library, but
nobody really used it to implement vectors of custom Metric
implementations. Now #796 has shown up with a fairly special use case
for which I'd prefer a custom implementation of a special
"auto-sampling histogram" outside of this library. Therefore, I'd like
to reinstate support for creating vectors of custom Metric
implementations.

I played around for quite some while with the option of a separate
package providing the tools one would need to create vectors of custom
Metric implementations. However, with the current structure of the
prometheus/client_golang/prometheus package, this leads to a lot of
complications with circular dependencies. (The new package would need
the primitives from the prometheus package, while the existing metric
vectors like GaugeVec need to import the new vector package to not
duplicate the implementation. Separating vector types from the main
prometheus package is out of the question at this point because that
would be a breaking change.)

Signed-off-by: beorn7 <beorn@grafana.com>
2020-10-15 14:32:54 +02:00
Oleg Zaytsev 38045061c3
GaugeFunc: ConstLabels godoc example
Document the example usage of ConstLabels to register several GaugeFuncs
on the same metric name. Also reference it from the NewCounterFunc
documentation as it's similar.

Ref: https://github.com/prometheus/client_golang/pull/736

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2020-04-24 09:33:20 +02:00
Shreyas Srivatsan bc9e50c39f Initialize now fn for counters
Signed-off-by: Shreyas Srivatsan <shreyas@chronosphere.io>
2020-02-05 22:25:47 -08:00
beorn7 f34b09877c Pull out ...WithExemplar methods into separate interfaces
This is, sadly, the only way to avoid a breaking change. The cost is
that anyone using exemplars has to perform a type assertion. This is,
however, a common pattern where interfaces turn out to need additional
methods in a stable library or only some implementations can provide
the additional methods (AKA "interface upgrade").

Needless to say that in v2 of this library, we'll do things in a more
straight forward way.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-27 15:41:13 +01:00
beorn7 c32ffd121f Add tests for examplars
Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-24 17:12:36 +01:00
beorn7 57d41259e1 Add exemplars to counter and histogram
Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-24 17:12:08 +01:00
Peter Jausovec b686a9e02e Remove fmt from import
Signed-off-by: Peter Jausovec <peter.jausovec@oracle.com>
2018-11-02 09:21:12 -07:00
Peter Jausovec ea348d7c20 Update code based on the PR feedback
Signed-off-by: Peter Jausovec <peter.jausovec@oracle.com>
2018-11-02 09:01:14 -07:00
Peter Jausovec 4fe949f2fd Add missing fmt import
Signed-off-by: Peter Jausovec <peter.jausovec@oracle.com>
2018-10-31 11:09:12 -07:00
Peter Jausovec 902733d080 Add more info to the inconsistent cardinality errors
Signed-off-by: Peter Jausovec <peter.jausovec@oracle.com>
2018-10-31 11:01:42 -07:00
beorn7 a2facc3074 Iterate on a proposed performance improvement for counters
Original discussion see
https://github.com/prometheus/client_golang/pull/362 .

Assuming that the most frequently used method of a `Gauge` is `Set`
and the most frequently used method of a `Conuter` is `Inc`, this
separates the implementation of both metric types. `Inc` and integral
`Add` of a counter is now handled in a separate `uint64`. This would
create a race in `Set`, but luckily, there is no `Set` anymore in a
counter.

All attempts to solve above race (to use the same idea for a `Gauge`)
slow down `Set`, So we just stick with the old implementation
(formerly `value`) for `Gauge`.
2018-01-19 19:06:43 +01:00
beorn7 dd20712622 Allow currying of metric vec's
The idea behind it is described in detail in
https://github.com/prometheus/client_golang/issues/320 .

This commit also updates the example given in
promhttp/instrument_server_test.go , which nicely illustrates the
benefit of this change.

So far, currying could be emulated by creating different metric vec's
with different values in their ConstLabels. This was quite difficult
to grasp - which is essentially what was done in the example mentioned
above. Now that this use case can be solved without ConstLabels, we
can safely declare ConstLabels as rarely used. (Perhaps we can
deprecate them entirely one day, but I'll take a raincheck on that
when the changes of v0.10 have materialized.) This commit thus also
updates the ConstLabel doc comments in the various Opts. (It contained
fairly outdated stuff anyway.)
2017-12-22 15:56:11 +01:00
beorn7 10c55533cb Rename the receiver of `...Vec` methods from `m` to `v` 2017-12-21 14:06:39 +01:00
beorn7 9e1588b2a2 Pull `With` and `WithLabelValues` up into exported types
The "panic in case of error" code was so far in metricVec. This pulls
it up into the exported types like CounterVec. This is code
replication, but it avoids an explicit type conversion. Mostly,
however, this is preparation to make the wrapped metricVec an
interface (required for curried vec's).
2017-12-21 14:06:39 +01:00
beorn7 ed379b7d99 Remove remaining references to MetricVec from doc comments
MetricVec is un-experted by now. godoc handles that correctly by
showing the methods of the embedded un-exported metricVec with the
exported type (CounterVec, SummaryVec, ...) that embeds metricVec.
2017-07-12 15:13:09 +02:00
beorn7 f66cdf0736 Un-export MetricVec
This is in preparation for "curried" metric vecs, as discussed.

And it's a good thing anyway. The exported MetricVec was from a time
when I thought people would define own Metric types and then create
Vecs of it. That has never happened.
2017-06-28 17:55:59 +02:00
beorn7 606b8f85e5 - Point from Inc and Dec to Add and Sub in doc comments.
- Deprecate Untyped for direct instrumentation.

- Add a SetToCurrentTime method to Gauge

Note that adding the SetToCurrentTime method is not really following
Go's principle of lean interfaces. However, the Gauge interface is
already quite fat. (The only methods really required are Set and
Add. Everything else could be expressed in terms of those two.) So we
have already quite a few "convenience" methods traditionally, so I
think we should stay consistent here.

The alternatives would be:

- Not support SetToCurrentTime at all (it's only a SHOULD in the
  guidelines).

- A top level function `SetToCurrentTime(Gauge)`.

- Just a helper `CurrentTime()` that returns the curent unix time in
  seconds as a float (which is pretty verbose using the standard
  library, see code in this commit). This would allow
  `myGauge.Set(CurrentTime)`.

Weighing all circumstances, I believe the way in this commit is the
least evil. Issue #223 could be used to rework interfaces more
fundamentally in a breaking change if feasible.
2016-11-18 19:32:10 +01:00
beorn7 2fee50beaa Remove deprecated features that are esay to replace
That's the "soft" part of the deprecation: Everything that has been
marked deprecated in v0.8 or earlier and is straight-forward to
replace by a non-deprecated way, is removed here.

Sadly, this does not include the HTTP part. We first need to provide a
replacement for HTTP instrumentation (as planned for v0.8) to then
remove the deprecated parts in v0.9.
2016-10-25 18:28:15 +02:00
beorn7 434a8ed85d Bring back zero-alloc label-value access for metric vecs
Also, fix mutex copy-by-value bug.
2016-08-17 14:01:11 +02:00
Stephen J Day c4004ef5f6
benchmark: measure label resolution in MetricVec
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-15 16:47:42 -07:00
beorn7 e5c6302150 Fix a number of doc comments and similar nits 2016-08-03 12:23:37 +02:00
beorn7 249069ec01 Unexport SelfCollector.
This is most likely used nowhere and can be unexported to clean up the
namespace.
2016-08-03 01:09:27 +02:00
beorn7 cf7e1caf17 Create a public registry interface and separate out HTTP exposition
General context and approch
===========================

This is the first part of the long awaited wider refurbishment of
`client_golang/prometheus/...`. After a lot of struggling, I decided
to not go for one breaking big-bang, but cut things into smaller steps
after all, mostly to keep the changes manageable and easy to
review. I'm aiming for having the invasive breaking changes
concentrated in as few steps as possible (ideally one). Some steps
will not be breaking at all, but typically there will be breaking
changes that only affect quite special cases so that 95+% of users
will not be affected. This first step is an example for that, see
details below.

What's happening in this commit?
================================

This step is about finally creating an exported registry
interface. This could not be done by simply export the existing
internal implementation because the interface would be _way_ too
fat. This commit introduces a qutie lean `Registry` interface
(compared to the previous interval implementation). The functions that
act on the default registry are retained (with very few exceptions) so
that most use cases won't see a change. However, several of those are
deprecated now to clean up the namespace in the future.

The default registry is kept in the public variable
`DefaultRegistry`. This follows the example of the http package in the
standard library (cf. `http.DefaultServeMux`, `http.DefaultClient`)
with the same implications. (This pattern is somewhat disputed within
the Go community but I chose to go with the devil you know instead of
creating something more complex or even disallowing any changes to the
default registry. The current approach gives everybody the freedom to
not touch DefaultRegistry or to do everything with a custom registry
to play save.)

Another important part in making the registry lean is the extraction
of the HTTP exposition, which also allows for customization of the
HTTP exposition. Note that the separation of metric collection and
exposition has the side effect that managing the MetricFamily and
Metric protobuf objects in a free-list or pool isn't really feasible
anymore. By now (with better GC in more recent Go versions), the
returns were anyway dimisishing. To be effective at all, scrapes had
to happen more often than GC cycles, and even then most elements of
the protobufs (everything excetp the MetricFamily and Metric structs
themselves) would still cause allocation churn. In a future breaking
change, the signature of the Write method in the Metric interface will
be adjusted accordingly. In this commit, avoiding breakage is more
important.

The following issues are fixed by this commit (some solved "on the
fly" now that I was touching the code anyway and it would have been
stupid to port the bugs):

https://github.com/prometheus/client_golang/issues/46
https://github.com/prometheus/client_golang/issues/100
https://github.com/prometheus/client_golang/issues/170
https://github.com/prometheus/client_golang/issues/205

Documentation including examples have been amended as required.

What future changes does this commit enable?
============================================

The following items are not yet implemented, but this commit opens the
possibility of implementing these independently.

- The separation of the HTTP exposition allows the implementation of
  other exposition methods based on the Registry interface, as known
  from other Prometheus client libraries, e.g. sending the metrics to
  Graphite.
  Cf. https://github.com/prometheus/client_golang/issues/197

- The public `Registry` interface allows the implementation of
  convenience tools for testing metrics collection. Those tools can
  inspect the collected MetricFamily protobufs and compare them to
  expectation. Also, tests can use their own testing instance of a
  registry.
  Cf. https://github.com/prometheus/client_golang/issues/58

Notable non-goals of this commit
================================

Non-goals that will be tackled later
------------------------------------

The following two issues are quite closely connected to the changes in
this commit but the line has been drawn deliberately to address them
in later steps of the refurbishment:

- `InstrumentHandler` has many known problems. The plan is to create a
  saner way to conveniently intrument HTTP handlers and remove the old
  `InstrumentHandler` altogether. To keep breakage low for now, even
  the default handler to expose metrics is still using the old
  `InstrumentHandler`. This leads to weird naming inconsistencies but
  I have deemed it better to not break the world right now but do it
  in the change that provides better ways of instrumenting HTTP
  handlers.
  Cf. https://github.com/prometheus/client_golang/issues/200

- There is work underway to make the whole handling of metric
  descriptors (`Desc`) more intuitive and transparent for the user
  (including an ability for less strict checking,
  cf. https://github.com/prometheus/client_golang/issues/47). That's
  quite invasive from the perspective of the internal code, namely the
  registry. I deliberately kept those changes out of this commit.

- While this commit adds new external dependency, the effort to vendor
  anything within the library that is not visible in any exported
  types will have to be done later.

Non-goals that _might_ be tackled later
---------------------------------------

There is a strong and understandable urge to divide the `prometheus`
package into a number of sub-packages (like `registry`, `collectors`,
`http`, `metrics`, …). However, to not run into a multitude of
circular import chains, this would need to break every single existing
usage of the library. (As just one example, if the ubiquitious
`prometheus.MustRegister` (with more than 2,000 uses on GitHub alone)
is kept in the `prometheus` package, but the other registry concerns
go into a new `registry` package, then the `prometheus` package would
import the `registry` package (to call the actual register method),
while at the same time the `registry` package needs to import the
`prometheus` package to access `Collector`, `Metric`, `Desc` and
more. If we moved `MustRegister` into the `registry` package,
thousands of code lines would have to be fixed (which would be easy if
the world was a mono repo, but it is not). If we moved everything else
the proposed registry package needs into packages of their own, we
would break thousands of other code lines.)

The main problem is really the top-level functions like
`MustRegister`, `Handler`, …, which effectively pull everything into
one package. Those functions are however very convenient for the easy
and very frequent use-cases.

This problem has to be revisited later.

For now, I'm trying to keep the amount of exported names in the
package as low as possible (e.g. I unexported expvarCollector in this
commit because the NewExpvarCollector constructor is enough to export,
and it is now consistent with other collectors, like the goCollector).

Non-goals that won't be tackled anytime soon
--------------------------------------------

Something that I have played with a lot is "streaming collection",
i.e. allow an implementation of the `Registry` interface that collects
metrics incrementally and serves them while doing so. As it has turned
out, this has many many issues and makes the `Registry` interface very
clunky. Eventually, I made the call that it is unlikely we will really
implement streaming collection; and making the interface more clunky
for something that might not even happen is really a big no-no. Note
that the `Registry` interface only creates the in-memory
representation of the metric family protobufs in one go. The
serializaton onto the wire can still be handled in a streaming fashion
(which hasn't been done so far, without causing any trouble, but might
be done in the future without breaking any interfaces).

What are the breaking changes?
==============================

- Signatures of functions pushing to Pushgateway have changed to allow
  arbitrary grouping (which was planned for a long time anyway, and
  now that I had to work on the Push code anyway for the registry
  refurbishment, I finally did it,
  cf. https://github.com/prometheus/client_golang/issues/100).
  With the gained insight that pushing to the default registry is almost
  never the right thing, and now that we are breaking the Push call
  anyway, all the Push functions were moved to their own package,
  which cleans up the namespace and is more idiomatic (pushing
  Collectors is now literally done by `push.Collectors(...)`).

- The registry is doing more consistency checks by default now. Past
  creators of inconsistent metrics could have masked the problem by
  not setting `EnableCollectChecks`. Those inconsistencies will now be
  detected. (But note that a "best effort" metrics collection is now
  possible with `HandlerOpts.ErrorHandling = ContinueOnError`.)

- `EnableCollectChecks` is gone. The registry is now performing some
  of those checks anyway (see previous item), and a registry with all
  of those checks can now be created with `NewPedanticRegistry` (only
  used for testing).

- `PanicOnCollectError` is gone. This behavior can now be configured
  when creating a custom HTTP handler.
2016-08-02 18:46:22 +02:00
Robert Vollmert 1312da4c0c Inline hash/fnv. 2015-11-09 15:16:26 +01:00
Julius Volz 066ab78410 Correct typo in Counter.Set() docstring. 2015-04-30 18:51:33 +02:00
beorn7 000ef45157 Replaced http by HTTP if used as the name of the protocol in English. 2015-02-19 15:54:26 +01:00
Bjoern Rabenstein d7f8eb1083 Change "Prometheus Team" to "The Prometheus Authors". 2015-02-02 15:14:36 +01:00
Bjoern Rabenstein 3798bbca12 Add const labels to counter. 2015-01-28 15:47:48 +01:00
Bjoern Rabenstein f9401ffab9 Added "callback" metrics, e.g. GaugeFunc.
Change-Id: I449b558207963ce60572bd04c8102f1db684dd4c
2014-06-23 14:35:01 +02:00
Bjoern Rabenstein 5d40912fd2 Complete rewrite of the exposition library.
This rewrite had may backs and forths. In my git repository, it
consists of 35 commits which I cannot group or merge into reasonable
review buckets. Gerrit breaks fundamental git semantics, so I have to
squash the 35 commits into one for the review.

I'll push this not with refs/for/master, but with refs/for/next so
that we can transition after submission in a controlled fashion.

For the review, I recommend to start with looking at godoc and in
particular the many examples. After that, continue with a line-by-line
detailed review. (The big picture is hopefully as expected after
wrapping up the discussion earlier.)

Change-Id: Ib38cc46493a5139ca29d84020650929d94cac850
2014-06-17 14:08:22 +02:00
Bjoern Rabenstein e5dc0421cd Move signature.go and related tests to the model package.
The LabelsToSignature function is now used outside of the prometheus
package, too. Leaving it in the prometheuos package is misleading
design and will lead to circulat import chains soon.

Change-Id: If1ca442d4023b33b138cf79fee68e82ff2a355be
2014-04-25 20:48:16 +02:00
Matt T. Proud 7efd34a6f8 Optimize fingerprinting and metric locks.
These are all simple changes we should have caught a long time ago:

1. The hashing mechanism for fingerprint label sets should have not
   allocated new objects for the actual hashing---at least not
   egregiously.  This simplifies the hash writing by just byte-
   dumping the string stream into the hasher.

2. The hashing mechanism within the scope of a metric does not care
   about the value of the label keys themselves but only of the label
   values.  The keys can be dropped from the calculation.

3. The locking mechanism for the metrics should not block on hash
   computation but rather solely on the actual mutation or critical
   section reads.

4. For scalar metrics (i.e., ones with niladic label signatures), we
   should rely on a preallocated map versus requesting a new one
   ad hoc.

This is tested with Go 1.1, so the results may yield other values
for us elsewhere:

BEFORE
BenchmarkLabelValuesToSignatureScalar	500000000	         3.97 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabelValuesToSignatureSingle	 5000000	       714 ns/op	      74 B/op	       4 allocs/op
BenchmarkLabelValuesToSignatureDouble	 1000000	      1153 ns/op	     107 B/op	       5 allocs/op
BenchmarkLabelValuesToSignatureTriple	 1000000	      1588 ns/op	     138 B/op	       6 allocs/op
BenchmarkLabelToSignatureScalar	500000000	         3.91 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabelToSignatureSingle	 2000000	       874 ns/op	      92 B/op	       5 allocs/op
BenchmarkLabelToSignatureDouble	 1000000	      1528 ns/op	     139 B/op	       7 allocs/op
BenchmarkLabelToSignatureTriple	 1000000	      2172 ns/op	     186 B/op	       9 allocs/op

AFTER
BenchmarkLabelValuesToSignatureScalar	500000000	         4.36 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabelValuesToSignatureSingle	 5000000	       378 ns/op	      89 B/op	       4 allocs/op
BenchmarkLabelValuesToSignatureDouble	 5000000	       574 ns/op	     142 B/op	       5 allocs/op
BenchmarkLabelValuesToSignatureTriple	 5000000	       758 ns/op	     186 B/op	       6 allocs/op
BenchmarkLabelToSignatureScalar	500000000	         4.06 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabelToSignatureSingle	 5000000	       472 ns/op	     106 B/op	       5 allocs/op
BenchmarkLabelToSignatureDouble	 2000000	       746 ns/op	     174 B/op	       7 allocs/op
BenchmarkLabelToSignatureTriple	 1000000	      1061 ns/op	     235 B/op	       9 allocs/op

In effect, a single metric mutation operation's lookup overhead will
move from Before::iBenchmarkLabelToSignature to
After::BenchmarkLabelValuesToSignature.  This MINIMALLY reduces
1/2 the overhead.  I would be hesitant in reading the memory
allocation statistics, for this was run with the GC still on and
thusly inaccurate per Go benchmarking documentation.

Before::BenchmarkLabelValuesToSignature never existed, so it is not
of any intrinsic value in itself.  That said, the cases that still
rely on LabelToSignature experience consistently a 1/2 drop in time.

Change-Id: Ifc9e69f718af65a59f5be8117473518233258159
2014-04-14 19:06:09 +02:00
Bernerd Schaefer 29ebb580db Add Reset(map[string]string) to Metric interface
Change-Id: I289cf8796adbd6ff55f23bba7730145329de00e1
2014-02-19 15:18:16 +01:00
Matt T. Proud 4956aea5ac Protocol Buffer negotiation support in handler. 2013-07-01 17:14:58 +02:00
Bernerd Schaefer d4ff2cc87a Fix race conditions in metric methods
Methods which expect to use a mutex must be defined for the pointer
value, because mutexes are not copyable.
2013-05-03 16:02:03 +02:00
Bernerd Schaefer 71dd60e431 Registry and Metrics implement json.Marshaler
* Drop `AsMarshallable()` from the Metric interface. Use
  `json.Marshaler` and `MarshalJSON()`, and leverage JSON struct tags
  where possible.

* Add `MarshalJSON()` to Registry and remove `dumpToWriter`, which
  makes the registry handler much simpler.

In addition to simplifying some of the marshalling behavior, this also
has the nice side effect of cutting down the number of
`map[string]interface{}` instances.
2013-04-19 15:07:24 +02:00
Bernerd Schaefer 0b30e065c8 Metrics explicitly implement Metric interface 2013-04-19 15:04:07 +02:00
Matt T. Proud f320d28a6c Rearrange file and package per convention.
WIP - Please review but do not merge.
2013-04-04 15:27:09 +02:00