Commit Graph

88 Commits

Author SHA1 Message Date
beorn7 77e97da564 histogram: Fix bug in bucket key calculation
The current code doesn't work fork negative schemas if the observed
value should go into a bucket with a non-positive key.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-05-25 19:03:43 +02:00
zhsj 7f99379117
Replace deprecated github.com/golang/protobuf package (#1183)
This replaces usage of proto.{Float64,Int32,Int64,String,Uint32,Uint64},
which doesn't break the interface.

And remove usage of proto.MarshalTextString in wrap_test.go

Updates: #1175

Signed-off-by: Shengjing Zhu <zhsj@debian.org>

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-22 16:14:00 +01:00
Quentin D fae2f6306b
Add constrained labels and Constrained variant for all MetricVecs (#1151)
* Introduce MetricVecOpts and add constraints to VariableLabels

MetricVecOpts exposes options specific to MetricVec initialisation. The
first option exposed by MetricVecOpts are constraints on VariableLabels,
allowing restrictions on the possible values a label can take, to
prevent cardinality explosion when the label value comes from a
non-trusted source (as a user input or HTTP header).

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>

* Add tests

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2022-12-13 13:47:52 +01:00
beorn7 043372ee04 Initialize atomic variables in histogramCounts with atomic.Store...
Signed-off-by: beorn7 <beorn@grafana.com>
2022-11-14 15:03:17 +01:00
beorn7 e92a8c7f48 Avoid the term 'sparse' where possible
This intends to avoid confusing users by the subtle difference between
a native histogram and a sparse bucket.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-31 16:23:08 +01:00
beorn7 d31f13b599 Add SparseBucketsZeroThresholdZero and groom doc comments
Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-19 19:22:46 +02:00
beorn7 111fae11e1 Merge branch 'main' into sparsehistogram 2022-10-19 18:09:21 +02:00
Balint Zsilavecz dcea97eee2
Fix `CumulativeCount` value of `+Inf` bucket created from exemplar (#1148)
* Fix `CumulativeCount` value of `+Inf` bucket created from exemplar

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Update prometheus/metric_test.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Clarify description of implicit `+Inf` bucket count

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Fix test variables

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-10-13 13:52:19 +02:00
beorn7 6942f9e454 sparse buckets: Fix handling of +Inf/-Inf/NaN observations
NaN observations now go to no bucket, but increment count (and
effectively set sum to NaN, too).

±Inf observations now go to the bucket following the bucket that would
have received math.MaxFloat64. The former is now the last bucket that
can be created.

The getLe is modified to return math.MaxFloat64 for the penultimate
possible bucket.

Also add a test for getLe.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-06 17:40:15 +02:00
beorn7 8cbcd4076a histograms: Move to new exposition protobuf format
Note that this is an incompatible change. To scrape this new format,
the Prometheus server needs to be updated at the same time. PR
incoming.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-07-19 16:54:05 +02:00
Björn Rabenstein eb59a7b3d7
Histogram: Fix bug with negative schemas (#1054)
* Histogram: Expose bug with negative schema

Signed-off-by: beorn7 <beorn@grafana.com>

* Histogram: Fix bug with negative schemas

Signed-off-by: beorn7 <beorn@grafana.com>
2022-05-15 23:52:30 +05:30
beorn7 b2372302ca Merge branch 'main' into sparsehistogram 2022-05-15 14:16:16 +02:00
William Perron 66837e3298
Add exemplar support for const histogram and const metric (#986)
* Add support for exemplars on constHistogram

Co-authored-by: William Perron <william.perron@shopify.com>
Signed-off-by: William Perron <william.perron@shopify.com>

* remove GetExemplars function

Signed-off-by: William Perron <william.perron@shopify.com>

* fixed linting warnings

reduce repetition in constHistogram w/ exemplar

Signed-off-by: William Perron <william.perron@shopify.com>

* Add values to correct bucket

Signed-off-by: William Perron <william.perron@shopify.com>

* Misc fixes

Co-authored-by: Francis Bogsanyi <francis.bogsanyi@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* avoid panic when there are fewer buckets than exemplars

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* Added MustNewMetricWithExemplars that wraps metrics with exemplar (#3)

Changes:
* Make sure to not "leak" dto.Metric
* Reused upper bounds we already have for histogram
* Common code for all types.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-03-17 11:30:41 +00:00
beorn7 70253f4dd0 Fix typo in doc comment
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 14:07:18 +01:00
beorn7 263be8dab7 Refactoring of sparse histograms
Signed-off-by: beorn7 <beorn@grafana.com>
2021-09-01 01:26:06 +02:00
beorn7 24099603bc Implement strategy to limit the sparse bucket count
Signed-off-by: beorn7 <beorn@grafana.com>
2021-08-31 13:28:27 +02:00
beorn7 84fcafffb1 Merge branch 'master' into sparsehistogram 2021-08-18 16:41:04 +02:00
Seth Bunce 2261d5cda1
add ExponentialBucketsRange function (#899)
This function calculates exponential buckets with different arguments
than the existing ExponentialBuckets function. Instead of specifying the
start and factor, the user can specify the min and max bucket value. We
have been doing it this way internally at my company for some time.

Signed-off-by: Seth Bunce <seth.bunce@getcruise.com>
2021-08-12 16:56:44 +01:00
beorn7 9ef5f90a76 Allow a zero threshold of zero
Signed-off-by: beorn7 <beorn@grafana.com>
2021-07-20 19:01:13 +02:00
beorn7 aa6f67a9e6 Add TODO about bucket search optimization
Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-29 14:52:37 +02:00
beorn7 6c4e0ef740 Add tests for sparse histogram
Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-23 21:56:26 +02:00
beorn7 31318b7523 Switch to base-2 buckets
This seem what OTel is converging towards, see
https://github.com/open-telemetry/oteps/pull/149 .

I see pros and cons with base-10 vs base-2. They are discussed in
detail in that OTel PR, and the gist of the discussion is pretty much
in line with my design doc. Since the balance is easy to tip here, I
think we should go with base-2 if OTel picks base-2. This also seems
to be in agreement with several proprietary solution (see again the
discussion on that OTel PR.)

The idea to make the number of buckets per power of 2 (or formerly 10)
a power of 2 itself was also sketched out in the design doc
already. It guarantees mergeability of different resolutions. I was
undecided between making it a recommendation or mandatory. Now I think
it should be mandatory as it has the additional benefit of playing
well with OTel's plans.

This commit also addresses a number of outstanding TODOs.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-23 16:48:03 +02:00
beorn7 5aa8534cd0 Merge branch 'master' into sparsehistogram 2021-06-11 14:58:43 +02:00
beorn7 f34145a85e Document implications of negative observations
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-26 23:41:30 +02:00
beorn7 553ed73917 Fix lint warning
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-03 16:58:07 +02:00
beorn7 d6983369d2 Merge branch 'master' into beorn7/histogram 2021-04-03 17:28:55 +02:00
beorn7 e92283d644 Fix linter ignores
Signed-off-by: beorn7 <beorn@grafana.com>
2021-03-16 17:19:03 +01:00
beorn7 08104a0ef9 Minor doc comment fixes
Signed-off-by: beorn7 <beorn@grafana.com>
2021-01-29 22:24:27 +01:00
beorn7 85aa957f63 Export MetricVec (again)
MetricVec was already exported in early versions of this library, but
nobody really used it to implement vectors of custom Metric
implementations. Now #796 has shown up with a fairly special use case
for which I'd prefer a custom implementation of a special
"auto-sampling histogram" outside of this library. Therefore, I'd like
to reinstate support for creating vectors of custom Metric
implementations.

I played around for quite some while with the option of a separate
package providing the tools one would need to create vectors of custom
Metric implementations. However, with the current structure of the
prometheus/client_golang/prometheus package, this leads to a lot of
complications with circular dependencies. (The new package would need
the primitives from the prometheus package, while the existing metric
vectors like GaugeVec need to import the new vector package to not
duplicate the implementation. Separating vector types from the main
prometheus package is out of the question at this point because that
would be a breaking change.)

Signed-off-by: beorn7 <beorn@grafana.com>
2020-10-15 14:32:54 +02:00
Mark Hansen 7687ac2137 histogram.go: fix copy/paste typo
This method calls NewConstHistogram.

Signed-off-by: Mark Hansen <mark@markhansen.co.nz>
2020-06-21 13:47:29 +10:00
beorn7 dba1478b8a Add lint:ignore for protobuf deprecation
`github.com/golang/protobuf/proto` is deprecated in lieu of
`google.golang.org/protobuf/proto`. However, we cannot simply
migrate. Types from the proto package are exposed to users of packages
in this repo. If we migrate here, users have to migrate to. Thus, we
could only migrate with a major version bump.

In different news, with all the inline lint:ignore comments, including
the existing ones, there is no need to repeat the exception in the
Makefile.

A current version of `staticcheck` is happy with the code after this
commit. golangci-lint is broken at the moment, however, and ignores
the lint:ignore comments in the code as well as those via envvar.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-05-14 20:11:22 +02:00
beorn7 a9d0066408 Add note about pow-of-10 precision issue
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-13 16:43:23 +02:00
beorn7 d1f5366b52 Fix span offset
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-13 15:50:40 +02:00
beorn7 abe540f8c0 Encode sparse histograms in protobuf
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-07 23:18:40 +02:00
beorn7 c98db4eccd Demo sparse histograms
Printf the structure of it instead of actually encoding it.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-07 23:18:09 +02:00
beorn7 f34b09877c Pull out ...WithExemplar methods into separate interfaces
This is, sadly, the only way to avoid a breaking change. The cost is
that anyone using exemplars has to perform a type assertion. This is,
however, a common pattern where interfaces turn out to need additional
methods in a stable library or only some implementations can provide
the additional methods (AKA "interface upgrade").

Needless to say that in v2 of this library, we'll do things in a more
straight forward way.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-27 15:41:13 +01:00
beorn7 c32ffd121f Add tests for examplars
Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-24 17:12:36 +01:00
beorn7 57d41259e1 Add exemplars to counter and histogram
Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-24 17:12:08 +01:00
Dean Coakley 508fc33903
Fix link to writing exporters guide
Signed-off-by: Dean Coakley <dean.s.coakley@gmail.com>
2019-11-28 15:01:46 +00:00
beorn7 679da921f7 Simplify code
`gofmt -s` taught me that this is possible.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-10-16 12:18:16 +02:00
beorn7 295f7e4861 Fix some comment and naming nits as leftover from #536
Signed-off-by: beorn7 <beorn@soundcloud.com>
2019-02-11 18:52:52 +01:00
Pascal S. de Kloe 19ff277483 Save memory on histograms: slightly faster with less code (#536)
Use the highest bit for buffer switch tracking in histograms

Signed-off-by: Pascal S. de Kloe <pascal@quies.net>
2019-02-11 18:29:02 +01:00
beorn7 226b83ac2e Provide lock-free implementation for Summary without objectives
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-12-24 11:23:13 +01:00
Peter Jausovec ea348d7c20 Update code based on the PR feedback
Signed-off-by: Peter Jausovec <peter.jausovec@oracle.com>
2018-11-02 09:01:14 -07:00
Peter Jausovec 902733d080 Add more info to the inconsistent cardinality errors
Signed-off-by: Peter Jausovec <peter.jausovec@oracle.com>
2018-10-31 11:01:42 -07:00
beorn7 7973e4709d Ensure 64bit alignment on 32bit platforms for histograms
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-24 13:28:13 +02:00
beorn7 663a9ad019 Make Help strings optional
This is in line with
https://prometheus.io/docs/instrumenting/writing_clientlibs/#metric-description-and-help

Since the zero value of a string in Go is `""`, we cannot distinguish
between a Help string not set and an empty Help string. Thus, we just
make it formally optional here with an encouragement to set it in the
doc comment.

In v0.10, the Help string will probably become a "normal" argument of
the constructor rather than a field in an Opts struct.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-17 12:07:31 +02:00
beorn7 713e6eb604 Let NewConst... functions detect invalid Desc
The error of the invalid Desc is returned in that case.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-17 11:50:42 +02:00
beorn7 1b2bd1d665 Make Histogram observations atomic while keeping them lock-free
Fixes #275

This is rather tricky and required some studying of the Go memory
model. I have added copious code comments to explain what's going on.

Benchmarks haven't changed significantly, despite the additional
atomic operations now required during Observe. Write performance is
noticable, but it is also much more involved now and has a mutex. (But
note that Write is supposed to be a relatively rare operation and thus
not in the hot path compared to Observe.) Allocs haven't changed at
all.

OLD:

BenchmarkHistogramWithLabelValues-4     10000000               151 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramNoLabels-4            50000000                36.0 ns/op             0 B/op          0 allocs/op
BenchmarkHistogramObserve1-4            50000000                28.1 ns/op             0 B/op          0 allocs/op
BenchmarkHistogramObserve2-4            10000000               160 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramObserve4-4             5000000               378 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramObserve8-4             2000000               768 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramWrite1-4               1000000              1589 ns/op             896 B/op         37 allocs/op
BenchmarkHistogramWrite2-4                500000              2973 ns/op            1792 B/op         74 allocs/op
BenchmarkHistogramWrite4-4                300000              6979 ns/op            3584 B/op        148 allocs/op
BenchmarkHistogramWrite8-4                100000             10701 ns/op            7168 B/op        296 allocs/op

NEW:

BenchmarkHistogramWithLabelValues-4     10000000               191 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramNoLabels-4            30000000                50.1 ns/op             0 B/op          0 allocs/op
BenchmarkHistogramObserve1-4            30000000                40.0 ns/op             0 B/op          0 allocs/op
BenchmarkHistogramObserve2-4            20000000                91.5 ns/op             0 B/op          0 allocs/op
BenchmarkHistogramObserve4-4             5000000               317 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramObserve8-4             2000000               636 ns/op               0 B/op          0 allocs/op
BenchmarkHistogramWrite1-4               1000000              2072 ns/op             896 B/op         37 allocs/op
BenchmarkHistogramWrite2-4                300000              3729 ns/op            1792 B/op         74 allocs/op
BenchmarkHistogramWrite4-4                200000              7847 ns/op            3584 B/op        148 allocs/op
BenchmarkHistogramWrite8-4                100000             16975 ns/op            7168 B/op        296 allocs/op

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-12 15:03:44 +02:00
beorn7 1ba60c7d58 Pull currying methods up into ObserverVec interface 2017-12-22 16:11:58 +01:00