Commit Graph

96 Commits

Author SHA1 Message Date
Björn Rabenstein cd8cba2ceb
histograms: Add timer to reset ASAP after bucket limiting has happened (#1367)
Fixes #1248. See issue description for all the details.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-10-19 14:22:02 +01:00
Oleksandr Redko bbab8fe770
Fix typos in comments, tests, and errors (#1346)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-09-21 12:31:08 +01:00
Arthur Silva Sens df7fa49417
Extend Counters, Summaries and Histograms with creation timestamp (#1313)
* Extend Counters, Summaries and Histograms with creation timestamp

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Backport created timestamp to existing tests

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Last touches (readability and consistency)

Changes:

* Comments for "now" are more explicit and not inlined.
* populateMetrics is simpler and bit more efficient without timestamp to time to timestamp conversionts for more common code.
* Test consistency and simplicity - the fewer variables the better.
* Fixed inconsistency for v2 and MetricVec - let's pass opt.now consistently.
* We don't need TestCounterXXXTimestamp - we test CT in many other places already.
* Added more involved test for counter vectors with created timestamp.
* Refactored normalization for simplicity.
* Make histogram, summaries now consistent.
* Simplified histograms CT flow and implemented proper CT on reset.

TODO for next PRs:
* NewConstSummary and NewConstHistogram - ability to specify CTs there.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Update prometheus/counter_test.go

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2023-09-21 10:46:54 +01:00
Bartlomiej Plotka 6f576e2c6f
histogram docs: Fixed minor nit. (#1324)
Signed-off-by: bwplotka <bwplotka@gmail.com>
2023-08-13 14:11:06 +01:00
beorn7 3d82c94432 histogram: Add a no-op span for an otherwise empty histogram
Fixes #1127.

If a native histogram has no observations and a zero threshold of
zero, then it is indistinguishable from a classic histogram. To give
scrapers a hint that it is indeed a native histogram, we add a no-op
span.

This needs follow-up PRs in prometheus/prometheus and
prometheus/client_model.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-20 17:34:00 +02:00
beorn7 f9db3821a8 histograms: Small code comment and code formatting improvements
Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-20 16:13:28 +02:00
Quentin D 644c80d136
Do not allocate memory when there's no constraints (#1296)
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2023-06-27 12:21:36 +01:00
Julius Volz 1b19d5f458 Fix docstring references to renamed native histogram fields / functions.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2023-06-13 10:36:40 +02:00
beorn7 77e97da564 histogram: Fix bug in bucket key calculation
The current code doesn't work fork negative schemas if the observed
value should go into a bucket with a non-positive key.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-05-25 19:03:43 +02:00
zhsj 7f99379117
Replace deprecated github.com/golang/protobuf package (#1183)
This replaces usage of proto.{Float64,Int32,Int64,String,Uint32,Uint64},
which doesn't break the interface.

And remove usage of proto.MarshalTextString in wrap_test.go

Updates: #1175

Signed-off-by: Shengjing Zhu <zhsj@debian.org>

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-22 16:14:00 +01:00
Quentin D fae2f6306b
Add constrained labels and Constrained variant for all MetricVecs (#1151)
* Introduce MetricVecOpts and add constraints to VariableLabels

MetricVecOpts exposes options specific to MetricVec initialisation. The
first option exposed by MetricVecOpts are constraints on VariableLabels,
allowing restrictions on the possible values a label can take, to
prevent cardinality explosion when the label value comes from a
non-trusted source (as a user input or HTTP header).

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>

* Add tests

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2022-12-13 13:47:52 +01:00
beorn7 043372ee04 Initialize atomic variables in histogramCounts with atomic.Store...
Signed-off-by: beorn7 <beorn@grafana.com>
2022-11-14 15:03:17 +01:00
beorn7 e92a8c7f48 Avoid the term 'sparse' where possible
This intends to avoid confusing users by the subtle difference between
a native histogram and a sparse bucket.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-31 16:23:08 +01:00
beorn7 d31f13b599 Add SparseBucketsZeroThresholdZero and groom doc comments
Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-19 19:22:46 +02:00
beorn7 111fae11e1 Merge branch 'main' into sparsehistogram 2022-10-19 18:09:21 +02:00
Balint Zsilavecz dcea97eee2
Fix `CumulativeCount` value of `+Inf` bucket created from exemplar (#1148)
* Fix `CumulativeCount` value of `+Inf` bucket created from exemplar

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Update prometheus/metric_test.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Clarify description of implicit `+Inf` bucket count

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Fix test variables

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-10-13 13:52:19 +02:00
beorn7 6942f9e454 sparse buckets: Fix handling of +Inf/-Inf/NaN observations
NaN observations now go to no bucket, but increment count (and
effectively set sum to NaN, too).

±Inf observations now go to the bucket following the bucket that would
have received math.MaxFloat64. The former is now the last bucket that
can be created.

The getLe is modified to return math.MaxFloat64 for the penultimate
possible bucket.

Also add a test for getLe.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-06 17:40:15 +02:00
beorn7 8cbcd4076a histograms: Move to new exposition protobuf format
Note that this is an incompatible change. To scrape this new format,
the Prometheus server needs to be updated at the same time. PR
incoming.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-07-19 16:54:05 +02:00
Björn Rabenstein eb59a7b3d7
Histogram: Fix bug with negative schemas (#1054)
* Histogram: Expose bug with negative schema

Signed-off-by: beorn7 <beorn@grafana.com>

* Histogram: Fix bug with negative schemas

Signed-off-by: beorn7 <beorn@grafana.com>
2022-05-15 23:52:30 +05:30
beorn7 b2372302ca Merge branch 'main' into sparsehistogram 2022-05-15 14:16:16 +02:00
William Perron 66837e3298
Add exemplar support for const histogram and const metric (#986)
* Add support for exemplars on constHistogram

Co-authored-by: William Perron <william.perron@shopify.com>
Signed-off-by: William Perron <william.perron@shopify.com>

* remove GetExemplars function

Signed-off-by: William Perron <william.perron@shopify.com>

* fixed linting warnings

reduce repetition in constHistogram w/ exemplar

Signed-off-by: William Perron <william.perron@shopify.com>

* Add values to correct bucket

Signed-off-by: William Perron <william.perron@shopify.com>

* Misc fixes

Co-authored-by: Francis Bogsanyi <francis.bogsanyi@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* avoid panic when there are fewer buckets than exemplars

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* Added MustNewMetricWithExemplars that wraps metrics with exemplar (#3)

Changes:
* Make sure to not "leak" dto.Metric
* Reused upper bounds we already have for histogram
* Common code for all types.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-03-17 11:30:41 +00:00
beorn7 70253f4dd0 Fix typo in doc comment
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 14:07:18 +01:00
beorn7 263be8dab7 Refactoring of sparse histograms
Signed-off-by: beorn7 <beorn@grafana.com>
2021-09-01 01:26:06 +02:00
beorn7 24099603bc Implement strategy to limit the sparse bucket count
Signed-off-by: beorn7 <beorn@grafana.com>
2021-08-31 13:28:27 +02:00
beorn7 84fcafffb1 Merge branch 'master' into sparsehistogram 2021-08-18 16:41:04 +02:00
Seth Bunce 2261d5cda1
add ExponentialBucketsRange function (#899)
This function calculates exponential buckets with different arguments
than the existing ExponentialBuckets function. Instead of specifying the
start and factor, the user can specify the min and max bucket value. We
have been doing it this way internally at my company for some time.

Signed-off-by: Seth Bunce <seth.bunce@getcruise.com>
2021-08-12 16:56:44 +01:00
beorn7 9ef5f90a76 Allow a zero threshold of zero
Signed-off-by: beorn7 <beorn@grafana.com>
2021-07-20 19:01:13 +02:00
beorn7 aa6f67a9e6 Add TODO about bucket search optimization
Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-29 14:52:37 +02:00
beorn7 6c4e0ef740 Add tests for sparse histogram
Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-23 21:56:26 +02:00
beorn7 31318b7523 Switch to base-2 buckets
This seem what OTel is converging towards, see
https://github.com/open-telemetry/oteps/pull/149 .

I see pros and cons with base-10 vs base-2. They are discussed in
detail in that OTel PR, and the gist of the discussion is pretty much
in line with my design doc. Since the balance is easy to tip here, I
think we should go with base-2 if OTel picks base-2. This also seems
to be in agreement with several proprietary solution (see again the
discussion on that OTel PR.)

The idea to make the number of buckets per power of 2 (or formerly 10)
a power of 2 itself was also sketched out in the design doc
already. It guarantees mergeability of different resolutions. I was
undecided between making it a recommendation or mandatory. Now I think
it should be mandatory as it has the additional benefit of playing
well with OTel's plans.

This commit also addresses a number of outstanding TODOs.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-23 16:48:03 +02:00
beorn7 5aa8534cd0 Merge branch 'master' into sparsehistogram 2021-06-11 14:58:43 +02:00
beorn7 f34145a85e Document implications of negative observations
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-26 23:41:30 +02:00
beorn7 553ed73917 Fix lint warning
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-03 16:58:07 +02:00
beorn7 d6983369d2 Merge branch 'master' into beorn7/histogram 2021-04-03 17:28:55 +02:00
beorn7 e92283d644 Fix linter ignores
Signed-off-by: beorn7 <beorn@grafana.com>
2021-03-16 17:19:03 +01:00
beorn7 08104a0ef9 Minor doc comment fixes
Signed-off-by: beorn7 <beorn@grafana.com>
2021-01-29 22:24:27 +01:00
beorn7 85aa957f63 Export MetricVec (again)
MetricVec was already exported in early versions of this library, but
nobody really used it to implement vectors of custom Metric
implementations. Now #796 has shown up with a fairly special use case
for which I'd prefer a custom implementation of a special
"auto-sampling histogram" outside of this library. Therefore, I'd like
to reinstate support for creating vectors of custom Metric
implementations.

I played around for quite some while with the option of a separate
package providing the tools one would need to create vectors of custom
Metric implementations. However, with the current structure of the
prometheus/client_golang/prometheus package, this leads to a lot of
complications with circular dependencies. (The new package would need
the primitives from the prometheus package, while the existing metric
vectors like GaugeVec need to import the new vector package to not
duplicate the implementation. Separating vector types from the main
prometheus package is out of the question at this point because that
would be a breaking change.)

Signed-off-by: beorn7 <beorn@grafana.com>
2020-10-15 14:32:54 +02:00
Mark Hansen 7687ac2137 histogram.go: fix copy/paste typo
This method calls NewConstHistogram.

Signed-off-by: Mark Hansen <mark@markhansen.co.nz>
2020-06-21 13:47:29 +10:00
beorn7 dba1478b8a Add lint:ignore for protobuf deprecation
`github.com/golang/protobuf/proto` is deprecated in lieu of
`google.golang.org/protobuf/proto`. However, we cannot simply
migrate. Types from the proto package are exposed to users of packages
in this repo. If we migrate here, users have to migrate to. Thus, we
could only migrate with a major version bump.

In different news, with all the inline lint:ignore comments, including
the existing ones, there is no need to repeat the exception in the
Makefile.

A current version of `staticcheck` is happy with the code after this
commit. golangci-lint is broken at the moment, however, and ignores
the lint:ignore comments in the code as well as those via envvar.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-05-14 20:11:22 +02:00
beorn7 a9d0066408 Add note about pow-of-10 precision issue
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-13 16:43:23 +02:00
beorn7 d1f5366b52 Fix span offset
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-13 15:50:40 +02:00
beorn7 abe540f8c0 Encode sparse histograms in protobuf
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-07 23:18:40 +02:00
beorn7 c98db4eccd Demo sparse histograms
Printf the structure of it instead of actually encoding it.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-07 23:18:09 +02:00
beorn7 f34b09877c Pull out ...WithExemplar methods into separate interfaces
This is, sadly, the only way to avoid a breaking change. The cost is
that anyone using exemplars has to perform a type assertion. This is,
however, a common pattern where interfaces turn out to need additional
methods in a stable library or only some implementations can provide
the additional methods (AKA "interface upgrade").

Needless to say that in v2 of this library, we'll do things in a more
straight forward way.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-27 15:41:13 +01:00
beorn7 c32ffd121f Add tests for examplars
Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-24 17:12:36 +01:00
beorn7 57d41259e1 Add exemplars to counter and histogram
Signed-off-by: beorn7 <beorn@grafana.com>
2020-01-24 17:12:08 +01:00
Dean Coakley 508fc33903
Fix link to writing exporters guide
Signed-off-by: Dean Coakley <dean.s.coakley@gmail.com>
2019-11-28 15:01:46 +00:00
beorn7 679da921f7 Simplify code
`gofmt -s` taught me that this is possible.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-10-16 12:18:16 +02:00
beorn7 295f7e4861 Fix some comment and naming nits as leftover from #536
Signed-off-by: beorn7 <beorn@soundcloud.com>
2019-02-11 18:52:52 +01:00
Pascal S. de Kloe 19ff277483 Save memory on histograms: slightly faster with less code (#536)
Use the highest bit for buffer switch tracking in histograms

Signed-off-by: Pascal S. de Kloe <pascal@quies.net>
2019-02-11 18:29:02 +01:00