Commit Graph

110 Commits

Author SHA1 Message Date
Shivanth ae8497940d Add native histograms
Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Apply suggestions from code review

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Fix references and existing tests

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

fmt

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

validation

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Count validation

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Validation

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Space formatting

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Rename SyncMaptoMap -> SyncMapToMap

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Remove exemplars from parameters for constNativeHistogram function

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Lint fix

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>
2024-11-13 14:19:40 +01:00
Ivan Goncharov a934c35951
Add: exponential backoff for CAS operations on floats (#1661)
* add: exponential backoff for CAS operations of floats

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* add: some more benchmark use cases (higher contention)

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* fmt: fumpted some files

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* add: license header

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* add: comment explaining origin of backoff constants

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

---------

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>
2024-11-11 16:26:17 +00:00
Ivan Goncharov 78d7a94e46 add: linear search implementation (+ benchmarks)
Signed-off-by: Ivan Goncharov <i.morph@gmail.com>
2024-11-04 20:26:59 +01:00
Arthur Silva Sens 39e7c23b95
Merge pull request #1627 from prometheus/release-1.20
Merge Release 1.20 back to main
2024-09-17 10:39:34 +02:00
György Krajcsovits 1e398ccb12 native histogram: Fix race between Write and addExemplar
Follow-up to 1608

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-06 11:01:04 +02:00
György Krajcsovits d6b8c8925b Update comments with more explanations
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits 504566f07c Use simplified solution from #1609 for the data race
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits dc8e9a4d8a fix: native histogram: Simplify and fix addExemplar
mdIdx was redundant when len(exemplars)>1, so got rid of it, rIdx
is enough.

Don't compare timestamp of incoming exemplar to timestamp of
minimal distance exemplar. Most of the time the incoming exemplar
will be newer. And if not, the previous code just replaced an
exemplar one index after the minimal distance exemplar. Which had
an index out of range bug, plus is essentially random.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits dc819ceb1b Use a trivial solution to #1605
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
Parth Lawania 850b6c0898
changed the name of all variables with min/max name (#1606)
* changed the name of all variables with min/max name

Signed-off-by: Parth Lawania <parthlawania@gmail.com>

* removed predeclared ignore condition for min and max identifiers

Signed-off-by: Parth Lawania <parthlawania@gmail.com>

---------

Signed-off-by: Parth Lawania <parthlawania@gmail.com>
Co-authored-by: Parth Lawania <parth.lawania@super.money>
2024-09-02 07:41:40 -03:00
Steven Swartz 1a5f005c91
Allow creating constant histogram and summary metrics with a created timestamp (#1537) 2024-07-01 17:35:47 -03:00
Ziqi Zhao 2754a4c204 fix for comments
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2024-05-16 18:16:36 +08:00
Ziqi Zhao d8c7074b1c refract the implementation
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2024-05-09 22:11:15 +08:00
Ziqi Zhao 494ccce4f1 add native histogram exemplar support
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2024-05-09 07:14:46 +08:00
Björn Rabenstein cd8cba2ceb
histograms: Add timer to reset ASAP after bucket limiting has happened (#1367)
Fixes #1248. See issue description for all the details.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-10-19 14:22:02 +01:00
Oleksandr Redko bbab8fe770
Fix typos in comments, tests, and errors (#1346)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-09-21 12:31:08 +01:00
Arthur Silva Sens df7fa49417
Extend Counters, Summaries and Histograms with creation timestamp (#1313)
* Extend Counters, Summaries and Histograms with creation timestamp

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Backport created timestamp to existing tests

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Last touches (readability and consistency)

Changes:

* Comments for "now" are more explicit and not inlined.
* populateMetrics is simpler and bit more efficient without timestamp to time to timestamp conversionts for more common code.
* Test consistency and simplicity - the fewer variables the better.
* Fixed inconsistency for v2 and MetricVec - let's pass opt.now consistently.
* We don't need TestCounterXXXTimestamp - we test CT in many other places already.
* Added more involved test for counter vectors with created timestamp.
* Refactored normalization for simplicity.
* Make histogram, summaries now consistent.
* Simplified histograms CT flow and implemented proper CT on reset.

TODO for next PRs:
* NewConstSummary and NewConstHistogram - ability to specify CTs there.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Update prometheus/counter_test.go

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2023-09-21 10:46:54 +01:00
Bartlomiej Plotka 6f576e2c6f
histogram docs: Fixed minor nit. (#1324)
Signed-off-by: bwplotka <bwplotka@gmail.com>
2023-08-13 14:11:06 +01:00
beorn7 3d82c94432 histogram: Add a no-op span for an otherwise empty histogram
Fixes #1127.

If a native histogram has no observations and a zero threshold of
zero, then it is indistinguishable from a classic histogram. To give
scrapers a hint that it is indeed a native histogram, we add a no-op
span.

This needs follow-up PRs in prometheus/prometheus and
prometheus/client_model.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-20 17:34:00 +02:00
beorn7 f9db3821a8 histograms: Small code comment and code formatting improvements
Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-20 16:13:28 +02:00
Quentin D 644c80d136
Do not allocate memory when there's no constraints (#1296)
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2023-06-27 12:21:36 +01:00
Julius Volz 1b19d5f458 Fix docstring references to renamed native histogram fields / functions.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2023-06-13 10:36:40 +02:00
beorn7 77e97da564 histogram: Fix bug in bucket key calculation
The current code doesn't work fork negative schemas if the observed
value should go into a bucket with a non-positive key.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-05-25 19:03:43 +02:00
zhsj 7f99379117
Replace deprecated github.com/golang/protobuf package (#1183)
This replaces usage of proto.{Float64,Int32,Int64,String,Uint32,Uint64},
which doesn't break the interface.

And remove usage of proto.MarshalTextString in wrap_test.go

Updates: #1175

Signed-off-by: Shengjing Zhu <zhsj@debian.org>

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-22 16:14:00 +01:00
Quentin D fae2f6306b
Add constrained labels and Constrained variant for all MetricVecs (#1151)
* Introduce MetricVecOpts and add constraints to VariableLabels

MetricVecOpts exposes options specific to MetricVec initialisation. The
first option exposed by MetricVecOpts are constraints on VariableLabels,
allowing restrictions on the possible values a label can take, to
prevent cardinality explosion when the label value comes from a
non-trusted source (as a user input or HTTP header).

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>

* Add tests

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2022-12-13 13:47:52 +01:00
beorn7 043372ee04 Initialize atomic variables in histogramCounts with atomic.Store...
Signed-off-by: beorn7 <beorn@grafana.com>
2022-11-14 15:03:17 +01:00
beorn7 e92a8c7f48 Avoid the term 'sparse' where possible
This intends to avoid confusing users by the subtle difference between
a native histogram and a sparse bucket.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-31 16:23:08 +01:00
beorn7 d31f13b599 Add SparseBucketsZeroThresholdZero and groom doc comments
Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-19 19:22:46 +02:00
beorn7 111fae11e1 Merge branch 'main' into sparsehistogram 2022-10-19 18:09:21 +02:00
Balint Zsilavecz dcea97eee2
Fix `CumulativeCount` value of `+Inf` bucket created from exemplar (#1148)
* Fix `CumulativeCount` value of `+Inf` bucket created from exemplar

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Update prometheus/metric_test.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Clarify description of implicit `+Inf` bucket count

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

* Fix test variables

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>

Signed-off-by: Balint Zsilavecz <balint.zsilavecz@skyscanner.net>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-10-13 13:52:19 +02:00
beorn7 6942f9e454 sparse buckets: Fix handling of +Inf/-Inf/NaN observations
NaN observations now go to no bucket, but increment count (and
effectively set sum to NaN, too).

±Inf observations now go to the bucket following the bucket that would
have received math.MaxFloat64. The former is now the last bucket that
can be created.

The getLe is modified to return math.MaxFloat64 for the penultimate
possible bucket.

Also add a test for getLe.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-06 17:40:15 +02:00
beorn7 8cbcd4076a histograms: Move to new exposition protobuf format
Note that this is an incompatible change. To scrape this new format,
the Prometheus server needs to be updated at the same time. PR
incoming.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-07-19 16:54:05 +02:00
Björn Rabenstein eb59a7b3d7
Histogram: Fix bug with negative schemas (#1054)
* Histogram: Expose bug with negative schema

Signed-off-by: beorn7 <beorn@grafana.com>

* Histogram: Fix bug with negative schemas

Signed-off-by: beorn7 <beorn@grafana.com>
2022-05-15 23:52:30 +05:30
beorn7 b2372302ca Merge branch 'main' into sparsehistogram 2022-05-15 14:16:16 +02:00
William Perron 66837e3298
Add exemplar support for const histogram and const metric (#986)
* Add support for exemplars on constHistogram

Co-authored-by: William Perron <william.perron@shopify.com>
Signed-off-by: William Perron <william.perron@shopify.com>

* remove GetExemplars function

Signed-off-by: William Perron <william.perron@shopify.com>

* fixed linting warnings

reduce repetition in constHistogram w/ exemplar

Signed-off-by: William Perron <william.perron@shopify.com>

* Add values to correct bucket

Signed-off-by: William Perron <william.perron@shopify.com>

* Misc fixes

Co-authored-by: Francis Bogsanyi <francis.bogsanyi@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* avoid panic when there are fewer buckets than exemplars

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>

Signed-off-by: William Perron <william.perron@shopify.com>

* Added MustNewMetricWithExemplars that wraps metrics with exemplar (#3)

Changes:
* Make sure to not "leak" dto.Metric
* Reused upper bounds we already have for histogram
* Common code for all types.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Arun Mahendra <arun.mahendra@shopify.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2022-03-17 11:30:41 +00:00
beorn7 70253f4dd0 Fix typo in doc comment
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 14:07:18 +01:00
beorn7 263be8dab7 Refactoring of sparse histograms
Signed-off-by: beorn7 <beorn@grafana.com>
2021-09-01 01:26:06 +02:00
beorn7 24099603bc Implement strategy to limit the sparse bucket count
Signed-off-by: beorn7 <beorn@grafana.com>
2021-08-31 13:28:27 +02:00
beorn7 84fcafffb1 Merge branch 'master' into sparsehistogram 2021-08-18 16:41:04 +02:00
Seth Bunce 2261d5cda1
add ExponentialBucketsRange function (#899)
This function calculates exponential buckets with different arguments
than the existing ExponentialBuckets function. Instead of specifying the
start and factor, the user can specify the min and max bucket value. We
have been doing it this way internally at my company for some time.

Signed-off-by: Seth Bunce <seth.bunce@getcruise.com>
2021-08-12 16:56:44 +01:00
beorn7 9ef5f90a76 Allow a zero threshold of zero
Signed-off-by: beorn7 <beorn@grafana.com>
2021-07-20 19:01:13 +02:00
beorn7 aa6f67a9e6 Add TODO about bucket search optimization
Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-29 14:52:37 +02:00
beorn7 6c4e0ef740 Add tests for sparse histogram
Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-23 21:56:26 +02:00
beorn7 31318b7523 Switch to base-2 buckets
This seem what OTel is converging towards, see
https://github.com/open-telemetry/oteps/pull/149 .

I see pros and cons with base-10 vs base-2. They are discussed in
detail in that OTel PR, and the gist of the discussion is pretty much
in line with my design doc. Since the balance is easy to tip here, I
think we should go with base-2 if OTel picks base-2. This also seems
to be in agreement with several proprietary solution (see again the
discussion on that OTel PR.)

The idea to make the number of buckets per power of 2 (or formerly 10)
a power of 2 itself was also sketched out in the design doc
already. It guarantees mergeability of different resolutions. I was
undecided between making it a recommendation or mandatory. Now I think
it should be mandatory as it has the additional benefit of playing
well with OTel's plans.

This commit also addresses a number of outstanding TODOs.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-23 16:48:03 +02:00
beorn7 5aa8534cd0 Merge branch 'master' into sparsehistogram 2021-06-11 14:58:43 +02:00
beorn7 f34145a85e Document implications of negative observations
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-26 23:41:30 +02:00
beorn7 553ed73917 Fix lint warning
Signed-off-by: beorn7 <beorn@grafana.com>
2021-05-03 16:58:07 +02:00
beorn7 d6983369d2 Merge branch 'master' into beorn7/histogram 2021-04-03 17:28:55 +02:00
beorn7 e92283d644 Fix linter ignores
Signed-off-by: beorn7 <beorn@grafana.com>
2021-03-16 17:19:03 +01:00
beorn7 08104a0ef9 Minor doc comment fixes
Signed-off-by: beorn7 <beorn@grafana.com>
2021-01-29 22:24:27 +01:00