* Introduce MetricVecOpts and add constraints to VariableLabels
MetricVecOpts exposes options specific to MetricVec initialisation. The
first option exposed by MetricVecOpts are constraints on VariableLabels,
allowing restrictions on the possible values a label can take, to
prevent cardinality explosion when the label value comes from a
non-trusted source (as a user input or HTTP header).
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
* Add tests
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
* WIP partial match
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Cleanup
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Comments
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Tests and comments
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Handle properly deleting multiple metrics, update tests
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Comments
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Try using curry values
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Skip curry value to demo
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Fix curry deletion, remove outdated comment.
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Fix logic for deletion of metrics from prior to currying
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Don't match curried values. Update tests.
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Remove unneccesasry helper and todo comments
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Comment about partial matching curried labels
Signed-off-by: Zach Stone <zach@giantswarm.io>
* Simplify curried value check
Signed-off-by: Zach Stone <zach@giantswarm.io>
The idea behind it is described in detail in
https://github.com/prometheus/client_golang/issues/320 .
This commit also updates the example given in
promhttp/instrument_server_test.go , which nicely illustrates the
benefit of this change.
So far, currying could be emulated by creating different metric vec's
with different values in their ConstLabels. This was quite difficult
to grasp - which is essentially what was done in the example mentioned
above. Now that this use case can be solved without ConstLabels, we
can safely declare ConstLabels as rarely used. (Perhaps we can
deprecate them entirely one day, but I'll take a raincheck on that
when the changes of v0.10 have materialized.) This commit thus also
updates the ConstLabel doc comments in the various Opts. (It contained
fairly outdated stuff anyway.)
After increasing unit test coverage, it was found that the split
function call nature of metric matching wasn't working well in many
cases. By increasing test coverage, we've ensured that both the fast
path and fallback collision path are working appropriately.
With these changes, there is a further performance hit, but now the
results are ensured to be correct.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
While hash collisions are quite rare, the current state of the client
library carries a risk of merging two separate label values into a
single metric bucket. The effects are near impossible to detect and the
result will be missing or incorrect counters.
This changeset handles hash collisions by falling back to collision
resolution if multiple label values hash to the same value. This works
similar to separate chaining using a slice. Extra storage is minimized
to only the value key slice to that metrics can be differentiated
within a bucket.
In general, the cost of handling collisions is completely minimized
under normal operation. Performance does show slight increases in
certain areas, but these are more likely statistically anomalies. More
importantly, zero allocation behavior for metrics is preserved on the
fast path. Minimal allocations may be made during collision handling but
this has minimal effect.
Benchmark comparisons with and without collision resolution follow.
```
benchmark old ns/op new ns/op delta
BenchmarkCounterWithLabelValues-4 99.0 107 +8.08%
BenchmarkCounterWithLabelValuesConcurrent-4 79.6 91.0 +14.32%
BenchmarkCounterWithMappedLabels-4 518 542 +4.63%
BenchmarkCounterWithPreparedMappedLabels-4 127 137 +7.87%
BenchmarkCounterNoLabels-4 19.5 19.1 -2.05%
BenchmarkGaugeWithLabelValues-4 97.4 110 +12.94%
BenchmarkGaugeNoLabels-4 12.4 10.3 -16.94%
BenchmarkSummaryWithLabelValues-4 1204 915 -24.00%
BenchmarkSummaryNoLabels-4 936 847 -9.51%
BenchmarkHistogramWithLabelValues-4 147 147 +0.00%
BenchmarkHistogramNoLabels-4 50.6 49.3 -2.57%
BenchmarkHistogramObserve1-4 37.9 37.5 -1.06%
BenchmarkHistogramObserve2-4 122 137 +12.30%
BenchmarkHistogramObserve4-4 310 352 +13.55%
BenchmarkHistogramObserve8-4 691 729 +5.50%
BenchmarkHistogramWrite1-4 3374 3097 -8.21%
BenchmarkHistogramWrite2-4 5310 5051 -4.88%
BenchmarkHistogramWrite4-4 12094 10690 -11.61%
BenchmarkHistogramWrite8-4 19416 17755 -8.55%
BenchmarkHandler-4 11934304 13765894 +15.35%
BenchmarkSummaryObserve1-4 1119 1105 -1.25%
BenchmarkSummaryObserve2-4 3679 3430 -6.77%
BenchmarkSummaryObserve4-4 10678 7982 -25.25%
BenchmarkSummaryObserve8-4 22974 16689 -27.36%
BenchmarkSummaryWrite1-4 25962 14680 -43.46%
BenchmarkSummaryWrite2-4 38019 35073 -7.75%
BenchmarkSummaryWrite4-4 78027 56816 -27.18%
BenchmarkSummaryWrite8-4 117220 132248 +12.82%
BenchmarkMetricVecWithLabelValuesBasic-4 138 133 -3.62%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4 150 144 -4.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4 263 256 -2.66%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4 145 155 +6.90%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4 606 634 +4.62%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4 746 641 -14.08%
benchmark old allocs new allocs delta
BenchmarkCounterWithLabelValues-4 0 0 +0.00%
BenchmarkCounterWithLabelValuesConcurrent-4 0 0 +0.00%
BenchmarkCounterWithMappedLabels-4 2 2 +0.00%
BenchmarkCounterWithPreparedMappedLabels-4 0 0 +0.00%
BenchmarkCounterNoLabels-4 0 0 +0.00%
BenchmarkGaugeWithLabelValues-4 0 0 +0.00%
BenchmarkGaugeNoLabels-4 0 0 +0.00%
BenchmarkSummaryWithLabelValues-4 0 0 +0.00%
BenchmarkSummaryNoLabels-4 0 0 +0.00%
BenchmarkHistogramWithLabelValues-4 0 0 +0.00%
BenchmarkHistogramNoLabels-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValuesBasic-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkCounterWithLabelValues-4 0 0 +0.00%
BenchmarkCounterWithLabelValuesConcurrent-4 0 0 +0.00%
BenchmarkCounterWithMappedLabels-4 336 336 +0.00%
BenchmarkCounterWithPreparedMappedLabels-4 0 0 +0.00%
BenchmarkCounterNoLabels-4 0 0 +0.00%
BenchmarkGaugeWithLabelValues-4 0 0 +0.00%
BenchmarkGaugeNoLabels-4 0 0 +0.00%
BenchmarkSummaryWithLabelValues-4 0 0 +0.00%
BenchmarkSummaryNoLabels-4 0 0 +0.00%
BenchmarkHistogramWithLabelValues-4 0 0 +0.00%
BenchmarkHistogramNoLabels-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValuesBasic-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4 0 0 +0.00%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4 0 0 +0.00%
```
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This rewrite had may backs and forths. In my git repository, it
consists of 35 commits which I cannot group or merge into reasonable
review buckets. Gerrit breaks fundamental git semantics, so I have to
squash the 35 commits into one for the review.
I'll push this not with refs/for/master, but with refs/for/next so
that we can transition after submission in a controlled fashion.
For the review, I recommend to start with looking at godoc and in
particular the many examples. After that, continue with a line-by-line
detailed review. (The big picture is hopefully as expected after
wrapping up the discussion earlier.)
Change-Id: Ib38cc46493a5139ca29d84020650929d94cac850
Bulat Khasanov