client_golang/prometheus
Stephen J Day 3cf50db5fd
metricvec: handle hash collision for labeled metrics
While hash collisions are quite rare, the current state of the client
library carries a risk of merging two separate label values into a
single metric bucket. The effects are near impossible to detect and the
result will be missing or incorrect counters.

This changeset handles hash collisions by falling back to collision
resolution if multiple label values hash to the same value. This works
similar to separate chaining using a slice. Extra storage is minimized
to only the value key slice to that metrics can be differentiated
within a bucket.

In general, the cost of handling collisions is completely minimized
under normal operation. Performance does show slight increases in
certain areas, but these are more likely statistically anomalies. More
importantly, zero allocation behavior for metrics is preserved on the
fast path. Minimal allocations may be made during collision handling but
this has minimal effect.

Benchmark comparisons with and without collision resolution follow.

```
benchmark                                                         old ns/op     new ns/op     delta
BenchmarkCounterWithLabelValues-4                                 99.0          107           +8.08%
BenchmarkCounterWithLabelValuesConcurrent-4                       79.6          91.0          +14.32%
BenchmarkCounterWithMappedLabels-4                                518           542           +4.63%
BenchmarkCounterWithPreparedMappedLabels-4                        127           137           +7.87%
BenchmarkCounterNoLabels-4                                        19.5          19.1          -2.05%
BenchmarkGaugeWithLabelValues-4                                   97.4          110           +12.94%
BenchmarkGaugeNoLabels-4                                          12.4          10.3          -16.94%
BenchmarkSummaryWithLabelValues-4                                 1204          915           -24.00%
BenchmarkSummaryNoLabels-4                                        936           847           -9.51%
BenchmarkHistogramWithLabelValues-4                               147           147           +0.00%
BenchmarkHistogramNoLabels-4                                      50.6          49.3          -2.57%
BenchmarkHistogramObserve1-4                                      37.9          37.5          -1.06%
BenchmarkHistogramObserve2-4                                      122           137           +12.30%
BenchmarkHistogramObserve4-4                                      310           352           +13.55%
BenchmarkHistogramObserve8-4                                      691           729           +5.50%
BenchmarkHistogramWrite1-4                                        3374          3097          -8.21%
BenchmarkHistogramWrite2-4                                        5310          5051          -4.88%
BenchmarkHistogramWrite4-4                                        12094         10690         -11.61%
BenchmarkHistogramWrite8-4                                        19416         17755         -8.55%
BenchmarkHandler-4                                                11934304      13765894      +15.35%
BenchmarkSummaryObserve1-4                                        1119          1105          -1.25%
BenchmarkSummaryObserve2-4                                        3679          3430          -6.77%
BenchmarkSummaryObserve4-4                                        10678         7982          -25.25%
BenchmarkSummaryObserve8-4                                        22974         16689         -27.36%
BenchmarkSummaryWrite1-4                                          25962         14680         -43.46%
BenchmarkSummaryWrite2-4                                          38019         35073         -7.75%
BenchmarkSummaryWrite4-4                                          78027         56816         -27.18%
BenchmarkSummaryWrite8-4                                          117220        132248        +12.82%
BenchmarkMetricVecWithLabelValuesBasic-4                          138           133           -3.62%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4        150           144           -4.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4        263           256           -2.66%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4       145           155           +6.90%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4      606           634           +4.62%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4     746           641           -14.08%

benchmark                                                         old allocs     new allocs     delta
BenchmarkCounterWithLabelValues-4                                 0              0              +0.00%
BenchmarkCounterWithLabelValuesConcurrent-4                       0              0              +0.00%
BenchmarkCounterWithMappedLabels-4                                2              2              +0.00%
BenchmarkCounterWithPreparedMappedLabels-4                        0              0              +0.00%
BenchmarkCounterNoLabels-4                                        0              0              +0.00%
BenchmarkGaugeWithLabelValues-4                                   0              0              +0.00%
BenchmarkGaugeNoLabels-4                                          0              0              +0.00%
BenchmarkSummaryWithLabelValues-4                                 0              0              +0.00%
BenchmarkSummaryNoLabels-4                                        0              0              +0.00%
BenchmarkHistogramWithLabelValues-4                               0              0              +0.00%
BenchmarkHistogramNoLabels-4                                      0              0              +0.00%
BenchmarkMetricVecWithLabelValuesBasic-4                          0              0              +0.00%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4        0              0              +0.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4        0              0              +0.00%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4       0              0              +0.00%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4      0              0              +0.00%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4     0              0              +0.00%

benchmark                                                         old bytes     new bytes     delta
BenchmarkCounterWithLabelValues-4                                 0             0             +0.00%
BenchmarkCounterWithLabelValuesConcurrent-4                       0             0             +0.00%
BenchmarkCounterWithMappedLabels-4                                336           336           +0.00%
BenchmarkCounterWithPreparedMappedLabels-4                        0             0             +0.00%
BenchmarkCounterNoLabels-4                                        0             0             +0.00%
BenchmarkGaugeWithLabelValues-4                                   0             0             +0.00%
BenchmarkGaugeNoLabels-4                                          0             0             +0.00%
BenchmarkSummaryWithLabelValues-4                                 0             0             +0.00%
BenchmarkSummaryNoLabels-4                                        0             0             +0.00%
BenchmarkHistogramWithLabelValues-4                               0             0             +0.00%
BenchmarkHistogramNoLabels-4                                      0             0             +0.00%
BenchmarkMetricVecWithLabelValuesBasic-4                          0             0             +0.00%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4        0             0             +0.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4        0             0             +0.00%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4       0             0             +0.00%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4      0             0             +0.00%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4     0             0             +0.00%
```

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-15 16:47:53 -07:00
..
promhttp Fix doc comment typo 2016-08-15 15:54:18 +02:00
push Create a "merge gatherer" 2016-08-12 21:34:17 +02:00
.gitignore Rearrange file and package per convention. 2013-04-04 15:27:09 +02:00
README.md Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
benchmark_test.go concurrency benchmark 2015-11-09 14:34:37 +01:00
collector.go Fix a number of doc comments and similar nits 2016-08-03 12:23:37 +02:00
counter.go benchmark: measure label resolution in MetricVec 2016-08-15 16:47:42 -07:00
counter_test.go Use non-rewritten Godep imports. 2015-02-27 16:49:40 +01:00
desc.go Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
doc.go Fix doc comment typo 2016-08-15 23:22:46 +02:00
example_clustermanager_test.go Fix a number of doc comments and similar nits 2016-08-03 12:23:37 +02:00
examples_test.go Create a "merge gatherer" 2016-08-12 21:34:17 +02:00
expvar_collector.go Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
expvar_collector_test.go Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
fnv.go use local fnv hash everywhere 2015-11-12 14:07:23 +01:00
gauge.go benchmark: measure label resolution in MetricVec 2016-08-15 16:47:42 -07:00
gauge_test.go Use non-rewritten Godep imports. 2015-02-27 16:49:40 +01:00
go_collector.go Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
go_collector_test.go Fix minor typos in comment. 2015-08-17 12:58:16 +02:00
histogram.go benchmark: measure label resolution in MetricVec 2016-08-15 16:47:42 -07:00
histogram_test.go Inline hash/fnv. 2015-11-09 15:16:26 +01:00
http.go Rename Deliver into Gather 2016-08-04 15:26:27 +02:00
http_test.go Move http stuff in its own package promhttp 2016-08-03 18:06:48 +02:00
metric.go Create a "merge gatherer" 2016-08-12 21:34:17 +02:00
metric_test.go Change "Prometheus Team" to "The Prometheus Authors". 2015-02-02 15:14:36 +01:00
process_collector.go Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
process_collector_test.go Rename Deliver into Gather 2016-08-04 15:26:27 +02:00
registry.go Create a "merge gatherer" 2016-08-12 21:34:17 +02:00
registry_test.go Create a "merge gatherer" 2016-08-12 21:34:17 +02:00
summary.go benchmark: measure label resolution in MetricVec 2016-08-15 16:47:42 -07:00
summary_test.go Create a public registry interface and separate out HTTP exposition 2016-08-02 18:46:22 +02:00
untyped.go benchmark: measure label resolution in MetricVec 2016-08-15 16:47:42 -07:00
value.go Unexport SelfCollector. 2016-08-03 01:09:27 +02:00
vec.go metricvec: handle hash collision for labeled metrics 2016-08-15 16:47:53 -07:00
vec_test.go metricvec: handle hash collision for labeled metrics 2016-08-15 16:47:53 -07:00

README.md

See go-doc.