Commit Graph

376 Commits

Author SHA1 Message Date
beorn7 c06fb788be Relax consistency checks during gathering
Also, clarify in the doc comment.

Previously, the assumption was that inconsistent label dimensions are
violating the exposition format spec. However, especially with the
knowledge that OpenMetrics will explicitly allow inconsistent label
dimensions in expositions, we should allow it in client_golang, too.

Note that registration with proper Descs provided will still check for
consistont label dimensions. However, you can "cheat" with custom
Collectors as you can collect metrics that don't follew the provided
Desc (this will be made more explicit and less cheaty once #47 is
fixed). You can also create expositions with inconsistent label
dimensions by merging Gatherers with the Gatherers slice type. (The
latter is used in the Pushgateway.)

Effectively, normal direct instrumentation will always have consistent
label dimensions in this way, but you can cover special use cases with
custom collectors or merging of different Gatherers.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-06-06 19:30:53 +02:00
beorn7 1b56b5c497 Be more robust about nil pointers in protobuf
While not strictly correct, it can easily happen that proto messages
are created that use nil pointers instead of pointers in empty strings
to denote an empty string.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-05-29 17:57:33 +02:00
beorn7 d66ac8f863 Document the stop-the-world implications of the Go collector
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-05-19 21:14:04 +02:00
Karsten Weiss d7590aab3c Fix three shadow variable warnings (govet -shadow)
Fixes:
http.go:118: declaration of "part" shadows declaration at http.go:117
http_test.go:50: declaration of "respBody" shadows declaration at http_test.go:25
promhttp/http.go:305: declaration of "part" shadows declaration at promhttp/http.go:304

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:25:14 +02:00
Karsten Weiss f3a13af35c http.go: Move helper function nowSeries() to test case
Fixes:
http.go:142:1⚠️ nowSeries is unused (deadcode)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:25:14 +02:00
Karsten Weiss 0a453dce84 Build process_collector_test.go only on Linux
The test case requires the /proc filesystem. The change prevents this skip
message during "go test -v" on platforms other than Linux:

=== RUN   TestProcessCollector
--- SKIP: TestProcessCollector (0.00s)
	process_collector_test.go:15: skipping TestProcessCollector, procfs not available: could not read /proc: stat /proc: no such file or directory

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:25:14 +02:00
Karsten Weiss 7a495a15b3 Type processCollector: Remove unused field 'pid'
Fixes:
process_collector.go:19:2: field pid is unused (U1000)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:25:14 +02:00
Karsten Weiss ea0d27e8cf Type responseWriterDelegator: Remove unused fields
Fixes:
http.go:355:2: field handler is unused (U1000)
http.go:355:11: field method is unused (U1000)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:25:14 +02:00
Karsten Weiss 958ea82988 Fix typos
Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:23:52 +02:00
Karsten Weiss 245fde70cb Simplify if expr in ExampleNewExpvarCollector()
Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:23:52 +02:00
Karsten Weiss 587d5265f8 metric.go: Remove unused type hashSorter
Fixes:
metric.go:130:6: type hashSorter is unused (U1000)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:23:52 +02:00
Karsten Weiss e01f7ec4fb Remove unnecessary conversion
Fixes:
go_collector.go:268:66⚠️ unnecessary conversion (unconvert)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:18:25 +02:00
Karsten Weiss 66b5a26cec Remove unnecessary for loop in makeLabelPairs()
Fixes:
value.go:155:2: should replace loop with labelPairs = append(labelPairs, desc.constLabelPairs...) (S1011)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:18:20 +02:00
Karsten Weiss daa9993ded Use direct API calls
Fixes:
prometheus/http_test.go:117:5⚠️ should use resp.Body.String() instead of string(resp.Body.Bytes()) (S1030) (megacheck)
prometheus/http_test.go:118:56⚠️ should use resp.Body.String() instead of string(resp.Body.Bytes()) (S1030) (megacheck)

Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-13 23:17:58 +02:00
Eric Boren ddd94f24c5 Fix unprotected write in metricMap 2018-04-09 10:10:15 -04:00
Luke Hanley f504d69aff promhttp: fix DNSDone should call Done, not Start (#395)
When tracing, it appears DNSDone is instead calling the supplied DNSStart function
2018-03-28 15:04:30 +02:00
beorn7 d7f8852e05 Fix TestHandler
prometheus/common has changed and now adds the charset to the
Content-Type header.
2018-03-19 14:04:12 +01:00
Adam Thomason 660e6909cc promauto: add NewCounterFunc and NewGaugeFunc 2018-03-17 14:01:36 -07:00
beorn7 b82bcfe55f Add a package promauto that provides auto-registering metrics
Finally, I found an easy solution to provide the "evil"
auto-registration without getting death threats from the wardens of Go
purity. The reasoning can be found in the package's doc comment.
2018-02-16 12:55:57 +01:00
Brian Brazil a331f1ea4a Add Godoc link at the top of the README
For those following that link, make it more obvious
which subpackage they probably want.
2018-02-15 11:45:41 +00:00
Björn Rabenstein 7e8a70d021
Merge pull request #381 from prometheus/beorn7/push
Add completely new push syntax
2018-02-14 19:20:02 +01:00
beorn7 361cb3a7d0 Delete another use of HostnameGrouping from a doc comment 2018-02-14 18:59:17 +01:00
beorn7 38de287a56 Remove obsolete copyright notices 2018-02-14 18:49:18 +01:00
beorn7 779cbe15b2 Remove HostnameGrouping method 2018-02-14 17:19:52 +01:00
beorn7 3c2baee2d1 Add completely new push syntax
This allows adding more options in elegant ways, showcased here by
HTTP basic auth and by injecting a custom http.Client.

Fixes #341 and #372.
2018-02-14 13:40:57 +01:00
Brian Brazil 02aee00fdb
Fix a typo 2018-02-14 11:10:46 +00:00
beorn7 c551c3c661 promhttp: Introduce limit for connections in flight and timeout 2018-02-09 17:05:10 +01:00
Björn Rabenstein a85074fc85
Merge pull request #377 from prometheus/beorn7/http
Provide an InstrumentedHandler
2018-02-08 17:52:34 +01:00
beorn7 8fd47d2e8f Provide an InstrumentedHandler
See https://github.com/prometheus/client_golang/issues/316 for details.
2018-02-08 16:27:51 +01:00
beorn7 154bb450e4 Document that the process collector only works on Linux 2018-02-08 14:17:54 +01:00
beorn7 e87046a87e Fix more interface upgrade bugs 2018-02-02 15:53:28 +01:00
beorn7 d892fd2b51 Add test to expose interface upgrade bug
In principle, we needed to iterate through all permutations, mirroring
the same that is happening in the code. For lack of time, I only
picked one of the cases currently buggy.

As said, this really needs code generation, should we ever find
ourselves touching this again.
2018-02-02 15:50:46 +01:00
Torin Sandall 44a8a1218c Fix delegator return value for single hijacker
Previously, the pickDelegator function was not returning a
*hijackerDelegator so the return value did not implement the Hijacker
interface. As a result, code that attempts to hijack the connection
would fail when using a type assertion.

All the other cases returned the hijackerDelegator correctly.
2018-02-01 10:57:42 -08:00
beorn7 4957f7bba4 Add a safety goroutine budget
This makes sure we don't spin up a possibly infinite number of
goroutines in `Gather`, which could theoretically happen with unlucky
scheduling.
2018-01-31 14:49:15 +01:00
beorn7 e04451f4be Create goroutines adaptively during metrics gathering 2018-01-26 19:58:07 +01:00
Stephen McQuay (smcquay) b77ed204f6
Add a benchmark for concurrent counter increments 2018-01-19 15:15:10 -08:00
beorn7 a2facc3074 Iterate on a proposed performance improvement for counters
Original discussion see
https://github.com/prometheus/client_golang/pull/362 .

Assuming that the most frequently used method of a `Gauge` is `Set`
and the most frequently used method of a `Conuter` is `Inc`, this
separates the implementation of both metric types. `Inc` and integral
`Add` of a counter is now handled in a separate `uint64`. This would
create a race in `Set`, but luckily, there is no `Set` anymore in a
counter.

All attempts to solve above race (to use the same idea for a `Gauge`)
slow down `Set`, So we just stick with the old implementation
(formerly `value`) for `Gauge`.
2018-01-19 19:06:43 +01:00
beorn7 c7029dc87d Merge branch 'counter/fixed-prec' of git://github.com/smcquay/client_golang into beorn7/counter 2018-01-19 13:12:07 +01:00
beorn7 f02bfc3484 Minor doc comment fix
This was forgotten to be included in the previous commit.
2018-01-11 16:59:30 +01:00
beorn7 9ac0bad606 Take into account curried labels in promhttp 2017-12-24 14:39:06 +01:00
beorn7 1ba60c7d58 Pull currying methods up into ObserverVec interface 2017-12-22 16:11:58 +01:00
beorn7 dd20712622 Allow currying of metric vec's
The idea behind it is described in detail in
https://github.com/prometheus/client_golang/issues/320 .

This commit also updates the example given in
promhttp/instrument_server_test.go , which nicely illustrates the
benefit of this change.

So far, currying could be emulated by creating different metric vec's
with different values in their ConstLabels. This was quite difficult
to grasp - which is essentially what was done in the example mentioned
above. Now that this use case can be solved without ConstLabels, we
can safely declare ConstLabels as rarely used. (Perhaps we can
deprecate them entirely one day, but I'll take a raincheck on that
when the changes of v0.10 have materialized.) This commit thus also
updates the ConstLabel doc comments in the various Opts. (It contained
fairly outdated stuff anyway.)
2017-12-22 15:56:11 +01:00
beorn7 10c55533cb Rename the receiver of `...Vec` methods from `m` to `v` 2017-12-21 14:06:39 +01:00
beorn7 9e1588b2a2 Pull `With` and `WithLabelValues` up into exported types
The "panic in case of error" code was so far in metricVec. This pulls
it up into the exported types like CounterVec. This is code
replication, but it avoids an explicit type conversion. Mostly,
however, this is preparation to make the wrapped metricVec an
interface (required for curried vec's).
2017-12-21 14:06:39 +01:00
Stephen McQuay (smcquay) 35559538c7
Implements review commentary
Specifically @beorn7 pointed out that the previous implementation had
some shortcomings around large numbers. I've changed the code to match
the suggestion in review, as well as added a few test cases.
2017-12-15 11:03:30 -08:00
Stephen McQuay (smcquay) ae6939214c
Adds a faster counter 2017-12-15 11:03:24 -08:00
Brian Brazil 9f5d03c01f Fix typo in comment 2017-10-04 17:44:49 +01:00
beorn7 50b3332fd6 Polishing some doc comments 2017-09-13 12:48:29 +02:00
Povilas Versockas 77a1417b06 Change summary docs 2017-09-12 20:27:18 +03:00
Matt Bostock b092a4bc11 Correct DefaultRegistry to DefaultRegisterer
`DefaultRegistry` no longer exists as of a6321dd0b.
2017-09-05 01:08:40 +01:00
beorn7 2cb8df16e3 Fix error reporting bug
That's the same bug fixed in #342 (which is the deprecated code only
replicated in the `prometheus` package until v0.10 is out).
2017-08-31 15:10:58 +02:00
Will Rouesnel a166207919 http.go: incorrect error message displayed when no metrics encoded error
The "No metrics encoded" error was erroneously displayed the value of err, not
lastErr.
2017-08-31 12:05:46 +10:00
beorn7 6164fff8ce Remove the deprecated uses of Untyped metrics
I couldn't find any use of the removed identifiers on sourcegraph.com
(outside of this repo itself).
2017-08-29 17:31:45 +02:00
Lynn Lin be904beebc fix spelling typo 2017-08-28 08:57:51 +08:00
Marco Jantke a956c5fdd6 improve validation function naming 2017-08-25 17:58:59 +02:00
Marco Jantke 0b8aef084e implement review feedback 2017-08-25 14:51:19 +02:00
Marco Jantke 6df742e132 improve formatting of invalid label value error messages 2017-08-20 00:54:11 +02:00
Marco Jantke 555018f3c9 make code compatible with go 1.6 2017-08-20 00:53:55 +02:00
Marco Jantke 685a3c90d4 fail Gather'ing when label value is not utf8 2017-08-20 00:10:32 +02:00
Marco Jantke 7ee20d77cb validate ConstLabels values in NewDesc 2017-08-20 00:09:51 +02:00
Marco Jantke 703c4a9c6f add label value validation to NewConstMetric and friends 2017-08-20 00:09:51 +02:00
Marco Jantke 459e88167e extract and refactor label validation functions
so that we can reuse them in other parts of the code, not only as part
of a metricVec.
2017-08-20 00:09:50 +02:00
Marco Jantke 957bba6f68 add label value validation to GetMetricWith and friends 2017-08-19 22:55:41 +02:00
hazey.dazey f36d4a3e73 Add new default metric go_info 2017-07-23 23:36:09 +02:00
beorn7 ed379b7d99 Remove remaining references to MetricVec from doc comments
MetricVec is un-experted by now. godoc handles that correctly by
showing the methods of the embedded un-exported metricVec with the
exported type (CounterVec, SummaryVec, ...) that embeds metricVec.
2017-07-12 15:13:09 +02:00
Maxime Song 90494ea7b1 fix typo in comments
Signed-off-by: Maxime Song <me@cppdo.com>
2017-07-07 23:28:24 +08:00
Björn Rabenstein 310ce84375 Merge pull request #319 from prometheus/beorn7/vec
Unexport MetricVec
2017-06-30 16:48:34 +02:00
beorn7 e04de4bfe3 Document the use of Go1.9+ for monotonic time where applicable 2017-06-29 16:07:12 +02:00
beorn7 f66cdf0736 Un-export MetricVec
This is in preparation for "curried" metric vecs, as discussed.

And it's a good thing anyway. The exported MetricVec was from a time
when I thought people would define own Metric types and then create
Vecs of it. That has never happened.
2017-06-28 17:55:59 +02:00
beorn7 721f93fda8 Deprecate UntypedVec 2017-06-28 17:00:33 +02:00
Alexey Palazhchenko f0c4c478a2 Minor documentation fix 2017-06-28 15:40:03 +03:00
beorn7 14730c70c0 promhttp: Bite the bullet and implement all 32 possible interface combos
As it turned out, it's not that esay to guess "common" combination of
interface upgrades. So I decided to just implement all 32 possible
combination of interface upgrades. (Only 16 with Go 1.7 and earlier.)

Clearly, this calls for code generation. But right now, we still need
to find out what's the best form of the code. For later additions,
implementing code generation might be useful.

Note that newDelegator is called for each HTTP request. Thus, this
commit aims to make the upgrade selection quick. (After the type
checks, it's just directly accessing an element in a slice.)
2017-06-02 19:03:08 +02:00
Norbert Tretkowski e7e903064f Import "log", it is required for log.Fatal(... (#305)
Import "log" in the example doc comment

Package log is required for log.Fatal.
2017-05-31 15:00:54 +02:00
stuart nelson 2b3ab50dcd Add time to write header handler middleware (#304) 2017-05-29 11:42:43 +02:00
beorn7 023c31fd59 Fix handling of ConstLabels in checkLabels 2017-05-10 20:39:36 +02:00
beorn7 753a259e20 Improve promhttp tests
- Use local registry to avoid conflicts between tests.
- Expose https://github.com/prometheus/client_golang/issues/299 by
  using ConstLabels in a test.
- Improve example: Buckets and help string must be consistent, even
  if the former is not enforced as of now, but see
  https://github.com/prometheus/client_golang/issues/222
2017-05-10 19:49:36 +02:00
beorn7 4ea620c5bf Point to promhttp in deprecation notices 2017-05-09 18:54:33 +02:00
stuart nelson d300d5cf21 Instrument RoundTripper via middleware (#295)
Instrument RoundTripper via middleware
2017-05-09 18:46:09 +02:00
stuart nelson d01fd62222 new handler instrumentation (#285)
Add new HTTP handler instrumentation
2017-04-24 15:13:19 -04:00
Björn Rabenstein 08fd2e1237 Merge pull request #281 from adjust/add_gc_cpu_fraction
Adding GCCPUFraction metric to goCollector
2017-04-01 12:34:46 +02:00
Tong 6d619ff5a8 Fix typo 2017-03-22 21:29:18 +08:00
Ivan Borshukov bea9149ebb Fix typo 2017-03-07 16:11:13 +02:00
Tobias Schmidt ae77d82d88 Fix invalid Timer (gauge) example
The example method is assumed to be used as main() function. As a main()
function doesn't have any return values, the example doesn't compile and
is invalid.
2017-02-28 11:13:17 -04:00
Julius Volz 69bb387064 Fix typo 2017-02-23 10:28:41 +01:00
Francis Stephens 56cf0be13f Adding GCCPUFraction metric to goCollector 2017-02-22 17:26:08 +01:00
Tobias Schmidt 70693bc297 Document graphite bridge 2017-02-15 12:19:46 -04:00
Peng Gao efb2f142f3 goCollector: change goroutines and treads type
Change gorountines and threads created Gauge to NewConstMetric.

Signed-off-by: Peng Gao <peng.gao.dut@gmail.com>
2017-02-15 13:06:22 +08:00
Peng Gao fa1cd67d1e goCollector: add thread count gauge in goCollector
Signed-off-by: Peng Gao <peng.gao.dut@gmail.com>
2017-02-13 19:39:00 +08:00
beorn7 9fdd167dba InstrumentHandler: Document issues with HTTP/2 2017-01-25 12:49:31 +01:00
beorn7 60e61927fc graphite: Adjust ExtractSamples call to new interface 2017-01-08 20:55:34 +01:00
Björn Rabenstein 575f371f78 Merge pull request #262 from prometheus/beorn7/summary
Allow Summaries with empty objectives and deprecate DefObjectives
2016-11-24 16:57:32 +01:00
Björn Rabenstein e83345f73f Merge pull request #250 from prometheus/timer-helper
Add timer helper function
2016-11-23 18:45:33 +01:00
beorn7 9c4b7780d7 Allow Summaries with empty objectives and deprecate DefObjectives
This also updates all tests and examples to use explicitly set
objectives.

In v0.10, DefObjectives will be completely removed, and the default
Summary will have no objectives then.

Fixes #118
2016-11-23 18:35:02 +01:00
beorn7 6bff9dc5a4 Make heap_released_bytes a Gauge.
It's not a counter after all.

Also, remove a misleading part of the sys_bytes help string.
2016-11-22 18:11:47 +01:00
beorn7 bc365741ee Improve timer examples 2016-11-21 19:16:56 +01:00
beorn7 89ca0458cb Change the Timer API
This finally makes the (presumably) common simple case as simple as it
gets.

The still quite common (but less common) case of using a Gauge is
slightly more verbose now, but not needing to provide a separate
constructor is totally worth it.

Finally, the advanced use case is not really more verbose as in my
original suggestion. However, the logic to decide which Observer to
use is now all in the ObserverFunc handed in at construction
time. This is deliberate and desired. It makes sure the selection
mechanism is all spelled out there. No surprises buried deep in the
function code somewhere.
2016-11-18 20:32:18 +01:00
beorn7 18c13ef63d Added doc comments 2016-11-18 20:32:18 +01:00
beorn7 8aba21aba3 Completed tests 2016-11-18 20:32:18 +01:00
beorn7 120be69578 Change timer naming from Start/Stop to New/ObserveDuration 2016-11-18 20:32:18 +01:00
beorn7 e63845e3ce Add observerFunc to observe with a Gauge 2016-11-18 20:32:18 +01:00
beorn7 8496756f6d Turn timer helper into a struct with methods 2016-11-18 20:32:18 +01:00
stuart nelson d845abb9f9 Add timer helper function 2016-11-18 20:32:18 +01:00
beorn7 e72c7946c6 Extend the example for pushing to pushgateway 2016-11-18 20:22:35 +01:00
beorn7 606b8f85e5 - Point from Inc and Dec to Add and Sub in doc comments.
- Deprecate Untyped for direct instrumentation.

- Add a SetToCurrentTime method to Gauge

Note that adding the SetToCurrentTime method is not really following
Go's principle of lean interfaces. However, the Gauge interface is
already quite fat. (The only methods really required are Set and
Add. Everything else could be expressed in terms of those two.) So we
have already quite a few "convenience" methods traditionally, so I
think we should stay consistent here.

The alternatives would be:

- Not support SetToCurrentTime at all (it's only a SHOULD in the
  guidelines).

- A top level function `SetToCurrentTime(Gauge)`.

- Just a helper `CurrentTime()` that returns the curent unix time in
  seconds as a float (which is pretty verbose using the standard
  library, see code in this commit). This would allow
  `myGauge.Set(CurrentTime)`.

Weighing all circumstances, I believe the way in this commit is the
least evil. Issue #223 could be used to rework interfaces more
fundamentally in a breaking change if feasible.
2016-11-18 19:32:10 +01:00
stuart nelson 7664178cb2 Graphite bridge (#244)
Add Graphite bridge
2016-11-14 14:43:07 +01:00
beorn7 dcbc5caa16 Remove local REs for label and metric names and use fast checks
All was a mess, we had duplicates of the REs for label name and metric
names here, and we sometimes used them, sometimes we used those from
common/model.

Now we are consistently using the fast checking functions from common/model.

(Tests for leading colons are included there, see
https://github.com/prometheus/common/pull/66 .)
2016-11-11 16:59:23 +01:00
Tobias Schmidt 99d64f8879 Fix namespace of process collector metrics 2016-11-02 14:09:24 -04:00
Marcus Franke 473bcd5aa1 [docs] enhanced the basic example
To protect people from starting their exporter multiple times, one has to
evaluate the error returned by http.ListenAndServe().
2016-11-02 10:48:36 +01:00
beorn7 2fee50beaa Remove deprecated features that are esay to replace
That's the "soft" part of the deprecation: Everything that has been
marked deprecated in v0.8 or earlier and is straight-forward to
replace by a non-deprecated way, is removed here.

Sadly, this does not include the HTTP part. We first need to provide a
replacement for HTTP instrumentation (as planned for v0.8) to then
remove the deprecated parts in v0.9.
2016-10-25 18:28:15 +02:00
beorn7 6450fc55b1 Convert process collector to const metrics
This is needed to remove the deprecated Set method from the Counter.
2016-10-25 18:19:24 +02:00
beorn7 6373fd9334 Adding a test for non-monotonic buckets 2016-10-17 13:57:02 +02:00
beorn7 c9325a4a67 Add goreport card and remove warnings where feasible 2016-09-16 19:59:04 +02:00
nghialv bc86b956d7 fix goroutine leak 2016-09-11 00:24:13 +09:00
beorn7 434a8ed85d Bring back zero-alloc label-value access for metric vecs
Also, fix mutex copy-by-value bug.
2016-08-17 14:01:11 +02:00
beorn7 1f823ab271 Refactor tests
Also, add a "real" collision to the tests.
2016-08-17 12:19:50 +02:00
beorn7 00d39222c2 Naming nit 2016-08-17 10:36:20 +02:00
Stephen J Day 4db77b04a8
metricvec: refactor collision handling to handle equality
After increasing unit test coverage, it was found that the split
function call nature of metric matching wasn't working well in many
cases. By increasing test coverage, we've ensured that both the fast
path and fallback collision path are working appropriately.

With these changes, there is a further performance hit, but now the
results are ensured to be correct.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-16 18:34:16 -07:00
Stephen J Day 3cf50db5fd
metricvec: handle hash collision for labeled metrics
While hash collisions are quite rare, the current state of the client
library carries a risk of merging two separate label values into a
single metric bucket. The effects are near impossible to detect and the
result will be missing or incorrect counters.

This changeset handles hash collisions by falling back to collision
resolution if multiple label values hash to the same value. This works
similar to separate chaining using a slice. Extra storage is minimized
to only the value key slice to that metrics can be differentiated
within a bucket.

In general, the cost of handling collisions is completely minimized
under normal operation. Performance does show slight increases in
certain areas, but these are more likely statistically anomalies. More
importantly, zero allocation behavior for metrics is preserved on the
fast path. Minimal allocations may be made during collision handling but
this has minimal effect.

Benchmark comparisons with and without collision resolution follow.

```
benchmark                                                         old ns/op     new ns/op     delta
BenchmarkCounterWithLabelValues-4                                 99.0          107           +8.08%
BenchmarkCounterWithLabelValuesConcurrent-4                       79.6          91.0          +14.32%
BenchmarkCounterWithMappedLabels-4                                518           542           +4.63%
BenchmarkCounterWithPreparedMappedLabels-4                        127           137           +7.87%
BenchmarkCounterNoLabels-4                                        19.5          19.1          -2.05%
BenchmarkGaugeWithLabelValues-4                                   97.4          110           +12.94%
BenchmarkGaugeNoLabels-4                                          12.4          10.3          -16.94%
BenchmarkSummaryWithLabelValues-4                                 1204          915           -24.00%
BenchmarkSummaryNoLabels-4                                        936           847           -9.51%
BenchmarkHistogramWithLabelValues-4                               147           147           +0.00%
BenchmarkHistogramNoLabels-4                                      50.6          49.3          -2.57%
BenchmarkHistogramObserve1-4                                      37.9          37.5          -1.06%
BenchmarkHistogramObserve2-4                                      122           137           +12.30%
BenchmarkHistogramObserve4-4                                      310           352           +13.55%
BenchmarkHistogramObserve8-4                                      691           729           +5.50%
BenchmarkHistogramWrite1-4                                        3374          3097          -8.21%
BenchmarkHistogramWrite2-4                                        5310          5051          -4.88%
BenchmarkHistogramWrite4-4                                        12094         10690         -11.61%
BenchmarkHistogramWrite8-4                                        19416         17755         -8.55%
BenchmarkHandler-4                                                11934304      13765894      +15.35%
BenchmarkSummaryObserve1-4                                        1119          1105          -1.25%
BenchmarkSummaryObserve2-4                                        3679          3430          -6.77%
BenchmarkSummaryObserve4-4                                        10678         7982          -25.25%
BenchmarkSummaryObserve8-4                                        22974         16689         -27.36%
BenchmarkSummaryWrite1-4                                          25962         14680         -43.46%
BenchmarkSummaryWrite2-4                                          38019         35073         -7.75%
BenchmarkSummaryWrite4-4                                          78027         56816         -27.18%
BenchmarkSummaryWrite8-4                                          117220        132248        +12.82%
BenchmarkMetricVecWithLabelValuesBasic-4                          138           133           -3.62%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4        150           144           -4.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4        263           256           -2.66%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4       145           155           +6.90%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4      606           634           +4.62%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4     746           641           -14.08%

benchmark                                                         old allocs     new allocs     delta
BenchmarkCounterWithLabelValues-4                                 0              0              +0.00%
BenchmarkCounterWithLabelValuesConcurrent-4                       0              0              +0.00%
BenchmarkCounterWithMappedLabels-4                                2              2              +0.00%
BenchmarkCounterWithPreparedMappedLabels-4                        0              0              +0.00%
BenchmarkCounterNoLabels-4                                        0              0              +0.00%
BenchmarkGaugeWithLabelValues-4                                   0              0              +0.00%
BenchmarkGaugeNoLabels-4                                          0              0              +0.00%
BenchmarkSummaryWithLabelValues-4                                 0              0              +0.00%
BenchmarkSummaryNoLabels-4                                        0              0              +0.00%
BenchmarkHistogramWithLabelValues-4                               0              0              +0.00%
BenchmarkHistogramNoLabels-4                                      0              0              +0.00%
BenchmarkMetricVecWithLabelValuesBasic-4                          0              0              +0.00%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4        0              0              +0.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4        0              0              +0.00%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4       0              0              +0.00%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4      0              0              +0.00%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4     0              0              +0.00%

benchmark                                                         old bytes     new bytes     delta
BenchmarkCounterWithLabelValues-4                                 0             0             +0.00%
BenchmarkCounterWithLabelValuesConcurrent-4                       0             0             +0.00%
BenchmarkCounterWithMappedLabels-4                                336           336           +0.00%
BenchmarkCounterWithPreparedMappedLabels-4                        0             0             +0.00%
BenchmarkCounterNoLabels-4                                        0             0             +0.00%
BenchmarkGaugeWithLabelValues-4                                   0             0             +0.00%
BenchmarkGaugeNoLabels-4                                          0             0             +0.00%
BenchmarkSummaryWithLabelValues-4                                 0             0             +0.00%
BenchmarkSummaryNoLabels-4                                        0             0             +0.00%
BenchmarkHistogramWithLabelValues-4                               0             0             +0.00%
BenchmarkHistogramNoLabels-4                                      0             0             +0.00%
BenchmarkMetricVecWithLabelValuesBasic-4                          0             0             +0.00%
BenchmarkMetricVecWithLabelValues2Keys10ValueCardinality-4        0             0             +0.00%
BenchmarkMetricVecWithLabelValues4Keys10ValueCardinality-4        0             0             +0.00%
BenchmarkMetricVecWithLabelValues2Keys100ValueCardinality-4       0             0             +0.00%
BenchmarkMetricVecWithLabelValues10Keys100ValueCardinality-4      0             0             +0.00%
BenchmarkMetricVecWithLabelValues10Keys1000ValueCardinality-4     0             0             +0.00%
```

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-15 16:47:53 -07:00
Stephen J Day c4004ef5f6
benchmark: measure label resolution in MetricVec
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-15 16:47:42 -07:00
Bjoern Rabenstein 25db044749 Fix doc comment typo 2016-08-15 23:22:46 +02:00
beorn7 390e874a3f Fix doc comment typo 2016-08-15 15:54:18 +02:00
beorn7 a6321dd0b1 Create a "merge gatherer"
This allows to finally get rid of the infamous injection hook in the
interface. The old SetMetricFamilyInjectionHook still exist as a
deprecated function but is now implemented with the new plumbing under
the hood.

Now that we have multiple Gatherer implementation, I renamed
push.Registry to push.FromGatherer.

This commit also improves the consistency checks, which happened as a
byproduct of the refactoring to allow checking in both the "merge
gatherer" Gatherers as well as in the normal Registry.
2016-08-12 21:34:17 +02:00
beorn7 1dc03a72f6 Replace hashicorp/go-multierror by own implementation
The own implementation is much easier as it only has to serve our one
use case.
2016-08-04 16:03:06 +02:00
beorn7 f0c45acc50 Rename Deliver into Gather 2016-08-04 15:26:27 +02:00
beorn7 f9c977b1d6 Doc: Make the AlreadyRegisteredError example a real example 2016-08-04 11:41:02 +02:00
beorn7 bc0b2a3b17 Move http stuff in its own package promhttp
To keep backwards compatibility while not creating circular import
chains, some code had to be duplicated. But all functions using it
have been declared deprecated hereby.

The new ways of instrumenting handlers will all go into the new
package, and ultimately, the prometheus package itself will be
completely igorant of HTTP.
2016-08-03 18:06:48 +02:00
beorn7 2e66d12fe5 Act on DefaultRegisterer and DefaultDeliverer where possible
Both point to the DefaultRegistry anyway, but users can now replace it
with interfaces rather than only other Registry instances.
2016-08-03 12:50:39 +02:00
beorn7 e5c6302150 Fix a number of doc comments and similar nits 2016-08-03 12:23:37 +02:00
beorn7 5a918da56d Tweaked push package
- Moved the Deliverer parameter to the end of the list to mirror
  Collectors in push.Collectors.
- Improved doc comment and added an example for push.Registry.
2016-08-03 12:00:19 +02:00
beorn7 249069ec01 Unexport SelfCollector.
This is most likely used nowhere and can be unexported to clean up the
namespace.
2016-08-03 01:09:27 +02:00
beorn7 9c3fe750dd Expose the registry implementation and add two interfaces.
Registry is now a struct, which implements two interfaces, Registrerer
and Deliverer. The latter is particularly important as it is now the
argument type for pushes and HTTP handler construction (i.e. it is
easy to implement a custom Deliverer for testing or other
purposes). The Registerer interface is not used as a parameter type
but can (and should) be used by users of custom registries so that
they can easily do things like mocking it out for testing purposes.

With the broken up interfaces, adding MustRegister to the interface is
not such a big deal anymore (interface is still small). And since
setting the injection hook is such a rare thing to happen, it is
acceptable to not have it in any of the interfaces.

The renaming from `Collect` to `Deliver` was done to avoid confusion
with Collectors. (The registry _collects_ from the Collectors, and
then _delivers_ to the exposition mechanism.)
2016-08-03 01:02:34 +02:00
beorn7 cf7e1caf17 Create a public registry interface and separate out HTTP exposition
General context and approch
===========================

This is the first part of the long awaited wider refurbishment of
`client_golang/prometheus/...`. After a lot of struggling, I decided
to not go for one breaking big-bang, but cut things into smaller steps
after all, mostly to keep the changes manageable and easy to
review. I'm aiming for having the invasive breaking changes
concentrated in as few steps as possible (ideally one). Some steps
will not be breaking at all, but typically there will be breaking
changes that only affect quite special cases so that 95+% of users
will not be affected. This first step is an example for that, see
details below.

What's happening in this commit?
================================

This step is about finally creating an exported registry
interface. This could not be done by simply export the existing
internal implementation because the interface would be _way_ too
fat. This commit introduces a qutie lean `Registry` interface
(compared to the previous interval implementation). The functions that
act on the default registry are retained (with very few exceptions) so
that most use cases won't see a change. However, several of those are
deprecated now to clean up the namespace in the future.

The default registry is kept in the public variable
`DefaultRegistry`. This follows the example of the http package in the
standard library (cf. `http.DefaultServeMux`, `http.DefaultClient`)
with the same implications. (This pattern is somewhat disputed within
the Go community but I chose to go with the devil you know instead of
creating something more complex or even disallowing any changes to the
default registry. The current approach gives everybody the freedom to
not touch DefaultRegistry or to do everything with a custom registry
to play save.)

Another important part in making the registry lean is the extraction
of the HTTP exposition, which also allows for customization of the
HTTP exposition. Note that the separation of metric collection and
exposition has the side effect that managing the MetricFamily and
Metric protobuf objects in a free-list or pool isn't really feasible
anymore. By now (with better GC in more recent Go versions), the
returns were anyway dimisishing. To be effective at all, scrapes had
to happen more often than GC cycles, and even then most elements of
the protobufs (everything excetp the MetricFamily and Metric structs
themselves) would still cause allocation churn. In a future breaking
change, the signature of the Write method in the Metric interface will
be adjusted accordingly. In this commit, avoiding breakage is more
important.

The following issues are fixed by this commit (some solved "on the
fly" now that I was touching the code anyway and it would have been
stupid to port the bugs):

https://github.com/prometheus/client_golang/issues/46
https://github.com/prometheus/client_golang/issues/100
https://github.com/prometheus/client_golang/issues/170
https://github.com/prometheus/client_golang/issues/205

Documentation including examples have been amended as required.

What future changes does this commit enable?
============================================

The following items are not yet implemented, but this commit opens the
possibility of implementing these independently.

- The separation of the HTTP exposition allows the implementation of
  other exposition methods based on the Registry interface, as known
  from other Prometheus client libraries, e.g. sending the metrics to
  Graphite.
  Cf. https://github.com/prometheus/client_golang/issues/197

- The public `Registry` interface allows the implementation of
  convenience tools for testing metrics collection. Those tools can
  inspect the collected MetricFamily protobufs and compare them to
  expectation. Also, tests can use their own testing instance of a
  registry.
  Cf. https://github.com/prometheus/client_golang/issues/58

Notable non-goals of this commit
================================

Non-goals that will be tackled later
------------------------------------

The following two issues are quite closely connected to the changes in
this commit but the line has been drawn deliberately to address them
in later steps of the refurbishment:

- `InstrumentHandler` has many known problems. The plan is to create a
  saner way to conveniently intrument HTTP handlers and remove the old
  `InstrumentHandler` altogether. To keep breakage low for now, even
  the default handler to expose metrics is still using the old
  `InstrumentHandler`. This leads to weird naming inconsistencies but
  I have deemed it better to not break the world right now but do it
  in the change that provides better ways of instrumenting HTTP
  handlers.
  Cf. https://github.com/prometheus/client_golang/issues/200

- There is work underway to make the whole handling of metric
  descriptors (`Desc`) more intuitive and transparent for the user
  (including an ability for less strict checking,
  cf. https://github.com/prometheus/client_golang/issues/47). That's
  quite invasive from the perspective of the internal code, namely the
  registry. I deliberately kept those changes out of this commit.

- While this commit adds new external dependency, the effort to vendor
  anything within the library that is not visible in any exported
  types will have to be done later.

Non-goals that _might_ be tackled later
---------------------------------------

There is a strong and understandable urge to divide the `prometheus`
package into a number of sub-packages (like `registry`, `collectors`,
`http`, `metrics`, …). However, to not run into a multitude of
circular import chains, this would need to break every single existing
usage of the library. (As just one example, if the ubiquitious
`prometheus.MustRegister` (with more than 2,000 uses on GitHub alone)
is kept in the `prometheus` package, but the other registry concerns
go into a new `registry` package, then the `prometheus` package would
import the `registry` package (to call the actual register method),
while at the same time the `registry` package needs to import the
`prometheus` package to access `Collector`, `Metric`, `Desc` and
more. If we moved `MustRegister` into the `registry` package,
thousands of code lines would have to be fixed (which would be easy if
the world was a mono repo, but it is not). If we moved everything else
the proposed registry package needs into packages of their own, we
would break thousands of other code lines.)

The main problem is really the top-level functions like
`MustRegister`, `Handler`, …, which effectively pull everything into
one package. Those functions are however very convenient for the easy
and very frequent use-cases.

This problem has to be revisited later.

For now, I'm trying to keep the amount of exported names in the
package as low as possible (e.g. I unexported expvarCollector in this
commit because the NewExpvarCollector constructor is enough to export,
and it is now consistent with other collectors, like the goCollector).

Non-goals that won't be tackled anytime soon
--------------------------------------------

Something that I have played with a lot is "streaming collection",
i.e. allow an implementation of the `Registry` interface that collects
metrics incrementally and serves them while doing so. As it has turned
out, this has many many issues and makes the `Registry` interface very
clunky. Eventually, I made the call that it is unlikely we will really
implement streaming collection; and making the interface more clunky
for something that might not even happen is really a big no-no. Note
that the `Registry` interface only creates the in-memory
representation of the metric family protobufs in one go. The
serializaton onto the wire can still be handled in a streaming fashion
(which hasn't been done so far, without causing any trouble, but might
be done in the future without breaking any interfaces).

What are the breaking changes?
==============================

- Signatures of functions pushing to Pushgateway have changed to allow
  arbitrary grouping (which was planned for a long time anyway, and
  now that I had to work on the Push code anyway for the registry
  refurbishment, I finally did it,
  cf. https://github.com/prometheus/client_golang/issues/100).
  With the gained insight that pushing to the default registry is almost
  never the right thing, and now that we are breaking the Push call
  anyway, all the Push functions were moved to their own package,
  which cleans up the namespace and is more idiomatic (pushing
  Collectors is now literally done by `push.Collectors(...)`).

- The registry is doing more consistency checks by default now. Past
  creators of inconsistent metrics could have masked the problem by
  not setting `EnableCollectChecks`. Those inconsistencies will now be
  detected. (But note that a "best effort" metrics collection is now
  possible with `HandlerOpts.ErrorHandling = ContinueOnError`.)

- `EnableCollectChecks` is gone. The registry is now performing some
  of those checks anyway (see previous item), and a registry with all
  of those checks can now be created with `NewPedanticRegistry` (only
  used for testing).

- `PanicOnCollectError` is gone. This behavior can now be configured
  when creating a custom HTTP handler.
2016-08-02 18:46:22 +02:00
Uwe Dauernheim c1e8a41238 Fix typo in doc string 2016-08-01 23:14:45 +02:00
Andrew Stuart e63e6e3db7
Extend MustRegister function to accept variadic args 2016-07-11 15:04:23 -07:00
Matt Bostock c7f1ecb643 Fix GoDoc indentation
The line after the first in each bullet point was appearing as
preformatted blocks when viewed as HTML in GoDoc.
2016-06-27 15:26:48 +01:00
Chris Lonng 7752efd9d6 comment contains some misspell 2016-06-27 17:49:34 +08:00
Björn Rabenstein 488edd04dc Merge pull request #202 from mattyw/01-docs
prometheus/docs: Added a link to the metric types page
2016-05-31 11:15:28 +02:00
mattyw 3c063fb533 prometheus/docs: Added a link to the metric types page 2016-05-31 10:07:50 +01:00
beorn7 f6b16ed256 Document the issues of InstrumentHandler
Obvious next step: Fix those issues.
2016-05-15 17:59:51 +02:00
Matt Harden e3340f3371 Fix max_fds process collector test
This test fails when max_fds is a large value; say 4.5e+06, for example. Change it to match virtual_memory_bytes, which also has to handle large values.
2016-05-12 16:55:21 -07:00
Brian Brazil 09374d92aa Remove anti-pattern of having target labels in instrumentation example. 2016-03-17 13:22:20 +00:00
Harmen e48dd8ee97 fix go_memstats_last_gc_time_seconds division 2016-02-08 01:28:30 +01:00
Michael Stapelberg c1370d07ca Strip trailing / from push URL.
This circumvents the following problem:
• The prometheus client library appends “/metrics/…” to the pushURL.
• When pushURL ends in a trailing slash, the URL becomes e.g.
  http://pushgateway.example.com:9091//metrics/…
• The pushgateway will reply with an HTTP 307 status code
  (temporary redirect).
• While Go’s net/http client follows redirects by default, it will only
  follow HTTP 302 (Found) and HTTP 303 (See Other) redirects for PUT and
  POST requests, which the prometheus client library uses.

Hence, when calling e.g.:

    prometheus.Push("foo", "bar", "http://pushgateway.example.com:9091/")

…your metrics would not actually get pushed successfully, but rather
you’d see the error message:

    2015/11/26 10:59:49 main.go:209: unexpected status code 307 while pushing to http://push...
2015-11-26 19:16:53 +01:00
Michael Stapelberg 1a09b46c2e fix typo: s/addr/url/ 2015-11-25 08:46:01 +01:00
Harmen 5d4fdca1a5 use local fnv hash everywhere 2015-11-12 14:07:23 +01:00
Björn Rabenstein cfd904193d Merge pull request #180 from dnesting/dnesting-sort-metrics
Make metric sort stable and sort by timestamp
2015-11-10 22:31:22 +01:00
David Nesting b0bd184e74 Simplify metricSorter timestamp comparison and improve comment 2015-11-10 09:37:58 -05:00
David Nesting 544f65f7fc metricSorter stops relying on time.Now and respects nil timestamps 2015-11-09 17:14:07 -05:00
David Nesting 5fb1b89678 Make metric sort stable and sort by timestamp 2015-11-09 16:55:26 -05:00
Robert Vollmert 8c96b12588 try getting metrics with a read lock, first 2015-11-09 15:16:26 +01:00