3ffd7c4a6c
So far we've been using Go's native time.Time for anything related to sample timestamps. Since the range of time.Time is much bigger than what we need, this has created two problems: - there could be time.Time values which were out of the range/precision of the time type that we persist to disk, therefore causing incorrectly ordered keys. One bug caused by this was: https://github.com/prometheus/prometheus/issues/367 It would be good to use a timestamp type that's more closely aligned with what the underlying storage supports. - sizeof(time.Time) is 192, while Prometheus should be ok with a single 64-bit Unix timestamp (possibly even a 32-bit one). Since we store samples in large numbers, this seriously affects memory usage. Furthermore, copying/working with the data will be faster if it's smaller. *MEMORY USAGE RESULTS* Initial memory usage comparisons for a running Prometheus with 1 timeseries and 100,000 samples show roughly a 13% decrease in total (VIRT) memory usage. In my tests, this advantage for some reason decreased a bit the more samples the timeseries had (to 5-7% for millions of samples). This I can't fully explain, but perhaps garbage collection issues were involved. *WHEN TO USE THE NEW TIMESTAMP TYPE* The new clientmodel.Timestamp type should be used whenever time calculations are either directly or indirectly related to sample timestamps. For example: - the timestamp of a sample itself - all kinds of watermarks - anything that may become or is compared to a sample timestamp (like the timestamp passed into Target.Scrape()). When to still use time.Time: - for measuring durations/times not related to sample timestamps, like duration telemetry exporting, timers that indicate how frequently to execute some action, etc. Change-Id: I253a467388774280c10400fda122369ff77c1730 |
||
|---|---|---|
| documentation | ||
| examples | ||
| extraction | ||
| model | ||
| prometheus | ||
| test | ||
| vendor/goautoneg | ||
| .gitignore | ||
| .travis.yml | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| TODO | ||
README.md
Overview
These Go packages are an extraction of pieces of instrumentation code I whipped-up for a personal project that a friend of mine and I are working on. We were in need for some rudimentary statistics to observe behaviors of the server's various components, so this was written.
The code here is not a verbatim copy thereof but rather a thoughtful re-implementation should other folks need to consume and analyze such telemetry.
N.B. --- I have spent a bit of time working through the model in my head and probably haven't elucidated my ideas as clearly as I need to. If you examine examples/{simple,uniform_random}/main.go and registry.go, you'll find several examples of what types of potential instrumentation use cases this package addresses. There are probably numerous Go language idiomatic changes that need to be made, but this task has been deferred for now.
Continuous Integration
Documentation
Please read the generated documentation for the project's documentation from source code.
Basic Overview
Metrics
A metric is a measurement mechanism.
Gauge
A Gauge is a metric that exposes merely an instantaneous value or some snapshot thereof.
Counter
A Counter is a metric that exposes merely a sum or tally of things.
Histogram
A Histogram is a metric that captures events or samples into Buckets. It exposes its values via percentile estimations.
Buckets
A Bucket is a generic container that collects samples and their values. It prescribes no behavior on its own aside from merely accepting a value, leaving it up to the concrete implementation to what to do with the injected values.
Accumulating Bucket
An Accumulating Bucket is a Bucket that appends the new sample to a queue such that the eldest values are evicted according to a given policy.
Eviction Policies
Once an Accumulating Bucket reaches capacity, its eviction policy is invoked. This reaps the oldest N objects subject to certain behavior.
####### Remove Oldest This merely removes the oldest N items without performing some aggregation replacement operation on them.
####### Aggregate Oldest This removes the oldest N items while performing some summary aggregation operation thereupon, which is then appended to the list in the former values' place.
Tallying Bucket
A Tallying Bucket differs from an Accumulating Bucket in that it never stores any of the values emitted into it but rather exposes a simplied summary representation thereof. For instance, if a values therein is requested, it may situationally emit a minimum, maximum, an average, or any other reduction mechanism requested.
Getting Started
- The source code is periodically indexed: Go Exposition Client.
- All of the core developers are accessible via the Prometheus Developers Mailinglist.
Testing
This package employs gocheck for testing. Please ensure that all tests pass by running the following from the project root:
$ go test ./...
The use of gocheck is summarily being phased out; however, old tests that use it still exist.