- Separate out the packages' files' documentation into a separate documentation file per package.
- Provide better examples in the examples subdirectory. - Make the comments consistent in terms of using multi-line format for future-proofing. - Extract major constants out.
This commit is contained in:
Matt T. Proud
parent
5ea9b1a0b5
commit
e0b92aec7a
23
README.md
23
README.md
|
|
@ -17,42 +17,47 @@ to be made, but this task has been deferred for now.
|
|||
# Continuous Integration
|
||||
[](http://travis-ci.org/matttproud/golang_instrumentation)
|
||||
|
||||
# Metrics
|
||||
# Documentation
|
||||
Please read the [generated documentation](http://go.pkgdoc.org/launchpad.net/gocheck)
|
||||
for the project's documentation from source code.
|
||||
|
||||
# Basic Overview
|
||||
## Metrics
|
||||
A metric is a measurement mechanism.
|
||||
|
||||
## Gauge
|
||||
### Gauge
|
||||
A Gauge is a metric that exposes merely an instantaneous value or some snapshot
|
||||
thereof.
|
||||
|
||||
## Histogram
|
||||
### Histogram
|
||||
A Histogram is a metric that captures events or samples into buckets. It
|
||||
exposes its values via percentile estimations.
|
||||
|
||||
### Buckets
|
||||
#### Buckets
|
||||
A Bucket is a generic container that collects samples and their values. It
|
||||
prescribes no behavior on its own aside from merely accepting a value,
|
||||
leaving it up to the concrete implementation to what to do with the injected
|
||||
values.
|
||||
|
||||
#### Accumulating Bucket
|
||||
##### Accumulating Bucket
|
||||
An Accumulating Bucket is a bucket that appends the new sample to a timestamped
|
||||
priority queue such that the eldest values are evicted according to a given
|
||||
policy.
|
||||
|
||||
#### Eviction Policies
|
||||
##### Eviction Policies
|
||||
Once an Accumulating Bucket reaches capacity, its eviction policy is invoked.
|
||||
This reaps the oldest N objects subject to certain behavior.
|
||||
|
||||
##### Remove Oldest
|
||||
###### Remove Oldest
|
||||
This merely removes the oldest N items without performing some aggregation
|
||||
replacement operation on them.
|
||||
|
||||
##### Aggregate Oldest
|
||||
###### Aggregate Oldest
|
||||
This removes the oldest N items while performing some summary aggregation
|
||||
operation thereupon, which is then appended to the list in the former values'
|
||||
place.
|
||||
|
||||
#### Tallying Bucket
|
||||
##### Tallying Bucket
|
||||
A Tallying Bucket differs from an Accumulating Bucket in that it never stores
|
||||
any of the values emitted into it but rather exposes a simplied summary
|
||||
representation thereof. For instance, if a values therein is requested,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,54 @@
|
|||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
Use of this source code is governed by a BSD-style license that can be found in
|
||||
the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
registry.go provides a container for centralized exposition of metrics to
|
||||
their prospective consumers.
|
||||
|
||||
registry.Register("human_readable_metric_name", metric)
|
||||
|
||||
Please try to observe the following rules when naming metrics:
|
||||
|
||||
- Use underbars "_" to separate words.
|
||||
|
||||
- Have the metric name start from generality and work toward specificity
|
||||
toward the end. For example, when working with multiple caching subsystems,
|
||||
consider using the following structure "cache" + "user_credentials" →
|
||||
"cache_user_credentials" and "cache" + "value_transformations" →
|
||||
"cache_value_transformations".
|
||||
|
||||
- Have whatever is being measured follow the system and subsystem names cited
|
||||
supra. For instance, with "insertions", "deletions", "evictions",
|
||||
"replacements" of the above cache, they should be named as
|
||||
"cache_user_credentials_insertions" and "cache_user_credentials_deletions" and
|
||||
"cache_user_credentials_deletions" and "cache_user_credentials_evictions".
|
||||
|
||||
- If what is being measured has a standardized unit around it, consider
|
||||
providing a unit for it.
|
||||
|
||||
- Consider adding an additional suffix that designates what the value represents
|
||||
such as a "total" or "size"---e.g., "cache_user_credentials_size_kb" or
|
||||
"cache_user_credentials_insertions_total".
|
||||
|
||||
- Give heed to how future-proof the names are. Things may depend on these
|
||||
names; and as your service evolves, the calculated values may take on
|
||||
different meanings, which can be difficult to reflect if deployed code depends
|
||||
on antique names.
|
||||
|
||||
Further considerations:
|
||||
|
||||
- The Registry's exposition mechanism is not backed by authorization and
|
||||
authentication. This is something that will need to be addressed for
|
||||
production services that are directly exposed to the outside world.
|
||||
|
||||
- Engage in as little in-process processing of values as possible. The job
|
||||
of processing and aggregation of these values belongs in a separate
|
||||
post-processing job. The same goes for archiving. I will need to evaluate
|
||||
hooks into something like OpenTSBD.
|
||||
*/
|
||||
package registry
|
||||
|
|
@ -1,16 +1,19 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// main.go provides a simple example of how to use this instrumentation
|
||||
// framework in the context of having something that emits values into
|
||||
// its collectors.
|
||||
//
|
||||
// The emitted values correspond to uniform, normal, and exponential
|
||||
// distributions.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
main.go provides a simple example of how to use this instrumentation
|
||||
framework in the context of having something that emits values into
|
||||
its collectors.
|
||||
|
||||
The emitted values correspond to uniform, normal, and exponential
|
||||
distributions.
|
||||
*/
|
||||
package main
|
||||
|
||||
import (
|
||||
|
|
@ -43,12 +46,12 @@ func main() {
|
|||
zed_rpc_calls := &metrics.GaugeMetric{}
|
||||
|
||||
metrics := registry.NewRegistry()
|
||||
metrics.Register("foo_rpc_latency_ms_histogram", foo_rpc_latency)
|
||||
metrics.Register("foo_rpc_call_count", foo_rpc_calls)
|
||||
metrics.Register("bar_rpc_latency_ms_histogram", bar_rpc_latency)
|
||||
metrics.Register("bar_rpc_call_count", bar_rpc_calls)
|
||||
metrics.Register("zed_rpc_latency_ms_histogram", zed_rpc_latency)
|
||||
metrics.Register("zed_rpc_call_count", zed_rpc_calls)
|
||||
metrics.Register("rpc_latency_foo_microseconds", foo_rpc_latency)
|
||||
metrics.Register("rpc_calls_foo_total", foo_rpc_calls)
|
||||
metrics.Register("rpc_latency_bar_microseconds", bar_rpc_latency)
|
||||
metrics.Register("rpc_calls_bar_total", bar_rpc_calls)
|
||||
metrics.Register("rpc_latency_zed_microseconds", zed_rpc_latency)
|
||||
metrics.Register("rpc_calls_zed_total", zed_rpc_calls)
|
||||
|
||||
go func() {
|
||||
for {
|
||||
|
|
|
|||
|
|
@ -1,12 +1,15 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// main.go provides a simple skeletal example of how this instrumentation
|
||||
// framework is registered and invoked.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
main.go provides a simple skeletal example of how this instrumentation
|
||||
framework is registered and invoked.
|
||||
*/
|
||||
package main
|
||||
|
||||
import (
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// distributions.go provides basic distribution-generating functions that are
|
||||
// used primarily in testing contexts.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package maths
|
||||
|
||||
|
|
@ -13,7 +12,9 @@ import (
|
|||
"math"
|
||||
)
|
||||
|
||||
// Go's standard library does not offer a factorial function.
|
||||
/*
|
||||
Go's standard library does not offer a factorial function.
|
||||
*/
|
||||
func Factorial(of int) int64 {
|
||||
if of <= 0 {
|
||||
return 1
|
||||
|
|
@ -28,9 +29,11 @@ func Factorial(of int) int64 {
|
|||
return result
|
||||
}
|
||||
|
||||
// Create calculate the value of a probability density for a given binomial
|
||||
// statistic, where k is the target count of true cases, n is the number of
|
||||
// subjects, and p is the probability.
|
||||
/*
|
||||
Create calculate the value of a probability density for a given binomial
|
||||
statistic, where k is the target count of true cases, n is the number of
|
||||
subjects, and p is the probability.
|
||||
*/
|
||||
func BinomialPDF(k, n int, p float64) float64 {
|
||||
binomialCoefficient := float64(Factorial(n)) / float64(Factorial(k)*Factorial(n-k))
|
||||
intermediate := math.Pow(p, float64(k)) * math.Pow(1-p, float64(n-k))
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
The maths package provides a number of mathematical-related helpers:
|
||||
|
||||
distributions.go provides basic distribution-generating functions that are
|
||||
used primarily in testing contexts.
|
||||
|
||||
helpers_for_testing.go provides a testing assistents for this package and its
|
||||
dependents.
|
||||
|
||||
maths_test.go provides a test suite for all tests in the maths package
|
||||
hierarchy. It employs the gocheck framework for test scaffolding.
|
||||
|
||||
statistics.go provides basic summary statistics functions for the purpose of
|
||||
metrics aggregation.
|
||||
|
||||
statistics_test.go provides a test complement for the statistics.go module.
|
||||
*/
|
||||
package maths
|
||||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// helpers_for_testing.go provides a testing assistents for this package and its
|
||||
// dependents.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package maths
|
||||
|
||||
|
|
@ -19,8 +18,10 @@ type isNaNChecker struct {
|
|||
*CheckerInfo
|
||||
}
|
||||
|
||||
// This piece provides a simple tester for the gocheck testing library to
|
||||
// ascertain if a value is not-a-number.
|
||||
/*
|
||||
This piece provides a simple tester for the gocheck testing library to
|
||||
ascertain if a value is not-a-number.
|
||||
*/
|
||||
var IsNaN Checker = &isNaNChecker{
|
||||
&CheckerInfo{Name: "IsNaN", Params: []string{"value"}},
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// maths_test.go provides a test suite for all tests in the maths package
|
||||
// hierarchy. It employs the gocheck framework for test scaffolding.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package maths
|
||||
|
||||
|
|
|
|||
|
|
@ -1,14 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// statistics.go provides basic summary statistics functions for the purpose of
|
||||
// metrics aggregation.
|
||||
|
||||
// TODO(mtp): Split this out into a summary statistics file once moving/rolling
|
||||
// averages are calculated.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package maths
|
||||
|
||||
|
|
@ -17,8 +13,15 @@ import (
|
|||
"sort"
|
||||
)
|
||||
|
||||
// ReductionMethod provides a method for reducing metrics into a given scalar
|
||||
// value.
|
||||
/*
|
||||
TODO(mtp): Split this out into a summary statistics file once moving/rolling
|
||||
averages are calculated.
|
||||
*/
|
||||
|
||||
/*
|
||||
ReductionMethod provides a method for reducing metrics into a given scalar
|
||||
value.
|
||||
*/
|
||||
type ReductionMethod func([]float64) float64
|
||||
|
||||
var Average ReductionMethod = func(input []float64) float64 {
|
||||
|
|
@ -37,7 +40,9 @@ var Average ReductionMethod = func(input []float64) float64 {
|
|||
return sum / count
|
||||
}
|
||||
|
||||
// Extract the first modal value.
|
||||
/*
|
||||
Extract the first modal value.
|
||||
*/
|
||||
var FirstMode ReductionMethod = func(input []float64) float64 {
|
||||
valuesToFrequency := map[float64]int64{}
|
||||
var largestTally int64 = math.MinInt64
|
||||
|
|
@ -58,7 +63,9 @@ var FirstMode ReductionMethod = func(input []float64) float64 {
|
|||
return largestTallyValue
|
||||
}
|
||||
|
||||
// Calculate the percentile by choosing the nearest neighboring value.
|
||||
/*
|
||||
Calculate the percentile by choosing the nearest neighboring value.
|
||||
*/
|
||||
func NearestRank(input []float64, percentile float64) float64 {
|
||||
inputSize := len(input)
|
||||
|
||||
|
|
@ -81,7 +88,10 @@ func NearestRank(input []float64, percentile float64) float64 {
|
|||
return copiedInput[preliminaryIndex]
|
||||
}
|
||||
|
||||
func NearestRankReducer(percentile float64) func(input []float64) float64 {
|
||||
/*
|
||||
Generate a ReductionMethod based off of extracting a given percentile value.
|
||||
*/
|
||||
func NearestRankReducer(percentile float64) ReductionMethod {
|
||||
return func(input []float64) float64 {
|
||||
return NearestRank(input, percentile)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// statistics_test.go provides a test complement for the statistics.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package maths
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// accumulating_bucket.go provides a histogram bucket type that accumulates
|
||||
// elements until a given capacity and enacts a given eviction policy upon
|
||||
// such a condition.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -29,9 +27,11 @@ type AccumulatingBucket struct {
|
|||
evictionPolicy EvictionPolicy
|
||||
}
|
||||
|
||||
// AccumulatingBucketBuilder is a convenience method for generating a
|
||||
// BucketBuilder that produces AccumatingBucket entries with a certain
|
||||
// behavior set.
|
||||
/*
|
||||
AccumulatingBucketBuilder is a convenience method for generating a
|
||||
BucketBuilder that produces AccumatingBucket entries with a certain
|
||||
behavior set.
|
||||
*/
|
||||
func AccumulatingBucketBuilder(evictionPolicy EvictionPolicy, maximumSize int) BucketBuilder {
|
||||
return func() Bucket {
|
||||
return &AccumulatingBucket{
|
||||
|
|
@ -42,8 +42,10 @@ func AccumulatingBucketBuilder(evictionPolicy EvictionPolicy, maximumSize int) B
|
|||
}
|
||||
}
|
||||
|
||||
// Add a value to the bucket. Depending on whether the bucket is full, it may
|
||||
// trigger an eviction of older items.
|
||||
/*
|
||||
Add a value to the bucket. Depending on whether the bucket is full, it may
|
||||
trigger an eviction of older items.
|
||||
*/
|
||||
func (b *AccumulatingBucket) Add(value float64) {
|
||||
b.mutex.Lock()
|
||||
defer b.mutex.Unlock()
|
||||
|
|
@ -98,9 +100,11 @@ func (b *AccumulatingBucket) ValueForIndex(index int) float64 {
|
|||
|
||||
sort.Float64s(sortedElements)
|
||||
|
||||
// N.B.(mtp): Interfacing components should not need to comprehend what
|
||||
// eviction and storage container strategies used; therefore,
|
||||
// we adjust this silently.
|
||||
/*
|
||||
N.B.(mtp): Interfacing components should not need to comprehend what
|
||||
eviction and storage container strategies used; therefore,
|
||||
we adjust this silently.
|
||||
*/
|
||||
targetIndex := int(float64(elementCount-1) * (float64(index) / float64(b.observations)))
|
||||
|
||||
return sortedElements[targetIndex]
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// accumulating_bucket_test.go provides a test complement for the
|
||||
// accumulating_bucket_go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -124,17 +123,23 @@ func (s *S) TestAccumulatingBucketValueForIndex(c *C) {
|
|||
c.Assert(b.ValueForIndex(i), maths.IsNaN)
|
||||
}
|
||||
|
||||
// The bucket has only observed one item and contains now one item.
|
||||
/*
|
||||
The bucket has only observed one item and contains now one item.
|
||||
*/
|
||||
b.Add(1.0)
|
||||
|
||||
c.Check(b.ValueForIndex(0), Equals, 1.0)
|
||||
// Let's sanity check what occurs if presumably an eviction happened and
|
||||
// we requested an index larger than what is contained.
|
||||
/*
|
||||
Let's sanity check what occurs if presumably an eviction happened and
|
||||
we requested an index larger than what is contained.
|
||||
*/
|
||||
c.Check(b.ValueForIndex(1), Equals, 1.0)
|
||||
|
||||
for i := 2.0; i <= 100; i += 1 {
|
||||
b.Add(i)
|
||||
// TODO(mtp): This is a sin. Provide a mechanism for deterministic testing.
|
||||
/*
|
||||
TODO(mtp): This is a sin. Provide a mechanism for deterministic testing.
|
||||
*/
|
||||
time.Sleep(1 * time.Millisecond)
|
||||
}
|
||||
|
||||
|
|
@ -144,13 +149,17 @@ func (s *S) TestAccumulatingBucketValueForIndex(c *C) {
|
|||
|
||||
for i := 101.0; i <= 150; i += 1 {
|
||||
b.Add(i)
|
||||
// TODO(mtp): This is a sin. Provide a mechanism for deterministic testing.
|
||||
/*
|
||||
TODO(mtp): This is a sin. Provide a mechanism for deterministic testing.
|
||||
*/
|
||||
time.Sleep(1 * time.Millisecond)
|
||||
}
|
||||
|
||||
// The bucket's capacity has been exceeded by inputs at this point;
|
||||
// consequently, we search for a given element by percentage offset
|
||||
// therein.
|
||||
/*
|
||||
The bucket's capacity has been exceeded by inputs at this point;
|
||||
consequently, we search for a given element by percentage offset
|
||||
therein.
|
||||
*/
|
||||
c.Check(b.ValueForIndex(0), Equals, 51.0)
|
||||
c.Check(b.ValueForIndex(50), Equals, 84.0)
|
||||
c.Check(b.ValueForIndex(99), Equals, 116.0)
|
||||
|
|
|
|||
|
|
@ -1,17 +0,0 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
|
||||
// base.go provides fundamental interface expectations for the various metrics.
|
||||
|
||||
package metrics
|
||||
|
||||
// A Metric is something that can be exposed via the registry framework.
|
||||
type Metric interface {
|
||||
// Produce a human-consumable representation of the metric.
|
||||
Humanize() string
|
||||
// Produce a JSON-consumable representation of the metric.
|
||||
Marshallable() map[string]interface{}
|
||||
}
|
||||
|
|
@ -1,32 +1,47 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// bucket.go provides fundamental interface expectations for various bucket
|
||||
// types.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
bucket.go provides fundamental interface expectations for various bucket
|
||||
types.
|
||||
*/
|
||||
package metrics
|
||||
|
||||
// The Histogram class and associated types build buckets on their own.
|
||||
/*
|
||||
The Histogram class and associated types build buckets on their own.
|
||||
*/
|
||||
type BucketBuilder func() Bucket
|
||||
|
||||
// This defines the base Bucket type. The exact behaviors of the bucket are
|
||||
// at the whim of the implementor.
|
||||
//
|
||||
// A Bucket is used as a container by Histogram as a collection for its
|
||||
// accumulated samples.
|
||||
/*
|
||||
This defines the base Bucket type. The exact behaviors of the bucket are
|
||||
at the whim of the implementor.
|
||||
|
||||
A Bucket is used as a container by Histogram as a collection for its
|
||||
accumulated samples.
|
||||
*/
|
||||
type Bucket interface {
|
||||
// Add a value to the bucket.
|
||||
/*
|
||||
Add a value to the bucket.
|
||||
*/
|
||||
Add(value float64)
|
||||
// Provide a humanized representation hereof.
|
||||
/*
|
||||
Provide a humanized representation hereof.
|
||||
*/
|
||||
Humanize() string
|
||||
// Provide a count of observations throughout the bucket's lifetime.
|
||||
/*
|
||||
Provide a count of observations throughout the bucket's lifetime.
|
||||
*/
|
||||
Observations() int
|
||||
// Provide the value from the given in-memory value cache or an estimate
|
||||
// thereof for the given index. The consumer of the bucket's data makes
|
||||
// no assumptions about the underlying storage mechanisms that the bucket
|
||||
// employs.
|
||||
/*
|
||||
Provide the value from the given in-memory value cache or an estimate
|
||||
thereof for the given index. The consumer of the bucket's data makes
|
||||
no assumptions about the underlying storage mechanisms that the bucket
|
||||
employs.
|
||||
*/
|
||||
ValueForIndex(index int) float64
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,22 @@
|
|||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
constants.go provides package-level constants for metrics.
|
||||
*/
|
||||
package metrics
|
||||
|
||||
const (
|
||||
valueKey = "value"
|
||||
gaugeTypeValue = "gauge"
|
||||
typeKey = "type"
|
||||
histogramTypeValue = "histogram"
|
||||
floatFormat = 'f'
|
||||
floatPrecision = 6
|
||||
floatBitCount = 64
|
||||
)
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
The metrics package provides general descriptors for the concept of exportable
|
||||
metrics.
|
||||
|
||||
accumulating_bucket.go provides a histogram bucket type that accumulates
|
||||
elements until a given capacity and enacts a given eviction policy upon
|
||||
such a condition.
|
||||
|
||||
accumulating_bucket_test.go provides a test complement for the
|
||||
accumulating_bucket_go module.
|
||||
|
||||
eviction.go provides several histogram bucket eviction strategies.
|
||||
|
||||
eviction_test.go provides a test complement for the eviction.go module.
|
||||
|
||||
gauge.go provides a scalar metric that one can monitor. It is useful for
|
||||
certain cases, such as instantaneous temperature.
|
||||
|
||||
gauge_test.go provides a test complement for the gauge.go module.
|
||||
|
||||
histogram.go provides a basic histogram metric, which can accumulate scalar
|
||||
event values or samples. The underlying histogram implementation is designed
|
||||
to be performant in that it accepts tolerable inaccuracies.
|
||||
|
||||
histogram_test.go provides a test complement for the histogram.go module.
|
||||
|
||||
metric.go provides fundamental interface expectations for the various metrics.
|
||||
|
||||
metrics_test.go provides a test suite for all tests in the metrics package
|
||||
hierarchy. It employs the gocheck framework for test scaffolding.
|
||||
|
||||
tallying_bucket.go provides a histogram bucket type that aggregates tallies
|
||||
of events that fall into its ranges versus a summary of the values
|
||||
themselves.
|
||||
|
||||
tallying_bucket_test.go provides a test complement for the
|
||||
tallying_bucket.go module.
|
||||
|
||||
timer.go provides a scalar metric that times how long a given event takes.
|
||||
|
||||
timer_test.go provides a test complement for the timer.go module.
|
||||
*/
|
||||
package metrics
|
||||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// eviction.go provides several histogram bucket eviction strategies.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -15,12 +15,16 @@ import (
|
|||
"time"
|
||||
)
|
||||
|
||||
// EvictionPolicy implements some sort of garbage collection methodology for
|
||||
// an underlying heap.Interface. This is presently only used for
|
||||
// AccumulatingBucket.
|
||||
/*
|
||||
EvictionPolicy implements some sort of garbage collection methodology for
|
||||
an underlying heap.Interface. This is presently only used for
|
||||
AccumulatingBucket.
|
||||
*/
|
||||
type EvictionPolicy func(h heap.Interface)
|
||||
|
||||
// As the name implies, this evicts the oldest x objects from the heap.
|
||||
/*
|
||||
As the name implies, this evicts the oldest x objects from the heap.
|
||||
*/
|
||||
func EvictOldest(count int) EvictionPolicy {
|
||||
return func(h heap.Interface) {
|
||||
for i := 0; i < count; i++ {
|
||||
|
|
@ -29,10 +33,10 @@ func EvictOldest(count int) EvictionPolicy {
|
|||
}
|
||||
}
|
||||
|
||||
// This factory produces an EvictionPolicy that applies some standardized
|
||||
// reduction methodology on the to-be-terminated values.
|
||||
//
|
||||
// TODO(mtp): Parameterize the priority generation since these tools are useful.
|
||||
/*
|
||||
This factory produces an EvictionPolicy that applies some standardized
|
||||
reduction methodology on the to-be-terminated values.
|
||||
*/
|
||||
func EvictAndReplaceWith(count int, reducer maths.ReductionMethod) EvictionPolicy {
|
||||
return func(h heap.Interface) {
|
||||
oldValues := make([]float64, count)
|
||||
|
|
@ -45,6 +49,9 @@ func EvictAndReplaceWith(count int, reducer maths.ReductionMethod) EvictionPolic
|
|||
|
||||
heap.Push(h, &utility.Item{
|
||||
Value: reduced,
|
||||
/*
|
||||
TODO(mtp): Parameterize the priority generation since these tools are useful.
|
||||
*/
|
||||
Priority: -1 * time.Now().UnixNano(),
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// eviction_test.go provides a test complement for the eviction.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// gauge.go provides a scalar metric that one can monitor. It is useful for
|
||||
// certain cases, such as instantaneous temperature.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -14,10 +13,12 @@ import (
|
|||
"sync"
|
||||
)
|
||||
|
||||
// A gauge metric merely provides an instantaneous representation of a scalar
|
||||
// value or an accumulation. For instance, if one wants to expose the current
|
||||
// temperature or the hitherto bandwidth used, this would be the metric for such
|
||||
// circumstances.
|
||||
/*
|
||||
A gauge metric merely provides an instantaneous representation of a scalar
|
||||
value or an accumulation. For instance, if one wants to expose the current
|
||||
temperature or the hitherto bandwidth used, this would be the metric for such
|
||||
circumstances.
|
||||
*/
|
||||
type GaugeMetric struct {
|
||||
value float64
|
||||
mutex sync.RWMutex
|
||||
|
|
@ -80,8 +81,8 @@ func (metric *GaugeMetric) Marshallable() map[string]interface{} {
|
|||
|
||||
v := make(map[string]interface{}, 2)
|
||||
|
||||
v["value"] = metric.value
|
||||
v["type"] = "gauge"
|
||||
v[valueKey] = metric.value
|
||||
v[typeKey] = gaugeTypeValue
|
||||
|
||||
return v
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// gauge_test.go provides a test complement for the gauge.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// histogram.go provides a basic histogram metric, which can accumulate scalar
|
||||
// event values or samples. The underlying histogram implementation is designed
|
||||
// to be performant in that it accepts tolerable inaccuracies.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -17,9 +15,11 @@ import (
|
|||
"strconv"
|
||||
)
|
||||
|
||||
// This generates count-buckets of equal size distributed along the open
|
||||
// interval of lower to upper. For instance, {lower=0, upper=10, count=5}
|
||||
// yields the following: [0, 2, 4, 6, 8].
|
||||
/*
|
||||
This generates count-buckets of equal size distributed along the open
|
||||
interval of lower to upper. For instance, {lower=0, upper=10, count=5}
|
||||
yields the following: [0, 2, 4, 6, 8].
|
||||
*/
|
||||
func EquallySizedBucketsFor(lower, upper float64, count int) []float64 {
|
||||
buckets := make([]float64, count)
|
||||
|
||||
|
|
@ -33,8 +33,10 @@ func EquallySizedBucketsFor(lower, upper float64, count int) []float64 {
|
|||
return buckets
|
||||
}
|
||||
|
||||
// This generates log2-sized buckets spanning from lower to upper inclusively
|
||||
// as well as values beyond it.
|
||||
/*
|
||||
This generates log2-sized buckets spanning from lower to upper inclusively
|
||||
as well as values beyond it.
|
||||
*/
|
||||
func LogarithmicSizedBucketsFor(lower, upper float64) []float64 {
|
||||
bucketCount := int(math.Ceil(math.Log2(upper)))
|
||||
|
||||
|
|
@ -47,36 +49,46 @@ func LogarithmicSizedBucketsFor(lower, upper float64) []float64 {
|
|||
return buckets
|
||||
}
|
||||
|
||||
// A HistogramSpecification defines how a Histogram is to be built.
|
||||
/*
|
||||
A HistogramSpecification defines how a Histogram is to be built.
|
||||
*/
|
||||
type HistogramSpecification struct {
|
||||
Starts []float64
|
||||
BucketMaker BucketBuilder
|
||||
ReportablePercentiles []float64
|
||||
}
|
||||
|
||||
// The histogram is an accumulator for samples. It merely routes into which
|
||||
// to bucket to capture an event and provides a percentile calculation
|
||||
// mechanism.
|
||||
//
|
||||
// Histogram makes do without locking by employing the law of large numbers
|
||||
// to presume a convergence toward a given bucket distribution. Locking
|
||||
// may be implemented in the buckets themselves, though.
|
||||
/*
|
||||
The histogram is an accumulator for samples. It merely routes into which
|
||||
to bucket to capture an event and provides a percentile calculation
|
||||
mechanism.
|
||||
|
||||
Histogram makes do without locking by employing the law of large numbers
|
||||
to presume a convergence toward a given bucket distribution. Locking
|
||||
may be implemented in the buckets themselves, though.
|
||||
*/
|
||||
type Histogram struct {
|
||||
// This represents the open interval's start at which values shall be added to
|
||||
// the bucket. The interval continues until the beginning of the next bucket
|
||||
// exclusive or positive infinity.
|
||||
//
|
||||
// N.B.
|
||||
// - bucketStarts should be sorted in ascending order;
|
||||
// - len(bucketStarts) must be equivalent to len(buckets);
|
||||
// - The index of a given bucketStarts' element is presumed to match
|
||||
// correspond to the appropriate element in buckets.
|
||||
/*
|
||||
This represents the open interval's start at which values shall be added to
|
||||
the bucket. The interval continues until the beginning of the next bucket
|
||||
exclusive or positive infinity.
|
||||
|
||||
N.B.
|
||||
- bucketStarts should be sorted in ascending order;
|
||||
- len(bucketStarts) must be equivalent to len(buckets);
|
||||
- The index of a given bucketStarts' element is presumed to match
|
||||
correspond to the appropriate element in buckets.
|
||||
*/
|
||||
bucketStarts []float64
|
||||
// These are the buckets that capture samples as they are emitted to the
|
||||
// histogram. Please consult the reference interface and its implements for
|
||||
// further details about behavior expectations.
|
||||
/*
|
||||
These are the buckets that capture samples as they are emitted to the
|
||||
histogram. Please consult the reference interface and its implements for
|
||||
further details about behavior expectations.
|
||||
*/
|
||||
buckets []Bucket
|
||||
// These are the percentile values that will be reported on marshalling.
|
||||
/*
|
||||
These are the percentile values that will be reported on marshalling.
|
||||
*/
|
||||
reportablePercentiles []float64
|
||||
}
|
||||
|
||||
|
|
@ -108,7 +120,9 @@ func (h *Histogram) Humanize() string {
|
|||
return string(stringBuffer.Bytes())
|
||||
}
|
||||
|
||||
// Determine the number of previous observations up to a given index.
|
||||
/*
|
||||
Determine the number of previous observations up to a given index.
|
||||
*/
|
||||
func previousCumulativeObservations(cumulativeObservations []int, bucketIndex int) int {
|
||||
if bucketIndex == 0 {
|
||||
return 0
|
||||
|
|
@ -117,12 +131,16 @@ func previousCumulativeObservations(cumulativeObservations []int, bucketIndex in
|
|||
return cumulativeObservations[bucketIndex-1]
|
||||
}
|
||||
|
||||
// Determine the index for an element given a percentage of length.
|
||||
/*
|
||||
Determine the index for an element given a percentage of length.
|
||||
*/
|
||||
func prospectiveIndexForPercentile(percentile float64, totalObservations int) int {
|
||||
return int(percentile * float64(totalObservations-1))
|
||||
}
|
||||
|
||||
// Determine the next bucket element when interim bucket intervals may be empty.
|
||||
/*
|
||||
Determine the next bucket element when interim bucket intervals may be empty.
|
||||
*/
|
||||
func (h *Histogram) nextNonEmptyBucketElement(currentIndex, bucketCount int, observationsByBucket []int) (*Bucket, int) {
|
||||
for i := currentIndex; i < bucketCount; i++ {
|
||||
if observationsByBucket[i] == 0 {
|
||||
|
|
@ -135,18 +153,24 @@ func (h *Histogram) nextNonEmptyBucketElement(currentIndex, bucketCount int, obs
|
|||
panic("Illegal Condition: There were no remaining buckets to provide a value.")
|
||||
}
|
||||
|
||||
// Find what bucket and element index contains a given percentile value.
|
||||
// If a percentile is requested that results in a corresponding index that is no
|
||||
// longer contained by the bucket, the index of the last item is returned. This
|
||||
// may occur if the underlying bucket catalogs values and employs an eviction
|
||||
// strategy.
|
||||
/*
|
||||
Find what bucket and element index contains a given percentile value.
|
||||
If a percentile is requested that results in a corresponding index that is no
|
||||
longer contained by the bucket, the index of the last item is returned. This
|
||||
may occur if the underlying bucket catalogs values and employs an eviction
|
||||
strategy.
|
||||
*/
|
||||
func (h *Histogram) bucketForPercentile(percentile float64) (*Bucket, int) {
|
||||
bucketCount := len(h.buckets)
|
||||
|
||||
// This captures the quantity of samples in a given bucket's range.
|
||||
/*
|
||||
This captures the quantity of samples in a given bucket's range.
|
||||
*/
|
||||
observationsByBucket := make([]int, bucketCount)
|
||||
// This captures the cumulative quantity of observations from all preceding
|
||||
// buckets up and to the end of this bucket.
|
||||
/*
|
||||
This captures the cumulative quantity of observations from all preceding
|
||||
buckets up and to the end of this bucket.
|
||||
*/
|
||||
cumulativeObservationsByBucket := make([]int, bucketCount)
|
||||
|
||||
var totalObservations int = 0
|
||||
|
|
@ -158,9 +182,11 @@ func (h *Histogram) bucketForPercentile(percentile float64) (*Bucket, int) {
|
|||
cumulativeObservationsByBucket[i] = totalObservations
|
||||
}
|
||||
|
||||
// This captures the index offset where the given percentile value would be
|
||||
// were all submitted samples stored and never down-/re-sampled nor deleted
|
||||
// and housed in a singular array.
|
||||
/*
|
||||
This captures the index offset where the given percentile value would be
|
||||
were all submitted samples stored and never down-/re-sampled nor deleted
|
||||
and housed in a singular array.
|
||||
*/
|
||||
prospectiveIndex := prospectiveIndexForPercentile(percentile, totalObservations)
|
||||
|
||||
for i, cumulativeObservation := range cumulativeObservationsByBucket {
|
||||
|
|
@ -168,15 +194,21 @@ func (h *Histogram) bucketForPercentile(percentile float64) (*Bucket, int) {
|
|||
continue
|
||||
}
|
||||
|
||||
// Find the bucket that contains the given index.
|
||||
/*
|
||||
Find the bucket that contains the given index.
|
||||
*/
|
||||
if cumulativeObservation >= prospectiveIndex {
|
||||
var subIndex int
|
||||
// This calculates the index within the current bucket where the given
|
||||
// percentile may be found.
|
||||
/*
|
||||
This calculates the index within the current bucket where the given
|
||||
percentile may be found.
|
||||
*/
|
||||
subIndex = prospectiveIndex - previousCumulativeObservations(cumulativeObservationsByBucket, i)
|
||||
|
||||
// Sometimes the index may be the last item, in which case we need to
|
||||
// take this into account.
|
||||
/*
|
||||
Sometimes the index may be the last item, in which case we need to
|
||||
take this into account.
|
||||
*/
|
||||
if observationsByBucket[i] == subIndex {
|
||||
return h.nextNonEmptyBucketElement(i+1, bucketCount, observationsByBucket)
|
||||
}
|
||||
|
|
@ -188,34 +220,42 @@ func (h *Histogram) bucketForPercentile(percentile float64) (*Bucket, int) {
|
|||
return &h.buckets[0], 0
|
||||
}
|
||||
|
||||
// Return the histogram's estimate of the value for a given percentile of
|
||||
// collected samples. The requested percentile is expected to be a real
|
||||
// value within (0, 1.0].
|
||||
/*
|
||||
Return the histogram's estimate of the value for a given percentile of
|
||||
collected samples. The requested percentile is expected to be a real
|
||||
value within (0, 1.0].
|
||||
*/
|
||||
func (h *Histogram) Percentile(percentile float64) float64 {
|
||||
bucket, index := h.bucketForPercentile(percentile)
|
||||
|
||||
return (*bucket).ValueForIndex(index)
|
||||
}
|
||||
|
||||
func formatFloat(value float64) string {
|
||||
return strconv.FormatFloat(value, floatFormat, floatPrecision, floatBitCount)
|
||||
}
|
||||
|
||||
func (h *Histogram) Marshallable() map[string]interface{} {
|
||||
numberOfPercentiles := len(h.reportablePercentiles)
|
||||
result := make(map[string]interface{}, 2)
|
||||
|
||||
result["type"] = "histogram"
|
||||
result[typeKey] = histogramTypeValue
|
||||
|
||||
value := make(map[string]interface{}, numberOfPercentiles)
|
||||
|
||||
for _, percentile := range h.reportablePercentiles {
|
||||
percentileString := strconv.FormatFloat(percentile, 'f', 6, 64)
|
||||
value[percentileString] = strconv.FormatFloat(h.Percentile(percentile), 'f', 6, 64)
|
||||
percentileString := formatFloat(percentile)
|
||||
value[percentileString] = formatFloat(h.Percentile(percentile))
|
||||
}
|
||||
|
||||
result["value"] = value
|
||||
result[valueKey] = value
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// Produce a histogram from a given specification.
|
||||
/*
|
||||
Produce a histogram from a given specification.
|
||||
*/
|
||||
func CreateHistogram(specification *HistogramSpecification) *Histogram {
|
||||
bucketCount := len(specification.Starts)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// histogram_test.go provides a test complement for the histogram.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,23 @@
|
|||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
/*
|
||||
A Metric is something that can be exposed via the registry framework.
|
||||
*/
|
||||
type Metric interface {
|
||||
/*
|
||||
Produce a human-consumable representation of the metric.
|
||||
*/
|
||||
Humanize() string
|
||||
/*
|
||||
Produce a JSON-consumable representation of the metric.
|
||||
*/
|
||||
Marshallable() map[string]interface{}
|
||||
}
|
||||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// metrics_test.go provides a test suite for all tests in the metrics package
|
||||
// hierarchy. It employs the gocheck framework for test scaffolding.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// tallying_bucket.go provides a histogram bucket type that aggregates tallies
|
||||
// of events that fall into its ranges versus a summary of the values
|
||||
// themselves.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -22,13 +20,17 @@ const (
|
|||
upperThird = 2.0 * lowerThird
|
||||
)
|
||||
|
||||
// A TallyingIndexEstimator is responsible for estimating the value of index for
|
||||
// a given TallyingBucket, even though a TallyingBucket does not possess a
|
||||
// collection of samples. There are a few strategies listed below for how
|
||||
// this value should be approximated.
|
||||
/*
|
||||
A TallyingIndexEstimator is responsible for estimating the value of index for
|
||||
a given TallyingBucket, even though a TallyingBucket does not possess a
|
||||
collection of samples. There are a few strategies listed below for how
|
||||
this value should be approximated.
|
||||
*/
|
||||
type TallyingIndexEstimator func(minimum, maximum float64, index, observations int) float64
|
||||
|
||||
// Provide a filter for handling empty buckets.
|
||||
/*
|
||||
Provide a filter for handling empty buckets.
|
||||
*/
|
||||
func emptyFilter(e TallyingIndexEstimator) TallyingIndexEstimator {
|
||||
return func(minimum, maximum float64, index, observations int) float64 {
|
||||
if observations == 0 {
|
||||
|
|
@ -39,23 +41,31 @@ func emptyFilter(e TallyingIndexEstimator) TallyingIndexEstimator {
|
|||
}
|
||||
}
|
||||
|
||||
// Report the smallest observed value in the bucket.
|
||||
/*
|
||||
Report the smallest observed value in the bucket.
|
||||
*/
|
||||
var Minimum TallyingIndexEstimator = emptyFilter(func(minimum, maximum float64, _, observations int) float64 {
|
||||
return minimum
|
||||
})
|
||||
|
||||
// Report the largest observed value in the bucket.
|
||||
/*
|
||||
Report the largest observed value in the bucket.
|
||||
*/
|
||||
var Maximum TallyingIndexEstimator = emptyFilter(func(minimum, maximum float64, _, observations int) float64 {
|
||||
return maximum
|
||||
})
|
||||
|
||||
// Report the average of the extrema.
|
||||
/*
|
||||
Report the average of the extrema.
|
||||
*/
|
||||
var Average TallyingIndexEstimator = emptyFilter(func(minimum, maximum float64, _, observations int) float64 {
|
||||
return maths.Average([]float64{minimum, maximum})
|
||||
})
|
||||
|
||||
// Report the minimum value of the index is in the lower-third of observations,
|
||||
// the average if in the middle-third, and the maximum if in the largest third.
|
||||
/*
|
||||
Report the minimum value of the index is in the lower-third of observations,
|
||||
the average if in the middle-third, and the maximum if in the largest third.
|
||||
*/
|
||||
var Uniform TallyingIndexEstimator = emptyFilter(func(minimum, maximum float64, index, observations int) float64 {
|
||||
if observations == 1 {
|
||||
return minimum
|
||||
|
|
@ -72,9 +82,11 @@ var Uniform TallyingIndexEstimator = emptyFilter(func(minimum, maximum float64,
|
|||
return maths.Average([]float64{minimum, maximum})
|
||||
})
|
||||
|
||||
// A TallyingBucket is a Bucket that tallies when an object is added to it.
|
||||
// Upon insertion, an object is compared against collected extrema and noted
|
||||
// as a new minimum or maximum if appropriate.
|
||||
/*
|
||||
A TallyingBucket is a Bucket that tallies when an object is added to it.
|
||||
Upon insertion, an object is compared against collected extrema and noted
|
||||
as a new minimum or maximum if appropriate.
|
||||
*/
|
||||
type TallyingBucket struct {
|
||||
observations int
|
||||
smallestObserved float64
|
||||
|
|
@ -119,7 +131,9 @@ func (b *TallyingBucket) ValueForIndex(index int) float64 {
|
|||
return b.estimator(b.smallestObserved, b.largestObserved, index, b.observations)
|
||||
}
|
||||
|
||||
// Produce a TallyingBucket with sane defaults.
|
||||
/*
|
||||
Produce a TallyingBucket with sane defaults.
|
||||
*/
|
||||
func DefaultTallyingBucket() TallyingBucket {
|
||||
return TallyingBucket{
|
||||
smallestObserved: math.MaxFloat64,
|
||||
|
|
@ -136,7 +150,9 @@ func CustomTallyingBucket(estimator TallyingIndexEstimator) TallyingBucket {
|
|||
}
|
||||
}
|
||||
|
||||
// This is used strictly for testing.
|
||||
/*
|
||||
This is used strictly for testing.
|
||||
*/
|
||||
func TallyingBucketBuilder() Bucket {
|
||||
b := DefaultTallyingBucket()
|
||||
return &b
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// tallying_bucket_test.go provides a test complement for the
|
||||
// tallying_bucket.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// timer.go provides a scalar metric that times how long a given event takes.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
@ -12,20 +12,33 @@ import (
|
|||
"time"
|
||||
)
|
||||
|
||||
// This callback is called upon the completion of the timer—i.e., when it stops.
|
||||
/*
|
||||
This callback is called upon the completion of the timer—i.e., when it stops.
|
||||
*/
|
||||
type CompletionCallback func(duration time.Duration)
|
||||
|
||||
// This is meant to capture a function that a StopWatch can call for purposes
|
||||
// of instrumentation.
|
||||
/*
|
||||
This is meant to capture a function that a StopWatch can call for purposes
|
||||
of instrumentation.
|
||||
*/
|
||||
type InstrumentableCall func()
|
||||
|
||||
/*
|
||||
StopWatch is the structure that captures instrumentation for durations.
|
||||
|
||||
N.B.(mtp): A major limitation hereof is that the StopWatch protocol cannot
|
||||
retain instrumentation if a panic percolates within the context that is
|
||||
being measured.
|
||||
*/
|
||||
type StopWatch struct {
|
||||
startTime time.Time
|
||||
endTime time.Time
|
||||
onCompletion CompletionCallback
|
||||
}
|
||||
|
||||
// Return a new StopWatch that is ready for instrumentation.
|
||||
/*
|
||||
Return a new StopWatch that is ready for instrumentation.
|
||||
*/
|
||||
func Start(onCompletion CompletionCallback) *StopWatch {
|
||||
return &StopWatch{
|
||||
startTime: time.Now(),
|
||||
|
|
@ -33,8 +46,10 @@ func Start(onCompletion CompletionCallback) *StopWatch {
|
|||
}
|
||||
}
|
||||
|
||||
// Stop the StopWatch returning the elapsed duration of its lifetime while
|
||||
// firing an optional CompletionCallback in the background.
|
||||
/*
|
||||
Stop the StopWatch returning the elapsed duration of its lifetime while
|
||||
firing an optional CompletionCallback in the background.
|
||||
*/
|
||||
func (s *StopWatch) Stop() time.Duration {
|
||||
s.endTime = time.Now()
|
||||
duration := s.endTime.Sub(s.startTime)
|
||||
|
|
@ -46,8 +61,10 @@ func (s *StopWatch) Stop() time.Duration {
|
|||
return duration
|
||||
}
|
||||
|
||||
// Provide a quick way of instrumenting a InstrumentableCall and emitting its
|
||||
// duration.
|
||||
/*
|
||||
Provide a quick way of instrumenting a InstrumentableCall and emitting its
|
||||
duration.
|
||||
*/
|
||||
func InstrumentCall(instrumentable InstrumentableCall, onCompletion CompletionCallback) time.Duration {
|
||||
s := Start(onCompletion)
|
||||
instrumentable()
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// timer_test.go provides a test complement for the timer.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package metrics
|
||||
|
||||
|
|
|
|||
84
registry.go
84
registry.go
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// registry.go provides a container for centralized exposition of metrics to
|
||||
// their prospective consumers.
|
||||
Use of this source code is governed by a BSD-style license that can be found in
|
||||
the LICENSE file.
|
||||
*/
|
||||
|
||||
package registry
|
||||
|
||||
|
|
@ -20,9 +19,18 @@ import (
|
|||
"time"
|
||||
)
|
||||
|
||||
// Boilerplate metrics about the metrics reporting subservice. These are only
|
||||
// exposed if the DefaultRegistry's exporter is hooked into the HTTP request
|
||||
// handler.
|
||||
const (
|
||||
jsonContentType = "application/json"
|
||||
contentType = "Content-Type"
|
||||
jsonSuffix = ".json"
|
||||
)
|
||||
|
||||
/*
|
||||
Boilerplate metrics about the metrics reporting subservice. These are only
|
||||
exposed if the DefaultRegistry's exporter is hooked into the HTTP request
|
||||
handler.
|
||||
*/
|
||||
|
||||
var requestCount *metrics.GaugeMetric = &metrics.GaugeMetric{}
|
||||
var requestLatencyLogarithmicBuckets []float64 = metrics.LogarithmicSizedBucketsFor(0, 1000)
|
||||
var requestLatencyEqualBuckets []float64 = metrics.EquallySizedBucketsFor(0, 1000, 10)
|
||||
|
|
@ -47,8 +55,10 @@ var requestLatencyEqualTallying *metrics.Histogram = metrics.CreateHistogram(&me
|
|||
ReportablePercentiles: []float64{0.01, 0.05, 0.5, 0.9, 0.99},
|
||||
})
|
||||
|
||||
// This callback accumulates the microsecond duration of the reporting
|
||||
// framework's overhead such that it can be reported.
|
||||
/*
|
||||
This callback accumulates the microsecond duration of the reporting framework's
|
||||
overhead such that it can be reported.
|
||||
*/
|
||||
var requestLatencyAccumulator metrics.CompletionCallback = func(duration time.Duration) {
|
||||
microseconds := float64(int64(duration) / 1E3)
|
||||
|
||||
|
|
@ -58,33 +68,43 @@ var requestLatencyAccumulator metrics.CompletionCallback = func(duration time.Du
|
|||
requestLatencyEqualTallying.Add(microseconds)
|
||||
}
|
||||
|
||||
// Registry is, as the name implies, a registrar where metrics are listed.
|
||||
//
|
||||
// In most situations, using DefaultRegistry is sufficient versus creating
|
||||
// one's own.
|
||||
/*
|
||||
Registry is, as the name implies, a registrar where metrics are listed.
|
||||
|
||||
In most situations, using DefaultRegistry is sufficient versus creating one's
|
||||
own.
|
||||
*/
|
||||
type Registry struct {
|
||||
mutex sync.RWMutex
|
||||
NameToMetric map[string]metrics.Metric
|
||||
}
|
||||
|
||||
// This builds a new metric registry. It is not needed in the majority of
|
||||
// cases.
|
||||
/*
|
||||
This builds a new metric registry. It is not needed in the majority of
|
||||
cases.
|
||||
*/
|
||||
func NewRegistry() *Registry {
|
||||
return &Registry{
|
||||
NameToMetric: make(map[string]metrics.Metric),
|
||||
}
|
||||
}
|
||||
|
||||
// This is the default registry with which Metric objects are associated. It
|
||||
// is primarily a read-only object after server instantiation.
|
||||
/*
|
||||
This is the default registry with which Metric objects are associated. It
|
||||
is primarily a read-only object after server instantiation.
|
||||
*/
|
||||
var DefaultRegistry = NewRegistry()
|
||||
|
||||
// Associate a Metric with the DefaultRegistry.
|
||||
/*
|
||||
Associate a Metric with the DefaultRegistry.
|
||||
*/
|
||||
func Register(name string, metric metrics.Metric) {
|
||||
DefaultRegistry.Register(name, metric)
|
||||
}
|
||||
|
||||
// Register a metric with a given name. Name should be globally unique.
|
||||
/*
|
||||
Register a metric with a given name. Name should be globally unique.
|
||||
*/
|
||||
func (r *Registry) Register(name string, metric metrics.Metric) {
|
||||
r.mutex.Lock()
|
||||
defer r.mutex.Unlock()
|
||||
|
|
@ -97,17 +117,19 @@ func (r *Registry) Register(name string, metric metrics.Metric) {
|
|||
}
|
||||
}
|
||||
|
||||
// Create a http.HandlerFunc that is tied to r Registry such that requests
|
||||
// against it generate a representation of the housed metrics.
|
||||
/*
|
||||
Create a http.HandlerFunc that is tied to r Registry such that requests
|
||||
against it generate a representation of the housed metrics.
|
||||
*/
|
||||
func (registry *Registry) YieldExporter() http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
var instrumentable metrics.InstrumentableCall = func() {
|
||||
requestCount.Increment()
|
||||
url := r.URL
|
||||
|
||||
if strings.HasSuffix(url.Path, ".json") {
|
||||
if strings.HasSuffix(url.Path, jsonSuffix) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set(contentType, jsonContentType)
|
||||
composite := make(map[string]interface{}, len(registry.NameToMetric))
|
||||
for name, metric := range registry.NameToMetric {
|
||||
composite[name] = metric.Marshallable()
|
||||
|
|
@ -126,9 +148,9 @@ func (registry *Registry) YieldExporter() http.HandlerFunc {
|
|||
}
|
||||
|
||||
func init() {
|
||||
DefaultRegistry.Register("requests_total", requestCount)
|
||||
DefaultRegistry.Register("request_latency_logarithmic_accumulating_microseconds", requestLatencyLogarithmicAccumulating)
|
||||
DefaultRegistry.Register("request_latency_equal_accumulating_microseconds", requestLatencyEqualAccumulating)
|
||||
DefaultRegistry.Register("request_latency_logarithmic_tallying_microseconds", requestLatencyLogarithmicTallying)
|
||||
DefaultRegistry.Register("request_latency_equal_tallying_microseconds", requestLatencyEqualTallying)
|
||||
DefaultRegistry.Register("requests_metrics_total", requestCount)
|
||||
DefaultRegistry.Register("requests_metrics_latency_logarithmic_accumulating_microseconds", requestLatencyLogarithmicAccumulating)
|
||||
DefaultRegistry.Register("requests_metrics_latency_equal_accumulating_microseconds", requestLatencyEqualAccumulating)
|
||||
DefaultRegistry.Register("requests_metrics_latency_logarithmic_tallying_microseconds", requestLatencyLogarithmicTallying)
|
||||
DefaultRegistry.Register("request_metrics_latency_equal_tallying_microseconds", requestLatencyEqualTallying)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,29 @@
|
|||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
/*
|
||||
The utility package provides general purpose helpers to assist with this
|
||||
library.
|
||||
|
||||
optional.go provides a mechanism for safely getting a set value or falling
|
||||
back to defaults a la a Haskell and Scala Maybe or Guava Optional.
|
||||
|
||||
optional_test.go provides a test complement for the optional.go module.
|
||||
|
||||
priority_queue.go provides a simple priority queue.
|
||||
|
||||
priority_queue_test.go provides a test complement for the priority_queue.go
|
||||
module.
|
||||
|
||||
test_helper.go provides a testing assistents for this package and its
|
||||
dependents.
|
||||
|
||||
utility_test.go provides a test suite for all tests in the utility package
|
||||
hierarchy. It employs the gocheck framework for test scaffolding.
|
||||
*/
|
||||
package documentation
|
||||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// optional.go provides a mechanism for safely getting a set value or falling
|
||||
// back to defaults.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package utility
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// optional_test.go provides a test complement for the optional.go module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package utility
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// priority_queue.go provides a simple priority queue.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package utility
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// priority_queue_test.go provides a test complement for the priority_queue.go
|
||||
// module.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package utility
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// test_helper.go provides a testing assistents for this package and its
|
||||
// dependents.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package utility
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +1,10 @@
|
|||
// Copyright (c) 2012, Matt T. Proud
|
||||
// All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file.
|
||||
/*
|
||||
Copyright (c) 2012, Matt T. Proud
|
||||
All rights reserved.
|
||||
|
||||
// utility_test.go provides a test suite for all tests in the utility package
|
||||
// hierarchy. It employs the gocheck framework for test scaffolding.
|
||||
Use of this source code is governed by a BSD-style
|
||||
license that can be found in the LICENSE file.
|
||||
*/
|
||||
|
||||
package utility
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue