Commit Graph

682 Commits

Author SHA1 Message Date
Lukas Vogel 1321e11795
Merge e7ef5df93d into 76b74e25d5 2024-11-20 14:58:12 +01:00
Ivan Goncharov 76b74e25d5
fix: add very small delay between observations to make test more realistic (and faster) (#1691) 2024-11-20 14:47:38 +01:00
Kemal Akkoyun 7d0a0fbb72
fix: replace fmt.Errorf with errors.New (#1689)
```shell
prometheus/process_collector_darwin.go:28:25: fmt.Errorf can be replaced with errors.New (perfsprint)
var notImplementedErr = fmt.Errorf("not implemented")
```

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2024-11-19 18:13:52 +01:00
Shivanth ae8497940d Add native histograms
Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Apply suggestions from code review

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Fix references and existing tests

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

fmt

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

validation

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Count validation

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Validation

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Space formatting

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Rename SyncMaptoMap -> SyncMapToMap

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Remove exemplars from parameters for constNativeHistogram function

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Update prometheus/histogram.go

Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Shivanth MP <shivanth.metchem@deliveryhero.com>

Review comments

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>

Lint fix

Signed-off-by: Shivanth <shivanth.metchem@deliveryhero.com>
2024-11-13 14:19:40 +01:00
Ivan Goncharov a934c35951
Add: exponential backoff for CAS operations on floats (#1661)
* add: exponential backoff for CAS operations of floats

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* add: some more benchmark use cases (higher contention)

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* fmt: fumpted some files

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* add: license header

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

* add: comment explaining origin of backoff constants

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>

---------

Signed-off-by: Ivan Goncharov <i.morph@gmail.com>
2024-11-11 16:26:17 +00:00
Matthieu MOREL bab92a7743
chore: enable usestdlibvars linter (#1680)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-11-11 15:19:33 +01:00
Arthur Silva Sens 2b11a4ba39
Merge pull request #1673 from imorph/faster_find_bucket
PERF: faster algorithm to discover bucket of an histogram observation
2024-11-08 07:22:56 -03:00
Matthieu MOREL fcfad5c0b9
[chore]: enable perfsprint linter (#1676) 2024-11-08 09:54:31 +01:00
Matt Harbison 3c21cc0ecf process_collector: avoid a compiler warning on macOS (fixes #1660)
The header has a warning when included, with no way to shut it off, and no
alternative to obtain these symbols.  They're technically architecture specific
values, but they aren't different between amd64 and arm64, so combine the
definitions.

Signed-off-by: Matt Harbison <mharbison72@gmail.com>
2024-11-07 17:22:22 -05:00
Ivan Goncharov 78d7a94e46 add: linear search implementation (+ benchmarks)
Signed-off-by: Ivan Goncharov <i.morph@gmail.com>
2024-11-04 20:26:59 +01:00
Ivan Goncharov 6d099da971 add: no skip in short mode for TestSummaryDecay
Signed-off-by: Ivan Goncharov <i.morph@gmail.com>
2024-11-03 14:12:24 +01:00
Ivan Goncharov 7877776aa6 fix: use injected now() instead of time.Now() in summary methods
Signed-off-by: Ivan Goncharov <i.morph@gmail.com>
2024-11-02 10:49:17 +01:00
Jan-Otto Kröpke ece74f8b01
optimize BuildFQName function
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
2024-10-31 14:09:19 +01:00
Arthur Silva Sens e265df7a36
Merge pull request #1647 from prometheus/release-1.20
Merge release 1.20 to main
2024-10-15 09:50:03 -03:00
bwplotka 504ad9bf5c Cut 1.20.5; update comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>

# Conflicts:
#	CHANGELOG.md
2024-10-14 21:56:44 +01:00
bwplotka 584a7ce3d9 Revert "testutil compareMetricFamilies: make less error-prone (#1424)"
This reverts commit c7c7509669.
2024-10-14 21:52:04 +01:00
Ying WANG 9c67f6190c
process_collector: Merge wasip1 and js into a single implementation (#1644)
Signed-off-by: Ying WANG <ying.wang@grafana.com>
2024-10-14 15:32:03 +01:00
Arthur Silva Sens 284ca0ff58
Optinally print OM created lines (#1408)
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Co-authored-by: Arthur Silva Sens <arthur.sens@coralogix.com>
2024-10-14 15:31:13 +01:00
Ying WANG b2ef833442
process_collector: Add Platform-Specific Describe for processCollector (#1625)
* process_collector: Add Platform-Specific Describe for processCollector

Signed-off-by: Ying WANG <ying.wang@grafana.com>

* add changelog entry

Signed-off-by: Ying WANG <ying.wang@grafana.com>

* Address comments

Signed-off-by: Ying WANG <ying.wang@grafana.com>

---------

Signed-off-by: Ying WANG <ying.wang@grafana.com>
2024-10-07 12:08:32 +02:00
Matt Harbison 25bda7ceb5
process_collector: fill in virtual and resident memory values on macOS using optional cgo (#1616)
Unfortunately, these values aren't available from getrusage(2), or any other
builtin Go API.  Go itself doesn't provide a mechanism (like on Windows) to call
into system libraries.  Using a 3rd party package[1] to dynamically call system
libraries was proposed and rejected, to avoid adding to the number of
dependencies.  That leaves using cgo, which is used here when available.  When
not available (either because of cross compiling or explicitly disabling it), a
stub function is linked instead, and the metrics are not exported.  That way,
cross compiling of other platforms is unaffected (and can also still be done
with Darwin too, but at the cost of not exporting these metrics).

Note that building an amd64 image on an arm64 mac or vice-versa is cross
compiling, and will use the stub method by default.  This can be avoided by
setting `CGO_ENABLED=1` in the environment to force the use of cgo for both
architectures.

I'm unsure of the usefulness of the potential adjustment made to the virtual
memory value after calling `mach_vm_region()`.  I've not seen that code get run
with a native amd64 or arm64 image, or with an amd64 image running under
Rosetta.  But that's what the `ps(1)` command does, and I think we should report
what the system tools do.

When I was testing this on a beta of macOS 15 with Go 1.21.13 (the current
minimum support for this module), the amd64 image ran fine under Rosetta, but
the arm64 image immediately printed a message that it was killed, even prior to
the cgo call.  This seems to be a recurring issue on macOS[2][3], and passing
`-ldflags -s` to `go build` avoided the issue.  Go 1.23.1 worked out of the box,
without fiddling with linker flags, so I don't think this is an issue- Go 1.21
is simply too old to support macOS 15, but I thought it was worth noting.  I
supposed we could gate the cgo code with an additional build flag, if anyone is
concerned about this.

[1] https://github.com/ebitengine/purego
[2] https://github.com/golang/go/issues/19841#issuecomment-293334802
[3] https://github.com/golang/go/issues/11887#issuecomment-125694604

Signed-off-by: Matt Harbison <mharbison72@gmail.com>
2024-09-27 17:29:44 +01:00
Arianna Vespri ff60566f1b
Refactor default runtime metrics tests for Go collector so that default runtime metric set autogenerates (#1631)
* Enable autogeneration for default runtime metrics list in collectors tests according to Go version

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Adapt withDefaultRuntimeMetrics function to work regardless of the Go version

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Autogenerate go collector test for go1.23

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Modify gen_go_collector_set.go to please linter and regenerate files

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Simplify gen_go_collector_set.go logic by modifying func computeMetricsList

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Slight simplification of withDefaultRuntimeMetrics func

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Refactor withDefaultRuntimeMetrics with generated default runtime metrics subsets

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

---------

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>
2024-09-23 14:55:38 +01:00
Arthur Silva Sens 39e7c23b95
Merge pull request #1627 from prometheus/release-1.20
Merge Release 1.20 back to main
2024-09-17 10:39:34 +02:00
György Krajcsovits 1e398ccb12 native histogram: Fix race between Write and addExemplar
Follow-up to 1608

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-06 11:01:04 +02:00
Owen Williams 4d20de5bfa Set allow-utf-8 in Format during tests to avoid escaping.
For https://github.com/open-telemetry/opentelemetry-go/pull/5755

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-09-05 15:16:46 -04:00
Lukas Vogel e7ef5df93d registry: drop duplicate checks
In processMetrics drop duplicate checks. If the metricFamily exists we
already have a type check with the call to checkMetricConsistency. The
help string is already checked in the checkDescConsistency albeit only
if pedantic mode is enabled. However this is probably what is desired
anyway.
2024-09-05 15:01:23 +02:00
György Krajcsovits d6b8c8925b Update comments with more explanations
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits 504566f07c Use simplified solution from #1609 for the data race
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits dc8e9a4d8a fix: native histogram: Simplify and fix addExemplar
mdIdx was redundant when len(exemplars)>1, so got rid of it, rIdx
is enough.

Don't compare timestamp of incoming exemplar to timestamp of
minimal distance exemplar. Most of the time the incoming exemplar
will be newer. And if not, the previous code just replaced an
exemplar one index after the minimal distance exemplar. Which had
an index out of range bug, plus is essentially random.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits dc819ceb1b Use a trivial solution to #1605
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
György Krajcsovits e061dfae88 native histogram: use exemplars in concurrency test
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-04 21:10:22 +02:00
Matt Harbison a5e134014f
process_collector: fill in most statistics on macOS (#1600)
* process_collector: fill in most statistics on macOS

Unfortunately, the virtual memory, resident memory, and network stats will
require access to undocumented C functions.  I was warned off of cgo in IRC
because it would then have to be enabled in a bunch of different projects that
use this module, but I already was against it because that would break the
ability to cross-compile.  There is no interface to `dlopen` built into golang.
The `github.com/ebitengine/purego` module looks promising (I can cross-compile
and call these methods), but I'm currently getting unexpected results.  I'll
follow up with that separately if I can get it working, but hopefully this stuff
is pretty uncontroversial.

Tested on macOS 10.14.6 (amd64), macOS 14.6.1 (amd64), and macOS 15.0 (arm64)
by spawning `/usr/bin/ulimit -a -S` and `/usr/sbin/lsof -c $my_process` from
the test exporter process, and `ps -o lstart,vsize,rss,utime,stime,command` from
the shell, and comparing results with the exported metrics.

I can't find documentation for `RLIMIT_AS` on macOS (specifically if it's in
bytes or pages).  It's currently being reported back as `RLIM_INFINITY`, which
seems reasonable, because I've come across reports that the value is ignored
anyway[1].  The bash 3.2 code for the built-in `ulimit` divides the value
reported by `getrusage(2)` by 1024 when printing, as it does for `RLIMIT_DATA`,
which is documented as being bytes in `getrusage(2)`.  The help for `ulimit`
indicates it prints both in kbytes, so it's reasonable to assume this is already
in bytes.

[1] https://issues.chromium.org/issues/40581251#comment3

Signed-off-by: Matt Harbison <mharbison72@gmail.com>

* Update prometheus/process_collector_darwin.go

Co-authored-by: Ben Kochie <superq@gmail.com>
Signed-off-by: Matt Harbison <57785103+mharbison72@users.noreply.github.com>

---------

Signed-off-by: Matt Harbison <mharbison72@gmail.com>
Signed-off-by: Matt Harbison <57785103+mharbison72@users.noreply.github.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-09-04 15:56:01 +01:00
Parth Lawania 850b6c0898
changed the name of all variables with min/max name (#1606)
* changed the name of all variables with min/max name

Signed-off-by: Parth Lawania <parthlawania@gmail.com>

* removed predeclared ignore condition for min and max identifiers

Signed-off-by: Parth Lawania <parthlawania@gmail.com>

---------

Signed-off-by: Parth Lawania <parthlawania@gmail.com>
Co-authored-by: Parth Lawania <parth.lawania@super.money>
2024-09-02 07:41:40 -03:00
SuperQ c85ae6b1fe
Update supported Go versions
Now that 1.23 is out, update the supported version matrix to Go 1.21
through 1.23. This allows us to start using `log/slog`.
* Update generated tests.

Signed-off-by: SuperQ <superq@gmail.com>
2024-08-31 22:47:31 +02:00
SuperQ e080e48d5b
Update linting
* Update golangci-lint from upstream prometheus repo.
* Ignore min/max in predeclared.
* Fix `Errorf()` use.

Signed-off-by: SuperQ <superq@gmail.com>
2024-08-30 18:09:51 +02:00
Arthur Silva Sens dbf72fc1a2
Merge pull request #1593 from prometheus/release-1.20
Merge release-1.20 back to main
2024-08-25 14:50:40 -03:00
Manuel Rüger f7f8f3a1e2 fix: Unset Content-Encoding header when uncompressed
Fixes: #1595
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2024-08-23 18:53:42 +02:00
bwplotka f2dd7b35fd Use pedantic registry in other places too, to double check.
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-08-20 11:44:35 +01:00
Arthur Silva Sens 261fe84cd4 bugfix: Pass network metrics to processCollector's Describe() function
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-08-20 11:44:35 +01:00
Arthur Silva Sens 5bf3341b66 Use NewPedanticRegistry in Process' Collector tests
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-08-20 11:44:35 +01:00
bwplotka 8514bd7695 gocollector: Tiny fix for help message with runtime/metrics source.
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-08-16 09:58:24 +01:00
Bartlomiej Plotka 7ce508988a
gocollector: Attach original runtime/metrics metric name to help. (#1578)
This helps in filtering capabilities and generally in discovery/knowledge sharing.

Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-08-14 12:25:55 +00:00
Bartlomiej Plotka 062300ede9
Remove go_memstat_lookups_total; added runtime/metrics calculation to memstat metric's help. (#1577)
* Removeed go_memstat_lookups_total which was always set to 0; added runtime/metrics info to memstat metric helps.

I know we ideally should not remove any metric from default list, but
this one is always zero, so let's save everyone's money.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Update prometheus/go_collector.go

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-08-14 12:11:51 +00:00
Arianna Vespri 3ad272204b
Add default Go runtime metrics for /gc/gogc:percent, /gc/gomemlimit:bytes, /sched/gomaxprocs:threads (#1559)
* Add go_gomaxprocs, go_gogc_percent and go_gomemlimit to the default Go runtime metrics

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Move newly added metrics out of base metrics and into goCollector

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Rethink struct for newly added metrics, adapt and add tests

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Simplify new metrics reading

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Correct loop, add debugging lines

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Make goRuntimeEnvVarsMetrics function Go version dependent

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Fix go mod

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Remove debuggin line

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Move default runtime metrics into the runtime metrics flow, change tests accordingly

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Go version expected default runtime metrics map for tests

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* 1.21 update.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments on Arianna's PR.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Use default GoCollector func in test

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

---------

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2024-08-09 20:27:25 +01:00
Bartlomiej Plotka 0ce6d719bc
go collector: add default metrics acceptance tests; adding more context to HELP (#1568)
* go collector: add default metrics acceptance tests; adding more context to HELP

The context and details for help were possible thanks to @vesari research, thanks for that!

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Update prometheus/go_collector.go

Co-authored-by: Arianna Vespri <36129782+vesari@users.noreply.github.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Arianna Vespri <36129782+vesari@users.noreply.github.com>
2024-08-01 09:24:31 +01:00
Huw Jones 10f4161c7d
process_collector: collect received/transmitted bytes (#1555)
Signed-off-by: Huw Jones <huw@pexip.com>
2024-07-17 12:19:40 +02:00
Ben Kochie fec6b222d9
Update Go tests (#1562)
Add new metric that appeared in 1.22.5 and 1.21.12.
* Improve test output with `cmp.Diff`.

Signed-off-by: SuperQ <superq@gmail.com>
2024-07-15 19:28:14 +02:00
Steven Swartz 1a5f005c91
Allow creating constant histogram and summary metrics with a created timestamp (#1537) 2024-07-01 17:35:47 -03:00
Ben Kochie bccd68204b
Remove inlined upstream code (#1539)
Avoid having to maintain inlined code by using Go modules for
`github.com/kylelemons/godebug`.

Signed-off-by: SuperQ <superq@gmail.com>
2024-06-20 13:05:41 +02:00
Manuel Rüger f08b10c5bc
feat: Support zstd compression (#1496)
* feat: Support zstd encoding

This allows endpoints to respond with zstd compressed metric data, if
the requester supports it.

I have imported a content-encoding parser from
https://github.com/golang/gddo which is an archived repository to
support different content-encoding headers.

Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Update prometheus/promhttp/http.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Update prometheus/promhttp/http.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Update prometheus/promhttp/http.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Integrate review comments

* String typed enum

Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Test with gzip compression

Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Update prometheus/promhttp/http.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Reorder error handling

Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Apply suggestions from code review

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manuel Rüger <manuel@rueg.eu>

* Include review suggestions

Signed-off-by: Manuel Rüger <manuel@rueg.eu>

---------

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-06-07 15:56:19 +01:00
Björn Rabenstein 542f7e6c6e
Merge pull request #1471 from fatsheep9146/native-histogram-exemplar
add native histogram exemplar support
2024-05-16 12:22:21 +02:00