Prometheus

Term 1 · Mar-May · 2 projects

• Improving Documentation for Prometheus and OpenTelemetry Interoperability

  LFX Mentorship · Listed in the official cohort

  @arthursensSource IssueLFX portal

  Prometheus and OpenTelemetry are commonly deployed together, yet many users struggle to understand how the two systems interoperate — especially around concepts such as resource attributes, label mapping, attribute promotion, and recommended integration patterns. Prior UX research identified documentation gaps as a primary source of confusion, and ongoing community discussions continue to surface similar questions. While both projects provide extensive documentation, guidance is often fragmented, highly technical, or lacks practical end-to-end explanations for real-world usage. This mentorship focuses on improving the clarity, usability, and consistency of documentation that explains how Prometheus and OpenTelemetry work together. The mentee will analyze existing documentation across both projects, identify high-impact gaps or friction points, and collaborate with mentors to design and deliver meaningful documentation improvements upstream. The project is intentionally exploratory and iterative. Part of the mentorship is learning how to evaluate documentation quality, prioritize improvements, and define ways to measure the impact of documentation changes made during the program.

  • Prior experience with technical writing or clear motivation to pursue a Tech Writing career.
  • Familiarity with Git and contributing via pull requests.

• Start scraping immediately on startup without waiting for WAL replay to finish

  LFX Mentorship · Listed in the official cohort

  @codesomeSource IssueLFX portal

  When a large Prometheus is running in HA mode (i.e. 2 identical Prometheus scraping same targets) and it needs to be restarted (for upgrades, etc), there can be a temporary gap in queries when the second Prometheus is restarting. This is because the first Prometheus did not scrape any metrics when it was restarting (this is the gap), and when second Prometheus is restarting, only the first Prometheus is serving queries. To mitigate this, we want to start scraping targets as soon as Prometheus starts up, without waiting for it to complete the WAL replay and restore the old in-memory state. However, querying is still only enabled after WAL replay is over, but there won't be gaps in scraped data anymore. This project includes careful designing of low level technical challenges, including ensuring the correct handling of WAL records and seamlessly stitching together the new scraped data and the data restored from WAL without a lot of memory overhead.

  • Basic understanding of Go.
  • Comfortable diving deep into low level system code.
  • Familiarity with unit testing.

Term 3 · Sep-Nov · 3 projects

• Prometheus Native Summaries

  LFX Mentorship · Listed in the official cohort

  @bwplotkaSource IssueLFX portal

  Prometheus Summaries are less but still widely used metric type. Currently, similar to classic histogram, a single summary is composed of multiple counter series. The native summary idea proposes to structure the summary type metric samples into a complex sample, similar to native histograms. As a result, bringing meanigful storage efficiency and transactionality gains. This project offers an opportunity to introduce relatively complex database change across Prometheus scraping, WAL, TSDB and querying; considering the precedense of native histograms. This project will allow mentee to learn about Prometheus, generally database internals as well as observabvility needs.

  • Go
  • Prometheus

• Prometheus Remote Write 2.0 stability

  LFX Mentorship · Listed in the official cohort

  @jmichalek132Source IssueLFX portal

  In the past few quarters, a lot of work has gone into the new Remote Write 2.0 (PROM-35) proposal, and a new spec has been successfully established But there is still a lot of work that needs to be done to declare it stable, in terms of stability and performance in Prometheus, and general adoption from the wider Prometheus ecosystem.

  • Go
  • Prometheus

• Prototyping Prometheus for exploratory use cases

  LFX Mentorship · Listed in the official cohort

  @ArthurSensSource IssueLFX portal

  Through a LFX Mentorship done in March-May, we performed UX research focusing on different ways Prometheus could deal with OpenTelemetry's Resource Attributes. One of the insights that this research showed us is that Prometheus and OpenTelemetry were primarily designed for different use-cases. While Prometheus focuses on monitoring metrics that the user knows that exist, one of OpenTelemetry's goals is to enable exploration of your data while the user doesn't really know what he/she is looking for. This mentorship will take the next steps on those research findings, exploring ideas about how Prometheus can enable the "Exploration" use case.

  • Interest or currently working in UX Research and Design.
  • Familiarities with databases and querying.
  • Being comfortable to talk with End-Users in English.

Term 1 · Mar-May · 2 projects

• Get PrometheusRemoteWriteReceiver in OTel-Collector to Alpha Maturity

  LFX Mentorship · Listed in the official cohort

  @ArthurSensSource IssueLFX portal

  In recent discussions with the team, we decided that Prometheus won't be exporting its data with the OTLP format, however, Prometheus is still committed to have good import/export compatibility with OpenTelemetry. Last year Prometheus release the second version of its Remote-Write protocol, which translates a lot better with the OTLP format and the team started working on a PRW receiver in the collector-contrib project. This project is about getting this component into the finish line and publish it as an stable component in the collector.

  • Prometheus Remote-Write
  • OTLP
  • Go.

• UX Research: How users expect to use OTLP Resource Attributes in Prometheus

  LFX Mentorship · Listed in the official cohort

  @ArthurSensSource IssueLFX portal

  In the last year Prometheus has tackled and solved many UX problems that OTel users had when sending OTLP data to Prometheus. One challenge that remains unsolved is how do users expect to use OTLP Resource Attributes in Prometheus. This project is about conducting a UX research that explores the main problems users are facing today with the current state of Resource Attributes and Prometheus and coming up with ideas how to solve them.

  • Interest or currently working in UX Research and Design.
  • Familiarity with databases and querying.
  • Being comfortable to talk with End-Users in English.

Term 3 · Sep-Nov · 2 projects

• Enhance Prometheus Benchmark Suite

  LFX Mentorship · Listed in the official cohort

  @bborehamSource IssueLFX portal

  PromBench is the automated system for benchmarking Prometheus, intended to be a realistic comparison for users. It sets up a Kubernetes cluster, configures Prometheus to monitor many fake services, and simulates users querying the data.

  • Go
  • Prometheus

• Prometheus Remote-Write v2 support in otel-collector's prometheusremotewriteexporter.

  LFX Mentorship · Listed in the official cohort

  @ArthurSensSource IssueLFX portal

  Prometheus recently announced the second version of it's remote-write protocol, with support for new features on top of performance and cost savings. The work to be done is contribute support for this new protocol in the OpenTelemtry-Collector-Contrib repository, more specifically to the prometheusremotewriteexporter component.

  • Go
  • Prometheus
  • OpenTelemetry.

Term 2 · Jun-Aug · 2 projects

• Mark Out-of-order ingestion as stable

  LFX Mentorship · Listed in the official cohort

  @bborehamSource IssueLFX portal

  Prometheus is known by not accepting out-of-order samples during ingestion, but recently (2 years ago) support was added behind a feature-flag. Since then, many improvements have been made to out-of-order ingestion and it has become one of the requirements for OTLP ingestion in Prometheus. We want to deliver Prometheus 3.0 in a few months, and that requires marking out-or-order ingestion as a stable feature. In this project we will clean up several smaller issues around out-of-order ingestion, hopefully marking it as stable by the end of the mentorship.

  • Go
  • familiarity with Prometheus TSDB.

• Prometheus Remote Write Benchmarking Capabilities

  LFX Mentorship · Listed in the official cohort

  @cstyanSource IssueLFX portal

  Prometheus remote write allows users to send their metrics to other time series databases. Though the Prombench tool has existed for a number of years, it has never been extended to support performance testing of Remote Write in a realistic production like environment. With the upcomming Remote Write 2.0 changes to both the underlying implementation as well as the wire format, the need for benchmarking of remote write beyond static Go bechmark tests has increased.

  • Go
  • some familiarity with Prometheus or metrics
  • basic Docker knowledge

Term 1 · Mar-May · 1 project

• Client_golang CI/CD improvements

  LFX Mentorship · Listed in the official cohort

  @ArthurSensSource IssueLFX portal

  Prometheus' client_golang is the Prometheus SDK for metrics instrumentation for Go applications. Client_golang promises full support for the 3 latests major Go versions, and for this task a lot of manual effort is executed by the community. Client_golang could receive several improvements around its CI/CD pipelines and automation: Golang version upgrades requires autogenerating go files that Go Collector uses to collect Go runtime metrics. Unit tests need to be run for the 3 latest Go versions, and running tests locally with different Go versions is hard at the moment. We can explore locally reproducible CI/CD. The changelog of new releases still requires a lot of manual work, like going through commit history and hand-picking commits that need to be advertised. We want to explore automation around semantic conventional commits that allows Changelog/Release automation.

  • Go
  • Shell
  • CI/CD

Term 0 · 2020 Q3–Q4 · 1 project

• Add various post processing steps in query API after PromQL execution

  CommunityBridge · Listed in the official cohort

  @bwplotkaSource IssueCommunityBridge

  The results from the query/query_range are often unordered w.r.t. the label values and sample value. Also, there is no limit on how many series the endpoint can return. In this project, we want to extend those query APIs to include options for post processing the output to do the following: Support nested sorting (ascending/descending) of time series based on (1) Label values (2) Sample values for /query endpoint. Support limiting the number of time series returned by query/query_range endpoint. If time permits, expore how we can do nested sorting of time series for query_range API.

  • Go
  • PromQL

Term 0 · 2020 Q2 · 2 projects

• Persist Retroactive Rule Reevaluations

  CommunityBridge · Listed in the official cohort

  @bwplotkaSource IssueCommunityBridge

  Right now one of the biggest issues with recording rules is that data is only available since the rule was created. Which means any dashboards that use the recording rule will not have data prior to the recording rules create time. We can already reevaluate queries on old data, but we should be able to persist that for a certain window from [Oldest, Now)

  • Go

• Remote Write WAL Pointer + Other Improvements

  CommunityBridge · Listed in the official cohort

  @cstyanSource IssueCommunityBridge

  Remote write has gone through a rewrite in the last year, but there are still open improvements and feature additions that can be explored. For example, because remote write now sends data via Prometheus' WAL, the pointer for each remote write queue into the WAL should be persisted across restarts. The addition of new record types, or a new remote write specific WAL, so that new data not currently part of the WAL can be remote written coulda also be investigated.

  • Go
  • database and dist. systems nice to have.

Term 0 · 2020 Q1 · 5 projects

• Benchmarks for TSDB

  CommunityBridge · Proposed, acceptance not recorded

  @krasi-georgievSource Issue

  The TSDB module used in Prometheus doesn’t have proper benchmarks yet, which means we cannot see the potential impact of the changes we are introducing. The idea is to build some automated benchmarking which can be added to the CI pipeline.

  • CI
  • Go
  • Kubernetes

• e2e test SD mechanisms

  CommunityBridge · Proposed, acceptance not recorded

  @gouthamveSource Issue

  We're not really excellent in the correctness / bug-free-ness department yet, partially because certain key components either lack tests or you'd have to run them in a real-world scenario for a while to discover certain bugs. I'm especially looking at our under-tested service discovery mechanisms here that require e.g. a Zookeeper or Consul as a dependency to test them for real. It'd be cool to have a test environment that runs a Prometheus release end-to-end (with different SD mechanisms) for a while and checks the results (evaluated expressions, alerts, etc.) for sanity.

  • Go

• Persist retroactive rule evaluations

  CommunityBridge · Proposed, acceptance not recorded

  @codesomeSource Issue

  We can already reevaluate queries on old data, but we should be able to persist that for a certain window from (Oldest, Now).

  • Go

• PromQL/Rule formatting tool

  CommunityBridge · Proposed, acceptance not recorded

  @codesomeSource Issue

  Like "gofmt" for Go, we ought to have a "promfmt" for Prometheus since we have a syntax tree. The idea being that the system produces uniform style that minimizes deviation and learning curve. Care should be taken to preserve comments.

  • Go

• Various React UI improvements and filtering label values API with matchers

  CommunityBridge · Listed in the official cohort

  Source