Background Imagine you’re writing a library to extend the prometheus python client and you needed to add some dynamic labels specific to the environment. Ideally you would add most of your labels at the collector end and avoid writing extensions.

Requirements for metrics system Multidimensional data model which can be sliced and diced along different dimensions as defined by the service(example: instance, service, endpoint, method) Operational simplicity Scalable data collection and decentralized architecture, so that independent teams can setup independent monitoring servers A powerful query language that leverages the data model for alerting and graphing Client libraries Client libraries take care of all the nitty gritty details like thread-safety, bookkeeping and producing the Prometheus text exposition format in response to HTTP request.