Development

Prometheus is a popular metric reporting framework. It utilizes a pull-based collection system where exporters expose endpoints and a centralized collection agent periodically requests metric values. This scheme has been validated by prolonged testing and scaled deployments. However, while there are certainly “best practices” in defining metrics, the exporter API does nothing to safeguard you from inefficiencies. In our applications, this means extremely high memory pressure on metric exporters using the Prometheus golang client....

In my daily role I am a software engineer at Union focusing on backend development. We are the core maintainers, and the most frequent contributors, to the open-source Flyte project. Per our documentation it is described as “The Workflow Automation Platform for Complex, Mission-Critical Data and Machine Learning Processes at Scale”. Basically, it is a framework to abstract execution of complex data workflows and the coinciding cloud infrastructure management. This enables teams to efficiently and effectively scale data processing....