The cloud security vendor called on Prometheus to provide users with additional safeguards to protect against misconfigurations discovered in the open source monitoring tool.
Aqua Security warned that at least 336,000 exposed Prometheus instances are vulnerable to alarming security flaws that could allow attackers to steal credentials and API keys, as well as launch DOS attacks.
Assaf Morag, director of threat intelligence at Aqua Security, and Yakir Kadkoda, lead security researcher at Aqua, published new research Thursday detailing several vulnerabilities and security flaws found in Prometheus, a popular open source monitoring and alert tool. The research highlighted three significant security risks, including information disclosure, DOS and remote code execution, that are associated with publicly accessible Prometheus servers and exporters.
Despite prior warnings and increased awareness around the risks of unauthenticated Prometheus servers, they continue to be present in user environments. Aqua researchers conducted a Shodan search and found at least 336,000 internet-exposed Prometheus exporters and servers, with the U.S. claiming the top spot among several countries.
"When Prometheus servers or exporters are connected to the public internet without authentication, they introduce a significant risk. Such misconfigurations allow anyone to query the exposed environments to list labels or metrics. Attackers can exploit this access to gather seemingly trivial data and, with the help of secret-scanning tools, uncover sensitive information, including credentials, passwords, authentication tokens, and API keys," Morag and Kadkoda wrote in the blog post.
The researchers discovered several examples of information disclosure risks related to unauthenticated Prometheus servers and warned it is "not a theoretical risk." The blog post highlighted how exposed Prometheus servers and the metrics can reveal sensitive company data such as subdomains, Docker registries and images.
Aqua provided one real-world example and said the company addressed the risk by blocking the vulnerable instance.
"For instance, we discovered an unauthenticated Prometheus instance associated with Skoda, one of the biggest European car manufacturers. In addition to exposing Docker registries and images linked to Skoda, this Prometheus server also revealed subdomains and paths of Skoda through the kube_ingress_path metrics," the blog post said.
Aqua also detailed the risks of DOS attacks. Researchers noted the Go debugging interface, known as pprof package, is commonly used for performance profiling. "Misconfigured Prometheus servers and exporters exposed to the internet provide HTTP access to the pprof endpoint, which is enabled by default in most Prometheus components," the blog post said.
While the /debug/pprof endpoint is designed to help users with remote profiling, the researchers found that attackers could exploit it to conduct DoS attacks, and victim organizations could incur service outages. The risk increases if proper access controls or resource limits are not implemented on Prometheus instances, they warned.
The blog post highlighted prior warnings of /debug/pprof exposures. After conducting an audit in 2020, Cure53, a Germany-based cybersecurity vendor, previously reported the flaw to Prometheus and flagged it again in 2022 on GitHub. However, the issue wasn't addressed.
"Exposing Prometheus, its components, and various exporters to the internet without authentication is considered poor practice. However, the /debug/pprof endpoint introduces a particularly concerning risk: the ability to directly impact the host machine/pod and serve as a vector for DoS attacks. In our view, this vulnerability demands attention and mitigation," the blog post said.
Aqua disclosed its findings to Prometheus and said it received the following response: "Supporting good production practices trumps protecting users from gross misconfigurations."