Start your free 14-day ContainIQ trial

Grafana vs. Prometheus | Tutorial and Setup

Prometheus and Grafana are popular open-source technologies for metrics, monitoring, and visualization. In this article, we provide a tutorial for using each and we explain how to best use these tools together.

July 28, 2022
Ashish Choudhary
Software Engineer

Prometheus, a popular open-source project, is used by thousands of organizations as a time-series database and monitoring system. Prometheus has a number of benefits including a large number of integrations, efficient storage, powerful alerting with Alertmanager, as well as the ability to run queries using PromQL.

While Prometheus does have some visualization capabilities, it is most often paired with another tool for visualization, like Grafana. Grafana, originally launched as an open-source project, is a popular visualization toolset and a popular integration with Prometheus.

Both of these tools have their own set of benefits, use cases, and capabilities. And for companies using Kubernetes, the leading container orchestration system, it makes a lot of sense to use these tools together.

In this article, we highlight both Prometheus and Grafana as tools to monitor workloads running on Kubernetes. We start by highlighting the setup and implementation process for each tool and then finish by discussing best practices for using these tools together.

Grafana and Prometheus Feature Comparison

At a Glance
OPEN-SOURCE
Yes
Yes
DEVELOPED
2014
2012
IMPLEMENTATION
Self-Hosted, or Grafana Cloud
Self-Hosted, or Managed (ex AWS, GCP)
IDEAL FOR
Visualization
Metrics collection

Getting Started with Grafana

Grafana is an open-source data visualization tool that helps users understand complex data through some great visualization. It comes with capabilities to query, visualize, and alert on metrics from different data sources, such as Prometheus (which I’ll discuss more in-depth later in this article), MySQL, MSSQL, PostgreSQL, InfluxDB, Graphite, and Elasticsearch. Grafana can also pull data from cloud-hosted services like AWS CloudWatch, Google Cloud Monitoring, and Azure Monitor.

Running Grafana on your local machine is as easy as executing a <terminal inline>docker run<terminal inline> command or running the <terminal inline>helm install grafana bitnami/grafana<terminal inline> Helm command. For example, you can use the following <terminal inline>docker run<terminal inline> command to start Grafana on your local:


docker run -d -p 3000:3000 grafana/grafana

Verify that the Grafana container image is running using the <terminal inline>docker ps<terminal inline> command.

$ docker ps

Container ID Image Command Created Status Ports Name
fcbd1815b4a4 grafana/grafana "/run.sh" 3 minutes ago Up 3 minutes 0.0.0.0:3000->3000/tcp dreamy_brattain

Now browse the URL <terminal inline>http://localhost:3000<terminal inline> and you will be redirected to the Grafana login page.

Grafana login
Grafana login


Enter default credentials admin/admin, and you will see the following screen with options given to create dashboard and data sources.

Grafana home
Grafana home


To verify further, you can even add a data source named as <terminal inline>TestData DB<terminal inline> which is a fake data source that comes with Grafana. On the sidebar, click Configuration, then click Add data source to add <terminal inline>TestData DB<terminal inline> as a data source.

Grafana configuration
Grafana configuration


Click Save & test. Under the Dashboards tab, you can import the Simple Streaming Example.

Grafana configuration
Grafana configuration


Navigate to your dashboard home and choose the Simple Streaming Example dashboard. You will see the following dashboard.

Grafana dashboard
Grafana dashboard


Grafana Features

Some of Grafana’s highlighted features include:

  • Data analytics through dashboards
  • Application monitoring
  • Integration with variety of data sources, ie, Prometheus, MySQL
  • Visualize system resources such as CPU and memory
  • Create, edit, and share dashboards
  • Alerting capabilities
  • Query your metrics

Grafana Support and Pricing

Grafana currently comes with three different support models:

  1. Grafana Open Source: The open-source and self-managed offering of Grafana. You have to manage your instances yourself, and if you run into any issues, you have to raise the problem with the Grafana community. However, if you want to contribute to the community, you can raise issues and fix them.
  2. Grafana Cloud: This is the fully managed service offered by Grafana. With Grafana Cloud, you don’t have to worry about managing infrastructure for the Grafana instances. Grafana Cloud comes in three flavors: Free ($0), Pro ($49 +usage), and Advanced (custom pricing).
  3. Grafana Enterprise: This offering caters to company-specific needs around data and privacy. It comes with features such as advanced security and compliance, 24/7 hands-on support teams, and access to enterprise plugins like New Relic, AppDynamics, Oracle, Dynatrace, ServiceNow, and Datadog.
K8s Metrics, Logging, and Tracing
Monitor the health of your cluster and troubleshoot issues faster with pre-built dashboards that just work.
Start Free Trial Book a Demo

Getting Started with Prometheus

Prometheus is an open-source metrics-based monitoring and alerting solution, and the de facto standard when it comes to monitoring Kubernetes. You can install Prometheus to your Kubernetes cluster with one command: <terminal inline>helm install Prometheus bitnami/kube-prometheus<terminal inline> using Helm. Prometheus is often self-hosted, but it has become increasingly popular to used a managed service.

In Prometheus terminology, the process to gather data from the cluster or application is called scraping. It scrapes the metrics by using an HTTP connection on a predefined port to get metrics from your applications and stores the data in time-series format in its database.

You can run query data using its PromQL query language. The Prometheus server uses a pull-based approach to collect metrics data from applications.

Prometheus has the following core components:

  • Server: Collects and stores metrics.
  • Alertmanager: Receives alerts from various data sources and processes them. Alerts are typically sent using Slack or email.
  • Client libraries: Supports client libraries for different languages that can be used to instrument your applications.

Prometheus and Grafana Feature Comparison

Prometheus Grafana
Some options to view metrics and basic trends. Maximum data visualization options using charts and graphs.
Provides storage for time-series data. Doesn't have its own storage.
Pull-based, scrapping HTTP endpoints. Provides plugins to get data from different data sources.

Grafana and Prometheus: Match Made in Heaven

However, capable as Grafana and Prometheus are on their own, a common use case for each is to work in tandem. Prometheus collects and stores data, and Grafana offers developers an elegant way to visualize that data, making it easier to analyze and act on.

In this section, you’ll be installing Prometheus and Grafana on an existing Kubernetes cluster for monitoring the cluster. You’ll use a minikube local Kubernetes cluster for this setup.

First, install Prometheus on your local Kubernetes cluster using Helm. You’ll use the <terminal inline>helm repo add bitnami https://charts.bitnami.com/bitnami<terminal inline> command to add a Bitnami Helm repository.

For this demo, you’ll install Prometheus with its default configuration using the <terminal inline>helm install prometheus bitnami/kube-prometheus<terminal inline> command. It takes a while for all the pods to come into RUNNING status. After the install, you should see the following pods in the output of <terminal inline>kubectl get all<terminal inline> command:

$ docker ps

Name Ready Status
pod/alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running
pod/prometheus-kube-prometheus-operator-86bbf564b8-rzx2h 1/1 Running
pod/prometheus-kube-state-metrics-78667fbf77-wj4w4 1/1 Running
pod/prometheus-node-exporter-8zpd4 1/1 Running
pod/prometheus-prometheus-kube-prometheus-prometheus-0 2/2 Running

In this default setup, you cannot access Prometheus outside the cluster as the service type is <terminal inline>ClusterIP<terminal inline>. To access Prometheus from outside the cluster, execute the following commands:


kubectl port-forward --namespace default svc/prometheus-kube-prometheus-prometheus 9090:9090

After that you can access it via <terminal inline>http://127.0.0.1:9090/<terminal inline>.

Next install Grafana using the <terminal inline>helm install grafana bitnami/grafana<terminal inline> command. You should see the following output on your terminal:


NAME: grafana
LAST DEPLOYED: Wed Jun 16 22:58:46 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **

1. Get the application URL by running these commands:
 echo "Browse to http://127.0.0.1:8080"
 kubectl port-forward svc/grafana 8080:3000 &

2. Get the admin credentials:
 echo "User: admin"
 echo "Password: $(kubectl get secret grafana-admin --namespace default -o jsonpath="{.data.GF_SECURITY_ADMIN_PASSWORD}" | base64 --decode)"

After the Grafana pod is created, you can run the <terminal inline>kubectl port-forward<terminal inline> command as mentioned in the output and navigate to <terminal inline>http://127.0.0.1:8080<terminal inline>.

In this setup, default admin/admin credentials don’t work, as the Helm chart generates a random password stored in Kubernetes secret. You can retrieve it using the <terminal inline>kubectl get secret<terminal inline> command as mentioned in the output. Log in using this password. At this point, both Grafana and Prometheus are up and running inside your Kubernetes cluster.

Prometheus has already started collecting metrics from your Kubernetes cluster, but you haven’t added it as a data source in Grafana. Let’s do that.

The previous section about Grafana already covered how to add a data source in Grafana, so the process is not much different for Prometheus. However, while adding Prometheus as a data source, you have to provide its server URL. Both Prometheus and Grafana are running in the same cluster, so use the <terminal inline>prometheus-kube-prometheus-prometheus.default.svc.cluster.local<terminal inline> DNS name mentioned in the following screenshot as server URL and port <terminal inline>9090<terminal inline>.

Grafana prometheus
Grafana prometheus


For visualization, you can also import ready-made Grafana dashboards from its vast repository of dashboards. Use a dashboard named <terminal inline>Cluster Monitoring<terminal inline> for Kubernetes and have ID <terminal inline>10000<terminal inline> for this demo.

Grafana dashboard import
Grafana dashboard import


After the successful import, you should see the following dashboard:

Grafana dashboard import
Grafana dashboard import

Conclusion

While Prometheus is good at scraping metrics and storing time-series data, Grafana scores in visualization using charts and graphs. Together, they’re a strong combo for your team’s ability to access and work from data that’s relevant and thorough. At this point, you should feel armed with some practical examples for combining these tools to monitor your Kubernetes clusters.

Observability and application monitoring are essential components of the DevOps philosophy. We tend to interchange these terms when used together, but in reality, they have different meanings. While observability helps you understand why your application/system is not working, monitoring tells how it is working. When we talk about monitoring distributed systems like Kubernetes, it’s hard to know if something is up or down through traditional monitoring.

Start your free 14-day ContainIQ trial
Start Free TrialBook a Demo
No card required
Ashish Choudhary
Software Engineer

Java, BigData, and Cloud enthusiast, Ashish likes learning new emerging technologies and trends in the DevOps space. Ashish has spoken about containerization at a variety of conferences including Free and Open-source Software Developers' European Meeting (Fosdem) and GitHub Satellite India. He has a Bachelor of Technology degree in Computer Science from SKIT Jaipur.

READ MORE