Cloudwatch Dashboards

Overview

Amazon Cloudwatch makes it possible to search Login.gov logs and AWS metrics to create visualizations or display statistics. Cloudwatch Insights is the tool that makes it possible to query logs. Information on how to build Cloudwatch Insights queries can be found in the AWS documentation.

Login.gov’s CloudWatch Dashboards can be found by navigating to the Cloudwatch service in the AWS Console and selecting “Dashboards” in the left pane. From here you will see a list of available dashboards and there is functionality for searching dashboards.

Experimental vs Terraformed Dashboards

The source for CloudWatch Dashboards should be maintained in Terraform code under version control in the identity-devops repo.

Creating dashboards can be tricky and require some experimentation that are impeded by the long feedback loop of making and applying changes to terraform code. As a result, Login.gov supports experimental dashboards. These are ephemeral dashboards which can be used to create source for long-lived dashboards.

Terraformed dashboards are distinguished from experimental dashboards by the prefix in the dashboard name. A terraformed dashboard will have a prefix which is an environment name. For example, a terraformed dashboard may be named prod-sample-dashboard. Conversely, an experimental dashboard will use a name that identifies the author. For example, my-example-dashboard.

Once an experimental dashboard reaches the desired state it should be converted into a terraformed dashboard. The identity-devops repo has tooling for converting dashboards to terraformed dashboards.

Given an experimental dashboard named my-sample-dashboard, run the following in the identity-devops repo to create prod-my-sample-dashboard, staging-my-sample-dashboard, etc.:

aws-vault exec prod-power -- bin/copy-cloudwatch-dashboard -i my-sample-dashboard

This will create a terraform file in the devops repo for the dashboard. When this file is checked in and applied by terraform a dashboard matching the experimental one will be created for all environments.

Technical Information

For technical information on creating dashboards and alerts, see Monitoring and observability: CloudWatch queries, alarms and dashboards.