Delivery dashboard
Create a public-facing dashboard with sprint and delivery metrics.
Summary details
| Field | Value |
|---|---|
Deliverable status | In Progress |
Link to GitHub issue | |
Key sections |
Overview
Summary
What: Release a basic public dashboard that allows internal and external stakeholders to track key sprint and delivery metrics for the project.
Why: Publishing our metrics publicly encourages us to define and track key measures for each deliverable and builds public trust in the approach we’re taking. Tracking metrics around sprint and delivery, in particular, will help us scope and plan future deliverables more accurately by enabling us to understand team capacity and delivery velocity.
Who:
Project maintainers who want to monitor team capacity and delivery velocity
Internal and external stakeholders who want to monitor our progress toward key deliverables
Business value
Problem
Establishing and actively tracking operational metrics is critical to the long-term success of software projects. Often projects will start strong, quickly delivering features that add value and improve the user experience of their products. Without the accountability and insight that operational metrics provide, though, delivery velocity and value can decrease over time due to a number of factors, such as technical debt and deferred maintenance. And without a way to meaningfully engage with these metrics, such as a dashboard, trends or signals in the data can easily go undetected for extended periods of time, making it harder to address the underlying cause.
Value
Establishing robust operational metrics and publishing them to a public-facing dashboard are important steps in creating a data-driven culture that is focused on delivery. Defining clear operational metrics helps align the team around shared delivery goals and strategies for measuring progress. Making these metrics publicly available encourages us to monitor our operational data and respond to changes in performance.
By setting up the appropriate foundation to collect and publish data for delivery metrics, we also make it easier to publish future, more important, metrics about operational and program outcomes.
Goals
Ensure that we are collecting the data needed to calculate proposed metrics
Begin to create the infrastructure required for more advanced data analytics
Begin to establish a framework to measure project impact and success
Build public trust in our approach through transparency around core metrics
Improve our ability to understand and manage team capacity and delivery velocity
Non-goals
We're not trying to measure the performance of the team using these metrics. They are meant to be signals that inform planning and requests for additional capacity.
The dashboard that is an output of this deliverable may not be the final or only dashboard that members of the public are meant to access long-term. This deliverable mainly proves our ability to publish metrics in a public dashboard, but additional design and research work is needed for the long-term dashboard.
User stories
As a project maintainer, I want:
to be able to provide input on which metrics we publish and how they are presented in the dashboard, so that I can use the dashboard to meaningfully plan and manage delivery on the project.
to be able to provide additional details about the metrics included in the dashboard, so that other stakeholders know how to interpret these metrics correctly in the context of the project.
to be able to easily add new charts and metrics to the dashboard, so that we can continue to take a data-driven approach to the project and publicly report on the measures of success defined for each deliverable.
As an internal stakeholder, I want:
the dashboard's key insights to be easy to understand at a glance, so that I don't need to spend a lot of time learning how to interpret the metrics correctly.
to be able to access the dashboard from the HHS network, so that I can easily monitor our metrics when I'm in the office.
to understand how the metrics are calculated, so that I can explain them to other stakeholders.
the data behind the dashboard to be updated regularly, so that I'm not sharing outdated information with other stakeholders.
As a member of the public, I want:
to be able to easily navigate between the dashboard and other project resources (e.g. Simpler.Grants.gov or GitHub), so that I don't have to bookmark or remember the links for each resource separately.
all of the key project dashboards to be accessible in a central location, so that I don't have bookmark or remember multiple links to view all of the metrics.
to know when the data was last refreshed, so that I can understand how up-to-date the information in the dashboard is.
As an open source contributor, I want:
to have access to the data behind the dashboard, so that I can explore and analyze the data myself.
to access the source code behind the dashboard, so that I can use this dashboard for my own project or submit code to improve upon the existing dashboard.
to understand how my contributions are reflected in the delivery metrics, so that I know what changes to expect when I contribute code or fix a bug.
Definition of done
Following sections describe the acceptance criteria for this deliverable.
Must have
Nice to have
Not in scope
List of functionality or features that are explicitly out of scope for this deliverable.
Additional metrics: While the goal of this deliverable is to create the infrastructure for publishing operational and program metrics for the public, only sprint and delivery metrics are in scope for this initial deliverable. Additional metrics or dashboards will need to be added in future deliverables.
Additional ETL pipelines: While other aspects of the project, e.g. transforming the current transactional data model to the new transactional data model, will likely require a similar ETL pipeline, this deliverable is only focused on building out the data pipeline needed for sprint and delivery metrics. That being said, the selection and implementation of an ETL infrastructure for this deliverable should take these future needs and use cases into consideration.
Email campaign: While the dashboard will be publicly available, this deliverable does not include sending an email campaign to public stakeholders publicizing the dashboard. Email communications around the dashboard will likely be scoped into a future deliverable, for example, once program-related metrics are added to the dashboard (e.g. place of performance data).
Measurement
Proposed metrics
Number of unique dashboard visitors
Total number of dashboard views
Dashboard availability
Number of failures to load data
Destination for live updating metrics
The metrics described above will not be immediately available in the dashboard we're publishing in this deliverable, but a future deliverable will involve adding those metrics to a new page in the dashboard. In the interim, we will share these metrics in the Analytics section of our public wiki.
Open questions
Logs
Change log
Major updates to the content of this page will be added here.
| Date | Update | Notes |
|---|---|---|
4/5/2024 | Added change log and implementation log | This is part of the April onsite follow-up |
4/9/2024 | Incorporated comments on change request |
|
4/10/2024 | Moved part of the content of this spec into a technical spec | Moved the following into this spec:
|
5/6/2024 | Moved deliverable status to "In Progress" |
Implementation log
Use this section to indicate when acceptance criteria in the "Definition of done" section have been completed, and provide notes on steps taken to satisfy this criteria when appropriate.
| Date | Criteria completed | Notes |
|---|---|---|
Last updated