# Welcome
Welcome to the public wiki for the Simpler.Grants.gov initiative.
This website is a shared resource for internal Simpler.Grants.gov team members, external collaborators, and community stakeholders.
### Interested in joining the Simpler Grants community? Start here:
{% content-ref url="/pages/FTAGGyvkcYgSFjR8T5S2" %}
[Why open source?](/get-involved/get-involved)
{% endcontent-ref %}
# Why open source?
At Simpler.Grants.gov, we are committed to transparency and collaboration. We embrace open-source practices to ensure all our software tools are accessible for public observation, input, and reuse.
We firmly believe that the most effective solutions are crafted through open collaboration. We are eager to engage with a community of grant-making professionals, applicants, and technology enthusiasts. Your feedback and contributions are not only welcome but integral to our mission. Together, we can simplify the grant-making process and make it more accessible to everyone.
### Contribute
Your contribution will help make the Simpler.Grants.gov experience better for everyone.
### Who we're looking for
* Contribute to coding and development
* Contribute to feedback and discussion
* Join the conversation across our wiki and chat to learn more.
### How to participate
| Interested in... | How to Get Involved |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Providing coding or development support |
Contribute code to GitHub - Contributing Guidelines
Open "Help Wanted" Issues: GitHub Issues
|
| Participating in discussions | Join our Slack to have access to real-time discussions on Simpler.Grants.gov information. Provide your perspective as we grow and develop this community together. |
### Change log
Major updates to the content of this page will be added here.
| Date | Update | Notes |
| --------- | ------------------- | -------------------------------------------------------- |
| 1/30/2024 | Rough Draft | Initial rough draft of content for welcome page |
| 2/5/2024 | Rough Draft Updated | Updated Get Involved based on feedback |
| 1/30/2025 | Links removed | Broken links to pages that no longer exist were removed. |
# Open source and security
Answer the most common questions we hear about how an open source software affects security, and explain why public code is an asset, not a liability, when it is paired with strong operational control
## Summary
Open source does not remove the need for strong security engineering. It does something more useful: it makes our code, dependencies, and architecture reviewable, so issues can be found, discussed, and fixed in the open.
For Simpler Grants, our security posture is based on controls that remain effective even when the code is fully visible:
* secrets stored outside the repository
* least-privilege access controls
* network boundaries and managed cloud services
* automated vulnerability scanning
* code review, audit trails, and reproducible dependency management
This approach is consistent with modern security practice and with federal guidance from CISA.Federal open source is not a fringe practice; it is increasingly the default:
* OMB M-16-21 (Federal Source Code Policy, 2016) requires federal agencies to release at least 20% of custom-developed code as open source and to make code reusable across agencies by default.
* The SHARE IT Act (2024, Public Law 118-187) reinforces and extends this direction: custom-developed federal code should be shared across the government and, where appropriate, with the public.
***
### How this applies to Simpler Grants
Our repository separates public design information from sensitive runtime data. Specifically:
* Architecture is documented publicly in the repo so reviewers, partners, and contributors can see how the system is built.
* Runtime secrets and sensitive configuration are managed outside the repository using AWS secrets management and environment-scoped parameters. No credentials, API keys, or private certificates live in the code.
* Dependencies are pinned and reproducible via lockfiles, so any change is reviewable and auditable.
* Production dependency footprint is minimized by separating dev-only packages from runtime dependencies and by using minimal release images where possible.
* Automated dependency maintenance runs through Renovate, including vulnerability alert handling.
* Vulnerability scanning runs pre-deploy via Anchore.
* Post-build and cloud-side scanning runs through AWS Inspector and Security Hub.
* Code review, audit trails, and branch protections mean that every change to production code is attributable, reviewable, and traceable.
***
### **What is not public**
Our open source approach does not mean every security-relevant detail is published. We do not publish:
* secrets, tokens, private keys, or credentials
* protected production data
* privileged operational access paths
* private incident handling details that would create unnecessary risk while an issue is being remediated
Security vulnerabilities should be reported through the HHS Vulnerability Disclosure Policy, not through public GitHub issues.
***
### Why this is a security advantage
Open source strengthens security when paired with mature operational controls:
* Security researchers, vendors, partners, and HHS reviewers can inspect the same code and documentation our team uses. No black box.
* When a new CVE is published, maintainers can quickly determine whether a vulnerable dependency is present and whether a fix has landed.
* Pull requests, commit history, and lockfile changes create an auditable record of what changed, when, by whom, and why.
* Sensitive protections live in secrets, access controls, infrastructure policy, and runtime defenses, not in the assumption that design details will stay hidden.
* HHS is not dependent on a single vendor's black-box implementation to understand, operate, or transition the software in the future.
This is consistent with[ CISA's Open Source Software Security Roadmap](https://www.cisa.gov/resources-tools/resources/cisa-open-source-software-security-roadmap), which treats secure open source usage as a core part of modern federal cybersecurity rather than an exception to it. CISA itself operates under an "open-by-default" software development policy and publishes its own code on GitHub.
***
### Industry and government examples of open source in high-risk environments
Public code and professionally operated, security-sensitive services are not mutually exclusive. This is especially well-established in the federal government.
**Federal government**
* Login.gov (GSA / Technology Transformation Services). The identity provider that secures authentication for dozens of federal agencies is fully open source. Its core codebase lives at[ github.com/18F/identity-idp](https://github.com/18F/identity-idp). Login.gov protects access to services handling SSNs, IRS records, veteran benefits, and more — and it does so with a public codebase.
* CISA. America's cyber defense agency operates under an open-by-default software development policy and[ publishes its own code publicly on GitHub](https://github.com/cisagov). This is the agency whose guidance most federal security programs follow.
**Industry**
Many organizations operating security-sensitive, mission-critical systems combine public codebases with strong hosted operations:
* GitLab. GitLab's core product is[ developed in public](https://gitlab.com/gitlab-org/gitlab), and GitLab.com runs as a multi-tenant SaaS on top of that public codebase. GitLab holds FedRAMP and other government certifications.
* HashiCorp Vault / Terraform. Vault, used for secrets management at many Fortune 500 companies and federal agencies, is developed in the open.
* Signal. End-to-end encrypted messaging for journalists, activists, and national security personnel is[ fully open source](https://github.com/signalapp) on both client and server.
* Bitwarden. Password manager handling credentials for millions of users and enterprise customers is open source.
* Discourse, Zulip, Mattermost. All operate public codebases alongside managed cloud offerings used by governments and regulated industries.
* Kubernetes, PostgreSQL, Linux, nginx, OpenSSL. The foundational infrastructure of essentially every modern cloud and government system, including classified environments, is open source.
# How to contribute code
GitHub is our primary platform for collaborative software development within the Simpler Grants community. This guide will help you get started and make the most out of your code contribution experience.
{% hint style="info" %}
**Looking for work?**
Our development team curates "help wanted" issues for open source developers like you to help out with on [GitHub](https://github.com/HHS/simpler-grants-gov/labels/help%20wanted).
{% endhint %}
## Contribution Guidelines
### A. Code style and standards
* Follow the established code style and standards.
* Use clear and descriptive variable names, comments, and documentation where necessary.
### B. Testing
* Write tests for your code changes to ensure they function as expected
* Run exiting tests and ensure they pass before submitting your pull request
### C. Documentation
* Update any relevant documentation in the README files to reflect changes
## Getting Started
### 1. Create a Fork
Create a new fork of the simpler grants.gov repository for your changes. Fork names should conform to the simpler grants naming convention
```
/-simple-ticket-description
```
For example
```
btabaska/1234-updating-api-with-new-header
```
Next, create a feature branch using the following command, replacing feature-branch-name with a name that follows our branch naming schema
```
git checkout -b feature-branch-name
```
### 2. Clone the repository
Clone the forked repository to your local machine using the following command:
```
git clone https://github.com//.git
```
Instruction on setting up the local development environment can be found in the README of the repository.
API Development Instructions [API Development Instructions](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/api/development.md)
Front End Development Instructions [Front End Development Instructions](https://github.com/HHS/simpler-grants-gov/blob/main/frontend/README.md)
{% hint style="info" %}
**Need help with your local development environment?**\
\
Post in the [Discourse forum](https://forum.simpler.grants.gov/) if you're having issues. We can walk you through the process of resolving blockers.
{% endhint %}
### 3. Make & Commit changes
Make your desired changes to the codebase using your preferred editor. Once you're done, stage and commit your changes:
```
git add .
git commit -m "Brief Description of changes"
```
### 4. Push Changes
* Push your changes to your forked repository on GitHub using the following command:
```
git push origin feature-branch-name
```
### 5. Run code quality checks to verify changes
Frontend:
On the frontend of the project we enforce formatting, linting, typescript checks and unit testing before a code change can be accepted.
To run frontend tests:
{% code overflow="wrap" %}
```
// run linting to check that lint rules are being met
npm run lint
// run formatting to ensure that document formatting is correct. This will attempt to auto-fix simple issues with formatting
npm run format
// run type checks to ensure that typescript rule requirements are being met
npm run ts:check
// run unit tests to validate that defined component tests are not failing
npm run test
```
{% endcode %}
API
In the API of the project we enforce formatting, linting and unit tests. These all must pass before code changes can be accepted.
To run API tests:
{% code overflow="wrap" %}
```
// run linting to check that lint rules are being met
make lint
// run formatting to ensure that document formatting is correct. This will attempt to auto-fix simple issues with formatting
make format
// run unit tests to validate that defined unit tests are not failing
make test
```
{% endcode %}
### 6. Create a Pull Request
a. Navigate to your forked repository on GitHub
b. Click on the Pull Request button
c. Fill out the pull request form
d. Validate that GitHub automated integration testing is not failing. If it is go back and make changes to your code until all tests are passing again.
e. Submit the pull request for review by other developers within the community
\
Example pull request form:
```
## Summary
Fixes #(ISSUE)
### Time to review: ___x mins___
## Changes proposed
> // Update with a short description of what you changed
## Context for reviewers
> // Provide context for the PR reviewer
```
### 7. Review process
#### a. Assign reviewers
* Assign reviewers to your pull request who are knowledgeable about the project and can provide valuable feedback
#### b. Feedback and iteration
* Address any feedback or comments from reviewers promptly
* Make necessary changes and iterate on your code until it meets the projects standard to be accepted
* Once your pull request has been approved it will be merged into the main branch and deployed as necessary.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
10/3/2024
Updated content
Updated content for hackathon
# How to file issues
Simpler.Grants.gov uses GitHub as our primary work tracker. If you encounter bugs, have feature requests, or want to provide feedback, filing an issue on GitHub is the way to go. This guide will help you understand how to file issues and contribute to improving our projects.
## Getting Started
Make sure that all issues follow the Simpler.Grants.gov [Code of Conduct](/get-involved/policies-and-guidelines/code-of-conduct). We understand that experiencing issues with software can be frustrating, especially when it prevents you from doing work. However, please remember that a human on the other side will be reading your issue and responding to it.
### 1. Navigate to Issues
* Go to the repository on GitHub where you want to file the issue.
### 2. Click on "Issues"
* Click on the "Issues" tab at the top of the repository to view existing issues or file a new one.
### 3. Check Existing Issues
* Before filing a new issue, check if a similar issue has already been reported.
* If you find a similar issue, instead of creating a new one, react to the existing one with a thumbs up and comment if you have any additional insights.
## Filing a New Issue
### 1. Click on "New Issue"
* Click the green "New Issue" button to create a new issue.
### 2. Choose Issue Template (if available)
* Some repositories may provide issue templates to help standardize the information provided.
* Choose the appropriate template.
Used to report a software security issue that you have noticed.
### 3. Submit the Issue
* Once you've filled out all the necessary information, click the "Submit new issue" button to create the issue.
* We will review the issue and prioritize it within the Simpler Grants Community. We may not respond immediately or directly to you, but we review all issues and appreciate your participation and patience.
## Tips for Filing Issues
* Be descriptive: Provide as much information as possible to help developers understand and address the issue.
* Search first: Before filing a new issue, search existing issues to avoid duplicates.
* Follow guidelines: If the repository has specific guidelines or templates for filing issues, follow them for consistency.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Report a bug
Using GitHub to fill out a bug report helps us keep track of and implement fixes to issues that you may find in our application. This short guide will help you understand what we look for when a user fills out this template.
* **Title**: Use a clear and descriptive title that summarizes the issue.
* **What Happened**: Provide detailed information about how the problem.
* Describe the behavior that you experienced
* Specify the expected behavior.
* **Steps to reproduce the behavior**:
* Detail a step-by-step process that you followed that led to the issue
* Include any error messages
* **Priority**: Assign a priority that you think matches the severity of the issue
* **Relevant log outpu**t: Include any log output that you have.
* This is optional; if you don't know what log output is or where to get it, please leave it blank.
* **Labels**: Select relevant labels to categorize the issue (e.g., bug, feature request, enhancement).
* **Screenshots and additional context**: Embed any screenshots, gifs, or external video links that can help explain the issue you are experiencing.
* **Assignees**: Optionally, assign the issue to specific team members if necessary.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Request a feature
Using GitHub to fill out a feature request issue helps us keep track of and implement improvements to issues you may find in our application. This short guide will help you understand what we look for when a user fills out this template.
* **Title**: Use a clear and descriptive title that summarizes the issue.
* **Contact Details**: Provide a good follow-up email address to contact if we need more info.
* **Is your feature request related to a problem? Please describe**:
* Provide a detailed description of the problem that the feature would solve.
* Include error messages, logs, etc., to help us understand the problem.
* **A clear and concise description of what you want to happen**:
* Provide as much detail as possible for how you would like the software to function with your new feature idea implemented.
* **Describe alternatives you've considered**:
* This will help us follow your thought process and rule out any alternatives you have researched and considered before filling out the request.
* **Screenshots and additional context**:
* Embed any screenshots, gifs, or external video links that can help explain the issue you are experiencing.
* **Assignees**: Optionally, assign the issue to specific team members if necessary.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Report a security vulnerability
We can't accept secure submissions via GitHub. You will be redirected to the [HHS Responsible Disclosure](https://hhs.responsibledisclosure.com/) form to submit vulnerabilities.
* Before submitting a report, please review the [HHS Disclosure Policy.](https://www.hhs.gov/vulnerability-disclosure-policy/index.html)
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Community guidelines
Describes our policies and guidelines around collaborating as a community.
## Code of Conduct
Please review our [Code of Conduct](https://github.com/HHS/simpler-grants-gov/blob/main/CODE_OF_CONDUCT.md) before engaging with the community. We advise that all community members adhere to these rules in all their communications with other community members. We will enforce this to ensure that all community members can do their best work in a safe environment.
## Open Source Policy
We adhere to the [HHS Open Source Policy](https://github.com/CMSGov/cms-open-source-policy). If you have any questions, [send us an email](mailto:simpler@grants.gov).
## Security and Responsible Disclosure Policy
The Simpler Grants project adheres to the same security and disclosure policies as the HHS. Review the HHS Disclosure Policy to understand the policy and all websites within its scope to learn more.
This policy describes *what systems and types of research* are covered, *how to send* us vulnerability reports, and *how long* we ask security researchers to wait before publicly disclosing vulnerabilities.
If you have other cybersecurity related questions, please contact us at [csirc@hhs.gov.](mailto:csirc@hhs.gov).
### To Submit a Vulnerability
{% content-ref url="/pages/8gbNak0xLW1QrT9VeIJv" %}
[Report a security vulnerability](/get-involved/github-planning/reporting-a-security-vulnerability)
{% endcontent-ref %}
## Public Domain
This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
All contributions to this project will be released under the CC0 dedication. By submitting a pull request or issue, you agree to comply with this waiver of copyright interest.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
##
# Code of Conduct
As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
We are committed to making participation in this project a harassment-free experience for everyone, regardless of experience level, gender, gender identity, expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions not aligned with this Code of Conduct.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more project maintainers.
This Code of Conduct is adapted from the [Contributor Covenant version 1.0.0](http://contributor-covenant.org/version/1/0/0).
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Content privacy
## Content Privacy
{% hint style="warning" %}
**Important**
Content that you share in Simpler.Grants.gov communications channels is **never** fully private. Administrators can view all of the content shared in each channel, and the content in these channels is subject to [Freedom of Information Act (FOIA) requests](https://www.foia.gov/how-to.html).
{% endhint %}
There are a few important considerations regarding the privacy of content that is shared in any of the channels listed above:
* **Admins can view all messages:** Administrators of each channel can view and delete or modify content shared within that channel. This includes channels with the semblance of privacy, such as direct messages within Slack.
* **Content is subject to FOIA:** All content shared in these channels is also subject to a [Freedom of Information Act (FOIA) request](https://www.foia.gov/how-to.html). That may include content categorized as "internal," pending FOIA request approval. This further underscores the importance of **never** sharing sensitive or prohibited information in these changes.
Date
Update
Notes
Initial Content
Updated with Initial content
# Reporting and removing content
If content that violates our community guidelines is posted in any of our communication channels, please follow the steps below to remove the content. If you are the one who posted it by accident, please remove it if possible. If someone else has posted it or you are in a channel you do not have privileges to delete, contact the appropriate Simpler.Grants.gov administrator to remove it.
{% hint style="info" %}
**Note:** Many of the steps to remove content involve contacting platform administrators.
{% endhint %}
## Slack
### If it was a message that *you* posted:
1. Select the `More options` menu (the three vertically stacked dots that appear when you hover over your post)
2. Choose either `Edit message` or `Delete message` to remove the content
### If it was a message that *someone else* posted:
1. Contact a Slack admin
2. The Slack admin will follow the steps above to delete the post from that channel
## GitHub
{% hint style="warning" %}
**Important**
Only *administrators* can remove sensitive content from GitHub because content added to GitHub will remain visible in the history of the codebase until an administrator scrubs it from the repository.
{% endhint %}
1. Contact a GitHub administrator who is a software engineer
2. They will scrub the content from the history of the code repository
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
1/31/2025
Removed out of date content
Removed "Google Groups" section which is no longer accurate to team processes.
# Incident response protocol
## Observing Inappropriate Content or Behavior
If any community member notices content or behavior that violates our Code of Conduct during any of our meetings or on any communication channel, they are encouraged to report it immediately according to the following process:
1. **Immediate Action**: If possible, the person witnessing the incident should ask the offending party to stop the inappropriate behavior.
2. **Report the Incident**: Report the incident to the admin team via a private message on the platform where the incident occurred or via email. Provide as much detail as possible, including screenshots or logs if available.
3. **Admin Review**: The admin team will review the report, investigate the incident, and determine the appropriate response, which may include a warning, temporary ban, or permanent expulsion from the community.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Community events
The Simpler.Grants.gov team periodically hosts public events to engage our community with the ongoing development work.
#### Coding Challenges
Opportunities for external contributors to help the Simpler.Grants.gov team solve problems and develop site functionality.
The format varies. Software engineers of all experience levels are encouraged. Some, but not all, Coding Challenges also have a role for designers and researchers. The event is virtual but can vary from a weekend to several weeks.
#### Simpler.Grants.gov Big Demo
About every four months, our team hosts a public virtual demonstration of the features and work we produced in the previous 4-month development cycle.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Fall 2024 Coding Challenge
### Welcome to the Simpler Grants.gov Challenge
This is a great starting point for contributing to this year's challenge. Below are some important resources and context to help you get involved. If you don't find what you're looking for here, feel free to explore the Wiki or the [README on GitHub](https://github.com/HHS/simpler-grants-gov/tree/main), or reach out to us with any questions.
***
#### Collaborative Coding & Design Challenge Code of Conduct
This pilot event fosters a collaborative, creative community for all participants and government partners. Please adhere to the following guidelines:
1. Foster an environment that encourages creativity and empowers teammates to bring their best selves to the challenge.
2. Encourage respectful questioning, active listening, and openness to diverse ideas.
3. Always assume positive intent in interactions.
4. Consider the global and local impact of your ideas.
5. Ensure a safe, respectful, and inclusive space, free from discrimination or harassment.
***
#### Event Schedule
The event will start on **Friday, October 25th**, and end on **Saturday, October 26th**. The full event schedule will be posted as it’s confirmed.
***
#### Repositories and Resources
**Simpler Grants.gov Repository**
* **GitHub Link**: [Simpler Grants.gov GitHub](https://github.com/HHS/simpler-grants-gov/tree/main)
**Live Site**
* **Simpler Grants.gov**: [Visit the Site](https://simpler.grants.gov/)
**Zoom Link**
The various sessions and breakout rooms will be hosted on Zoom. We encourage you to have Zoom installed and set up prior to the event. The link will be updated here on the day of the event.
* **Zoom Link**: ``
**Developer Documentation**
* **Full Documentation**: [View Developer Docs](https://github.com/HHS/simpler-grants-gov/tree/main/documentation)
**Frontend**
The frontend is developed using React with Next.js. To get started, follow the instructions in the README below:
* **Frontend README**: [Read Here](https://github.com/HHS/simpler-grants-gov/blob/main/frontend/README.md)
**API**
Our API is built with Flask (Python) and containerized with Docker. To set up your development environment, follow the steps in the documentation below:
* **API Setup**: [API Development Guide](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/api/development.md)
* **API Endpoints Documentation (Swagger)**: [API Docs](http://api-dev-1839587515.us-east-1.elb.amazonaws.com/docs)
***
#### Design Documentation
Our designers have created guidelines for contributing design mockups and content for Simpler Grants.gov. All designs are based on the USWDS component library and follow our voice and tone guide.
* **Voice and Tone Guide**: [View Guide](https://app.gitbook.com/o/cFcvhi6d0nlLyH2VzVgn/s/Pm7UEzeiS1tbLCV1SFRu/product/voice-and-tone-guide)
* **USWDS Components**: [USWDS Design System](https://designsystem.digital.gov/)
***
#### Get Involved Now!
Can't wait until October 25th? Join the Simpler Grants.gov open-source community and start contributing today!
* **Get Involved Now**: [Join the Community](/get-involved/get-involved)
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# Event Submissions & Winners
The first Collaborative Code & Design Challenge was a short weekend "hackathon" event where competitors came together to create proposals. The winner was selected by a "crowd favorite" vote from all participants.
## Winner
[Dante's Login](https://docs.google.com/presentation/d/12fRxDYeAqVTG4xeNhd5GjITdgcaA1H8PbI-NSYHIfjg/edit?usp=share_link) - Kari Aanestad and Brian Krohn
## Submitted Projects
[Improving Search](https://docs.google.com/document/d/1RH-9ij7fjPhIqzM0qCZwdnGWt1hVKao6/edit?usp=sharing\&ouid=108978626373867578681\&rtpof=true\&sd=true) - Sarah Simister
[Subscriptions](https://docs.google.com/document/d/1RH-9ij7fjPhIqzM0qCZwdnGWt1hVKao6/edit?usp=sharing\&ouid=108978626373867578681\&rtpof=true\&sd=true) - Brian Krohn, Chantal Forster, Kari Aanestad, Liz Hoffman, Paul Gehrig, Sukhi Chuhan
[Search & Subscribe](https://docs.google.com/presentation/d/1BAhUlzwG6ET7DSWJHNL3JLGMC5b9j29K-GqIv_1Hr_U/edit#slide=id.p) - Kari Aanestad, Sukhi Chuhan, Chantal Forster, Paul Gehrig, Liz Hoffman, Brian Krohn
[Improving Site Clarity](https://docs.google.com/presentation/d/1erkSjAWb-O4T5U8I-LCYX9TRGd1qaypkC_FqHsotYYE/edit#slide=id.p) - Rachel, Judy, Aura
[Data Consolidation Over Site Redesign](https://drive.google.com/file/d/1re6jhEf2onoRzo_RvFljjTIk3qhk3603/view) - Judy Wieber
# Spring 2025 Collaborative Coding Challenge
Welcome to the Spring 2025 Collaborative Coding Challenge
The Collaborative Coding Challenge hosted by Simpler.Grants.gov starts on Monday, March 10th! Make sure you don’t miss out on it. This will be a great chance to collaborate with developers, subject matter experts, product managers, and designers from around the country. Winners will be chosen by community favorite voting among all participants, so start planning how you’ll make your submission stand out.
Who Should Participate?
* Software Developers – Building functional prototypes or MVPs
* Subject Matter Experts (SMEs) – Managers, analysts in state/local government who search and apply for federal grants
* Product Managers & Designers – Helping shape user experience and project direction
* Non-Technical Contributors – Brainstorming, planning, project management, etc.
Whether you have extensive coding experience or simply a passion for problem-solving, your insights will help shape the next generation of federal grants solutions.
\
If you are interested and have not already, please sign up [here](https://forms.gle/d7u6RzHYboZ1gvJ18) to participate.
***
## Goal
Develop simple projects using the Search & Opportunities API that showcase innovative or creative ways that could improve the federal grantseeking experience.
***
## Submissions
These must be added to this Google Drive folder by midnight EST on March 31st 2025. All submissions must contain the following critera to be accepted.
[Google Drive Folder](https://drive.google.com/drive/u/0/folders/1A4AlwA3fVsc40m1yjBBBiTd4NmyVL4u8)
**A document or slideshow in a readable format (.docx, .rtf, .txt, .pdf, etc) which contains:**
1. Team / Project Name
2. Names of all contributors
3. Link to a code repository that can be accessed by the public (open source)
4. A short description of the goals of the project
5. A short demonstration of the project in action.
[Optional Submission Template Link](https://docs.google.com/presentation/d/1F60a_gwmgKEhOpGoZXYx2T22bi0Mn11-O8KIyXSHBrI/edit?usp=sharing)
Looking for examples? Check out the [submissions](/get-involved/community-events/fall-2024-coding-challenge/event-submissions-and-winners) from our 24 hour Fall 2024 Collaborative Code & Design Challenge
***
## Prize Pool
A prize pool will be awarded to winners at the conclusion of the event. Awards will be decided by **community favorite voting** from all participants. Prize money must be divided up equally among all members of the winning team.
### Over $4,000 total
* 1st Place: **$1,500**
* 2nd Place: **$1,000**
* 3rd Place: **$500**
* Top 10 Proposals (beyond 1st–3rd): **$150 each**
***
## Event Schedule
Submissions open on **March 10, 2025**, and remain open for 3 weeks. We’ll kick off with an optional virtual meeting on **March 10th at 3 PM EST**.
{% stepper %}
{% step %}
#### **Kickoff Meeting**
* **Monday, March 10th at 3 PM EST**
* [Zoom Link](https://us02web.zoom.us/j/82924948517)
* Agenda: Welcome, overview of challenge rules, networking, team formation
{% endstep %}
{% step %}
#### Weekly Zoom Workshops
* **March 14th @ 2 PM EST**
* Topics: How to request an API key, setting up your development environment, live Q\&A with mentors
* [Zoom Link](https://us02web.zoom.us/j/84619443667)
* **March 19th @ 8 PM EST**
* Topics: Ongoing project support, advanced mentor Q\&A, troubleshooting tips
* [Zoom Link](https://us02web.zoom.us/j/83065296796)
* **March 22nd @ 5 PM EST**
* Topics: Finalizing project structure, best practices for code submissions, last-minute questions
* [Zoom Link](https://us02web.zoom.us/j/82578998872)
{% endstep %}
{% step %}
#### Submission Due Date
* **March 31st @ 12AM EST**
* All submissions must be turned in by this time. Late submissions will be accepted, but will not be eligible for awards.
{% endstep %}
{% step %}
#### **Voting Period Begins**
* **April 1st @ 12PM EST**
* Submissions will be voted on by participants to determine who the "Crowd Favorite" is. The top 10 submissions will be eligible for prize money.
{% endstep %}
{% step %}
#### **Voting Period Begins**
* **April 4th @ 12AM EST**
* All votes must be in by this time. The Simpler.Grants.gov team will then tabulate the results and announce them to the community.
{% endstep %}
{% step %}
#### Awards Ceremony
* **April 4th @ 4PM EST**
* We will announce winners, celebrate participants and discuss next steps for successful projects.
* [Zoom Link](https://us02web.zoom.us/j/84005217256)
{% endstep %}
{% endstepper %}
All virtual sessions and breakout rooms will be hosted on Zoom. We encourage you to have Zoom installed and set up prior to the event.
***
## Code of Conduct
Our events foster a collaborative, creative community for all participants and government partners. Please adhere to the following guidelines:
1. Foster an environment that encourages creativity and empowers teammates to bring their best selves.
2. Encourage respectful questioning, active listening, and openness to ideas.
3. Always assume positive intent in interactions.
4. Consider the global and local impact of your ideas.
5. Ensure a harassment-free space that encourages free speech.
***
## Repositories and Resources
**Simpler Grants.gov Repository:** [**GitHub**](https://github.com/HHS/simpler-grants-gov/tree/main)
**Simpler Grants.gov:** [**Visit the Site**](https://simpler.grants.gov/)
**Developer Documentation:** [**View Developer Docs**](https://github.com/HHS/simpler-grants-gov/tree/main/documentation)
**API**
* Built with Flask (Python), containerized with Docker.
* API Setup: [API Development Guide](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/api/development.md)
* API Endpoints (Swagger): [API Docs](http://api-dev-1839587515.us-east-1.elb.amazonaws.com/docs)
**Need an API Key?** Reach out to us [directly](mailto:opensource@grants.gov).
# Event Submissions & Winners
We concluded the second [Simpler.Grants.gov](http://simpler.grants.gov/) hackathon last week! This collaborative coding challenge took place over 3 weeks in the month of march. Participants were asked to submit proposals and code which would demonstrate an extension or improvement on our grant seeking API. Our participants were able to deliver some incredible work, which you can find on our Wiki. We want to congratulate the winners of this event:
## 1st Place - $1500
GrantFinder- Glenn Tatum
GrantFinder is a tool built onto the existing Simpler.Grants.gov which makes the process of grant discovery simpler.
[Proposal](https://drive.google.com/file/d/1T0bJ4Fm0AchTWDn0iQDL8eSW1WnuGOhN/view?usp=share_link)
[Code](https://github.com/GlennTatum/simpler-grants-gov)
## 2nd Place - $1000
GrantMatch AI+ - InterGemm LLC
GrantMatch AI+ is an AI-powered platform designed to streamline the process of discovering and applying for grant opportunities.
[Proposal](https://drive.google.com/file/d/1s3iDGmFfxUA1dkCLPaOEj-cTYBQNVtvy/view?usp=sharing)
[Code](https://drive.google.com/file/d/1Ep-GOCvyb5-x6oulZ2PecQS1zWSECpoj/view?usp=share_link)
[Full Submission Folder](https://drive.google.com/drive/folders/1giGPommhqziAp3oXxYhxwKFPgOGjnT6S?usp=sharing)
## 3rd Place - $500
GrantWise - Caleb Okpaheifufue & Emmanuel Ajayi
AI-Powered Grant Discovery & Application Assistance
[Proposal](https://drive.google.com/file/d/1fNt9vIa6WswSVHh6jLPQ9F46A3WfAsz4/view?usp=share_link)
## Participant Testimonials
**Glenn Tatum** wrote a piece about their experience in the hackathon, as well as contributing to our open source community.
{% embed url="" %}
**Bassirou Goumbane** had the following to say about their experience participating in the hackathon
> Just a quick reflection after receiving the latest [Simpler.Grants.gov](http://simpler.grants.gov/) Newsletter — and wow, it aligns so well with the spirit of what I shared earlier!
>
> The message about collaboration, open-source contributions, and building for the public good really resonated with me. Especially the example of two participants teaming up — one handling the technical side and the other focusing on UX/UI — to create something impactful together. That’s the kind of synergy I was referring to: different paths, different strengths, same destination.
>
> Whether we’re walking away with $500 or $1,500, the real prize might be what we can build together if we combine our unique insights. As they said, this platform is more than a challenge — it’s a space to connect, grow, and contribute to a bigger mission.
>
> Thanks again to everyone participating and supporting this journey. Let’s keep learning and building for something greater!
# Communication channels
We collaborate on the project using various public and private channels. Find out what information or conversations you'll find in each.
## Public channels
{% hint style="warning" %}
**Content privacy**
Content that you share in Simpler.Grants.gov communications channels is **never** fully private. Administrators can view all of the content shared in each channel, including those with the semblance of privacy, such as direct messages within Slack. The content in these channels is also subject to [Freedom of Information Act (FOIA) requests](https://www.foia.gov/how-to.html).
{% endhint %}
### Discourse forum
We use Discourse as our primary platform for discussing open-source development.
### All public channels
{% hint style="danger" %}
**Prohibited activity**
Prohibited activity (e.g., violations of our Code of Conduct or the Federal Hatch Act) is never allowed in any channel. Project maintainers have the right and responsibility to moderate and remove prohibited content from all communication channels.
{% endhint %}
Open source repository that stores the source code and technical documentation for the API, static site, and other tools/services related to the initiative.
Planning tool that helps the team organize project tickets into sprints that deliver pieces of functionality described in the product roadmap.
## Private channels
Channel
Platform
Description
Internal forum
Slack
The platform internal Simpler.Grants.gov team members use to discuss sensitive topics and manage day-to-day work.
## Zoom for public meetings
Zoom is our primary tool for virtual meetings within the Simpler Grants community. This guide will help you get started and make the most out of your Zoom experience.
Getting started with Zoom
**1. Joining Zoom meetings**
* Click on the Zoom meeting link provided by the meeting organizer.
* If it's your first time using Zoom, you may be prompted to download and install the Zoom application. Follow the on-screen instructions to do so.
**2. Testing audio and video**
* Before joining a meeting, test your audio and video settings to ensure they're working correctly.
* Click on "Test Speaker & Microphone" or "Test Video" in the Zoom settings menu to check your audio and video devices.
Using Zoom for meetings
**Meeting etiquette**
* Follow the[ code of conduct](https://wiki.simpler.grants.gov/get-involved/policies-and-guidelines) that has been set by the Simpler Grants community
* Keep discussions relevant to the meeting agenda and avoid disruptive behavior.
* Mute your microphone when not speaking to minimize background noise.
* Use the "Raise Hand" feature to indicate when you want to speak or have a question.
**2. Participating in discussions**
* Contribute actively to discussions by sharing your insights and ideas.
* Use the chat feature to ask questions or provide feedback in text format during the meeting.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
3/19/2025
Updated the channels
4/15
Updated privacy and prohibited activity warnings
Consolidated with information previously found on stand-alone "Content privacy" page.
# Zoom - Public Meetings
User guide for participating in our public meetings.
Welcome to our Zoom meetings platform! Zoom is our primary tool for virtual meetings within the Simpler Grants community. This guide will help you get started and make the most out of your Zoom experience.
## Getting Started
### 1. Joining Zoom Meetings
* Click on the Zoom meeting link provided by the meeting organizer.
* If it's your first time using Zoom, you may be prompted to download and install the Zoom application. Follow the on-screen instructions to do so.
### 2. Testing Audio and Video
* Before joining a meeting, test your audio and video settings to ensure they're working correctly.
* Click on "Test Speaker & Microphone" or "Test Video" in the Zoom settings menu to check your audio and video devices.
## Using Zoom for Meetings
### 1. Meeting Etiquette
* Follow the[ code of conduct](/get-involved/policies-and-guidelines) that has been set by the Simpler Grants community
* Keep discussions relevant to the meeting agenda and avoid disruptive behavior.
* Mute your microphone when not speaking to minimize background noise.
* Use the "Raise Hand" feature to indicate when you want to speak or have a question.
### 2. Participating in Discussions
* Contribute actively to discussions by sharing your insights and ideas.
* Use the chat feature to ask questions or provide feedback in text format during the meeting.
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
# API
Updated 9/17/2025 See our OpenAPI docs for most recent changes.
## API Gateway Developer Guide
### Overview
The Simpler Grants API Gateway provides developers with programmatic access to search federal grant opportunities through a RESTful API. This guide walks you through generating an API key via our web interface and integrating with your applications to search for funding opportunities.
{% hint style="danger" %}
The API is currently in early development. Features are under active development and subject to change. To see the latest changes please take a look at our [OpenAPI doc](https://api.simpler.grants.gov/docs#/Opportunity%20v1/post_v1_opportunities_search).
{% endhint %}
### Getting Started
{% hint style="info" %}
Are you looking for more guidance? [Check out our full API tutorial](/product/api/simpler-grants-api-tutorial) to get started creating applications that leverage the Simpler.Grants.gov API.
{% endhint %}
#### Prerequisites
* A user account on the Simpler Grants platform
* Basic understanding of REST APIs and HTTP requests
* Access to make HTTP requests from your application
#### Step 1: Generate Your API Key
1. **Log into the Platform**
* Navigate to the Simpler Grants website
* Sign in with your Login.gov credentials
2. **Access the API Dashboard**
* Go to the developer page under the community dropdown or via this [link](https://simpler.grants.gov/developer)
* Click "Manage API Keys" after reading through the developer page.
3. **Create a New API Key**
* Click "Create API Key"
* Provide a descriptive name for your key (e.g., "My Grant Search App")
* Click "Create API Key" to create the key
#### Step 2: API Authentication
All API requests must include your API key in the `X-API-Key` header:
```
X-API-Key: YOUR_API_KEY_HERE
```
### API Endpoints
#### Base URL
* **Production**: `https://api.simpler.grants.gov`
* **Development**: Contact [support](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/wiki/product/simpler-grants.gov-analytics) for development endpoints
#### **Core Endpoints**
{% content-ref url="/pages/jPXQQLJ0smfK9voDtVyA" %}
[Search opportunities](/product/api/search-opportunities)
{% endcontent-ref %}
{% content-ref url="/pages/2zWu0sQNhHpPNx7SMmel" %}
[Extracts](/product/api/extracts)
{% endcontent-ref %}
#### Common Issues
1. **Invalid API Key**
* Ensure key is included in `X-API-Key` header
* Verify key is active and not expired
* Check for typos in the key
2. **Request Format Errors**
* Ensure `Content-Type: application/json` header
* Validate JSON syntax
* Check required fields (pagination is required)
3. **Parameter Validation**
* `page_size` must be between 1 and 100
* Date formats must be YYYY-MM-DD
* Enum values must match exactly (case-sensitive)
### Best Practices
#### Rate Limiting
* The API implements rate limiting to ensure fair usage
* If you receive 429 responses, implement exponential backoff
* Consider caching results to reduce API calls
* If you are searching all opportunities then use the extracts endpoint
#### Efficient Searching
* Use specific filters to reduce result sets
* Implement pagination for large result sets
* Consider using CSV format for bulk data downloads
#### Security
* Never expose API keys in client-side code
* Store keys securely using environment variables
* Rotate keys periodically
* Use HTTPS for all requests
#### Error Handling
* Always check HTTP status codes
* Implement retry logic with backoff for transient errors
* Log errors for debugging but don't expose sensitive information
### Example Use Cases
{% tabs %}
{% tab title="Grant Discovery Dashboard" %}
Build a dashboard that shows relevant opportunities based on user preferences:
def get_relevant_grants(user_interests, applicant_type):
filters = {
"opportunity_status": {"one_of": ["posted", "forecasted"]},
"applicant_type": {"one_of": [applicant_type]}
}
# Search for each interest area
all_opportunities = []
for interest in user_interests:
payload = {
"query": interest,
"filters": filters,
"pagination": {"page_offset": 1, "page_size": 10}
}
# Make API call and collect results
opportunities = search_opportunities(payload)
all_opportunities.extend(opportunities)
return deduplicate_opportunities(all_opportunities)
{% endtab %}
{% tab title="Deadline Monitoring System" %}
Monitor approaching deadlines for relevant opportunities:
```python
def check_approaching_deadlines(days_ahead=30):
end_date = datetime.now() + timedelta(days=days_ahead)
payload = {
"filters": {
"opportunity_status": {"one_of": ["posted"]},
"close_date": {
"start_date": datetime.now().strftime("%Y-%m-%d"),
"end_date": end_date.strftime("%Y-%m-%d")
}
},
"pagination": {
"page_offset": 1,
"page_size": 100,
"sort_order": [{"order_by": "close_date", "sort_direction": "ascending"}]
}
}
return search_opportunities(payload)
```
{% endtab %}
{% tab title=" Funding Analytics" %}
Analyze funding trends and patterns:
```python
def analyze_funding_by_agency():
payload = {
"filters": {
"opportunity_status": {"one_of": ["posted"]},
"post_date": {"start_date": "2024-01-01"}
},
"pagination": {"page_offset": 1, "page_size": 1000}
}
response = search_opportunities(payload)
# Analyze facet counts for agency distribution
agency_stats = response.get("facet_counts", {}).get("agency_name", {})
return sorted(agency_stats.items(), key=lambda x: x[1], reverse=True)
```
{% endtab %}
{% endtabs %}
### Support and Resources
#### Getting Help
* **Documentation**: Check this guide and the OpenAPI documentation
* **Issues**: Report bugs or request features through the appropriate channels
* **Community**: Join developer discussions and share experiences
#### Additional Resources
* [OpenAPI Specification](https://api.simpler.grants.gov/docs) - Interactive API documentation
* [GitHub Repository](https://github.com/HHS/simpler-grants-gov) - Source code and issue tracking
* [Release Notes](https://github.com/HHS/simpler-grants-gov/releases) - API updates and changes
***
**Note**: This API is under active development. Please refer to the latest documentation and release notes for the most current information. We welcome feedback and contributions from the developer community.
# Search opportunities
Updated 9/17/2025 see our OpenAPI doc for most recent changes.
### **Endpoint**: `POST /v1/opportunities/search`
Search for grant opportunities using various filters and criteria.
### Caveats
* Search will only return a maximum of 10,000 opportunities. Any opportunties past that will be culled for performance. If you are receiving the maximum amount of opportunities as a response there is a good chance that you are not getting all possible results returned. In that case it is recommended to add more filters to get a smaller subset of opportunities. If you want to get an export of all opportunities see the [extracts endpoint.](/product/api/extracts)
* Search should return the same data as GET opportunity, except attachments are not included and the data is cached in search hourly.
### **Get Opportunity Details**
**Endpoints**:
* `GET /v1/opportunities/{opportunity_id}` (UUID format)
Retrieve detailed information about a specific opportunity.
### Search Parameters
The opportunity search endpoint accepts the following parameters:
#### Query Parameters
* **`query`** (string, optional): Free-text search across multiple fields
* Example: `"research"`, `"education funding"`
* Maximum length: 100 characters
* **`query_operator`** (string, optional): How to combine search terms
* Values: `"AND"` (default), `"OR"`
#### Filters
**Agency & Organization**
* **`top_level_agency`**: Filter by agency code
* Example: `{"one_of": ["USAID", "DOC"]}`
**Funding Details**
* **`funding_instrument`**: Type of funding
* Values: `"cooperative_agreement"`, `"grant"`, etc.
* Example: `{"one_of": ["grant"]}`
* **`funding_category`**: Category of funding
* Values: `"recovery_act"`, `"arts"`, `"natural_resources"`, etc.
**Eligibility**
* **`applicant_type`**: Who can apply
* Values: `"state_governments"`, `"county_governments"`, `"individuals"`, etc.
* Example: `{"one_of": ["state_governments", "nonprofits"]}`
**Status & Timing**
* **`opportunity_status`**: Current status
* Values: `"forecasted"`, `"posted"`, `"closed"`, `"archived"`
* Example: `{"one_of": ["posted", "forecasted"]}`
* **`post_date`**: When opportunity was posted
* Example: `{"start_date": "2024-01-01", "end_date": "2024-12-31"}`
* **`close_date`**: Application deadline
* Example: `{"start_date": "2024-06-01"}`
**Financial Filters**
* **`award_floor`**: Minimum award amount
* Example: `{"min": 10000}`
* **`award_ceiling`**: Maximum award amount
* Example: `{"max": 1000000}`
* **`expected_number_of_awards`**: Expected number of awards
* Example: `{"min": 5, "max": 25}`
* **`estimated_total_program_funding`**: Total program funding
* Example: `{"min": 100000, "max": 250000}`
**Other Filters**
* **`assistance_listing_number`**: Specific CFDA number
* Format: `##.##` (e.g., "45.C9")
* Example: `{"one_of": ["45.C9"]}`
* Format: `##.###` (e.g., "45.1C9")
* Example: `{"one_of": ["45.1C9"]}`
* **`is_cost_sharing`**: Whether cost sharing is required
* Example: `{"one_of": [true]}`
#### Pagination & Sorting
* **`pagination`** (required): Controls result pagination and sorting
```json
{
"page_offset": 1,
"page_size": 25,
"sort_order": [
{
"order_by": "opportunity_id",
"sort_direction": "descending"
}
]
}
```
**Sort Options**:
* `relevancy`, `opportunity_id`, `opportunity_number`
* `opportunity_title`, `post_date`, `close_date`
* `agency_code`, `agency_name`, `top_level_agency_name`
* `award_floor`, `award_ceiling`
#### Response Format
* **`format`** (optional): Response format
* Values: `"json"` (default), `"csv"`
* CSV format returns a downloadable file
### Code Examples
{% tabs %}
{% tab title="Python" %}
```python
import requests
import json
# Your API configuration
API_KEY = "your_api_key_here"
BASE_URL = "https://api.simpler.grants.gov"
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Basic search request
search_payload = {
"query": "research",
"filters": {
"opportunity_status": {"one_of": ["posted", "forecasted"]},
"funding_instrument": {"one_of": ["grant"]},
"agency": {"one_of": ["NSF", "NIH"]}
},
"pagination": {
"page_offset": 1,
"page_size": 25,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Make the request
response = requests.post(
f"{BASE_URL}/v1/opportunities/search",
headers=headers,
json=search_payload
)
if response.status_code == 200:
data = response.json()
opportunities = data["data"]
print(f"Found {len(opportunities)} opportunities")
for opp in opportunities:
print(f"- {opp['opportunity_title']}")
print(f" Agency: {opp['agency_name']}")
print(f" Deadline: {opp.get('close_date', 'No deadline specified')}")
print()
else:
print(f"Error: {response.status_code} - {response.text}")
```
{% endtab %}
{% tab title="JavaScript/Node.js" %}
{% endtab %}
{% tab title="cURL" %}
```bash
# Basic search
curl -X POST "https://api.simpler.grants.gov/v1/opportunities/search" \
-H "X-API-Key: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"query": "climate",
"filters": {
"opportunity_status": {"one_of": ["posted"]},
"funding_category": {"one_of": ["environment", "natural_resources"]}
},
"pagination": {
"page_offset": 1,
"page_size": 5,
"sort_order": [
{
"order_by": "relevancy",
"sort_direction": "descending"
}
]
}
}'
# Download results as CSV
curl -X POST "https://api.simpler.grants.gov/v1/opportunities/search" \
-H "X-API-Key: your_api_key_here" \
-H "Content-Type: application/json" \
-o "opportunities.csv" \
-d '{
"format": "csv",
"filters": {
"opportunity_status": {"one_of": ["posted"]}
},
"pagination": {
"page_offset": 1,
"page_size": 100,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}'
```
{% endtab %}
{% endtabs %}
### Response Format
#### JSON Response Structure
```json
{
"message": "Success",
"data": [
{
"opportunity_id": "12345678-1234-1234-1234-123456789012",
"opportunity_number": "EPA-R9-SFUND-23-003",
"opportunity_title": "Superfund Site Remediation Research",
"agency_code": "EPA",
"agency_name": "Environmental Protection Agency",
"post_date": "2024-01-15",
"close_date": "2024-06-30",
"opportunity_status": "posted",
"funding_instrument": "grant",
"funding_category": "environment",
"award_floor": 50000,
"award_ceiling": 500000,
"estimated_total_program_funding": 2000000,
"expected_number_of_awards": 4,
"applicant_types": ["nonprofits", "universities"],
"summary": "Funding for research into innovative remediation technologies...",
"is_cost_sharing": false
}
],
"pagination_info": {
"page_offset": 1,
"page_size": 25,
"total_pages": 15,
"total_records": 367
},
"facet_counts": {
"agency_name": {
"EPA": 45,
"NSF": 32,
"NIH": 28
},
"funding_instrument": {
"grant": 89,
"cooperative_agreement": 16
}
}
}
```
### Error Handling
#### Common HTTP Status Codes
* **200 OK**: Request successful
* **400 Bad Request**: Invalid request parameters
* **401 Unauthorized**: Missing or invalid API key
* **403 Forbidden**: API key lacks required permissions
* **429 Too Many Requests**: Rate limit exceeded
* **500 Internal Server Error**: Server error
#### Error Response Format
```json
{
"message": "Error description",
"status_code": 400,
"errors": [
{
"field": "pagination.page_size",
"message": "Must be between 1 and 100"
}
]
}
```
# Extracts
Updated 9/17/2025 see our OpenAPI doc for most recent changes.
### Core Endpoints
**List Extract Metadata**
**Endpoint**: `POST /v1/extracts`
Retrieve metadata about available data extracts, including file information, creation dates, and download URLs.
### Extract Types
The system provides the following types of data extracts:
#### Available Extract Types
* **`opportunities_json`**: Complete opportunity data in JSON format
* **`opportunities_csv`**: Complete opportunity data in CSV format
### Extract Metadata API
#### Request Parameters
The extract metadata endpoint accepts the following parameters:
**Filters**
* **`extract_type`** (enum, optional): Filter by specific extract type
* Values: `"opportunities_json"`, `"opportunities_csv"`
* Example: `"opportunities_json"`
* **`created_at`** (date range, optional): Filter by extract creation date
```json
{
"start_date": "2024-01-01",
"end_date": "2024-12-31"
}
```
**Pagination (Required)**
* **`pagination`**: Controls result pagination and sorting
```json
{
"page_offset": 1,
"page_size": 25,
"sort_order": [
{
"order_by": "created_at",
"sort_direction": "descending"
}
]
}
```
**Sort Options**:
* `created_at`: When the extract was created
### Code Examples
{% tabs %}
{% tab title="Python" %}
```python
import requests
import json
from datetime import datetime, timedelta
# Your API configuration
API_KEY = "your_api_key_here"
BASE_URL = "https://api.simpler.grants.gov"
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
def get_latest_extracts(extract_type=None, days_back=30):
"""Get extract metadata for the last N days"""
end_date = datetime.now()
start_date = end_date - timedelta(days=days_back)
filters = {
"created_at": {
"start_date": start_date.strftime("%Y-%m-%d"),
"end_date": end_date.strftime("%Y-%m-%d")
}
}
if extract_type:
filters["extract_type"] = extract_type
payload = {
"filters": filters,
"pagination": {
"page_offset": 1,
"page_size": 50,
"sort_order": [
{
"order_by": "created_at",
"sort_direction": "descending"
}
]
}
}
response = requests.post(
f"{BASE_URL}/v1/extracts",
headers=headers,
json=payload
)
if response.status_code == 200:
data = response.json()
extracts = data["data"]
print(f"Found {len(extracts)} extracts")
for extract in extracts:
print(f"- {extract['extract_type']} created {extract['created_at']}")
print(f" File: {extract.get('file_name', 'N/A')}")
print(f" Size: {extract.get('file_size', 'Unknown')} bytes")
if extract.get('download_url'):
print(f" Download: {extract['download_url']}")
print()
return extracts
else:
print(f"Error: {response.status_code} - {response.text}")
return []
def download_extract_file(extract_metadata, local_filename):
"""Download an extract file to local storage"""
download_url = extract_metadata.get('download_url')
if not download_url:
print("No download URL available for this extract")
return False
try:
# Note: Download URLs may be pre-signed and not require API key
response = requests.get(download_url, stream=True)
response.raise_for_status()
with open(local_filename, 'wb') as f:
for chunk in response.iter_content(chunk_size=8192):
f.write(chunk)
print(f"Downloaded {local_filename}")
return True
except requests.exceptions.RequestException as e:
print(f"Error downloading file: {e}")
return False
# Usage examples
print("Getting latest opportunity JSON extracts...")
json_extracts = get_latest_extracts(extract_type="opportunities_json", days_back=7)
if json_extracts:
latest_extract = json_extracts[0]
filename = f"opportunities_{latest_extract['created_at'][:10]}.json"
download_extract_file(latest_extract, filename)
print("\nGetting all recent extracts...")
all_extracts = get_latest_extracts(days_back=14)
```
{% endtab %}
{% tab title="JavaScript/Node.js" %}
```javascript
const fetch = require('node-fetch');
const fs = require('fs');
const API_KEY = 'your_api_key_here';
const BASE_URL = 'https://api.simpler.grants.gov';
async function getExtractMetadata(extractType = null, daysBack = 30) {
const endDate = new Date();
const startDate = new Date();
startDate.setDate(endDate.getDate() - daysBack);
const filters = {
created_at: {
start_date: startDate.toISOString().split('T')[0],
end_date: endDate.toISOString().split('T')[0]
}
};
if (extractType) {
filters.extract_type = extractType;
}
const payload = {
filters: filters,
pagination: {
page_offset: 1,
page_size: 50,
sort_order: [
{
order_by: "created_at",
sort_direction: "descending"
}
]
}
};
try {
const response = await fetch(`${BASE_URL}/v1/extracts`, {
method: 'POST',
headers: {
'X-API-Key': API_KEY,
'Content-Type': 'application/json'
},
body: JSON.stringify(payload)
});
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
console.log(`Found ${data.data.length} extracts`);
data.data.forEach(extract => {
console.log(`- ${extract.extract_type} created ${extract.created_at}`);
console.log(` File: ${extract.file_name || 'N/A'}`);
if (extract.download_url) {
console.log(` Download available`);
}
});
return data.data;
} catch (error) {
console.error('Error fetching extract metadata:', error);
return [];
}
}
async function downloadExtract(extractMetadata, localFilename) {
const downloadUrl = extractMetadata.download_url;
if (!downloadUrl) {
console.log('No download URL available for this extract');
return false;
}
try {
const response = await fetch(downloadUrl);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const fileStream = fs.createWriteStream(localFilename);
response.body.pipe(fileStream);
return new Promise((resolve, reject) => {
fileStream.on('finish', () => {
console.log(`Downloaded ${localFilename}`);
resolve(true);
});
fileStream.on('error', reject);
});
} catch (error) {
console.error('Error downloading extract:', error);
return false;
}
}
// Usage
async function main() {
console.log('Getting latest CSV extracts...');
const csvExtracts = await getExtractMetadata('opportunities_csv', 7);
if (csvExtracts.length > 0) {
const latestExtract = csvExtracts[0];
const filename = `opportunities_${latestExtract.created_at.split('T')[0]}.csv`;
await downloadExtract(latestExtract, filename);
}
console.log('\nGetting all recent extracts...');
const allExtracts = await getExtractMetadata(null, 14);
}
main();
```
{% endtab %}
{% tab title="cURL" %}
```bash
# Get metadata for all extracts from the last 7 days
curl -X POST "https://api.simpler.grants.gov/v1/extracts" \
-H "X-API-Key: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"filters": {
"created_at": {
"start_date": "2024-01-01",
"end_date": "2024-01-08"
}
},
"pagination": {
"page_offset": 1,
"page_size": 25,
"sort_order": [
{
"order_by": "created_at",
"sort_direction": "descending"
}
]
}
}'
# Get metadata for JSON extracts only
curl -X POST "https://api.simpler.grants.gov/v1/extracts" \
-H "X-API-Key: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"filters": {
"extract_type": "opportunities_json"
},
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "created_at",
"sort_direction": "descending"
}
]
}
}'
# Download an extract file (replace URL with actual download URL from metadata response)
curl -o "opportunities_2024-01-01.json" \
"https://example-bucket.s3.amazonaws.com/extracts/opportunities_2024-01-01.json?signature=..."
```
{% endtab %}
{% endtabs %}
### Response Format
#### Extract Metadata Response Structure
```json
{
"message": "Success",
"data": [
{
"extract_metadata_id": "12345678-1234-1234-1234-123456789012",
"extract_type": "opportunities_json",
"file_name": "opportunities_2024-01-15.json",
"file_size": 15728640,
"download_url": "https://example-bucket.s3.amazonaws.com/extracts/opportunities_2024-01-15.json?signature=...",
"created_at": "2024-01-15T02:30:00Z",
"updated_at": "2024-01-15T02:35:00Z"
},
{
"extract_metadata_id": "87654321-4321-4321-4321-210987654321",
"extract_type": "opportunities_csv",
"file_name": "opportunities_2024-01-15.csv",
"file_size": 8294400,
"download_url": "https://example-bucket.s3.amazonaws.com/extracts/opportunities_2024-01-15.csv?signature=...",
"created_at": "2024-01-15T02:30:00Z",
"updated_at": "2024-01-15T02:35:00Z"
}
],
"pagination_info": {
"page_offset": 1,
"page_size": 25,
"total_pages": 3,
"total_records": 67
}
}
```
#### Extract File Formats
**JSON Extract Structure**
The opportunities JSON extract contains an array of opportunity objects with complete data:
```json
[
{
"opportunity_id": "12345678-1234-1234-1234-123456789012",
"opportunity_number": "EPA-R9-SFUND-23-003",
"opportunity_title": "Superfund Site Remediation Research",
"agency_code": "EPA",
"agency_name": "Environmental Protection Agency",
"post_date": "2024-01-15",
"close_date": "2024-06-30",
"opportunity_status": "posted",
"funding_instrument": "grant",
"funding_category": "environment",
"award_floor": 50000,
"award_ceiling": 500000,
"estimated_total_program_funding": 2000000,
"expected_number_of_awards": 4,
"applicant_types": ["nonprofits", "universities"],
"summary": "Funding for research into innovative remediation technologies...",
"is_cost_sharing": false,
"attachments": [
{
"attachment_id": "attachment-123",
"file_name": "funding_announcement.pdf",
"download_url": "https://example.com/attachments/funding_announcement.pdf"
}
]
}
]
```
**CSV Extract Structure**
The opportunities CSV extract contains the same data as the JSON output in tabular format with the following columns:
* `opportunity_id`
* `opportunity_number`
* `opportunity_title`
* `agency_code`
* `agency_name`
* `post_date`
* `close_date`
* `opportunity_status`
* `funding_instrument`
* `funding_category`
* `award_floor`
* `award_ceiling`
* `estimated_total_program_funding`
* `expected_number_of_awards`
* `applicant_types` (pipe-separated values)
* `summary`
* `is_cost_sharing`
### Error Handling
#### Common HTTP Status Codes
* **200 OK**: Request successful
* **400 Bad Request**: Invalid request parameters
* **401 Unauthorized**: Missing or invalid API key
* **403 Forbidden**: API key lacks required permissions
* **404 Not Found**: Extract not found
* **429 Too Many Requests**: Rate limit exceeded
* **500 Internal Server Error**: Server error
#### Error Response Format
```json
{
"message": "Error description",
"status_code": 400,
"errors": [
{
"field": "filters.extract_type",
"message": "Must be one of: opportunities_json, opportunities_csv"
}
]
}
```
# Simpler Grants API tutorial
### Welcome, Developer! 👋
This guide will walk you through making your very first API call to the Simpler Grants system. Whether you're new to APIs or just new to our platform, we'll get you up and running in just a few minutes.
### What You'll Learn
By the end of this guide, you'll:
* Have an API key generated
* Understand how to authenticate requests
* Make your first successful API call
* Know where to go next for advanced features
# Prerequisites
Before we start, make sure you have:
* [ ] A computer with internet access
* [ ] A Login.gov account (free to create)
* [ ] Basic familiarity with using a terminal/command line
* [ ] One of these tools installed:
* `curl` (usually pre-installed on Mac/Linux)
* A REST client like [Postman](https://www.postman.com/) or [Insomnia](https://insomnia.rest/) (optional)
* A programming environment (Python, JavaScript, etc.) (optional)
* [ ] Python 3 (If you would like to follow the "Write Your First Program")
# Create your account & get your API key
#### 1.1 Sign Up for an Account
1. Go to the [Simpler Grants website](https://simpler.grants.gov/)
2. Click "Sign In" and create a Login.gov account if you don't have one
3. Complete the account setup process
#### 1.2 Generate Your API Key
1. **Log into Simpler.Grants.gov** using your Login.gov credentials
2. **Navigate to the API Dashboard**:
* Look for "Developer" under the "Community" dropdown on the main navigation
* Select "Manage API Keys" on the "Developer" page
3. **Create your first API key**:
* Select the "Create API Key" button
* Give it a name like "My First API Key" or "Testing Key"
* Click "Create API Key"
4. **Get ready to use your key**:
* Copy the generated key (it looks like: `abcd1234efgh5678ijkl9012mn`)
* The key is exactly 25 characters long
> 💡 **Pro Tip**: Treat your API key like a password. Don't share it publicly or commit it to version control!
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Test your setup
Let's make sure everything is working with a simple test call.
#### Option A: Using cURL (Recommended for Beginners)
Open your terminal and copy and paste in and then run this command (replace `YOUR_API_KEY_HERE` with your actual key):
```bash
curl -X POST "https://api.simpler.grants.gov/v1/opportunities/search" \
-H "X-API-Key: YOUR_API_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{
"pagination": {
"page_offset": 1,
"page_size": 5,
"sort_order": [
{
"order_by": "opportunity_id",
"sort_direction": "descending"
}
]
}
}'
```
#### Option B: Using Postman
1. **Create a new request**:
* Method: `POST`
* URL: `https://api.simpler.grants.gov/v1/opportunities/search`
2. **Add headers**:
* `X-API-Key`: `YOUR_API_KEY_HERE`
* `Content-Type`: `application/json`
3. **Add request body** (select "raw" and "JSON"):
```json
{
"pagination": {
"page_offset": 1,
"page_size": 5,
"sort_order": [
{
"order_by": "opportunity_id",
"sort_direction": "descending"
}
]
}
}
```
4. **Click Send**
#### What Should Happen
If everything worked, you should see a response like this:
```json
{
"message": "Success",
"data": [
{
"opportunity_id": "12345678-1234-1234-1234-123456789012",
"opportunity_number": "EPA-R9-SFUND-23-003",
"opportunity_title": "Environmental Research Grant Program",
"agency_name": "Environmental Protection Agency",
"post_date": "2024-01-15",
"close_date": "2024-06-30",
"opportunity_status": "posted"
}
],
"pagination_info": {
"page_offset": 1,
"page_size": 5,
"total_pages": 247,
"total_records": 1234
}
}
```
{% hint style="success" %}
**Congratulations**\
You just made your first API call and retrieved grant opportunities!
{% endhint %}
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Understanding what just happened
Let's break down that API call:
#### The URL
* `https://api.simpler.grants.gov` - This is our API base URL
* `/v1/opportunities/search` - This endpoint searches for grant opportunities
* `v1` means this is version 1 of our API
#### The Headers
* `X-API-Key: YOUR_API_KEY_HERE` - This authenticates your request
* `Content-Type: application/json` - This tells the server we're sending JSON data
#### The Request Body
```json
{
"pagination": {
"page_offset": 1, // Start with the first page
"page_size": 5, // Return 5 opportunities per page
"sort_order": [ // Sort by opportunity ID, newest first
{
"order_by": "opportunity_id",
"sort_direction": "descending"
}
]
}
}
```
#### The Response
* `message`: Always "Success" when things work
* `data`: An array of opportunities (5 in this case)
* `pagination_info`: Information about the total results and pages
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Try some variations
Now that you've made your first call, let's try a few variations to see different features:
#### Variation 1: Search for Specific Opportunities
This searches for education-related opportunities that are currently posted, sorted by deadline.
```bash
curl -X POST "https://api.simpler.grants.gov/v1/opportunities/search" \
-H "X-API-Key: YOUR_API_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{
"query": "education",
"filters": {
"opportunity_status": {"one_of": ["posted"]}
},
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "close_date",
"sort_direction": "ascending"
}
]
}
}'
```
#### Variation 2: Filter by Agency
This finds opportunities from the National Science Foundation or National Institutes of Health.
```bash
curl -X POST "https://api.simpler.grants.gov/v1/opportunities/search" \
-H "X-API-Key: YOUR_API_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{
"filters": {
"agency": {"one_of": ["NSF", "NIH"]},
"opportunity_status": {"one_of": ["posted", "forecasted"]}
},
"pagination": {
"page_offset": 1,
"page_size": 15,
"sort_order": [
{
"order_by": "agency_name",
"sort_direction": "ascending"
}
]
}
}'
```
Next we are going to create a Python script together which will tie in everything that we have learned so far. By the end of it you will be ready to create and manage your own software which leverages Simpler.Grants.gov's API.
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Let's make a python script to use the API
Now let's apply everything we have learned so far to build a simple script together. Over the next few steps in the tutorial we will work together step by step to create a working Python script which can call and display responses from the Simpler.Grants.gov API. Below you can see the final project code that we will build together. This capstone project in the tutorial will give you everything you need to develop software which leverages our powerful federal grants API.
### Final Project Code (for reference)
```python
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Add search term if provided
if search_term:
payload["query"] = search_term
# Make the API call
response = requests.post(
f"{BASE_URL}/v1/opportunities/search",
headers=headers,
json=payload
)
# Check if the request was successful
if response.status_code == 200:
data = response.json()
opportunities = data["data"]
print(f"Found {len(opportunities)} opportunities:")
print("-" * 50)
for opp in opportunities:
print(f"Title: {opp['opportunity_title']}")
print(f"Agency: {opp['agency_name']}")
print(f"Posted: {opp['post_date']}")
if opp.get('close_date'):
print(f"Deadline: {opp['close_date']}")
print("-" * 50)
else:
print(f"Error: {response.status_code}")
print(response.text)
# Run the search
if __name__ == "__main__":
print("\nSearching for health-related opportunities...")
search_opportunities("health")
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Make a project folder
First let's make a folder to keep our project in. Open your terminal and navigate to a directory that you'd like to save this project to. Then we will create a directory and move into it.
```bash
mkdir sgg-quickstart
cd sgg-quickstart
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Create and activate a virtual environment
We are going to import Python packages to make accessing the API easier. This will keep those project packages isolated to this project. You’ll know it worked if you see `(.venv)` at the start of your terminal prompt.
{% tabs %}
{% tab title="Windows (PowerShell)" %}
```powershell
python -m venv .venv
.venv\Scripts\Activate.ps1
```
{% endtab %}
{% tab title="macOS/Linux" %}
python3 -m venv .venv
source .venv/bin/activate
{% endtab %}
{% endtabs %}
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Install the only dependency we need
We need to install the requests package to make it easier to interact with the API. This command will use the python package manager pip to install requests into our isolated virtual environment.
```bash
pip install requests
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Create a new Python file
**Option A:**\
Open your code editor and create a file named **`search_opportunities.py`**\
\
**Option B:**\
Use your terminal to run the command `touch search_opportunities.py`
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Add the imports
Once we have the file open in our editor we can get started coding our project. First we are going to import both the default package json to manage data in the JSON format and the requests package that we installed.
```python
import requests
import json
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Add the API configuration
Now under the imports we are going to add the API configuration. This is where we will have the API key that we generated earlier and the base url that we will be calling.
```python
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
```
{% hint style="info" %}
If you wanted to improve this project in the future you may want to look into moving the API\_KEY and BASE\_URL into a .env file so that they can be kept secret. You don't want to commit your API\_KEY to version control where others may be able to see it.
{% endhint %}
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Make a function to perform the search
Since we have the needed packages imported and the API configuration variables set, we can now start to do things with them. First let's define a function that will perform the search.
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
{% hint style="info" %}
The default `search_term=""` means “no keyword filter.”
{% endhint %}
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Add the request headers
Now that we have a function defined, let's go ahead and add the request headers. You should be familiar with these from the tests that we did earlier. They are used to authenticate the application with the Simpler.Grants.gov API.
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
{% hint style="info" %}
* `X-API-Key` authenticates you.
* `Content-Type` tells the server we’re sending JSON.
{% endhint %}
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Build the request body
Still inside the function we are now going to build out the "payload" which is the body of the request. In here we will add parameters which instructs the API what we are searching for.
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
# This asks for page 1, 10 results, newest first by post_date.
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Add a search term
Now that we have our payload defined, lets also add a search term. This will be used like a Google search to instruct the API what we are looking for.
```python
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
# This asks for page 1, 10 results, newest first by post_date.
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Add search term if provided
if search_term:
payload["query"] = search_term
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Get ready to run the script
In order to run this script we need to add a "main" section so Python knows where to start executing our code. At the bottom outside the function that we have defined lets go ahead and create the main section which will execute the search function that we wrote. When the script is executed it will search for all opportunities and then opportunities with the word "health".
```python
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
# This asks for page 1, 10 results, newest first by post_date.
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Add search term if provided
if search_term:
payload["query"] = search_term
# Run the search
if __name__ == "__main__":
print("\nSearching for health-related opportunities...")
search_opportunities("health")
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Make the API call
Now that we have both our search term and the payload defined, as well as our headers for authentication it is finally time to make an API call. We are going to add a POST request which will reach out to the Simpler.Grants.gov API with our code and will save the response into the new response variable.
Reminder that we are setting up the call with the following information:
* `json=payload` makes `requests` send the body as JSON.
* We’re calling `/v1/opportunities/search` on the base URL.
```python
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
# This asks for page 1, 10 results, newest first by post_date.
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Add search term if provided
if search_term:
payload["query"] = search_term
# Make the API call
response = requests.post(
f"{BASE_URL}/v1/opportunities/search",
headers=headers,
json=payload
)
# Run the search
if __name__ == "__main__":
print("\nSearching for health-related opportunities...")
search_opportunities("health")
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Check the response status
Now let's run the script to make sure that everything is working. We are going to add error validation to the response so that when the script executes it will either return a success message or an error message.
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
# This asks for page 1, 10 results, newest first by post_date.
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Add search term if provided
if search_term:
payload["query"] = search_term
# Make the API call
response = requests.post(
f"{BASE_URL}/v1/opportunities/search",
headers=headers,
json=payload
)
# Check if the request was successful
# If it’s a success (200), we print the JSON and "Success".
if response.status_code == 200:
data = response.json()
print("Success")
print(data)
# Otherwise, we show the error code and the server’s message.
else:
print(f"Error: {response.status_code}")
print(response.text)
# Run the search
if __name__ == "__main__":
print("\nSearching for health-related opportunities...")
search_opportunities("health")
Now let's run the code that we have written to test the response from the Simpler.Grants.gov API. Save what you have written and then go back to your terminal. You can execute the command you see below which will run the script once.
{% tabs %}
{% tab title="Windows(PowerShell)" %}
```powershell
python search_opportunities.py
```
{% endtab %}
{% tab title="MacOS/Linux" %}
python3 search_opportunities.py
{% endtab %}
{% endtabs %}
If you are getting an error message please check to make sure that your code matches the example. if you continue to have issues check out the [common issues & solutions section](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions) to see if it is applicable to the error you are seeing.
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Handle the response
Now that we know that everything is working, we are getting a JSON response to our API call which is saved to the response variable that we created. From there we can manipulate the variable and print out information from the response to view when we run the script. Lets clean up the test run that we wrote earlier and use print to format the JSON into something that is easier to read.
Try playing around with different search terms and see what the API responds with. For example, If you replace `"health"` in the main function call with `"space"`, the API will filter results by that term instead.
```python
import requests
import json
# Your API configuration
API_KEY = "YOUR_API_KEY_HERE" # Replace with your actual key
BASE_URL = "https://api.simpler.grants.gov"
# The function that will contain search logic
def search_opportunities(search_term=""):
"""Search for grant opportunities"""
# These headers will authenticate your API call
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
# Build the search request
# This asks for page 1, 10 results, newest first by post_date.
payload = {
"pagination": {
"page_offset": 1,
"page_size": 10,
"sort_order": [
{
"order_by": "post_date",
"sort_direction": "descending"
}
]
}
}
# Add search term if provided
if search_term:
payload["query"] = search_term
# Make the API call
response = requests.post(
f"{BASE_URL}/v1/opportunities/search",
headers=headers,
json=payload
)
# Check if the request was successful
# If it’s a success (200), we parse JSON and loop through the list at data["data"].
if response.status_code == 200:
data = response.json()
opportunities = data["data"]
# Format and print out the response
print(f"Found {len(opportunities)} opportunities:")
print("-" * 50)
# We print a few helpful fields from each opportunity.
for opp in opportunities:
print(f"Title: {opp['opportunity_title']}")
print(f"Agency: {opp['agency_name']}")
print(f"Posted: {opp['post_date']}")
if opp.get('close_date'):
print(f"Deadline: {opp['close_date']}")
print("-" * 50)
# Otherwise, we show the error code and the server’s message.
else:
print(f"Error: {response.status_code}")
print(response.text)
# Run the search
if __name__ == "__main__":
print("\nSearching for health-related opportunities...")
search_opportunities("health")
```
{% hint style="warning" %}
Something not working as expected? Check out [common issues & solutions](/product/api/simpler-grants-api-tutorial/common-issues-and-solutions).
{% endhint %}
# Next Steps (optional)
Now that you have a working script to access the Simpler.Grants.gov API let's make some changes and see how it effects the response data.
* Change `page_size` to 25 to see more results per request
* Replace `"health"` with you own keyword(s), e.g. `"education"` or `"rural broadband"`
* When you run the script, try piping the output to a file to easily save it. `python3 search_opportunities.py > results.txt`
## Next-step ideas to level up your script
### Read the API key from an environment variable
It is safer than hard-coding your API key into the script. Try adding this import and variable to the top of your script.
```python
import os
API_KEY = os.getenv("SGG_API_KEY", "")
```
Then when you run it, you first need to add your API key to the environment:
{% tabs %}
{% tab title="Windows(PowerShell)" %}
```powershell
$env:SGG_API_KEY='your_key_here'
```
{% endtab %}
{% tab title="macOS/Linux" %}
```bash
export SGG_API_KEY=your_key_here
```
{% endtab %}
{% endtabs %}
### Add a real command-line interface
Try adding these changes to your code so that users can pass query, page size and sort options without needing to edit the code.
```python
import argparse
def parse_args():
p = argparse.ArgumentParser()
p.add_argument("query", nargs="*", help="optional keywords")
p.add_argument("--page-size", type=int, default=10)
p.add_argument("--sort-by", default="post_date")
p.add_argument("--sort-dir", choices=["ascending","descending"], default="descending")
return p.parse_args()
if __name__ == "__main__":
args = parse_args()
term = " ".join(args.query)
search_opportunities(term, page_size=args.page_size, order_by=args.sort_by, sort_direction=args.sort_dir)
```
You could then run the script with command line arguments like `python search_opportunities.py health --sort-dir ascending`
# You did it!
You've successfully taken your first steps to become a full fledged Simpler Grants API developer! This is just the beginning of what you can build with access to federal grant data.
We're excited to see what you'll create, and we're here to help along the way. Don't hesitate to reach out with questions, ideas, or feedback.
Happy coding!
### What's Next?
Now that you've made your first API call, here's where you can go next:
#### Learn More About Our APIs
* [**Opportunity Search API**](/product/api/search-opportunities) - Deep dive into searching for grants
* [**Extract API**](/product/api/extracts) - Download bulk data for analysis
#### Get Involved
* [**Join the Simpler.Grants.gov Community**](https://forum.simpler.grants.gov/)
* [**Help us improve by reporting bugs or problems**](https://github.com/HHS/simpler-grants-gov/issues/new?template=1_bug_report.yml)
* [**Contibute to help wanted issues on our open source repository**](https://github.com/HHS/simpler-grants-gov/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22help%20wanted%22%20)
#### 📖 Additional Resources
* [**OpenAPI Documentation**](https://api.simpler.grants.gov/docs)
* [**GitHub Repository**](https://github.com/HHS/simpler-grants-gov)
# Common issues & solutions
### Quick Reference
#### Essential Information
* **API Base URL**: `https://api.simpler.grants.gov`
* **Authentication Header**: `X-API-Key: YOUR_API_KEY_HERE`
* **Content Type**: `application/json`
* **Main Search Endpoint**: `POST /v1/opportunities/search`
#### Required Fields
Every search request must include:
```json
{
"pagination": {
"page_offset": 1,
"page_size": 25,
"sort_order": [
{
"order_by": "opportunity_id",
"sort_direction": "descending"
}
]
}
}
```
#### Common Status Codes
* **200**: Success! Your request worked
* **400**: Bad request - check your JSON format
* **401**: Unauthorized - check your API key
* **429**: Too many requests - slow down
* **500**: Server error - try again later
#### Problem: "401 Unauthorized" Error
**Solution**: Check your API key
* Make sure you copied it correctly (all 25 characters)
* Verify you're using the header `X-API-Key`
* Ensure there are no extra spaces
#### Problem: "400 Bad Request" Error
**Solution**: Check your JSON format
* Use a JSON validator to make sure your request body is valid JSON
* Ensure `pagination` is always included - it's required
* Check that all field names match exactly (case-sensitive)
#### Problem: No Results Returned
**Solution**: Try broader search criteria
* Remove filters to see if there are any opportunities at all
* Try different search terms
* Check if you're filtering by dates that might exclude current opportunities
#### Problem: "Too Many Requests" Error
**Solution**: Slow down your requests
* Wait a few seconds between API calls
* The API has rate limits to ensure fair usage for everyone
# Roadmap
A high-level overview of the functionality we're planning to build and test on simpler.grants.gov.
{% hint style="warning" %}
**Important**
This section contains forward-looking statements and descriptions of functionality that are subject to change. These statements involve known and unknown risks, uncertainties, and other factors that may cause our actual results to differ from what is described here.
Forward-looking statements give our expectations or forecasts of future events, and you should not place undue reliance on these statements.
{% endhint %}
## Objectives
The goal of the SimplerGrants initiative is to **make Grants.gov...**
* **Easier for grant seekers**\
Reduce applicant burden by simplifying the process of discovering and applying for grants.
* **More intuitive for non-experts**\
Build capacity for all grant-seeking communities by increasing their efficiency and effectiveness.
* **More efficient for grantors**\
Reduce grantor burden by streamlining the process of publishing and managing grants.
* **More transparent and participatory for the public**
Build stakeholder trust by actively engaging the public in planning, design, implementation, and reporting.
## Product roadmap
The features and functionality we plan to deliver as part of the SimplerGrants initiative are currently tracked on a public [product roadmap in GitHub](https://github.com/orgs/HHS/projects/12/views/8). There's also a version prepared for a general audience available at [Simpler.Grants.gov/roadmap](https://simpler.grants.gov/roadmap).
We currently plan in 4-month increments, but hope to build out a longer-term backlog of anticipated features as the project continues.
## Key product terms
The following terms are used to categorize items in our roadmap. We've defined them below and provided examples to help folks understand
Term
Description
Examples
Quad
Four-month planning increments that we use to prioritize items in our roadmap.
Quad 1.1:
Base Year 1, Quad 1
Deliverable
A product feature or update that aims to improve functionality for users.
Major updates to the content of this page will be added here.
Date
Update
Notes
Initial Content
Updated with initial content
# Release Notes
## [Simpler.Grants.Gov](#simplergrantsgov-hosted-a-public-demonstration) Release Notes - May 18, 2026
May 18th’s update brings [several new changes](https://github.com/HHS/simpler-grants-gov/releases/tag/2026.05.13-1):
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### Save Opportunities to Organizations
Users can now save opportunities to their organization so that all members can see what others are interested in to increase collaboration. Users can also receive email notifications for newly saved opportunities by members of your organization. After saving an opportunity, users can select ‘Sharing Options’ to add it to their organization.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### Added a new filter category
To support agencies like the [U.S. Economic Development Agency](https://www.eda.gov/funding/programs/american-rescue-plan/travel-tourism-and-outdoor-recreation) with a new classification of grants that support infrastructure projects, park development, and more, the [Simpler.Grants.gov](http://simpler.grants.gov) team added a new "[Recreation and Tourism](https://github.com/HHS/simpler-grants-gov/pull/9716)" category for users to filter by as new opportunities are posted for this type of grant.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Announcement**
{% endcolumn %}
{% column width="75%" %}
#### Updated [Simpler.Grants.gov](http://simpler.grants.gov) website content
The [Simpler.Grants.gov](http://simpler.grants.gov) team has updated language across several static pages to better reflect the team’s progress on Simpler Grants and provide more clarity on where users can find the information they need.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Shout Outs**
{% endcolumn %}
{% column width="75%" %}
#### Open Source Contributors
The [Simpler.Grants.gov](http://simpler.grants.gov) team extends a big thank you to:
[vijaygovindaraja](https://github.com/vijaygovindaraja) for [Fixing the search loading state by adding Suspense boundary around filters](https://github.com/HHS/simpler-grants-gov/pull/9295),
[monapdx](https://github.com/monapdx) for [Adding a documentation improvement issue type](https://github.com/HHS/simpler-grants-gov/pull/9576#pullrequestreview-4107434147),
[bobyu7941](https://github.com/bobyu7941) for [Fixing a documentation issue on our CONTRIBUTING.md](https://github.com/HHS/simpler-grants-gov/commit/85e4aee8e7738886989b9e1dc3b0e4aad5a86a52),
[MozamilS](https://github.com/MozamilS) for [Adding a comprehensive example local frontend .env example](https://github.com/HHS/simpler-grants-gov/pull/9577/changes),
[AsMurphy-student](https://github.com/AsMurphy-student) for [fixing frontend dev docs typos](https://github.com/HHS/simpler-grants-gov/pull/9535).
{% endcolumn %}
{% endcolumns %}
## [Simpler.Grants.Gov](#simplergrantsgov-hosted-a-public-demonstration) Release Notes - April 3, 2026
April 3rd’s update brings [several new changes](https://github.com/HHS/simpler-grants-gov/releases/tag/2026.04.01-1):
{% columns %}
{% column width="25%" %}
**Announcement**
{% endcolumn %}
{% column width="75%" %}
#### More consistent breadcrumb navigation
Since [Simpler.Grants.gov](http://simpler.grants.gov) has added more features and nested pages, it's important that users can quickly understand where they are in relation to other content, how they got there, and how various functionalities relate to each other. This latest release makes our use of breadcrumbs consistent at the top of pages to show the location of the current page within the site structure. The team has removed breadcrumbs where they were not necessary, and standardized them where they are helpful — such as pages nested under the improved Workspace menu.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Bug Fix**
{% endcolumn %}
{% column width="75%" %}
#### Fixed root cause of 404 pages on [Simpler.Grants.gov](http://simpler.grants.gov)
Pages for opportunities, applications, and forms were returning HTTP 200 instead of 404 when resources weren't found. The root cause was incorrect rendering of the function that sets the 404 status code, which was addressed and fixed across 4 pages that had the same bug pattern.
This is a result of a great [open source contribution](https://github.com/HHS/simpler-grants-gov/pull/9301) made by user [vijaygovindaraja](https://github.com/vijaygovindaraja) to address this bug in our backlog - thank you for your contribution!
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Bug Fix**
{% endcolumn %}
{% column width="75%" %}
#### Opportunity Forecasts now correctly populate agency contact information
Previously, when an opportunity forecast was published in Grants.gov, some of the Grantor contact information [was not being properly transferred](https://github.com/HHS/simpler-grants-gov/issues/9186) to [Simpler.Grants.gov](http://simpler.grants.gov) due to how the data was structured in [grants.gov](http://grants.gov). To correct this, we [made a change](https://github.com/HHS/simpler-grants-gov/pull/9187) to capture this information when the forecast is published, in addition to the already-captured information on the opportunity itself.
As a result of this change, the Simpler.Grants.gov team recently did a data update backfilling description information resulting in additional emails being sent to users related to forecasted opportunities. There are no further changes or responses required from users, and functionality is back to normal.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### Updated workflow emails
We [updated workflow emails](https://github.com/HHS/simpler-grants-gov/pull/9104) to render correctly and modified the email format to handle new lines more effectively. Additionally, we made minor logging adjustments based on needs for the dashboard.
{% endcolumn %}
{% endcolumns %}
## [Simpler.Grants.Gov](http://simpler.grants.gov) Release Notes - February 23, 2026
February 23rd’s update brings [several new changes](https://github.com/HHS/simpler-grants-gov/releases/tag/2026.02.18-1):
{% columns %}
{% column width="25%" %}
**Announcement**
{% endcolumn %}
{% column width="75%" %}
#### Watch the [Simpler.Grants.Gov](http://simpler.grants.gov) public demonstration
The [Simpler.Grants.gov](http://simpler.grants.gov) team gave a public demonstration of our newest features and functionality hosted by the [Council on Federal Financial Assistance (COFFA)](https://www.coffa.gov/). This virtual session highlighted our progress, shared user research insights, and showcased community engagement efforts. [You can watch the video from January 29, 2026 here](https://simpler.grants.gov/events).
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### Accessibility and navigation improvements
This release includes accessibility improvements to help users who navigate with the keyboard — especially people that rely on screen readers and keyboard-first users seeking navigation efficiency. Improvements include [making the "Skip to main" content more visually accessible](https://github.com/HHS/simpler-grants-gov/pull/8565), and [improving the focus styles that make it easier to navigate between search filters](https://github.com/HHS/simpler-grants-gov/pull/8550).
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### A better header menu design
On [Simpler.Grants.gov](http://simpler.grants.gov), both mobile and desktop, the team reformatted the account dropdown menu to be consistent with the rest of the navigation dropdown items and moved ‘Sign in’ after other menu items for more logical keyboard navigation. On mobile, the ‘Sign In’ was moved to the burger menu with other aesthetic and dropdown formatting improvements.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Bug Fix**
{% endcolumn %}
{% column width="75%" %}
#### ‘Start Application’ is disabled when organization data fails to load
This update prevents users from submitting the ‘Start Application’ modal when organization data fails to load, preventing users from interacting with an incomplete form and avoiding a misleading user experience.
{% endcolumn %}
{% endcolumns %}
## [Simpler.Grants.Gov](http://simpler.grants.gov) Release Notes - February 4, 2026
February 4th’s update brings [several new changes:](https://github.com/HHS/simpler-grants-gov/releases/tag/2026.02.04-1)
{% columns %}
{% column width="25%" %}
**Announcement**
{% endcolumn %}
{% column width="75%" %}
#### [Simpler.Grants.Gov](http://simpler.grants.gov) hosted a public demonstration
The [Simpler.Grants.gov](http://simpler.grants.gov) team hosted a public demonstration of our newest features and functionality with the [Council on Federal Financial Assistance (COFFA)](https://www.coffa.gov/) and the Department of Health and Human Services (HHS). This virtual session highlighted our progress, shared user research insights, and showcased community engagement efforts.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Announcement**
{% endcolumn %}
{% column width="75%" %}
#### The [Simpler.Grants.gov Roadmap](https://simpler.grants.gov/roadmap) has been updated
The team updated the roadmap content to reflect updated goals and recent accomplishments to date. [Check out the updated roadmap](https://simpler.grants.gov/roadmap) to see what we’re working on, what we’ve delivered, and more.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### Filtering Search Results by Posted Date
In response to a highly requested feature, users may now filter their search queries by Posted Date. Users may now search and click ‘Filter’ to reveal an accordion drop-down menu that can filter funding opportunities within a posted date of 3, 7, 14, 30, or 60 days.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### Submitted applications are available to download from application page
Users can now download their submitted application from the application page. This enables applicants to retain their submitted applications offline and provides transparency into what is sent to grantor agencies.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### Primary EBIZ POC Role Protections
The team has made sure that users cannot be removed from an organization if they are also the EBIZ POC for that organization by disabling the ‘Remove’ button. This will ensure that organizations do not lose access to the EBIZ POC administrative functions required to use and access Simpler Grants. If your EBIZ POC changes in [Sam.gov](https://sam.gov/), that will automatically be reflected in [Simpler.Grants.gov](http://simpler.grants.gov).
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### Updated logic after applications are submitted
The team has improved the view of submitted applications so selections cannot be altered after application is complete and submitted.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**Improvement**
{% endcolumn %}
{% column width="75%" %}
#### Simplified URL structures for applications
The team has updated all application and form pages to point to new routes defined by new information architecture. For example, the previous URL of
> *simpler.grants.gov/workspace/applications/application/\[applicationId]*
will now be
> *simpler.grants.gov/applications/\[applicationId].*
All previous URL routes will automatically redirect to the new routes.
{% endcolumn %}
{% endcolumns %}
## [Simpler.Grants.Gov](http://simpler.grants.gov) Release Notes - December 23, 2025
December 23rd’s update brings [several new changes](https://github.com/HHS/simpler-grants-gov/releases/tag/2025.12.23-1):
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### [Simpler.Grants.Gov](http://simpler.grants.gov) is now the default for searching for opportunities.
With Simpler Search launching as the default experience, we can get a wider array of users providing feedback about their experience. It also solidifies Simpler.Grants.Gov as the standard for search and helps to iteratively build and deliver working software methodically. Just over the last week, we have seen a 200% increase in users visiting the search page.
Users can still access the classic search by first logging into Grants.gov and then searching for opportunities. Some funding opportunities may be removed or modified by the agency that created them, and are no longer accessible from any Grants.gov systems (Classic or Simpler). These changes are unrelated to the switch over to the Simpler Search.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### Activity history is now available on applications.
The new activity history section of the application page shows what changes have been made to an application, when they were made, and who made each change. This surfaces important information to applicants collaborating on applications.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**New Improvement**
{% endcolumn %}
{% column width="75%" %}
#### A clearer, persistent confirmation after applying.
Instead of a temporary success alert, applicants will now see a permanent confirmation message at the top of their application page after successfully submitting an application. This message provides important information about the submission, what to expect next, and who to contact if they have any questions.
{% endcolumn %}
{% endcolumns %}
## Launching [Simpler.Grants.Gov](http://simpler.grants.gov) Release Notes - December 10, 2025!
We're excited to announce the launch of release notes for Simpler.Grants.Gov! Our goal for these updates is to regularly showcase new features, important bug fixes, improvements to enhance your experience, and more. This update for December 10th brings [several new changes](https://github.com/HHS/simpler-grants-gov/releases/tag/2025.12.10-1):
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### We’ve launched our first iteration of user profile and permissions.
Our research showed that users previously struggled to determine who within their organization had the necessary permissions, information, and access which resulted in a confusing chain of misdirected inquiries. These new improvements set the stage that will allow our users to collaborate together when submitting applications and clearly identify individuals with key roles who are responsible for critical tasks like renewing the Unique Entity Identifier (UEI) and inviting users to build out their teams.
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column width="25%" %}
**New Feature**
{% endcolumn %}
{% column width="75%" %}
#### We launched new changes to the Activity Dashboard.
We have created a hub that allows users to quickly access their frequently visited features like saved searches and queries, and introduced applications for ease of access and navigation. To solve for the key pain point of finding active grants and managing work, this Activity Dashboard centralizes where you can manage active work, giving users a clearer, personalized way to keep track of their progress.
{% endcolumn %}
{% endcolumns %}
# Deliverables
Describes our process for prioritizing, planning, and completing deliverables within the Simpler.Grants.gov initiative.
## Latest reporting
Since September 2024, we've been tracking our deliverables in a [GitHub-based product roadmap](https://github.com/orgs/HHS/projects/12). Please refer to that roadmap to view the deliverables we've planned or worked on since September.
## Previous deliverables
Prior to adopting the current process for tracking deliverables in GitHub, we used the following pages in our wiki to scope and report on deliverables.
* [Static site soft launch](/product/deliverables/static-site-soft-launch)
* [Static site public launch](/product/deliverables/static-site-public-launch)
* [GET opportunities endpoint](/product/deliverables/get-opportunities)
* [Open source onboarding](/product/deliverables/open-source-onboarding)
* [Co-design group planning](/product/deliverables/co-design-group)
## Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
2/12/2024
Initial Content
Updated with Initial content
12/19/2024
Removed stale content
Removed references to draft deliverables that were replaced with GitHub issues and reorganized the page.
# Static site soft launch
Deploy a static site with information about the Simpler.Grants.gov initiative.
## Summary details
## Overview
### Summary
* **What:** Deploy a static site to a public-facing URL that contains information about the Simpler Grants.gov initiative
* **Why:** Proves the successful completion of several front-end technical milestones and builds key infrastructure for future UI improvements
* **Who**
* Internal development team
* Internal HHS stakeholders
### Business value
The launch of a static site for the Simpler Grants.gov project represents the culmination of multiple internally focused deliverables and serves as a landing page where key stakeholders can access information about the project.
By sharing this information in a publicly accessible format and investing early in the infrastructure used to host it, this deliverable aims to demonstrate the following value propositions:
* Establishes simpler.grants.gov as the primary location that stakeholders can visit for project updates and previews of deliverables for the Simpler Grants.gov project
* Proves the successful completion of technical deliverables that enable faster development without sacrificing code quality or security
* Delivers an early win that both internal and external stakeholders can rally around, which helps build momentum and enthusiam for the project
* Facilitates a parallel approach to development, in which new features can be built and tested on `simpler.grants.gov` without risking or disrupting the existing functionality of legacy grants.gov
### User stories
* As a **full-time HHS staff member**, I want:
* the site to be accessible to members of the public and the Federal government, so we can use it to share information about the project with both internal and external stakeholders.
* the site to adopt modern branding and user interface (UI), so that stakeholders are excited to visit the page and can find the information they need more easily.
* As a **grantor**, I want:
* to be able to access information about the Simpler Grants.gov project in a central location, so that I don't have to rely exclusively on email for updates about the project.
* the site to be user friendly and easy to navigate, so that I don't have to spend a lot of time looking for information that is relevant to me.
* As a **prospective grant applicant**, I want:
* the site to be user friendly and easy to navigate, so that I don't have to spend a lot of time looking for information that is relevant to me.
* an opportunity to provide feedback or ask questions about the project, so that I can voice my concerns and help shape the direction of the project.
* As **maintainer of the project** I want:
* most of the critical development infrastructure to be in place when we officially launch the site, so that we can deploy bug fixes or new features quickly once the site is live.
* As an **open source contributor**, I want:
* the site to link to resources like the repository, support email, etc., so that I can easily learn where and how to participate in the project.
## Technical description
### Infrastructure Requirements
The infrastructure developed to deploy and host the site should balance:
* Code quality
* Security
* Delivery velocity
* Cost & maintenance
### User Experience Requirements
The design and structure of the site should balance:
* Usability
* Accessibility
* Site performance
* Brand identity
### Content Requirements
Process for drafting and updating the content of the site should balance:
* Speed & ease of content management
* Need for review and approval
### Definition of done
Following sections describe the conditions that must be met to consider this deliverable "done".
* [x] The following infrastructure requirements are satisfied:
* [x] The code needed to build and deploy the site is merged to `main`
* [x] The site is built and hosted with the tools selected in the [Front-end Planning deliverable](https://github.com/HHS/simpler-grants-gov/issues/49)
* [x] All code quality checks set up in the [Developer Tools deliverable](https://github.com/HHS/simpler-grants-gov/issues/50) are passing
* [x] The resources required to deploy and host the site are provisioned programmatically
* [x] Code changes are deployed using a CI/CD pipeline
* [x] The following user experience (UX) requirements are satisfied:
* [x] Anyone can access a live version of the site at simpler.grants.gov
* [x] The site adopts the UI principles and framework established
* [x] Anyone can access a live version of the site at simpler.grants.gov
* [x] The site has completed a 508 compliance reiview
* [x] Web traffic data for the site is actively being collected
* [x] Additional development tickets have been created for collecting other data needed to calculate the metrics below
* [x] The following content requirements are satisfied:
* [x] All content is deployed to simpler.grants.gov
* [x] The content on the site has been been reviewed and approved by the relevant stakeholders within each workstream
* [x] The site also links to external resources related to the project (if they are available), including:
* [x] The legacy grants.gov site
* [x] The open source repository
### Proposed metrics
* Number of unique site visitors
* Total number of site visits
* Uptime service
* Deployment build time
* Deployment/hosting costs
* Number of visits to outbound links to the following external resources (once added to the site)
* Open source repository
* grants.gov
### Destination for live updating metrics
Page on the public wiki. **Note:** This will likely change once we deliver [the Public Measurement Dashboard deliverable](https://github.com/HHS/simpler-grants-gov/issues/65).
## Planning
### Assumptions and dependencies
What functionality do we expect to be in place ***by the end of*** work on this deliverable?
* [x] [**Front-end Planning**](https://github.com/HHS/simpler-grants-gov/issues/49)**:** Determines the language, framework, and deployment service used to build and host the site.
* [x] [**Developer Tools**](https://github.com/HHS/simpler-grants-gov/issues/50)**:** Establishes a suite of tools used to ensure the quality and security of the site codebase.
* [x] [**simpler.grants.gov Domain**](https://github.com/HHS/simpler-grants-gov/issues/51)**:** Secures access to the `simpler.grants.gov` domain which is where the site will be hosted.
* [x] [**Security Approval**](https://github.com/HHS/simpler-grants-gov/issues/53)**:** Ensures that the site and the infrastructure that hosts it are comply with HHS security standards and practices.
* [x] [**Infrastructure-as-Code**](https://github.com/HHS/simpler-grants-gov/issues/123)**:** Programmatically provisions the resources needed to deploy and host this site.
* [x] [**Front-end CI/CD**](https://github.com/orgs/HHS/projects/12/views/3?pane=issue\&itemId=31950276)**:** Sets up a CI/CD pipeline that will be used to test and publish code changes to the site.
* [x] [**Foundational UI**](https://github.com/HHS/simpler-grants-gov/issues/60)**:** Determines the UI framework that the site will adopt before launch.
* [x] [**Web Analytics**](https://github.com/HHS/simpler-grants-gov/issues/63)**:** Enables tracking key success metrics for this deliverable, e.g. site traffic and number of unique visitors.
Is there any notable functionality we do ***not*** expect to be in place before works starts on this deliverable?
* **Internationalization:** While there will be content delivered within this deliverable that needs to be translated in the future, we do not expect to have a framework for managing translations set up by the time this deliverable is delivered.
* **CMS:** While in the long-term we may want to support a Content Management Service (CMS) that allows non-technical users to update and manage copy for the website, we do not expect a CMS to be selected and implemented when we launch this site.
### Not in scope
List of functionality or features that are explicitly out of scope for this deliverable.
* **Translating site contents:** Site translations will be the focus of a future deliverable.
* **Legacy web analytics:** Updating the existing analytics recorded on legacy grants.gov in order to establish a baseline for comparing the site traffic for `simpler.grants.gov` will happen in a later deliverable.
## Integrations
### Translations
Does this deliverable involve delivering any content that needs translation?
* Yes, the site contents will need to be translated.
If so, when will English-language content be locked? Then when will translation be started and completed?
* The initial process for translation is slotted for release in a future deliverable.
### Services going into PROD for the first time
This can include services going into PROD behind a feature flag that is not turned on.
* **Static Site:** This deliverable represents the official launch of the static site
* **simpler.grants.gov Domain:** The static site is the first service to officially use the `simpler.grants.gov` domain
* **Stakeholder Feedback Form:** This is the first time we're collecting feedback directly from stakeholders on `simpler.grants.gov`
* **Web Analytics:** This will most likely be the first service for which we are configuring web analytics
### Services being integrated in PROD for the first time
Are there multiple services that are being connected for the first time in PROD?
* **Static Site + Feedback Form:** The feedback form should be accessible directly from the site, preferably embedded directly on the page
* **Static Site + Web Analytics:** All of the public pages on the static site should be configured to track web analytics
* **Static Site + Communications Platforms:** The static site should link to the relevant communication platforms that are available at the time of launch
### Data being shared publicly for the first time
Are there any fields being shared publicly that have never been shared in PROD before?
* No, the content of the static site in this deliverable will be limited to general information about the Simpler Grants.gov project. It does not include exposing any production data from the new simpler.grants.gov data model.
### Security considerations
Does this deliverable expose any new attack vectors or expand the attack surface of the product?
* **Deployment Services:** Automating our deployment process using a CI/CD platform exposes the deployment process as a potential attack vector if the deployment secrets/tokens are compromised or if malicious code through a supply chain attack.
* **Form Submissions:** While the majority of the site content will be static, accepting user input through a feedback form does expose a potential attack vector.
If so, how are we addressing these risks?
* **Security Approval:** Before the official launch of the static site to the public, we will be reviewing our infrastructure and code security practices with the HHS team to ensure that they adhere to HHS standards.
* **Developer Tools:** As part of the Developer Tools deliverable, the team is setting up a series of tools that will enforce certain code quality standards and security checks. These include things like secrets management, code linting, dependency monitoring, etc.
* **Form Submissions:** The implementation plan for form submissions will evaluate and consider common security practices for validating and sanitizing user input. Form submissions will also likely be stored in a system that is separate from the production database with grant data.
## Logs
### Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
3/5/2024
Added change log and implementation log
This is part of the April onsite follow-up
### Implementation log
Use this section to indicate when acceptance criteria in the "Definition of done" section have been completed, and provide notes on steps taken to satisfy this criteria when appropriate.
Date
Criteria completed
Notes
3/5/2024
All criteria
All criteria were previously marked as completed in GitHub when 30k was delivered in September of 2023
# Static site public launch
Deploy a static site with information about the Simpler.Grants.gov initiative.
## Summary details
## Overview
### Summary
* **What:** Make a series of updates to the content and the infrastructure to the static site, so that we are prepared to launch to a wider audience publicly and effectively communicate the project's goals, ways of working, and ongoing efforts.
* **Why:** Continues to improve front-end infrastructure, demonstrate ongoing value both internally and publicly, and begins to build public excitement for the Simpler Grants.gov initiative. These updates underscore our agile methodology of releasing often, even with minimal updates. We also want to ensure that our site can handle an increase in traffic.
* **Who:** The site will be announced directly with a small group of applicants, grantors, and internal stakeholders. These include:
* Federal Demonstration Partnership (mostly applicants)
* S2S Federal User Group (grantors)
* Grantor S2S CGMO (Grants Management Officers and others within HHS)
* Internal content managers
### Business value
#### Problem
In our initial static site release, we released quickly to get learnings and release iteratively. We have limited content on our site that does not properly communicate enough about our project to the public. We also have room for improvement on the infrastructure and operations so that we can confidently handle more traffic.
#### Value
We want to build on our initial release and enhance the site to provide a more comprehensive communication about our project's efforts, ongoing initiatives, and avenues for public involvement. We also aim to validate and reinforce our infrastructure and operations, ensuring they are robust enough to effectively handle the anticipated surge in site traffic.
This allows us to continue to deliver iteratively, improve the site's infrastructure for sustained maintenance, and communicate our project vision, principles, and the way we work more clearly to anyone coming to the site.
#### Goals
* Begin sharing simpler.grants.gov with a targeted set of public stakeholders
* Develop a communication strategy for future stakeholder engagement
* Create the systems and processes needed to ensure the site remains available under heavier traffic
### User stories
* As an **HHS staff member**, I want:
* to approve the content on the site before we share it with the public, so that I know what information will be visible to external stakeholders.
* to know when I can share the site publicly, so that I can direct key stakeholders to a centralized location where they can learn more about the Simpler.Grants.gov initiative.
* to have a communications strategy for stakeholder engagement, so that we have clear expectations about which groups will receive updates on the Simpler.Grants.gov initiative and can review proposed messaging to those groups.
* clear and concise content that effectively communicates our project's goals, vision, and way of working to the public. This user story is essential to ensure that we uphold our commitment to transparency, fostering open communication and alignment with our overarching project objectives.
* As an **internal developer**, I want:
* to be notified when the site goes down, so that I can work to troubleshoot the issue and minimize downtime.
* the front-end infrastructure to handle the expected traffic with limited/no performance degradation, so that the site remains available and responsive after the launch.
* As a **site visitor**, I want:
* the site to load quickly when I visit it, so that I don't have to wait a long time to access the information I'm looking for.
* there to be minimal planned or unplanned downtime, so that I can trust the new Simpler.Grants.gov site will be reliable when I use it to apply for grants.
## Technical description
### Content updates
Prior to the public launch of the site, update the content to include:
* Summary of user research findings that have informed our product roadmap
* Description of our process and approach to building simpler.grants.gov
* Summaries of upcoming deliverables in the roadmap
* More targeted calls-to-action for site visitors
These content updates should be approved following the content review process outlined in our wiki.
### Infrastructure improvements
The focus of infrastructure improvements for this deliverable should be on changes that will help maintain the responsiveness and availability of the site as traffic increases. While the initial public announcement of the site will be targeted to an audience of around 1,000 users, we should should have a plan in place that allows us to easily support increases in site traffic.
In particular, it would be helpful to review our approach to the following:
* Employing static site generation (SSG) when possible
* Caching the site via a content delivery network (CDN)
* Load balancing and/or proxying requests
* Auto-scaling infrastructure resources based on volume of traffic
* Other strategies as recommended by the engineering team
### Incident response
This deliverable also involves adopting or updating a set of systems and procedures to support with incident response in the event that the site experiences downtime.
In particular, it would be helpful to review our approach to the following:
* **Systems**
* Heartbeat monitoring of internal services and public urls to ensure that the website is live and publicly accessible
* Automated notifications when:
* services are down
* 4xx or 5xx requests reach a certain threshold
* resources reach a certain usage threshold or are auto-scaled
* Logging capabilities (e.g. search, filtering) and retention policies to assist with troubleshooting
* **Procedures**
* Documented incident response plan with steps and roles identified
* Documented plan for receiving and responding to alerts
* Staff trainings, tabletop and/or dry-runs for common types of incidents
### Communication strategy
The main communication output needed for this deliverable is an email campaign that is sent to the people with whom we plan to share the site.
An additional stretch goal is to develop a communications plan that outlines future plans for stakeholder engagement about the Simpler.Grants.gov initiative. This plan should answer the following questions:
* Which stakeholder groups are we planning to engage?
* Who is responsible for managing the relationship with each group?
* What is the primary channel and cadence for those communications?
Both the initial email communication and the communication plan should be reviewed and approved by key stakeholders in the recurring communications meetings.
### Email subscription mechanism
Another goal for this deliverable is to adopt a service that allows site visitors to subscribe to a mailing list to receive updates about the Simpler.Grants.gov initiative. As part of this deliverable the team should answer the following questions:
* Which email marketing provider will we use for email-based communications?
* How will visitors submit their information to be added to a mailing list for Simpler.Grants.gov communications?
### Definition of done
Following sections describe the conditions that must be met to consider this deliverable "done".
#### **Must have**
* [x] Basic requirements
* [x] Code is deployed to main & PROD through our CI/CD pipeline
* [x] Services are live in PROD (may be behind feature flag)
* [x] All new services have passed a security review (as needed)
* [x] All new services have completed a 508 compliance review (as needed)
* [x] Data needed for metrics is actively being captured in PROD
* [x] Key architectural decisions made about this deliverable are documented publicly (as needed)
* [x] Functional requirements
* [x] The static site has been updated with content approved by internal stakeholders
* [x] The front-end infrastructure can handle site traffic of up to 1,000 concurrent users
* [x] Engineering staff are notified when the site goes down
* [x] Engineering staff have been trained on a documented incident response plan that describes a set of actions to take when the site goes down
* [x] Communications requirements
* [x] An email has been sent out to the subset of public stakeholders with whom we plan to share the static site
* [x] Key internal stakeholders (e.g. help desk staff, HHS leadership) have been notified when the email is sent out so that they can prepare for questions from public stakeholders
* [x] Help Desk is notified and trained for any potential support issues that may come through
* [x] Site visitors can subscribe to mailing list to receive updates about the Simpler.Grants.gov initiative
#### **Nice to have**
* [ ] A longer-term stakeholder engagement plan has been drafted that describes future phases of engagement
* [ ] Internal stakeholders can preview a live version of content changes before they are visible to everyone on simpler.grants.gov
* [ ] Site visitors can sign up for user research opportunities
* [ ] Tickets and/or documentation have been created that describe the infrastructure changes needed to support future site traffic at the following levels:
* [ ] 10,000 users
* [ ] 100,000 users
* [ ] 1 million users
### Proposed metrics
* Number of unique site visitors
* Total number of site visits
* Total number of visits per page
* Total number of visits to outbound links to external resources
* Site availability
* Deployment build time
* Deployment/hosting costs
* Number of visitors who subscribe to the mailing list
### Destination for live updating metrics
Page on the public wiki with these metrics or a link to our page on . **Note:** This will likely change once we deliver [the public measurement dashboard](https://github.com/HHS/simpler-grants-gov/issues/65).
## Planning
### Assumptions & dependencies
What functionality do we expect to be in place ***before*** work starts on this deliverable?
* [**Static site soft launch**](https://github.com/HHS/simpler-grants-gov/issues/62)**:** Delivered the following functionality that is needed by this deliverable:
* **Front-end CI/CD:** Enables us to run a series of code quality and security checks and deploy front-end code automatically
* **Foundational UI:** Allows us to continue to make site updates using a consistent design system and set of UI components
* **Domain access:** Secured the simpler.grants.gov where the public site will be hosted
What functionality do we expect to be in place by ***the end*** of work on this deliverable?
* **Communications strategy:** Allows us to create a schedule for future release announcements and estimate site traffic
Is there any notable functionality we do ***not*** expect to be in place before works starts on this deliverable?
* **Content management:** A more formal content management process will be defined in its own 30k deliverable.
* **Translation process:** Both translations and internationalization are being worked on in a separate 30k deliverable for static site improvements.
* **Feedback mechanism:** A feedback mechanism will be worked on in a future 30k deliverable alongside the translation process.
### Not in scope
The following work will *not* be completed as part of this deliverable:
* **Translations:** Because translating the content of the site depends on formalizing the translation process, translating site contents into multiple languages is out of scope for this deliverable.
* **User survey:** Because a feedback mechanism won't be adopted during this deliverable, a user survey will also be descoped from the public launch.
* **Testable features from users:** The site will functionally remain a static site. There will be methods for the public to get involved with the project and follow our progress but there is currently no API or functionality on the site for users to engage with in this release.
### Open questions
Who is the audience for the initial public release and what is the estimated size of that audience?
The initial audience will primarily be a set of federal stakeholders and system-to-system (S2S) users who currently use grants.gov often and who will want to stay up to date about new features being tested on simpler.grants.gov.
This initial group is estimated to be around 1,000 stakeholders, but the communications plan will likely involve sharing the site with successively larger groups of public stakeholders shortly afterward. As a result, we'll want to plan for infrastructure updates needed to support site traffic at multiple levels (e.g. 1k users, 10k users, 100k users, etc.)
What are the expectations about site availability, responsiveness, and traffic?
We have not defined explicit SLAs for site availability or responsiveness yet, but beginning to think about targets for responsiveness and uptime is one of goals of this deliverable.
In identifying those targets, however, the team should consider the importance of maintaining a high rate of availability for this site in building both public and federal trust in the Simpler.Grants.gov initiative.
## Integrations
### Translations
Does this deliverable involve delivering any content that needs translation?
* Yes, the site contents will need to be translated -- but not as part of this deliverable.
If so, when will English-language content be locked? Then when will translation be started and completed?
* The site contents will be translated when we complete the [static site improvements deliverable](https://github.com/HHS/simpler-grants-gov/issues/568)
### Services going into PROD for the first time
This can include services going into PROD behind a feature flag that is not turned on.
* **Subscription to mailing list:** This deliverable *may* include deploying a service that allows site visitors to sign up for a mailing list for updates about the Simpler.Grants.gov initiative.
### Services being integrated in PROD for the first time
Are there multiple services that are being connected for the first time in PROD?
* **Static site + mailing list:** If this deliverable includes a mechanism for visitors to subscribe to a mailing list, it will need to be integrated in the static site for the first time.
### Data being shared publicly for the first time
Are there any fields being shared publicly that have never been shared in PROD before?
* No, the content of the static site in this milestone will be limited to general information about the Simpler.Grants.gov project. It does not include exposing any production data from the new simpler.grants.gov data model.
### Security considerations
Does this deliverable expose any new attack vectors or expand the attack surface of the product?
* If it is included in this deliverable, the mechanism to allow site visitors to sign up for our mailing list will expose a new attack vector through the form submission for this service.
If so, how are we addressing these risks?
* The implementation plan for email subscriptions will evaluate and consider common security practices for validating and sanitizing user input. Where possible, we should adopt an existing system that HHS uses to manage email subscriptions.
## Logs
### Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
4/5/2024
Added change log and implementation log
This is part of the April onsite follow-up
### Implementation log
Use this section to indicate when acceptance criteria in the "Definition of done" section have been completed, and provide notes on steps taken to satisfy this criteria when appropriate.
Date
Criteria completed
Notes
3/5/2024
All criteria
All criteria were previously marked as completed in GitHub when 30k was delivered in December of 2023
# GET Opportunities
Deploy an API endpoint that shares public information about every opportunity on grants.gov.
## Summary details
## Overview
### Summary
* **What:** Deploy a public API endpoint to PROD that allows users to see at least one field per grant opportunity listed in grants.gov.
* **Why:** Build iteratively, validate security approval and connection to the current production database, and set the groundwork for continued work on the API.
* **Who**
* Internal development team
* Internal HHS stakeholders
### Business value
#### Problem
A previous effort to modernize grants.gov attempted to replicate *all* of the existing functionality in a sandbox environment, then replace the legacy system in production at a fixed cutover date. By isolating the release of new features from production data, this approach reduced the ability to conduct meaningful usability testing and significantly increased the overall risk associated with the migration.
However, getting the necessary security approvals to expose production data in a new software system takes a fair amount of effort and documentation. For this reason, many federal software projects opt to defer working with production data for as long as possible, despite the risk it adds to the success of the project overall.
#### Value
Publishing live data from grants.gov in a new simpler.grants.gov public API proves that we have successfully established a connection between grants.gov and simpler.grants.gov environments *and* received the security approvals required to share production data through the simpler.grants.gov API publicly.
Additionally, by building features on top of production data, we enable users to test this new functionality against real-life scenarios and needs. This approach significantly reduces the long-term risk of the project relative to building and testing features against mock data and then attempting to replace all of the existing functionality in a single migration multiple years into the project.
#### Goals
* Select a URL for the API
* Expose the chosen URL name as the public API for the project that technical stakeholders can access and where future endpoints and/or features will be released
* Prove the successful completion of technical deliverables that enable faster development on the API without sacrificing code quality or security
* Prove that we have gotten the necessary security approval to host and share grants.gov production data in simpler.grants.gov environments
### User stories
* As an **HHS staff member**, I want:
* the API to adopt the proper security practices, so that we have a strategy for preventing and responding to security vulnerabilities before the API is launched
* published data about opportunities to be consistent between legacy grants.gov and `simpler.grants.gov`, so that users won't be confused by discrepancies between these sources
* As a **consumer of the API**, I want:
* clear documentation and a user guide for the API, so that I don't have to rely on reading the source code to learn how to interact with and consume from it
* changes made to a given endpoint to be backward-compatible, so that I can start building against this API without worrying about breaking changes
* a clear and relatively intuitive data model to represent the opportunities in the API, so that the data returned by the endpoints match my expectations and are easy to work with
* reliable uptime for the API, so that I don't have to worry about interruptions in API service breaking any systems that I try to integrate with it
* As a **project maintainer**, I want:
* most of the critical development infrastructure to be in place when we officially launch the API, so that we can deploy bug fixes or new features quickly once the API is live
* to be alerted when the API is down, so that I can troubleshoot the issue with minimal downtime or interruption in service
* to automatically collect data on API usage and service availability, so that I can better understand usage patterns and identify opportunities to improve the API performance and reliability
* the API code to be designed to minimize repetitive configuration for the data model, so that updates to the definition of a field in the API does need to be manually changed in the core data classes, API documentation, DB migration scripts etc.
* As an **open source contributor**, I want:
* to be able run the code for the API locally, so that I can test my contributions to the codebase.
## Technical description
### Infrastructure requirements
The infrastructure developed to deploy and host the API should balance:
* Code quality
* Security
* Delivery velocity
* Cost & maintenance
### Developer experience requirements
The API should be developed and versioned in a way that balances:
* Discoverability
* Ease of adoption
* Backwards compatibility
* Clear and intuitive data model
* Minimal repetitive configuration for data model
### Data requirements
The way that data is stored and delivered through the API should balance:
* Eventual consistency with legacy Grants.gov
* Improvements to the existing data model
* Ease of managing schema changes
### Definition of done
Following sections describe the conditions that must be met to consider this deliverable "done".
* [x] The following infrastructure requirements are satisfied:
* [x] The code needed to build and deploy the site is merged to `main`
* [x] The site is built and hosted with the tools selected in the [API Planning](https://github.com/HHS/simpler-grants-gov/issues/42) and [DB Planning](https://github.com/HHS/simpler-grants-gov/issues/48) deliverables
* [x] All code quality checks set up in the [Developer Tools deliverable](https://github.com/HHS/simpler-grants-gov/issues/50) are passing
* [x] The resources required to deploy and host the API are provisioned programmatically using the [Infrastructure-as-Code deliverable](https://github.com/HHS/simpler-grants-gov/issues/123) framework
* [x] Code changes are deployed using the CI/CD pipeline set up in [the Back-end CI/CD deliverable](https://github.com/HHS/simpler-grants-gov/issues/57)
* [x] DB migrations are automatically configured through scripts that enable upgrading/downgrading database quickly and easily (e.g., by using Alembic )
* [x] The API has been load tested using the framework established in the [Peformance Testing Framework deliverable](https://github.com/HHS/simpler-grants-gov/issues/69) to ensure that it remains performant under heavy user traffic
* [x] Logging/monitoring is configured, and it both records the metrics defined below and alerts the development team when the API is down or other key monitoring thresholds are met (e.g. frequency of 4xx requests, response times, etc.) per the [API Logging & Monitoring deliverable](https://github.com/HHS/simpler-grants-gov/issues/370)
* [x] An incident response protocol is in place and the on-call team have followed that protocol in at least one training or simulation per the [Incident Response deliverable](https://github.com/HHS/simpler-grants-gov/issues/373)
* [x] The chosen API URL sub-domain has been secured for future deployment of the API and we've contacted the teams working on the existing service (if any) that is currently accessed through this sub-domain
* [x] The following developer experience (DX) requirements are satisfied:
* [x] The API is live at the chosen URL
* [x] Developers can learn how to interact with the API by referencing the API documentation
* [x] The endpoint path indicates which major version of the API the developer is consuming
* [x] Breaking changes to the API follow a predictable protocol that is documented within the [API Versioning deliverable](https://github.com/HHS/simpler-grants-gov/issues/68)
* [x] The endpoint is available when legacy grants.gov experiences planned or unplanned downtime, maintenance, and upgrades
* [x] Test data is scripted to provide consistent and reliable test fixtures for integration tests and local development per the [Test Data and Schema deliverable](https://github.com/HHS/simpler-grants-gov/issues/)
* [x] All developers (including open source contributors) are able to spin up either database replica or test fixture data so that they can conduct local development.
* [x] The following data requirements are satisfied:
* [x] The endpoint returns all of the grant opportunities that are available on grants.gov
* [x] The endpoint returns at least one (1) field per opportunity
* [x] Updates to the data in legacy Grants.gov are propagated to the new endpoint within 1 hour
* [x] Our desired project metrics are captured and displayed in a public place
### Proposed metrics
* Number of API keys
* Total number of API calls made
* Error rate of API calls
* Uptime of service
* Deployment/hosting costs
* Average response time
### Destination for live updating metrics
Page on the public wiki that is updated at the end of each sprint. **Note:** This will likely change once we deliver [the Public Measurement Dashboard deliverable](https://github.com/HHS/simpler-grants-gov/issues/65)
## Planning
### Assumptions and dependencies
What functionality do we expect to be in place ***before*** work starts on this deliverable?
* [**simpler.grants.gov domain**](https://github.com/HHS/simpler-grants-gov/issues/)**:** Secures access to the `simpler.grants.gov` domain from which the API endpoints will be routed.
Are there any notable capabilities / deliverables we **do** **not** expect to be in place by the completion of work on this deliverable?
* **AuthN/AuthZ:** While the implementation of rate limiting or other API security measures may require some basic authentication, the full AuthN/AuthZ framework will be developed in a later deliverable.
### Not in scope
List of functionality or features that are explicitly out of scope for this deliverable.
* **User Interface:** Because this deliverable is focused on the API endpoint, it will not include delivering a user interface for non-technical users to access a list of opportunities. That work will be incorporated in the Search UI deliverable instead.
* **Translating API Docs:** Translation of key documents will be covered in an upcoming deliverable.
* **Public communications:** This deliverable does *not* include making a public announcement about the launch of the API. An announcement to targeted groups of system-to-system (S2S users) will be scoped into a subsequent deliverable.
## Integrations
### Translations
Does this deliverable involve delivering any content that needs translation?
* Yes, portions of the API user guide and docs will need to be translated.
If so, when will English-language content be locked? Then when will translation be started and completed?
* Timeline and strategy for translation is still TBD.
### Services going into PROD for the first time
This can include services going into PROD behind a feature flag that is not turned on.
* **API:** This deliverable is the official release of the `simpler.grants.gov/api`
* **Replica Database:** A replica of relevant tables from the legacy database
* **Updated Data Model:** An updated data model that will provide the data for the GET Opportunities endpoint
* **ETL Pipeline:** An ETL pipeline that both replicates data from legacy grants.gov and then transforms that data into the new `simpler.grants.gov` data model
### Services being integrated in PROD for the first time
Are there multiple services that are being connected for the first time in PROD?
* **API + Static Site or Wiki:** We will need to host the API docs and user guide on either the wiki platform or the static site.
### Data being shared publicly for the first time
Are there any fields being shared publicly that have never been shared in PROD before?
* **Opportunity Field(s):** This deliverable will expose at least one field from the opportunity resource in production.
### Security considerations
Does this deliverable expose any new attack vectors or expand the attack surface of the product?
* **Legacy DB Access:** Because this deliverable requires replicating data from the legacy database, it exposes a new potential attack vector to that database.
* **Replica Database Access:** This deliverable expands the attack surface of the application by introducing the replica database as another data store that needs to be secured against unauthorized access.
* **API:** This deliverable deliverable also expands the attack surface of the application by launching the API, which needs to be secured against Denial of Service (DoS) attackes.
If so, how are we addressing these risks?
* **Security Approval:** Before the official launch of the API to the public, we will be reviewing our infrastructure and code security practices with the HHS team to ensure that they adhere to HHS standards.
* **Developer Tools:** As part of the Developer Tools deliverable, the team is setting up a series of tools that will enforce certain code quality standards and security checks. These include things like secrets management, code linting, dependency monitoring, etc.
* **API Security Planning:** As part of the API Security Planning deliverable, we will specifically be identifying and evaluating strategies to mitigate security risks for the API, such as the use of API tokens and/or rate limiting API requests.
## Logs
### Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
4/5/2024
Added change log and implementation log
This is part of the April onsite follow-up
4/5/2024
Adds acceptance criteria for publish metrics publicly to match GitHub issue
This acceptance criteria was added in GitHub on 3/14/24 to clarify the expectation around how stakeholders would access metrics for this 30k
4/10/2024
Changed metric for "Number of unique users" to "Number of API keys"
Change was made during 30k review meeting
### Implementation log
Use this section to indicate when acceptance criteria in the "Definition of done" section have been completed, and provide notes on steps taken to satisfy this criteria when appropriate.
Date
Criteria completed
Notes
4/5/2024
All criteria except for:
Our desired project metrics are captured and displayed in a public place
Criteria were previously marked as completed in GitHub, with the exception of publishing metrics.
4/10/2024
Completed and checked off:
Our desired project metrics are captured and displayed in a public place
Updated in 30k review meeting on 4/10/24
# Open source onboarding
Template page for deliverable specifications.
## Summary details
## Overview
### Summary
* **What:** Set up the tools and processes needed to build an open source community around the Simpler Grants.gov initiative
* **Why:** Ensures that open source contributors and the general public can easily participate in the project and provide input and code
* **Who:** Any public members who want to contribute to the open source project. Focus is on:
* Open source contributors
* Other external stakeholders
### Business value
#### Problem
Currently, the Simpler.Grants.gov project lacks a ready-to-use onboarding experience for individuals interested in joining the open source community. The absence of essential tools, processes, onboarding steps, and guidelines poses a challenge to meeting business goals and mission alignment.
#### Value
We seek to build stakeholder trust, enhance project transparency, and establish a dynamic open source community. Our focus is on foundational components that facilitate future expansion and productive collaboration, ensuring efficient community management and promoting transparency in project development.
**Goals**
This effort allows us to...
* Set up the foundational set of communication channels for continuous user input, feedback, and engagement for an open source project
* Ensure that public code contributions meet code quality and security standards
* Build stakeholder trust in our product roadmap and our approach to development
* Streamline and standardize the onboarding process for new open source contributors
### User stories
* As a **full-time HHS staff member**, I want to:
* have assurance that the open source channels and methods are configured in a way that eliminates security risks.
* As a **member of an HHS contractor team**, I want to:
* have streamlined tools and processes for managing contributions, reviewing code, and ensuring code quality, so I can efficiently maintain and enhance the project while upholding its standards.
* to have a standardized process for onboarding new open source contributors, so that members of the public can be added to our communication channels smoothly and efficiently.
* to have the ability to monitor the health and inclusivity of the project by tracking metrics like community engagement, issue resolution times, and the diversity of contributors, so I can make informed decisions for community growth and sustainability.
* As a **member of the public**, I want to:
* have easy access to clear documentation and guidelines for joining, contributing, and engaging with the open source community so I can quickly become a productive contributor and understand the community's values and expectations.
* have effective communication channels, regular updates, and opportunities for networking and collaboration, so I can stay informed, participate in discussions, and contribute to the community's goals, fostering an active and thriving open source ecosystem.
## Technical description
### Communication tools onboarding
There are some basic communication tools that are required for a good onboarding experience for an open source contributor. Configuration and setting up an obboarding experinece for communications and contributing tools such as a Github and Slack are required. Specify the platforms, integration methods, user access levels, onboarding experience, and any technical considerations for making these tools fully operational for the community.
### Team meetings and open source events
Detail the aspects of scheduling and conducting team scrum meetings. This can include information about the chosen video conferencing tools, meeting frequency, methods for the general public to join open meetings, and any requirements for ensuring effective remote communication during these meetings.
### Getting Started Guide Development
There is clear documentation and guidelines for the general public to get started and contribute. Documentation typically should include project overviews, installation instructions, usage guidelines, contribution procedures, and community engagement details, fostering a collaborative and informed open source community.
### Developer Tools Setup
Provide technical instructions for setting up developer tools and environments. This can encompass version control systems, code review tools, and any collaborative platforms for coding and testing.
### Definition of done
Following sections describe the conditions that must be met to consider this deliverable "done".
* **Must have**
* [x] The following tools have been procured and implemented for internal and public use:
* [x] **Slack** - chat-based community engagement
* [x] **GitBook** - public-facing wiki and knowledge base
* [x] **Zoom** - webinars, office hours, or other open source events
* [x] **Google group** - email-based community engagement
* [x] The following conditions have been satisfied for *all* tools:
* [x] Services are accessible to all people on the HHS network, public internet, and *preferably* the White House network and most or all agency networks
* [x] Instructions for the internal team that assists with onboarding new users are clearly and accessibly documented in our public or internal wiki
* [x] Code for managing these services is deployed to `main` & PROD (if necessary)
* [x] An ADR has been recorded that documents why the tool was chosen
* [x] There is a public page of information that explains how to onboard to all of the communication tools for the open source community (e.g., this could be hosted in the public wiki)
* [x] The onboarding page is linked from simpler.grants.gov
* [x] The onboarding page is linked from the \`simpler-grants-gov\` GitHub repo
* [x] Onboarding to communication channels ready for the public this includes:
* [x] onboarding to the selected chat tool, Slack
* [x] onboarding to Github
* [x] onboarding to Google Groups
* [x] materials for getting started with the project
* [x] onboarding guidance for developer tools and environments
* [x] We have onboarded three members of the general public to the following communication tools:
* [x] Slack
* [x] Google groups
* [x] GitHub
* [x] We have a system for tracking the onboarding process for all new members, ensuring that they have agreed to the terms of use (such as community agreements and content sensitivity guidelines) and completed all necessary onboarding steps
* [x] Tool-specific requirements:
* [x] The public wiki can be accessed from a custom domain that is easy to remember and specific to the Simpler.Grants.gov initiative
* [x] The public wiki includes copies of our deliverable specs and information about our roadmap and planning process
* [x] The public wiki includes a user guides for all public-facing communication tools
* [x] Public Slack channels have been reviewed and sensitive content has been removed or the channel has been made private
* [x] At least three GitHub issues have been labeled with "help wanted"
* [x] We've documented an information architecture for the public wiki, so that content has clear organizing principles and it is easy to know where to add or look for content
* [x] We have a system for "offboarding" members of the community from all tools
* [x] We are able to "block" users who violate community guidelines and offboard them from all tools without their participation in the offboarding process
* [x] We have a process documented for what to do if anyone notices inappropriate content or behaviors and how to escalate and remove the content
* **Nice to have**
* [ ] Members of the public can suggest changes to content in the wiki through a pull request
* [ ] Links to each of these communication tools are available on simpler.grants.gov and in the repository's main README
### Proposed metrics
* Number of users onboarded to the open source community
* Time to onboard to the open source community. For example, lead time between opening and closing a ticket
* Slack metrics
* Number of monthly active users in slack, total
* Number of monthly active users in slack, external
* Weekly volume of chat messages
* Wiki metrics
* Total number of visitors
* Number of unique visitors
* Number of visitors per page
* GitHub metrics
* Number of stars
* Number of forks
## Planning
### Assumptions and dependencies
What functionality do we expect to be in place ***before*** work starts on this deliverable?
* [x] **Static site:** The static site should be publicly deployed so that we can direct open source contributors to learn more about the program at simpler.grants.gov. We may also want to use the site to solicit requests to join the open source group.
* [x] **Tool procurement:** Procurement of the following tools tools should be in place as the procurement process can delay delivery.
* [x] Slack
* [x] GitBook
* [x] Github
* [x] Zoom
* [x] Google Groups
Is there any notable functionality we do ***not*** expect to be in place before works starts on this deliverable?
* **Participant advisory council (PAC):** We will not have a participant advisory council in place when work begins on this deliverable. Instead, this deliverable will be required to onboard members of the PAC once we begin work on setting it up.
* **Site content translations:** We will not yet have a process in place to translate the content of our static site or repository documents. That will be addressed in a future 30k ft deliverable.
### Not in scope
List of functionality or features that are explicitly out of scope for this deliverable.
* We will not be hosting an open source kickoff within this effort. However, we do plan to onboard members of the public within this effort.
### Open questions
Is it important to have a public wiki available and ready in this deliverable?
Yes, there should be a minimal version of the public wiki available to share with the general public.
Is it important to have a public API documentation ready for consumption in this deliverable?
No, we will handle public API documentation in another 30k deliverable.
## Integrations
### Translations
Does this deliverable involve delivering any content that needs translation?
* **User guides in wiki** - User guides for our main communication tools would ideally be translated. And we should consider translating other relatively static content in the wiki moving forward.
* **Repository documents** - Relatively static and central documentation in GitHub such as the main repository README, code of conduct, and contributing guidelines should eventually be translated.
If so, when will English-language content be locked? Then when will translation be started and completed?
* Translations will need to happen *after* the onboarding process is delivered. We'll track those translations in the process defined by the content translation process deliverable.
### Services going into PROD for the first time
This can include services going into PROD behind a feature flag that is not turned on.
* **Zoom** for video conferencing. Only internal teams will have a Zoom license, but the public will join Zoom
* **Google group:** We'll use this for email-based communication with our open source community.
* **GitBook:** for the public wiki. We'll be using this to share public information about the project, such as onboarding guides, presentations, and meeting notes for public meetings.
### Services being integrated in PROD for the first time
Are there multiple services that are being connected for the first time in PROD?
1. **Chat + Ticket tracking:** Option to receive updates on key tickets in chat
2. **Wiki + Chat:** Option to receive updates on key document changes in chat
3. **Video Conference + Shared Calendar:** Option to add video conference details to events
4. **Shared Calendar + Wiki:** Option to embed public calendar events in the wiki
### Data being shared publicly for the first time
Are there any fields being shared publicly that have never been shared in PROD before?
* No, this 30k deliverable does not involve sharing any new production data.
### Security considerations
Does this deliverable expose any new attack vectors or expand the attack surface of the product? If so, how are we addressing these risks?
* Adding members of the public to Slack, Google Groups, and Zoom can pose a risk. Each tool contains their own risks:
* Slack contains a mix of public and private channels.
* GitBook contains both an internal wiki and a public wiki.
* Slack, Google Groups, and Zoom requires the public to sign up by giving their name and email address
* Mitigation strategies:
* Review the user agreements and/or terms of service for the different tools (Google Groups, Zoom, etc) to ensure they state that any data collected (like name and email) will be shared with the owner of the instance being used. This ensures we legally own the data in case of a breach. Julius and Lucas have reviewed this risk and agreed that no SIA is needed in this case
* We have policy guidelines around where to post sensitive content for all forums where the general public can post. If PII data is posted in public channels, we will have a plan to remove the sensitive data. We will need to consider how each of the tools handle versioning as well. For example, in Github, since data is still in the git history, we will not be able to just delete the data from the repo and update commits. We need to have a mechanism and plan to remove sensitive data
* We've trained internal staff on those policies
* We'll review public channels for sensitive content and remove that content prior to inviting open source contributors.
* We will review with the security team to ensure that we can collect data using these tools before we do so.
## Logs
### Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
4/5/2024
Added change log and implementation log
This is part of the April onsite follow-up
4/26/2024
Updated specification for completion of the 30k.
### Implementation log
Use this section to indicate when acceptance criteria in the "Definition of done" section have been completed, and provide notes on steps taken to satisfy this criteria when appropriate.
Date
Criteria completed
Notes
4/17/2024
Checked off all criteria except for Onboarding 3 members of the public and Nice to Have Criteria
Only at 1/3 members of the public onboarded. Working to get the rest across the finish line.
4/24/2024
Updated and checked off onboarding criteria.
# Co-Design Group planning
Establish a co-design group plan to prepare to engage representatives from underserved communities to collaborate with HHS to ensure a more accessible grant experience
## Summary details
## Overview
### Summary
* **What:** Define scope and establish the necessary administrative work to prepare for onboarding target underserved communities to build a Co-Design Group around the Simpler.Grants.gov initiative
* **Why:** Ensures that Simpler.Grants.gov HHS decision makers design with the most underserved communities in the grants ecosystem and create continuous feedback loops to ensure equitable product development. This includes framing strategic problems, roadmap development, designing solutions, and iteration.
* **Who**
* The most underserved applicant individuals/groups within the grants ecosystem
* Internal Simpler.Grants.gov stakeholders
**Out of scope**
* Defined plans for individual Co-Design Group activities
* We will have other forms of user research for more diverse user communities that will be handled outside of this 30k deliverable
* Onboard participants to the Co-Design Group and host a Co-Design Group session. We will onboard participants and kick-off a Co-Design Group in a future, separate deliverable
### Business value
#### Problem
Setting up a Co-Design Group with HHS requires planning and coordination with various stakeholders. We want to set up the planning processes so that we can more smoothly transition into a future deliverable of kicking off the Co-Design Group sessions.
Setting up a Co-Design Group helps us address the following problems:
* Firstly, the current grant making system privileges certain groups above others according to access to power and resources, making grants distribution highly inaccessible to specific groups.
* Secondly, the most impacted underserved communities have not been concretely identified or involved in the Simpler.Grants.gov project, which inhibits Simpler.Grants.gov stakeholders from framing and solving specific contextual problems faced by these underserved communities in tandem with product build.
* Thirdly, there are no research protocols or process for obtaining participant consent that proactively mitigates potential participant harm.
* Finally, the product roadmap has not been designed to integrate opportunities for executing and implementing insights from user research, which would mitigate risks in user adoption and usage.
#### Value
By completing the administrative and planning efforts to establish the Co-Design Group, we are able to remove blockers and start the work to recruit and host Co-Design Group sessions.
By identifying and directly involving representatives of the Novice archetype, [identified through previous research](https://simpler.grants.gov/research), along with other underserved communities in project design, Simpler.Grants.gov HHS decision makers increase the likelihood of identifying and solving the right problems, those that will increase grants access to underserved communities. Moreover, building relationships with these communities will allow internal stakeholders to identify and uplift existing community designed solutions with Simpler.Grants.gov work, thereby removing the risk of duplication of effort and increasing the chance of user adoption. To this end, value is rooted in [Design Justice Principles](https://designjustice.org/read-the-principles), which centers the experiences of underserved end users throughout the entire framing, execution, and evaluation of project planning and delivery and acknowledges and extends the reach of already existing community designed solutions. By improving the experience and access for the most underserved communities, we improve the experience and access for everyone who uses the service.
Once created, the Co-Design Group will enable Simpler.Grants.gov internal stakeholders to more effectively implement best practices in user research by identifying opportunities for research ahead of time, mitigating risk, and more quickly standing up generative and evaluative studies with a pre-screened target user panel. The Co-Design Group will develop research protocols and obtain participant consent to ensure that participants are protected from harm that might arise from government stakeholders obtaining potentially sensitive data.
#### Goals
This effort allows us to…
* Set the foundations to bring the most impacted applicants into partnership with grants.gov internal stakeholders to frame, plan, design, and solution grants experience
* Create continuous feedback loop between community groups already working on behalf of marginalized community and allows grants.gov internal stakeholders to contribute to their existing solutions and build with (not for)
* Increase speed of participant recruitment for studies that aim to gather insights center user needs and behaviors throughout the product development lifecycle
* Improve strategy and proactive planning for future grants.gov work with Co-Design Group participants to frame problems and contribute insights that will guide roadmap development based on the real challenges users navigate in their lives
### User stories
* As a **HHS staff member**, I want to:
* center the voices of those who are directly impacted by the outcome of the design process so that I can ensure that our solutions lead to sustainable positive outcomes
* As a **Co-Design Group administrator and project maintainer**, I want:
* a streamlined process for collaborating with Co-Design Group members so that I can ensure that there is continuous feedback loop throughout strategic planning and product development
* to ensure that the Co-Design Group includes representatives from a range of communities facing limited access so that they can reflect diverse voices and perspectives of those impacted by the outcomes
* a longer-term view and tactical plan for how we will leverage the Co-Design Group so I can build trust with the group, set expectations, and right-size capacity needed to run the group for a defined duration of time
* a manageable size of participants that matches the need of the project and capacity of the team
* ensure that participants understand how their data will be used, obtain their consent, and that I have a way of protecting and anonymizing their data
* As a **grantor**, I want:
* Simpler Grants.gov to be simple, effective, and accessible and work for all communities and individuals in the grants ecosystem so that it’s easier to attract a wide range of candidates
* As an **applicant and a member of the general public**, I want:
* Simpler Grants.gov to be simple, effective, and accessible and work for all communities and individuals in the grants ecosystem so that it’s easier to find and apply for grants
### Definition of done
Following sections describe the conditions that must be met to consider this deliverable "done".
#### Must have
* [x] General requirements
* [x] ~~All new services have been submitted for a 508 compliance review (if necessary)~~
* [x] Key architectural decisions made about this deliverable are documented publicly
* [x] Finalize the name of what we will call the participatory group and approval from HHS
* [x] A screening and consent process has been drafted and documented for the Co-Design Group, and this draft includes:
* [x] How to screen applicants against an explicitly documented set of selection criteria
* [x] Clear guidelines and expectations for participation
* [x] Documentation needed for onboarding participants has been created
* [x] How to form a community that represents underserved communities that have been determined to have unequal access to grants
* [x] Compensation guidance has been provided to HHS based on best practices and industry standards, so that HHS can determine a budget
* [x] The following requirements satisfied to facilitate collaboration with members of the Co-Design Group:
* [x] Ethn.io has been procured and configured
* [x] HHS security team has approved Simpler.Grants.gov to collect and store Personally Identifying Information (PII) in our systems
* [x] Project maintainers and Co-Design Group facilitators have identified opportunities in the product roadmap where the Co-Design Group will be leveraged in decision making by HHS
#### Nice to have
* [ ] HHS approval of the compensation budget received
* [ ] Information and resources for the Co-Design Group are available in a central location, and this knowledge base:
* [ ] Is built on top of one of our existing communication tools, e.g. wiki page or Google Drive
* [ ] Should be accessible by members of the Co-Design Group at no cost to them
* [ ] Can be accessed by (a subset of) members of the Simpler.Grants.gov team
* [ ] A centralized communication channel is selected for Co-Design Group collaboration, and this channel:
* [ ] Is built on top of one of our existing communication tools, e.g. wiki page or Google Drive
* [ ] Should be accessible by members of the Co-Design Group at no cost to them
* [ ] Can be accessed by (a subset of) members of the Simpler.Grants.gov team
* [x] A proposal for including agency and other diverse actors in continuous feedback loops that enable anyone to give feedback on grants.gov has been documented, and this proposal:
* [x] Answers a series of open questions about how we involve agency partners and others in a robust way throughout the project
* [ ] Identifies potential changes that need to be made to the screening and consent process for agency representatives
## Planning
### Assumptions and dependencies
What functionality do we expect to be in place ***before*** work starts on this deliverable?
* Method/platform for holding and protecting participant PII, Ethn.io, will need to be procured
Is there any notable functionality we do ***not*** expect to be in place before works starts on this deliverable?
* Translations
### Not in scope
List of functionality or features that are explicitly out of scope for this deliverable.
* Defined plans for individual Co-Design Group activities
* This effort will not include more than one Co-Design Group engagement schedule beyond applicants
* Onboarding participants to the Co-Design Group
* Conduct sessions with participants for the Co-Design Group
### Open questions
What are the PRA requirements?
We can fast track specific approvals under the overall existing generic clearance. Fast tracks usually take 3-5 days. Felix Lorenzo at HHS owns this process.
Will we provide translations for people for whom English is not their first language?
It is likely that people whose first language is not English are more disadvantaged than other communities. We may be limited in our ability to provide real-time translations in this initial initiative, but we should plan how we can engage those communities. We should be able to provide text materials in other languages but real-time translations may be difficult.
We could potentially focus on a demographic segment where users who are fluent in English but not their first language attend sessions that help us get some input into this underserved population
Would we need a 508-compliance review of any materials (PDFs, slide decks, etc.) we're preparing for the Co-Design Group?
No 508 compliance review is required during planning. We'll be using standard, tested, accessible tools (email, Ethnio, etc.). We will evaluate subsequent materials and deliverables as needed during kickoff and ongoing operations.
## Implementation
### Translations
Does this deliverable involve delivering any content that needs translation?
* No, as we will not be onboarding participants in this deliverable, we will not know what translations will be needed by participants.
If so, when will English-language content be locked? Then when will translation be started and completed?
* n/a
### Services going into PROD for the first time
This can include services going into PROD behind a feature flag that is not turned on.
* The tools for procurement and dispersing compensation will be a new tools utilized on the project
### Services being integrated in PROD for the first time
Are there multiple services that are being connected for the first time in PROD?
* The tool for procurement and dispersing compensation and managing recruitment
### Data being shared publicly for the first time
Are there any fields being shared publicly that have never been shared in PROD before?
* n/a
### Security considerations
Does this deliverable expose any new attack vectors or expand the attack surface of the product?
* The recruitment and participant management tool will store PII data and we may need to have security approval to use the tool.
If so, how are we addressing these risks?
* We will check with MH and HHS security to ensure that we have permissions to use the tool selected
## Logs
### Change log
Major updates to the content of this page will be added here.
Date
Update
Notes
4/5/2024
Added change log and implementation log
This is part of the April onsite follow-up
4/15/2024
Updates name of deliverable to "Co-Design Group"
4/19/2024
Updated the deliverable spec to change scope to exclude onboarding activities as they are blocked by compensation
Deliverable spec title name changed (added "planning" to the name)
Deliverable spec short description updated
What section is updated
Out of scope section
Goals section updated
Technical description removed
DoD updated in must-haves and nice-to-haves to remove onboarding language
Proposed metrics removed
Planning section - solution agnostic language updates, questions updated because of descope of onboarding
5/2/2024
Moved deliverable status to "Done"
30k was identified as completed at our weekly 30k review meeting.
### Implementation log
Use this section to indicate when acceptance criteria in the "Definition of done" section have been completed, and provide notes on steps taken to satisfy this criteria when appropriate.
Date
Criteria completed
Notes
April 8, 2024
Finalize the name of what we will call the participatory group and approval from HHS
Project maintainers and co-design group facilitators have identified opportunities in the product roadmap where the Co-Design Group will be leveraged in decision making by HHS
Compensation guidance has been provided to HHS based on best practices and industry standards, so that HHS can determine a budget
Guidance has been sent to HHS. There has been a memo sent for approval of the compensation budget
April 26, 2024
Key architectural decisions made about this deliverable are documented publicly
Not applicable, no ADRs were or will be created. The Ethn.io ADR does inform this decision but was decided under the Search UI deliverable
April 30, 2024
…508 compliance…
n/a (We're using standard/tested tools, email etc. No 508 compliance required)
April 30, 2024
How to screen applicants against an explicitly documented set of selection criteria
"Intake form" (screener) written and sent thru approval process. Criteria defiend by SME interviews; they have made recommendations for representation of underserved communities.
April 30, 2024
Clear guidelines and expectations for participation
Documentation needed for onboarding participants has been created
"Statement of expectations", "Intake Script", "Invitation email", and "Participation Agreement Form" written and sent thru approval process. These will define expectations for participants.
April 30, 2024
How to form a community that represents underserved communities that have been determined to have unequal access to grants
SME interviews determined representative participant candidates (verbatim notes from each meeting), contacts gathered from interviews (recorded in spreadsheet)
Info sharing w/ team on radical participatory design & user research (via Slack)
April 30, 2024
A proposal for including agency and other diverse actors in continuous feedback loops that enable anyone to give feedback on grants.gov has been documented, and this proposal:
Answers a series of open questions about how we involve agency partners and others in a robust way throughout the project
# Decisions
## About ADRs
### Why do we need to document architecture decisions?
Developing software involves making lots of decisions. Documenting these decisions and their outcomes achieves several important goals:
1. **Knowledge Preservation:** Architectural decisions capture the rationale behind important choices made during the design and development process. They serve as a reference point for future developers, allowing them to understand the thinking and context behind the architecture. Without proper documentation, critical knowledge can be lost over time as team members change or move on to other projects.
2. **Communication and Collaboration:** Documentation helps facilitate communication among team members and stakeholders. It allows the product owner, developers, and other stakeholders to share a common understanding of the software's architecture, its components, and the reasoning behind specific decisions.
3. **Decision Transparency:** ADRs provide greater insight into the choices made throughout the project. It helps team members and stakeholders understand the factors considered, alternatives evaluated, and the trade-offs involved in the decision-making process. This transparency fosters trust, enables better collaboration, and allows for informed discussions and feedback.
4. **Risk Management:** Documenting key decisions also allows project stakeholders to assess the potential impact of architectural changes, identify dependencies, and evaluate the consequences of modifying specific components. By having a clear understanding of the trade-offs associated with each option, teams can make more informed decisions and mitigate potential risks associated with changes or enhancements.
5. **Improved Maintainability and Adaptability:** Software projects evolve over time due to changing requirements, new features, or technological advancements. Documentation ensures that future developers can understand the architecture's design principles, patterns, and constraints. It enables future maintainers to modify or extend the codebase in a consistent and coherent manner, reducing the risk of introducing unintended side effects or architectural drift.
6. **Onboarding and Knowledge Transfer:** When new team members join a project, ADRs serve as a valuable resource for onboarding. They provide insights into the design choices, patterns, and best practices employed in the codebase. Documentation reduces the ramp-up time for new team members and helps them become productive more quickly.
### What constitutes an architecture decision?
Almost all of engineering and code development involves making choices, so how do you know when a particular choices rises to the level of an architecture decision and requires an ADR?
> An Architectural Decision (AD) is a software design choice that addresses a functional or non-functional requirement that is architecturally significant.
>
> Source: [ADR GitHub Organization](/product/decisions/adr)
Another way to recognize when making a decision related to your project constitutes an architecture decision is to ask yourself three questions about that decision:
1. Does this decision affect the structure, direction, or outcome of the project?
2. Would someone unfamiliar with the project ask me to explain why I made this decision?
3. Were there other viable alternatives I could have chosen?
If you answer "Yes" to at least one of these questions, then you've likely just made an architectural decision, and you should create an ADR to explain why you made that choice. Other contributors (and your future self) will thank you for it.
### Examples of architecture decisions
While architecture decisions come in all shapes and sizes, some common examples include:
* Setting up the repository's main file structure
* Selecting a critical library, tool, or platform
* Adopting a certain analytical framework or algorithm
* Choosing *not* to build a particular feature
## ADR Process
When an architectural decision needs to be recorded, please use the following steps:
1. [Create a new ADR ticket in GitHub](https://github.com/HHS/simpler-grants-gov/issues/new/choose)
2. Identify the approvers for this decision, i.e. individuals or groups who must sign off on the decision before the ADR can be merged (ideally this happens before work starts on the ADR)
3. Draft an ADR in a branch named after the ticket, then open a Pull Request (PR)
* Use the `{YYYY-MM-DD}-{description}.md` naming convention and copy the [template](/product/decisions/template) into the [adr](/product/decisions/adr) directory (name the file with the date it was created, which may differ from the date the decision was made, PR was merged, or file was last modified)
* The ADR should use the status "Active"
* If this decision deprecates an ADR, update its status to indicate the reason (e.g. `"Superseded By [xxx](yyyymmdd-xxx.md)"`), and move it to the [deprecated](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/wiki/product/decisions/deprecated/README.md) directory
4. Invite the approvers to review the PR and provide feedback or approve the decision
5. Once the approvers have signed off on the decision, merge the PR into the `main` branch
6. Create any follow-up tickets that are needed to implement the decision outlined in the ADR (if necessary)
## Acknowledgements and Further Reading
* [ADR GitHub Organization](https://adr.github.io/)
* [Joel Parker Henderson's ADR repo](https://github.com/joelparkerhenderson/architecture-decision-record)
* [GitHub Blog - Why Write ADRs](https://github.blog/engineering/architecture-optimization/why-write-adrs/)
# ADR Template
* **Status:** {Active | deprecation reason, e.g. "Superseded By `[xxx](decisions/yyyymmdd-xxx.md)`"}
* **Last Modified:** {YYYY-MM-DD when the decision was last updated}
* **Related Issue:** \[#{issue number}]\( number})
* **Deciders:** {list everyone involved in the decision}
* **Tags:** {space and/or comma separated list of tags}
## Context and Problem Statement
{Describe the context and problem statement, e.g., in free form using two to three sentences. You may want to articulate the problem in form of a question.}
## Decision Drivers
* {driver 1, e.g., a constraint, priority, condition, etc.}
* {driver 2, e.g., a constraint, priority, condition, etc.}
* ...
## Options Considered
* {option 1}
* {option 2}
* ...
## Decision Outcome
Chosen option: "{option 1}", because {justification. e.g., only option which meets a key decision driver | which satisfies x condition | ... }.
### Positive Consequences
* {e.g., improved performance on quality metric, new capability enabled, ...}
* ...
### Negative Consequences
* {e.g., decreased performance on quality metric, risk, follow-up decisions required, ...}
* ...
## Pros and Cons of the Options
### {option 1}
{example | description | pointer to more information | ...}
* **Pros**
* Good, because {argument a}
* Good, because {argument b}
* ...
* **Cons**
* Bad, because {argument c}
* ...
### {option 2}
{example | description | pointer to more information | ...}
* **Pros**
* Good, because {argument a}
* Good, because {argument b}
* ...
* **Cons**
* Bad, because {argument c}
* ...
## Links
* \[{Link name}]\(link to external resource)
* ...
# ADRs
[ADRs we've adopted](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/wiki/SUMMARY.md)
# Recording Architecture Decisions
* **Status:** Active
* **Last Modified:** 2023-06-26
* **Related Issue:** [#34](https://github.com/HHS/simpler-grants-gov/issues/34)
* **Deciders:** Lucas, Aaron, Billy
* **Tags:** ADRs
## Context and Problem Statement
Important decisions about the structure of a codebase, the tools or platforms adopted, and the scope or vision of a product are made all throughout a project's lifecycle. Often these decisions are made and changed blindly with little record of why that decision was made, making it difficult for other collaborators and even the original decision makers to understand the justification behind that decision in the future.
*What is the best way to document key architectural decisions made within the project so that future contributors can understand the justification for those decisions?*
## Decision Drivers
* **Explicit:** Important decisions about the project architecture should be clear and unambiguous
* **Documented:** There should be a written record describing which decisions are made
* **Accessible:** Decision records should be centrally located, searchable, and accessible to non-technical audiences
* **Portable:** The decision records should be able to travel with the code when it's forked, cloned, etc.
* **Maintainable:** The system we adopt should be easy to maintain or amend as needed.
* **Publicly Documented:** Decisions should be part of the public record so that external stakeholders (including open source contributors) understand the rationale behind each decision.
## Options Considered
* ADRs in the repo `documentation/decisions/adr` folder
* Comments and docstrings in the code itself
* Articles in the project's Wiki
## Decision Outcome
Decisions will be documented using Architecture Decision Records (ADRs) as described by [Joel Parker Henderson](https://github.com/joelparkerhenderson/architecture-decision-record) and the [ADR GitHub organization](https://adr.github.io/). The template for this project's ADRs will be adapted from the [MADR template](/product/decisions/template).
**NOTE:** We may want to revisit this decision to reconsider storing them in the Project Wiki once that wiki is established. Ideally ADRs could be drafted/edited and visible in the wiki but also synced with the repo -- some wiki solutions provide this option.
### Positive Consequences
* ADRs travel with the repo when it's cloned, forked, etc.
* ADRs can be incorporated in the Issue and PR workflow
* Changes to ADRs will be listed in the project's commit history
### Negative Consequences
* Certain urgent decisions may take longer to finalize if they need to be documented and agreed upon, slowing down the process of finalizing key decisions with the project
* Unless regularly maintained and complied with, it could be easy for the ADRs to become out of sync with the actual decisions made about the architecture
* If the project is organized as a set of microservices with different repositories, we'll have to decide whether to keep ADRs in a central repo or record those decisions in the repo to which they are relevant which may reduce discoverability and accessibility
* Non-technical users may have a more challenging time creating and editing ADRs if they are not familiar with Markdown or GitHub
## Pros and Cons of the Options
### Comments and Docstrings
* **Pro**
* Decisions live alongside the code that implements them
* Docstrings travel with the code when it's forked, cloned, etc.
* Changes to docstrings and comments are part of the commit history
* **Cons**
* Not very accessible to non-technical audiences
* Not centralized and difficult to search for
* Hard to see the sequential evolution of decisions
### Project Wiki
Documenting decisions in the Wiki tab on the project repository in GitHub.
* **Pro**
* Wikis are fairly searchable and accessible to non-technical audiences
* **Cons**
* Wikis are not forked with the main repo
* Changes to the Wiki aren't easily integrated with the PR and Issue workflow
* Changes to the Wiki aren't a part of the project's commit history
## Links
* [ADR GitHub Organization](https://adr.github.io/)
* [Joel Parker Henderson's ADR repo](https://github.com/joelparkerhenderson/architecture-decision-record)
* [GitHub Blog - Why Write ADRs](https://github.blog/2020-08-13-why-write-adrs/)
# Task Runner for the CI / CD Pipeline
* **Status:** Active
* **Last Modified:** 2023-06-29
* **Related Issue:** [ADR: Task Runner and CI / CD interface #92](https://github.com/HHS/simpler-grants-gov/issues/92)
* **Deciders:** Lucas, Aaron, Billy
* **Tags:** Continuous Integration, Continuous Deployment
## Context and Problem Statement
A task runner needs to be selected and an interface described to initiate tasks so that the project can perform necessary testing, linting, other continuous integration tasks, and continous deployment.
The task runner should be able to run on commits and pull requests and support both testing of code and deployment to various environments. The task runner should be able to run tasks within its own environment as well as initiate remote tasks.
## Decision Drivers
* **speed:** The task runner should produce fast results compared to other options. Some of the components that produce a speedy result include time to initiate the task, time to prepare environments, speed of running similar types of steps (I.E. spin up docker containers), caching capabilities, and ability to parallelize tasks. The project will not have time to test the speed with different tools.
* **ease of use:** The task runner and interface should be easy to use for developers. Tasks run in the CI tool should be able to be run on local environments where possible. The tool and interface should be well documented. The task runner should have tools, communities, or patterns that make creating new tests or deployments easier.
* **cost:** The task runner should be cost-competitive.
* **security and authorization:** The project should have relevant security credentials like Fedramp authorization and should be authorized specifically for the project.
## Options Considered
* [Github Actions](https://github.com/features/actions)
* [Travis CI](https://www.travis-ci.com/)
* [Circle CI](https://circleci.com/)
## Decision Outcome
Github Actions offers competitive speed, developer support, shared / resusable actions, is part of Github which is already approved on the project and part of the [Fedramp marketplace](https://marketplace.fedramp.gov/products/FR1812058188), and is free.
# API Language
* **Status:** Active
* **Last Modified:** 2023-06-30
* **Related Issue:** [#27](https://github.com/HHS/simpler-grants-gov/issues/27)
* **Deciders:** Lucas, Aaron, Sammy, Daphne, Billy
## Context and Problem Statement
This ADR is to decide what programming language the API server for simpler.grants.gov will use. This API server will initially be responsible for responding to queries to search a replicated database, but will grow to encompass all the back-end responsibilities of grants.gov
In addition to the Nava and HHS teams ability to develop and maintain an API in the chosen language, it is also important that this language lends itself to open source engagement.
## Decision Drivers
* Nava and HHS familiarity with language and common libraries
* Performance of language and associated frameworks at scale
* Support of Open Source community
* Cost of necessary licenses, support, etc.
## Options Considered
* Python
* Javascript and Node
* Java
## Decision Outcome
Chosen option: Python, because it is free and open source and designed to be easy for anyone to learn quickly and contribute. Additionally, while all the languages can support the technical needs, Python in particular is optimized to handle the large and complex grants.gov dataset, and makes other parts of the project simpler, including ETL and data analysis for the analytics endpoints.
### Positive Consequences
* We're hopeful that this choice will lead to open source contribution and adoption
* We can make use of python libraries designed for complex data queries to quickly deliver features and analytics
### Negative Consequences
* Will require some context switching when going from work on the API to the front-end
## Pros and Cons of the Options
### Python
Python as a scripting language is fast and productive. Web-based applications can be created quickly and the code is highly readable. Python syntax is easy to understand, well-defined and extensible. APIs built over Python are highly scalable, reliable, and fast.
Because of its popularity in the open source community, Python makes it easy for open source contributors to support our work.
* **Pros**
* Nava and HHS experience with Python and APIs built with Python
* Number of available libraries and frameworks
* Robust data science and algorithmic libraries
* Free and open source to use
* Ideal for data computation and complex database queries
* **Cons**
* Single threaded locking of GIL can make computationally expensive operations slower than other options
### Javascript and Node
Node JS is used to build fast, highly scalable network applications based on an event-driven non-blocking input/output model, single-threaded asynchronous programming. It also suits well for microservice-based products. Such Node.js utility is accomplished by such characteristics as support of a non-blocking input-output and a significant number of connections.
Because of its popularity in the open source community, Javascript and Node makes it easy for open source contributors to support our work.
* **Pros**
* Nava experience with Javascript language and Node based API servers
* Fast single threaded but asynchronous event driven operation
* Free and open source to use
* Same language as most front-end apps requiring less context switching for developers
* Could use the same framework for front end, simplifying infrastructure
* **Cons**
* Not as mature as other options
* Responsiveness may be slowed by any CPU intensive computation
### Java
Java is widely used for building enterprise-scale web applications as it is one of the most stable languages on the market. Java’s advantages include platform independence, multi-threaded processing, automatic garbage collection, and security.
Because of its proprietary nature, Java is not as popular with the open source community, which adds some barriers for open source contributors to support our work.
* **Pros**
* Multi-threaded processing
* Long track record of reliability and stability as an Enterprise solution
* **Cons**
* Commercial use requires expensive licenses and not as open source friendly
* Compilation and abstraction by the Java Virtual Machine makes performance slower
* Nava team is not as familiar with Java as they are with the other options
# Use Figma for design prototyping
* **Status:** Active
* **Last Modified:** 2026-04-27, 2023-07-03
* **Related Issue:** [#103](https://github.com/HHS/simpler-grants-gov/issues/103)
* **Deciders:** Andy, Emily, Lucas
* **Tags:** design
## Context and Problem Statement
Which tool should be used for wireframing and prototyping? The tool should facilitate design conversation, be easilty integrated into research activities, and provide artifacts for stakeholder review.
## Decision Drivers
* Platform availability
* Live collborative editing
* assets can be integrated into design research tools
* tool and assets must be accessible to internal partners (HHS or employed by HHS) and external partners (general public).
* Nava familiarity
## Options Considered
* Figma
* Adobe XD
* Sketch
* Invision Studio
## Decision Outcome
Chosen option: Figma, because it is not platform specific and its multi-user collaboration allows for easy remote pairing. It's assets can be easily shared, embedded, or integrated into research tools. Additionaly, Nava has the most familiarity with this option.
**Update (Aug 26, 2025):** Designers should have a "Full seat." Engineers and Product Managers should have a "Dev seat."
**Update (Apr 27, 2026):** Frontend implementation tickets in GitHub should include screenshots that detail the exact design changes. Figma is a living document. Screenshots help us focus on the changes outlined in the GitHub Issue and avoid distraction of unrelated changes. This is especially important for open-source contributors since we now use Dev Mode, which is not accessible to anyone without a license.
### Positive Consequences
* Truss has developed [Figma assets using USWDS](https://www.figma.com/community/file/836611771720754351/U.S.-Web-Design-System-\(USWDS\))
* Figma's free tier does not require FOSS contributors to purchase licenses
* Figma has a robust community of plugins and integrations
* Includes Figjam (tho this may be redundant, depending on chosen whiteboarding tool)
### Negative Consequences
* Collaborative nature of Figma requires connectivity and working online
* Tho version control and file management is possible, it can be difficult to track *who* made recent changes
## Pros and Cons of the Options
### Figma
* **Pro**
* Browser-based (works on Mac, Windows, Linux…)
* Free forever
* Nava familiarity
* Robust plugin community
* **Cons**
* Limited files (3) in free tier
* Requires internet connectivity
### Adobe XD
* **Pro**
* Part of Adobe Creative Suite
* **Cons**
* Native app, not browser-based
* No free tier (7-day trial)
### Sketch
* **Pro**
* Lowest subscription cost
* **Cons**
* Mac only (nonstarter?)
* No free tier (30-day trial)
### Invision Studio
* Nonstarter: [tool is being sunset](https://help.invisionapp.com/hc/en-us/community/posts/11525657213965-SUNSET-NOTIFICATION-Studio)
## Links
* [Figma pricing](https://www.figma.com/pricing/)
# ADR: Chat
* **Status:** Accepted
* **Last Modified:** 2023-08-17
* **Related Issue:** #105 ()
* **Deciders:** Lucas and Billy
* **Tags:** docs:adr topic:comms
## Context and Problem Statement
We want to explore and evaluate options for group chat in order to facilitate effective cross-team and communication. This chat tool will be utilized by both internal individuals at HHS and the general public, requiring usability and effectiveness across diverse user needs. Additionally, the selected tool should have the capacity to scale and adapt as the project expands. There is currently a lack of a transparent chat tool that meets our requirements.
**Assumptions** We believe that a chat tool can enhance communication and collaboration in real-time for our open-source community. A chat tool can provide a more immediate and interactive means of communication, allowing community members to discuss ideas, ask questions, and provide feedback in a more dynamic manner. They can facilitate quicker decision-making, foster a sense of community, and enable easier coordination among contributors.
## Decision Drivers
* **Functionality:** consider the features and functionalities offered with each chat tool. Consider whether the chat solution provides essential capabilities like direct messaging, public channels for messaging, public and private channels, replying to messages in threads that other users can see, emojis to react to message, file sharing, search functionality, integrations.
* **Usability:** chat solution should be user-friendly and intuitive. IT should be easy for team members, including both technical and non-technical individuals, to navigate and utilize effectively.
* **Scalability:** solution should be able to accommodate the projected growth in the number of users, conversations, and data as the project expands. Ensure that the tool can handle demand without compromising performance
* **Integration capabilities:** solution is able to integrate with other tools and platforms that the project relies on. Integration with project management tools, issue trackers, wiki, document sharing, sprint retro.
* **Accessibility:** solution should be accessible for all individuals, including but not limited to individuals with visual, hearing or motor impairments. A chat solution should support keyboard navigation, screen reader compatibility, color contrast options, and support assistive technologies.
* **Community and support:** consider the size and activity of the user community. An active community, similarly to an FOSS community, can provide valuable support, resources, and community-developed plugins or integrations.
* **Cost:** evaluate the pricing structure, considering growth of users and additional features in the future.
* **Self-hosting vs. cloud-based:** evaluate whether the solutions are self-hosted where we have complete control over data and infrastructure or cloud-based that handles hosting and maintenance.
* **Open-source:** a solution that offers open-source is more likely to align with HHS values and provide transparency, customizability, community collaboration, longevity and continuity, cost effectiveness, and overall autonomy for our project.
## Options Considered
* [Slack](https://slack.com/features)
* [Discord](https://discord.com/)
* [Microsoft Teams](https://www.microsoft.com/en-us/microsoft-teams/group-chat-software)
* [GitHub Discussions](https://docs.github.com/en/discussions/collaborating-with-your-community-using-discussions/about-discussions)
* [Rocket.chat](https://www.rocket.chat/)
* [Mattermost](https://mattermost.com/)
* [Zulip](https://zulip.com/)
## Decision Outcome
At this time, it is recommended to use a basic paid version of Slack for internal and public users. Slack offers a wide range of functionality and features, making it a popular and widely adopted chat tool. It has robust integration capabilities and a user-friendly experience. The team's familiarity with Slack reduces the learning curve. Inviting the public to our paid version of Grants.gov Slack instance will allow us to get an understanding of number of users joining and usage patterns for chat. However, it's important to be mindful of pricing implications as the project grows. We should reassess if there is growth beyond 400 users. Moving to Slack’s free version is an option for the public, open-source community. Slack's free version has limitations like data retention but would be more cost-efficient.
### Positive Consequences
* Enhanced collaboration
* Increased engagement
* Faster onboarding
* Improved accessibility
* Smoother integration
* Positive user experience
### Negative Consequences
* Limited functionality
* Vendor lock-in
* Security concerns
* Learning curve
* Communication fragmentation
* Incompatible with existing workflows
## Pros and Cons of the Options
✅ Feature available, meets requirement ❌ Feature not available, does not meet requirement 🔄 Partial feature, limited feature availability, feature in progress or undergoing improvements 1-3 Strength level, (1 being lowest, 3 being strongest) ❓Unknown
| Evaluation criteria | [Slack](https://slack.com/features) | [Discord](https://discord.com/) | [Microsoft Teams](https://www.microsoft.com/en-us/microsoft-teams/group-chat-software) | [GitHub Discussions](https://docs.github.com/en/discussions/collaborating-with-your-community-using-discussions/about-discussions) | [Rocket.chat](https://www.rocket.chat/) | [Mattermost](https://mattermost.com/) | [Zulip](https://zulip.com/) |
| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| Experience with current team | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
| Self-hosting | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
| Open-source | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
| Voice | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
| Video | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
| Text | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Accessibility options | ✅ | 🔄 | ✅ | ✅ | 🔄 | ✅ | 🔄 |
| Accessibility: keyboard navigation | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Accessibility: Screen reader compatibility | ✅ | 🔄 | ✅ | ✅ | 🔄 | ✅ | ✅ |
| Accessibility: Color contrast options | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Accessibility: Assistive technologies | ✅ | 🔄 | ✅ | ✅ | ✅ | ✅ | 🔄 |
| Public users | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
| Metrics available | 2 | 1 | 2 | 2 | 2 | 2 | 2 |
| Community and support | 3 | 2 | 3 | 3 | 2 | 2 | 2 |
| Usability | 3 | 3 | 2 | 2 | 3 | 3 | 2 |
| Integration | 3 | 2 | 2 | 2 | 3 | 3 | 2 |
| Security | 3 | 2 | 3 | 2 | 2 | 3 | 3 |
| Public channels | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Private channels | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ |
| Threaded replies | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Emoji reactions | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ |
| Cost |
Free, Pro $7.25, Business $12.50 USD per active user per month billed annually. Or, $15 USD per active user per month, billed monthly. FOSS community users must request access
| Free to run, free to use |
Part of MS365 subscription plans $12.50/user/month
| $3.67 per user/month, billed annually; |
Free + own hosting costs; paid version available $7 /user/month Minimum of 25 users or $35 /agent/month Minimum of 5 agents
| free + own hosting costs; paid version available $10 per user/month, billed annually; enterprise pricing with requested quote | Free + own hosting costs; paid version available$6.67 per user per month, billed annually |
## Pricing breakdown for top choices
To understand the cost as we grow and expand chat to become accessible to public users, we want to evaluate costs for our top chat options: Slack and Rocket.chat
**Slack:**
For free version of Slack:
* No limit to channels or users
* Data retention for 90 days
* Limitations to message and file visibility limit at 90 days
* You can add up to 10 third-party or custom apps.
For paid versions of Slack:
* Guest Roles (External users that are not part of a paid workspace). Host pays for the guest roles for mult-channel use. Single channel based off number of members on host workspace.
* Slack Connect (requires both your workspace and their workspace be on a paid plan or start a trial). There is no additional charge on either workspace. Slack Connect (More for users that already have their own paid workspace). These users can have multi-channel access at no additional charge.
* Guest users (Allows for the host workspace to invite users under the host plan)
* Single Channel guests would be no charge and multi-channel guests would be billed like regular members of your workspace.
* Single-Channel Guests are free and can only access one channel. For every paid active member in your workspace, you can add up to 5 guests. For example, if you have 10 members, you can invite up to 50 Single-Channel Guests.
* Multi Channel guests would be billed like a regular member of your workspace.
**Rocket.chat**
* Rocket.chat has a couple different pricing models:
* Registered users (appropriate for a small number of users)
* Omni-channel model: that charges by unique monthly active users
* Model to charge for monthly active users (not unique)
* Rocket.chat does not currently have a use case where public users need to access or view channels and interact with channels, but could be willing to work out a solution.
* Rocket.Chat's self-hosted community edition is free forever. There are limitations to the free version including high availability and scalability. - Rocket.chat’s free community edition is limited to a single instance and designed to support 150-200 users. We have the ability to build the screen reader functionality on our own instance.
## Links
Government offerings
* [Slack for Government](https://slack.com/resources/why-use-slack/a-closer-look-at-govslack)
* [Microsoft](https://learn.microsoft.com/en-us/microsoftteams/expand-teams-across-your-org/teams-for-government-landing-page)
* [Github](https://government.github.com/)
* [Rocket.chat](https://www.rocket.chat/government)
* [Mattermost](https://mattermost.com/solutions/industries/government/)
Offerings
* [Enterprise](https://www.rocket.chat/enterprise)
* [Community](https://www.rocket.chat/community)
* [Slack Admins: guests vs channels in Slack Connect](https://slack.com/resources/slack-for-admins/guests-vs-channels-in-slack-connect)
* [Guest roles for paid version](https://slack.com/help/articles/202518103-Understand-guest-roles-in-Slack)
* [Slack Connect Guide for external organizations](https://slack.com/help/articles/115004151203-Slack-Connect-guide--work-with-external-organizations)
* [Slack Free workspace usage limits](https://slack.com/help/articles/115002422943-Usage-limits-for-free-workspaces)
* [Discord](https://discord.com/open-source)
# DB Choices
* **Status:** Active
* **Last Modified:** 2023-07-05
* **Related Issue:** [#27](https://github.com/HHS/simpler-grants-gov/issues/104)
* **Deciders:** Lucas, Aaron, Gina, Andy
## Context and Problem Statement
This ADR is to formalize a series of architectural decisions about how data is stored, including the type of database we will use and the platform we'll use host it. This ADR will contain the evaluation and selection of the type of the database, the database management system (DBMS), and database hosting service.
## Decision Drivers
Type of Database:
* Nature of the NOFO and grant data that we're storing
* Volume of data and performance considerations
DBMS Selection:
* Hosting options
* Expected access patterns (i.e. heavy read or write transactions)
* Ease of creating mock data
* Trusted libraries that support interfacing with this database in the language we've chosen for our API
* An open source version of this DBMS is available for self hosting or running locally
Database Hosting Service:
* FedRAMP compliant deployment option
* Ease of use and support
* Cost considerations related to hosting the database, in terms of direct operating costs as well as team resources
* Advanced service offerings such as read replicas and automatic backups
## Decision Outcome
### Type of Database
The target type of database selected for Grants.gov is a relational database management system. This was evaluated and selected based on several factors including the relational nature of the NOFO and grant data that we're storing as well as the ACID compliance and flexibility that a relational database can offer.
### Database Management System
The RDBMS selected for Grants.gov is PostgreSQL. This is due to a variety of factors, including:
* Resource familiarity with PostgreSQL as a database system
* Improved performance for high-frequency write operations and complex queries as opposed to MySQL
* PostgreSQL support of most advanced database features such as materialized views
* PostgreSQL trusted Python libraries (API language of choice)
* PostgreSQL is open source, in alignment with the Grants.gov strategy
### Database Hosting Service
The database hosting service selected is Amazon RDS. This is due to several factors:
* FedRAMP compliant deployment
* Reduction of many tasks and IT labor savings when compared to EC2 or on-premise including backups, server patching, automatic backups, scaling, etc.
## Links
* [Comparison of MySQL and PostgreSQL](https://aws.amazon.com/compare/the-difference-between-mysql-vs-postgresql/)
* [What is Amazon RDS?](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Welcome.html)
# API Framework and Libraries
* **Status:** Active
* **Last Modified:** 2023-07-07
* **Related Issue:** [#28](https://github.com/HHS/simpler-grants-gov/issues/28)
* **Deciders:** Lucas, Aaron, Billy, Sammy, Daphne, Gina, Sumi, Eshter
## Context and Problem Statement
This ADR is to decide what python framework to use for the back-end API of grants.gov. Python was chosen as the language for the back-end API in [ADR #3](/product/decisions/adr/2023-06-30-api-language).
## Decision Drivers
* Is this framework well established with a broad community of developers
* Does it have good documentation for developers to get up to speed quickly
* Does this language have a track record for reliability and continued support
* How familiar is Nava and HHS with the framework
* How effectively can the chosen framework support the back-end needs of grants.gov
## Options Considered
* Flask
* Flask + Connexion
* Flask + APIFlask
* FastAPI
* Django
## Decision Outcome
Chosen option: Flask + APIFlask, because it is well established with a broad community of developers and provides good tooling to move quickly. It has great documentation to help folks contribute quickly. Additionally the Nava Flask template recently adopted it, so we can leverage the template to get going quickly with a well engineered solution.
### Positive Consequences
* Leverages the Nava open source template
* OpenAPI specs can be auto generated from models in code
* Flexibility in defining schemas
### Negative Consequences
* Code first paradigm, we should auto generate api documentation in the CI/CD pipeline to ensure it stays up to date with the code
* This is a relatively new library, so we should ensure code modularity in case we need to swap it out in the future
## Pros and Cons of the Options
### Flask
Flask is a simple, but extensible, micro web framework for python created in 2010 that is easy to learn and build with. While it is capable on its own, it relies on other libraries to add core functionality. It was written to be pythonic, meaning it leverages python's unique features and follows python's principles of being readable and maintainable. Flask depends on the Werkzeug WSGI toolkit, the Jinja template engine, and the Click CLI toolkit.
* **Pros**
* Shallow learning curve, great documentation, and pythonic style makes it easy for contributors to support
* Flexible and scalable microframework means it can adjust as the needs of the project changes
* Designed for back-end API services
* **Cons**
* Tech stack can get complicated over time with various libraries
* Lack of standardization means more decision making and good code quality is very important
* Async operation takes additional planning and work
### Flask + Connexion
This deserves its own option because it fundamentally changes the way that we would develop with python and flask. With connexion, you first write your API contract, using the Swagger/OpenAPI standard. Then, the endpoints you defined will be mapped to your python view functions, ensuring that your python code does what your API contract says it does. This makes it rather unique in the landscape of python web frameworks, as most other tools start from your code instead of the other way around.
* **Pros**
* Same as Flask
* API first means all teams can work with a single contract, instead of having multiple sources of truth
* Documentation in OpenAPI or Swagger means it's easier to understand and integrate with other tools
* **Cons**
* Same as Flask
### Flask + APIFlask
This deserves its own option because it adds a lot of support to Flask for a more traditional code first api approach, in contrast to connexion. APIFlask is a lightweight Python web API framework based on Flask and marshmallow-code projects. It's easy to use, highly customizable, ORM/ODM-agnostic, and 100% compatible with the Flask ecosystem. The Nava Flask template repo has recently replaced connexion with apiflask, documented in [connexion replacement decision record](https://github.com/navapbc/template-application-flask/blob/main/docs/decisions/0001-connexion-replacement.md).
APIFlask relies on [Marshmallow](https://marshmallow.readthedocs.io/en/stable/) to define the request & response schemas. Marshmallow is a well tested serialization/deserialization, and validation library for converting between JSON and python dictionaries. Marshmallow schemas are defined using classes in-code, and polymorphism can be used to quickly make small adjustments to schemas for new endpoints. Fields can be marked as just for serialization or deserialization (ie. response / request) which also minimizes how much needs to be defined. Marshmallow also supports the concept of "partial" schemas. When attaching a schema to a route, if you mark it as partial, all fields are marked as not required. This makes building schemas for update endpoints simpler as you don't need to duplicate an entire schema just to mark fields as optional. This means we can potentially define a single schema for create, read, and update endpoints, and avoid a significant amount of duplication.
APIFlask uses [apispec](https://apispec.readthedocs.io/en/latest/) to convert the Marshmallow schema into the OpenAPI/Swagger specification.
* **Pros**
* Same as Flask
* Simplifies boilerplate code necessary for a Flask API by pulling the best features of many libraries
* Very well documented
* Marshmallow is very flexible, and allows for significant customization in defined schemas
* **Cons**
* Same as Flask
* Marshmallow only converts JSON to dictionaries - if you want to work with typed/IDE auto-complete-able objects, you'll need to use a library like Pydantic and define models again
* Relatively young project
### FastApi
FastAPI is a modern, fast (high-performance), web framework created in 2018 for building APIs with python 3.7+ based on standard python type hints. This implementation-first library is designed as a performant and intuitive alternative to existing python API frameworks.
Schemas are defined using [Pydantic](https://docs.pydantic.dev/latest/) which is a model definition and validation library that thoroughly leverages Python typing features
* **Pros**
* Designed for speed and can perform asynchronous operations natively (if needed)
* Can generate OpenAPI or Swagger documentation from the code
* **Cons**
* FastAPI maintenance and updates can be sporadic and have long spans between them, maintained by a single person
* Documentation for more advanced cases is lacking
* As the newest of the frameworks, it has the smallest community of support
* Can have memory management issues
### Django
Django is a full stack python web framework created in 2005 that follows the model–template–views (MTV) architectural pattern. It is maintained by the Django Software Foundation (DSF), an independent organization established in the US as a 501(c)(3) non-profit. Django is well documented and includes everything you may need in a full stack application already installed.
* **Pro**
* Maintained by an independent 501(c)(3) non-profit organization
* Longest running and most popular python framework in consideration
* **Cons**
* Monolithic style, steep learning curve, and large codebase make it difficult for collaborators
* Assumes both front end and back end are included in the monolith application
## Links \
* [Nava flask template repository](https://github.com/navapbc/template-application-flask)
* [Nava decision to switch from Connexion to APIFlask](https://github.com/navapbc/template-application-flask/blob/main/docs/decisions/0001-connexion-replacement.md)
# Back-end Code Quality Tools
* **Status:** Active
* **Last Modified:** 2023-07-07
* **Related Issue:** [#101](https://github.com/HHS/simpler-grants-gov/issues/101)
* **Deciders:** Aaron, Daphne, Sammy, Gina, Lucas, Billy
* **Tags:** ADR
## Context and Problem Statement
Back-end code quality tools should facilitate and efficiently enforce linting, auto-formatting, type-checking and security concerns. They should be easily configurable to suit our specific use-cases, uniformly applying conventions while alleviating the need for individual intervention.
## Decision Drivers
* **Ease of use and configurability:** Code quality tools should be configurable to meet the specific needs of our project and enforce standards uniformly. Running the tool should be simple, and feedback should be easily implementable and understandable.
* **Speed:** Code quality tools should be able to run in an timely manner, ideally in a pre-commit hook.
* **Documentation and resources:** Code quality tools should have robust and helpful documentation, specifically around usage and configuration.
* **Lean:** Selection of code quality tools should cover all use-cases with little overlap for ease of use and to minimize mental surface area required to understand the system. As in, each tool should have a specific, meaningful purpose.
* **Enforces security:** Tooling should assist us in identifying security concerns and vulnerabilities.
* **Well-maintained:** We have a preference towards libraries that are widely adopted and have active maintainers.
## Options Considered
1. Use a curated collection of Python libraries from Flask template repository (described below)
2. Use a different or modified set of Python libraries
### Dependency Management
[**Poetry**](https://python-poetry.org/docs/)**:** Python packaging and dependency management.
### Code Linting
[**Ruff**](https://beta.ruff.rs/docs/)**:** An extremely fast Python linter, written in Rust. Preferred for its speed and growing community adoption.
### Auto-formatting
[**Black**](https://github.com/psf/black)**:** Format Python code. Compatible with [Ruff](https://beta.ruff.rs/docs/faq/#is-ruff-compatible-with-black) out of the box, however, Ruff *may* replace the need for Black at some point. See this [issue](https://github.com/astral-sh/ruff/issues/1904).
### Type Checking
[**Mypy**](https://mypy-lang.org/)**:** Validate and enforce static type checking in Python.
### Security
[**Safety**](https://docs.pyup.io/docs/getting-started-with-safety-cli)**:** Safety first! Safety scans dependencies for vulnerabilities and security concerns.
### License Checking[^1]
[**pip-licenses**](https://github.com/raimon49/pip-licenses)**:** CLI tool for checking the software license of installed Python packages with `pip`.
### Interface
[**Make**](https://www.gnu.org/software/make/manual/make.html)**:** Run scripts, linters and formatters.
## Decision Outcome
Option #2 is preferred. We would like to use Ruff for linting and add some additional libraries for security and license checks.
We will be using the Flask template repository for initial project set up, which already relies on Flake8 and several additional extensions (bugbear, alfred, bandit) that would be redundant with Ruff. Some extra work will need to be done to migrate away from Flake8 and to Ruff without any regression. Recommend using [flake8-to-ruff](https://pypi.org/project/flake8-to-ruff/) to convert existing configuration.
There are some additional packages that we desire to use that are not included in the Flask template: safety & pip-licenses.
As we iterate on the tools that work for us, we would like to investigate a possible switch to Pyright in the future as well.
## Other Options
Adopting [Tox](https://tox.wiki/en/latest/) as a testing / linting manager with some of the libraries.
**Dependency Management:** [Pipenv](https://pipenv.pypa.io/en/latest/)
**Code Linting:** [Flake8](https://flake8.pycqa.org/en/latest/): Much slower than Ruff. Requires additional extentions like [bugbear](https://pypi.org/project/flake8-bugbear/) that are built into Ruff. [Pylint](https://pypi.org/project/pylint/)
**Auto-formatting:** [autopep8](https://pypi.org/project/autopep8/)
**Type Checking:** [Pyright](https://microsoft.github.io/pyright/#/): [Comparison of MyPy and Pyright](https://github.com/microsoft/pyright/blob/main/docs/mypy-comparison.md). Language service through Pylance. [Pyre](https://pyre-check.org/)
**Security:** [Bandit](https://bandit.readthedocs.io/en/latest/): Security checking tool used to identify common concerns in Python code. Redundant because Ruff implements `flake8-bandit`. [dependency-check](https://pypi.org/project/dependency-check/)
**License Checking:** [licensecheck](https://pypi.org/project/licensecheck/)
**Interface:** Bash, Poetry
[^1]: Addition to the existing curated collection
# Front-end Language
* **Status:** Active
* **Last Modified:** 2023-07-10
* **Related Issue:** [#96](https://github.com/HHS/simpler-grants-gov/issues/96)
* **Deciders:** Lucas, Aaron, Billy, Sammy
## Context and Problem Statement
The goal of this ADR is to select a language that we'll use to implement the front end for simpler.grants.gov. The front end will only support static content at first, but will grow to include the new search, supported by the API and eventually the entire grants.gov functionality. Therefore, while a simple solution might work in the short term, it will quickly become insufficient to meet our needs.
## Decision Drivers
* Active Maintenance: The language is actively maintained with patches and minor releases delivered on a regular basis
* Community of Users: The language has an active community of open source users and is commonly used for front-end development
* Usability: The language is relatively easy to learn for developers without prior experience and there are plenty of resources and training materials available
* Language Features: The language supports important features like concurrency, static type checking, immutable data types, etc. or there are well established libraries which provide these features
* Reusability: The language selected for the front end can be also used to support other parts of the project (e.g. ETL & data analysis, API development)
* HHS and Nava familiarity with the language
## Options Considered
* JavaScript
* TypeScript
* Python
* Java
* Go
## Decision Outcome
Chosen option: TypeScript, because it is the ideal language for front-end applications because some form of JavaScript is required for client side functionality anyway. This allows us to reduce the amount of context switching between langauges. Additionally, with node and npm TypeScript and JavaScript are fully featured platforms for building and running applications. We chose TypeScript over JavaScript because of the value that strong typing provides with promomting code quality and developer support.
### Positive Consequences
* Getting to stay in Javascript for the entire front end means less energy spent context switching
* Strong typing helps with code quality and guidance to developers, but is only helpful if it is used properly
### Negative Consequences
* Code should be written in such a way that if TypeScript loses longer supported, we can easily pare it down to regular JavaScript
## Pros and Cons of the Options
### JavaScript
JavaScript is a lightweight, interpreted programming language with first-class functions. While it's universally used for client side website functionality, with tools like npm and node, it is a powerful application language as well.
* **Pros**
* Keeping the entire front end in one language requires less context switching
* Large communinity of users, with lots of updates, and rich functionality
* HHS and Nava are very familiar with the language and its frameworks
* **Cons**
* Loosely typed
### TypeScript
TypeScript is a syntactic superset of JavaScript which adds static typing and is transpiled down to regular JavaScript when building the application. While TypeScript is a relatively new "language" it's gaining popularity quickly as it's more of a enhancement to JavaScript than just another new language.
* **Pros**
* Same as JavaScript
* Strong typing
* **Cons**
* More complex than JavaScript
### Python
Python is a high-level, dynamically typed general-purpose programming language. Its design philosophy emphasizes code readability but supports multiple programming paradigms.
* **Pros**
* Same language as back end reducing context switching
* **Cons**
* Will still require JavaScript for client side functionality
* Python is not as well supported for front-end tasks as it is for back-end tasks
### Java
Java is widely used for building enterprise-scale web applications as it is one of the most stable languages on the market. Java’s advantages include platform independence, multi-threaded processing, automatic garbage collection, and security.
* **Pros**
* Existing HHS website and tools are built with Java
* Will still require JavaScript for client side functionality
* **Cons**
* Commercial use requires expensive licenses and not as open source friendly
* Compilation and abstraction by the Java Virtual Machine makes performance slower
* Nava team is not as familiar with Java as they are with the other options
### Go
Go is an open source programming language supported by Google.
* **Pros**
* Rapidly gaining popularity
* **Cons**
* Will still require JavaScript for client side functionality
* Not a lot of front-end support
# Communications Tooling: Wiki Platform
* **Status:** Accepted
* **Last Modified:** 2023-07-10
* **Related Issue:** [#30](https://github.com/HHS/simpler-grants-gov/issues/30)
* **Deciders:** Lucas, Aaron, Billy, Sarah, Sumi
* **Tags:** communucations, open source, wiki
## Context and Problem Statement
The communications platform deliverable identifies a series of platforms through which the Grants API project needs to engage both internal and external stakeholders. One of these platforms is a wiki for storing notes, documents, and other content about the project. Ideally we would select a platform that balances ease of use and flexibility with the cost of implementing and maintaining the wiki.
The goal of this ADR is to evaluate a series of potential wiki platforms and determine which one best fits the needs and objectives of this project based on the decision criteria outlined below.
## Decision Drivers
#### Must Have
* **Usability:** Non-technical users should be able to access and create content with minimal training or guidance.
* **Public Access:** Members of the public should be able to read public documentation in the wiki without needing to sign up or login to a service.
* **Content Review:** Collaborators should be able to review and edit draft content before those changes are published.
* **Comments:** Reviewers should be able to leave in-line comments on content that they are reviewing.
* **Version History:** Editors should be able to see and restore previous versions of a given page.
* **Multi-Media:** The platform should support multiple types of media (e.g. videos, images, file uploads, tables, diagrams) with minimal configuration.
* **Internationalization (i18n):** The platform should provide support for displaying content in multiple languages.
* **Web Analytics:** The platform should provide support for tracking site usage and other web analytics.
* **Onboarding Costs:** Onboarding new members to the platform should be relatively inexpensive, both in terms of staff time/resources and direct costs (e.g. licensing fees).
* **Maintenance Costs:** It should not be prohibitively expensive to maintain the wiki, both in terms of staff time/resources and direct costs (e.g. hosting fees).
#### Nice to Have
* **External Contributions:** Members of the public should be able to suggest changes to wiki content and internal stakeholders should be able to review those contributions before they are published.
* **Data Access:** Content generated and stored in the wiki should be accessible outside of the wiki platform, either through syncing content to an HHS owned repository or through an official API.
* **Machine Readability:** The wiki platform should also support storing and exposing content in a machine-readable format so that certain types structured data can be managed within and accessed from the wiki without parsing.
* **Open Source:** The tool used to manage and host the wiki content should be open source, if possible.
* **Authority to Operate (ATO):** Because the wiki is a support tool rather than a production service, it doesn't *need* to be covered under the Grants.gov ATO. However, being covered under the existing ATO is an advantage if, in the future, we want to use it to support our production service (e.g. hosting training materials for grant applicants or grantors)
## Options Considered
* [Confluence](https://www.atlassian.com/software/confluence) - *NOT chosen* because of limits around data access and content review
* [Notion](https://www.notion.so/) - *NOT chosen* because of limits on version history and content review
* [GitHub Wiki](https://docs.github.com/en/communities/documenting-your-project-with-wikis) - *NOT chosen* because of limited feature set and issues with usability
* [GitBook](https://www.gitbook.com/) - *Chosen* because of support for content review and GitHub syncing
* [WikiJS](https://js.wiki/) - *NOT chosen* because of issues with usability and requirements for ongoing maintenance
## Decision Outcome
We have decided to use **GitBook** as our wiki platform because it balances the usability and maintainability of a SaaS offering like Confluence with key features around data access and content review.
Although it is a proprietary tool, it has become a standard platform for managing documentation within open source projects, mainly because it emphasizes version control within the documentation and enables bi-directional syncing of content between GitBook and GitHub.
### Positive Consequences
* Wiki content can be presented in a more usable format for editing and reading, while still being version-controlled alongside our code in GitHub
* Non-technical users who are not familiar editing content directly in markdown can easily create and modify pages (if they have a GitBook license)
* Contributions and changes to existing documentation can be reviewed before they are published
### Negative Consequences
* Users who don't have a license to edit and manage content in GitBook (i.e. members of the public) can only make suggested edits or contributions through creating Pull Requests (PRs) in GitHub which can present a high barrier to entry for non-technical users
* Because GitBook isn't covered under the existing Grants.gov ATO, we will not be able to use it for production services. If in the future we want to use GitBook as part of our production service, we'll need to seek ATO approval.
### Back-up Options
If we can't get coverage for GitBook under the existing ATO, we should pursue one of the following solutions:
* **Wiki.js** if we want to prioritize data access and open source tools but are willing to compromise on maintenance costs and usability
## Comparison Matrix
* ✅ Feature available, meets requirement
* ❌ Feature not available, does not meet requirement
* 🔄 Partial feature, limited feature availability, feature in progress or undergoing improvements
* 1-3 Strength level
* ❓Unknown
| Factor | Confluence | Notion | GitHub Wiki | GitBook | Wiki.js |
| --------------------------- | :--------: | :----: | :---------: | :-----: | :-----: |
| Usability | 3 | 2 | 1 | 2 | 1 |
| Public Access | 🔄 | 🔄 | ✅ | ✅ | ✅ |
| Content Review | 🔄 | ❌ | ❌ | ✅ | 🔄 |
| Comments | ✅ | ✅ | ❌ | ✅ | 🔄 |
| Version History | ✅ | 🔄 | ✅ | ✅ | 🔄 |
| Multi-Media | ✅ | ✅ | ❌ | ✅ | ✅ |
| I18n | 🔄 | 🔄 | ❌ | ✅ | ✅ |
| Web Analytics | ✅ | ✅ | ❌ | ✅ | 🔄 |
| Onboarding Cost Efficiency | 2 | 2 | 3 | 2 | 1 |
| Maintenance Cost Efficiency | 2 | 2 | 3 | 2 | 1 |
| External Contributions | ✅ | ✅ | 🔄 | 🔄 | 🔄 |
| Data Access | 🔄 | 🔄 | ✅ | ✅ | ✅ |
| Machine Readability | 🔄 | ✅ | ❌ | 🔄 | ✅ |
| Open Source | ❌ | ❌ | ❌ | ❌ | ✅ |
| Authority to Operate | ❌ | ❌ | ✅ | ❌ | ✅ |
## Pros and Cons of the Options
### Confluence
[Confluence](https://www.atlassian.com/software/confluence) is a Software as a Service (SaaS) documentation and collaboration tool offered by Atlassian that organizes content into "spaces" and offers a series of templates and components that can be used to create custom documentation for internal and external stakeholders.
#### Details
* **Hosting:** SaaS
* **Pricing:** [$5.75 (standard) or $11 (premium) per user per month](https://www.atlassian.com/software/confluence/pricing)
* **Public Access:** Supported, but limited to individual pages or entire spaces
* **Content Review:** Partial support, limited to drafts and not enforceable
* **Version History:** Supported by default
* **Supported Media**
* Markdown style text
* File uploads
* Image embedding
* Video embedding
* Diagrams
* **I18n:** Limited third party plugins for automating translation
* **Web Analytics:** [Google Analytics plugin](https://marketplace.atlassian.com/apps/1216936/google-analytics-in-confluence) available, also native analytics with premium tier
* **Open Source Status:** Propietary
* **External Contributions:** Only supported in public spaces (without review)
* **Data Access:** Limited access via API
#### Pros
* Relatively user friendly for non-technical users
* Supports a wide variety of media and page content
* Supports drafts, which allow edits to be made without publishing
* Supports page history and comparison of previous versions
* Limited support for public pages and spaces
* Minimimal ongoing maintenance costs due to SaaS hosting
* Most affordable per user cost for a given tier of features
#### Cons
* Does not support a formal review process for drafts before they can be published
* Supports fewer content types than Notion, especially in terms of structured data
* Content can only be made public at the level of an individual page or an entire space
* Content API is limited in capability and less intuitive than Notion API
* No direct support for internationalization and localization
* Closed source proprietary tool
* Data is controlled by Confluence, only accessible via API
### Notion
[Notion](https://www.notion.so/) is a Software as a Service (SaaS) documentation and collaboration tool that also allows users to add structured and semi-structured content to pages. Because Notion offers a fully-featured API for reading and managing content, it also has a robust set of community integrations that extend Notion's core functionality.
#### Details
* **Hosting:** SaaS
* **Pricing:** [$8 (pro) or $15(business) per user per month](https://www.notion.so/pricing)
* **Public Access:** Supported, but limited to individual pages
* **Content Review:** Not supported
* **Version History:** Limited support, past 30-90 days
* **Supported Media**
* Markdown style text
* Tabular/structured data
* Image embedding
* Video embedding
* Diagrams
* **I18n:** Partial support, Third-party beta plugin for translation
* **Web Analytics:** Native page analytics available
* **Open Source Status:** Propietary
* **External Contributions:** Only supported on public pages (without review)
* **Data Access:** Access via API
#### Pros
* Relatively user friendly for non-technical users
* Allows publishing pages to the web for external user access
* Supports the widest variety of page content and media
* Supports public comments and edits in the app
* Minimimal ongoing maintenance costs due to SaaS hosting
* Exposes wiki content via an API
* Robust plugin and add-on community
#### Cons
* Slightly more complicated interface than Confluence
* Pages published to the web aren't organized as clearly as public pages in GitBook
* No content review process, all changes and comments are published automatically
* No direct support for internationalization and localization
* Page history is limited to 30 (pro) or 90 (business) days
* Closed source proprietary tool
* Data is controlled by Notion, only accessible via API
### GitHub Wiki
[GitHub Wiki](https://docs.github.com/en/communities/documenting-your-project-with-wikis/about-wikis) is a free feature for public repositories that allows maintainers of the repository to host documents and other content that isn't stored directly within the repository itself.
#### Details
* **Hosting:** SaaS
* **Pricing:** Free for public repositories
* **Public Access:** Supported by default
* **Content Review:** Not supported
* **Version History:** Supported by default
* **Supported Media**
* Markdown style text
* Image embedding
* **I18n:** No support
* **Web Analytics:** No support
* **Open Source Status:** Propietary
* **External Contributions:** Supported with GitHub login (without review)
* **Data Access:** Can be exported or cloned from repo
#### Pros
* Available for free with public repositories
* Supports public access to view wiki content by default
* Minimimal ongoing maintenance costs due to SaaS hosting
* All of the wiki data can be exported with the GitHub repo
* Supports contributions from anyone with GitHub license (based on wiki settings)
#### Cons
* One of the hardest-to-use tools for non-technical audiences
* Supports a very limited set of media formats, mainly markdown and images
* Does not support web analytics
* Does not support internationalization
* Closed source proprietary tool
### GitBook
[GitBook](https://www.gitbook.com/) is a Software as a Service (SaaS) platform for creating and managing public documentation for a project. GitBook prioritizes version control and collaboration by offering first class support for reviewing and merging content changes. It has become a common documentation and wiki tool for many open source projects.
#### Details
* **Hosting:** SaaS
* **Pricing:** [$6.70 (plus) or $12.50 (pro) per user per month](https://www.gitbook.com/pricing)
* **Public Access:** Supported by default
* **Content Review:** Supported by default
* **Version History:** Supported by default
* **Supported Media**
* Markdown style text
* File uploads
* Image embedding
* Video embedding
* Diagrams
* **I18n:** Native support with [page collections and variants](https://gitbook.com/docs/guides/content-organization-and-localization/localize-your-docs-with-variants-in-gitbook)
* **Web Analytics:** [Google Analytics integration](https://www.gitbook.com/integrations/googleanalytics) available
* **Open Source Status:** Propietary
* **External Contributions:** Limited support, only through GitHub PRs
* **Data Access:** Full access with GitHub sync, limited access with API
#### Pros
* Relatively friendly for non-technical users
* Most robust content review process across all wikis, enforceable on a page-by-page basis
* Public spaces and pages are well organized and searchable
* Supports a wide variety of media and content types
* Supports Google Analytics integration as well as limited native analytics
* Supports external contributions through GitHub PRs
* Supports beginning-of-time page history and version comparison
* Supports itnernationalization across all pages
* Minimimal ongoing maintenance costs due to SaaS hosting
* Exposes limited API for content and space management
* Data can be synced to GitHub repo owned by HHS
#### Cons
* Slightly more complicated interface than Notion or Confluence
* Supports fewer content types than Notion, especially in terms of structured data
* Content API is limited in capability and less intuitive than Notion API
* External users cannot directly comment or suggest changes in the app
* Native web analytics are less robust than Confluence or Notion
* More expensive than Confluence for similar feature set
* Not currently covered under the Grants.gov ATO
### Wiki.js
[Wiki.js](https://js.wiki/) is an open source wiki platform that seeks to replicate many of the basic features found in SaaS offerings like Confluence or GitBook.
#### Details
* **Hosting:** Self-hosted
* **Pricing:** Free to use, cost of self-hosting
* **Public Access:** Supported by default
* **Content Review:** Limited support, via GitHub PRs
* **Version History:** Limited support
* **Supported Media**
* Markdown style text
* File uploads
* Image embedding
* Video embedding
* Diagrams
* **I18n:** Native support
* **Web Analytics:** Google Analytics support available
* **Open Source Status:** Open Source
* **External Contributions:** Requires sign up (but no license cost)
* **Data Access:** Total control over data, Git-sync and API available
#### Pros
* Open source and self-hosted
* Covered under the existing Grants.gov ATO
* Robust control over access and permissions
* Supports Google Analytics integration
* Supports external contributions through GitHub PRs
* Supports itnernationalization across all pages
* Supports for git syncing and API access
* Full control over data
#### Cons
* *Much* less intuitive than other wiki options
* Documentation for the tool is lacking
* Requires *significant* investment of staff time for initial configuration and ongoing maintenance
* Features are less mature than other SaaS offerings like Confluence or GitBook
## Links
* [Confluence](https://www.atlassian.com/software/confluence)
* [Confluence Pricing](https://www.atlassian.com/software/confluence/pricing)
* [Confluence Permissions](https://www.atlassian.com/software/confluence/resources/guides/get-started/manage-permissions#manage-global-permissions)
* [Confluence Google Analytics](https://marketplace.atlassian.com/apps/1216936/google-analytics-in-confluence)
* [Confluence Public Spaces](https://support.atlassian.com/confluence-cloud/docs/what-are-space-permissions/)
* [Confluence Public Pages](https://support.atlassian.com/confluence-cloud/docs/share-content-externally-with-public-links/)
* [Notion](https://www.notion.so/)
* [Notion Pricing](https://www.notion.so/pricing)
* [Notion Internationalization](https://www.notion.com/templates/product-localization-w-notion-ai)
* [Notion Permissions](https://www.notion.com/help/sharing-and-permissions)
* [GitHub Wiki](https://docs.github.com/en/communities/documenting-your-project-with-wikis/about-wikis)
* [GitBook](https://www.gitbook.com/)
* [GitBook Pricing](https://www.gitbook.com/pricing)
* [GitBook Open Source Pricing](https://www.gitbook.com/pricing#faq-open-source)
* [GitBook Internationalization](https://gitbook.com/docs/guides/content-organization-and-localization/localize-your-docs-with-variants-in-gitbook)
* [GitBook Google Analytics](https://www.gitbook.com/integrations/googleanalytics)
# Use Mural for design diagrams and whiteboarding
* **Status:** Active
* **Last Modified:** 2023-07-11
* **Related Issue:** [#116](https://github.com/HHS/simpler-grants-gov/issues/116)
* **Deciders:** Andy, Emily, Sumi, Aaron, Billy, Lucas
* **Tags:** design
## Context and Problem Statement
Which tool should be used for diagramming and whiteboarding? This tool is a primary concern for design in the creation of low-fidelity wireframes, process/logic flows, journey maps, service blueprints.
This diagramming tool will not be used only by designers. The right option will double as a whiteboarding tool for all disciplines, eliminating the need for a separate whiteboard tool. However, the goal here is not to prevent duplicative tools. There are programmatic methods of generating diagrams (e.g. MermaidJS) that will likely be used for creating architectural diagrams and tecnhical documentation in a machine-readable, version-controled format.
Note that the intended use for this particular tool is collaborative drawing.
## Decision Drivers
* Collborative editing
* Functions as both a digramming tool and a general whiteboard
* Useful to all disciplines (design, engineering, product, project)
## Options Considered
* Mural
* Lucidchart
* Miro
* Visio
* Figjam
* Draw\.io
* ClickUp
## Decision Outcome
Chosen option: Mural, because HHS has existing licenses and Nava has extensive experience using it for multiple purposes (diagrams, whiteboard, research synthesis, brainstorming, etc.). Mural is very effective as a collaborative drawing canvas. And it will be valuable to all disciplines for various purposes.
### Positive Consequences
* Mural can be used for sprint retrospective board, eliminating the need for an retro-specific tool
* Nava has found that using diagrams as the artifact engineers reference in implementation is an especially fast way of working with a design system (USDWS), preventing misinterpretation of visual design intent.
### Negative Consequences
None, realy. There's a possible duplication of tool capabilities — Figma comes with Figjam, which has similar features; engineers may prefer MermaidJS for diagraming — which may justify further analysis in the future.
## Pros and Cons of the Options
### Mural
* **Pro**
* Can begin using immediately:
* Existing HHS licenses
* Existing Nava licenses
* Nava familiarity
* Large selection of templates for many purposes (brainstorms, retros, research, planning…)
* Integrations with other products (Slack, GSuite…)
* All the standard whiteboarding features:
* Sharing / commenting
* Real-time user following (useful in presentations)
* Timer for collaborative sessions
* etc.
* **Cons**
* Free tier limited to 3 Murals
### Lucidchart
Note: If in the use of Mural, it becomes evident that the project would benefit from an additional tool that's more specifically made for drawing diagrams, Lucidchart should be chosen over the remaining options. However, choosing Lucidchart for that purpose would not require reevaluating Mural as a whiteboarding tool.
* **Pro**
* Excells at flow charts, ER models, UML diagrams
* Great user/access/file management
* **Cons**
* Not effective as a whiteboarding tool
## Miro
* **Pro**
* Basically Mural
* Includes video conferencing (not necessary?)
* **Cons**
* Fewer facilitator tools than Mural
## Visio
* **Pro**
* Integrates with Office 365 products
* Great selection of diagramming symbols
* **Cons**
* Pricey / not accessible for FOSS community
## Figjam
* **Pro**
* Comes with Figma
* Freemium version
* Simple UI
* Actively/regularly improved by Figma
* **Cons**
* Less mature product (recent acquisition of Diagram)
## Draw\.io
* **Pro**
* Totally free
* Confluence integration (among others)
* **Cons**
* Missing some drawing features
* Not effective as a whiteboarding tool
* Nava has experienced UI bugginess resulting in lost work
* File management is not intuitive (especially when synced to Sharepoint)
## ClickUp
* **Pro**
* Includes project management features, tasks, whiteboards, dashboards, chat, wiki…
* **Cons**
* More of a suite of products trying to compete with Atlassian, 365, etc
* "One app to replace them all" requires relying on whole suite to get the most of its features
# Ticket Tracking
* **Status:** Accepted
* **Last Modified:** 2023-07-11
* **Related Issue:** [#98](https://github.com/HHS/simpler-grants-gov/issues/98)
* **Deciders:** Lucas, Esther, Sumi, Sarah, Aaron, Billy
* **Tags:** communications, sprint planning, agile
## Context and Problem Statement
The project needs a system for tracking ongoing development work within the project, preferably as a series of tickets that can be organized into sprints. This system would both enable internal stakeholders to prioritize key tasks and assignments throughout the project and help communicate those priorities to external stakeholders.
The goal of this ADR is to evaluate a series of ticket tracking systems and select the one we will be using for the project.
## Decision Drivers
#### Must Have
* **Ticket Tracking:** Tickets can be organized into sprints and tracked as part of larger deliverables
* **Public Access:** Without logging in, members of the public can see tickets that are being worked on
* **Public Requests:** Members of the public can submit bug reports and feature requests and track how that work is being prioritized
* **Templates:** The system supports default templates for different types of tickets which prompts the person creating the ticket for a specific set of information
* **Authority to Operate (ATO):** The platform already must be authorized under the Grants.gov ATO (Authority to Operate) or ATO coverage must be requested
* **Reporting:** Users can report on the following key metrics:
* Sprint burndown -- % of tickets and/or points remaining over the course of the sprint
* Deliverable burdown -- % of tickets and/or points remaining per deliverable over time
* Sprint allocation -- # of tickets and/or points allocated to each team member per sprint
* Sprint completion -- # of tickets and/or points created vs planned per sprint
* Sprint velocity -- Average # of tickets and/or points completed per sprint
* Lead/cycle time -- # of days between creation of a ticket and when that ticket is closed
#### Nice to Have
* **Level of Effort Estimates:** Tickets can be assigned an estimated level of effort (e.g. story points, t-shirt size, etc.)
* **Custom Views:** Users can create custom views for managing tickets with multiple layouts (e.g. kanban board, tabular, roadmap)
* **Custom Fields:** Users can create custom fields and views to manage their projects
* **Automation:** Users can automate basic workflows like adding and moving tickets, linking PRs to their originating PRs, etc.
* **Open Source:** The tool used to manage and host the wiki content should be open source, if possible
## Options Considered
* [GitHub Issues + Zenhub](#github-issues--zenhub)
* [GitHub Issues + GitHub Projects](#github-issues--github-projects)
* [Jira](#jira)
* [OpenProject](#openproject)
## Decision Outcome
We are planning to use **GitHub issues with GitHub projects** because it is the only solution that allows members of the public to submit feature requests and bug reports and then track how those issues are being prioritized within upcoming sprints. Additionally, the use of these tools is free with public repositories and offers other helpful features such as custom fields and views.
**NOTE:** Given some of the constraints around reporting, we may want to continue to explore options that extend the reporting capabilities of GitHub projects. Similarly, if team productivity is meaningfully affected by the absence of the more robust features that Jira or Zenhub offers, we may want to re-evaluate this decision after an initial trial period.
### Positive Consequences
* We do not need to purchase licenses or seek ATO approval to start begin tracking tickets and planning sprints
* Members of the public can submit feature requests or bug tickets and track how those requests are being prioritized and worked on
* We can manage all of our sprint planning and development within the same platform
* We can extend GitHub project functionality with custom-built automations
### Negative Consequences
* We will need to develop custom reports to track some of the same metrics that Jira or Confluence offers out of the box. Our current proposed strategy for doing so is to create a Python script that exports the tickets and produces the following charts at the end of each sprint:
* Sprint burndown -- % of tickets and/or points remaining over the course of the sprint
* Deliverable burdown -- % of tickets and/or points remaining per deliverable over time
* Sprint allocation -- # of tickets and/or points allocated to each team member per sprint
* Sprint completion -- # of tickets and/or points created vs planned per sprint
* Sprint velocity -- Average # of tickets and/or points completed per sprint
* Lead/cycle time -- # of days between creation of a ticket and when that ticket is closed
* We will need to spend a bit more time setting up the GitHub project to replicate some of the features that Jira or Confluence offers (e.g. story points, sprints, epics, etc.)
* Team members who are familiar with Jira and Zenhub will need to spend a bit more time becoming familiar with GitHub Projects
## Comparison Matrix
* ✅ Feature available, meets requirement
* ❌ Feature not available, does not meet requirement
* 🔄 Partial feature, limited feature availability, feature in progress or undergoing improvements
* 1-3 Strength level
* ❓Unknown
| Factor | Zenhub | GitHub Projects | Jira | OpenProject |
| ------------------------- | :-----------: | :-------------: | :-----------: | :-----------: |
| Cost | $8.33/user/mo | Free | $7.75/user/mo | $7.25/user/mo |
| Ticket Tracking | ✅ | ✅ | ✅ | ✅ |
| Public Access | 🔄 | ✅ | 🔄 | ❌ |
| Public Requests | ✅ | ✅ | ❌ | ❌ |
| Issue Templates | 🔄 | ✅ | ✅ | ❌ |
| Reporting | ✅ | 🔄 | ✅ | ✅ |
| Authority to Operate | 🔄 | ✅ | ✅ | ✅ |
| Level of Effort Estimates | ✅ | ✅ | ✅ | ✅ |
| Custom Views | 🔄 | ✅ | ✅ | 🔄 |
| Custom Fields | ❌ | ✅ | ✅ | 🔄 |
| Automation | 🔄 | ✅ | ✅ | ❌ |
| Open Source | ❌ | ❌ | ❌ | ✅ |
## Pros and Cons of the Options
### GitHub Issues + Zenhub
Use [GitHub Issues](https://docs.github.com/en/issues) to create and manage development tickets and use Zenhub to organize those tickets into sprints.
* **Pros**
* Built off of existing GitHub tickets and functionality
* Robust [reporting](https://www.zenhub.com/reporting) (e.g. burndown charts, velocity, etc.) out of the box
* Supports key planning features like [story points](https://www.zenhub.com/blog-posts/how-to-estimate-software-development-projects-with-story-points) and [epics](https://www.zenhub.com/blog-posts/working-with-epics-in-github)
* Supports issue templates (through GitHub)
* Chrome extension to view Zenhub attributes in GitHub
* Team has experience working with Zenhub
* **Cons**
* Licenses have a monthly fee, even for Government-backed open source projects
* Sprint boards can't be viewed without Zenhub login
* Moving tickets requires both Zenhub and GitHub logins and write access to the repository
* Can be difficult to onboard existing Zenhub users to a new workspace
* GitHub form-based templates don't work when creating issues from Zenhub
* Limited support for custom views (e.g. no tabular layout)
* No support for custom fields
### GitHub Issues + GitHub Projects
Use [GitHub Issues](https://docs.github.com/en/issues) to create and manage development tickets and use GitHub projects to organize those tickets into sprints.
* **Pros**
* Keeps ticket creation and sprint planning in the same platform alongside code
* Free for open source repositories
* GitHub project boards can be viewed without a GitHub login
* Supports issue templates (through GitHub)
* Supports multiple [views of tickets](https://docs.github.com/en/issues/planning-and-tracking-with-projects/customizing-views-in-your-project/managing-your-views) (e.g. tabular, kanban board, roadmap)
* Supports custom fields with multiple data types (e.g. numbers, drop downs, iterations, text fields, etc.)
* Supports [custom reporting](https://docs.github.com/en/issues/planning-and-tracking-with-projects/viewing-insights-from-your-project/about-insights-for-projects) based on those fields
* Supports [built-in workflow automation](https://docs.github.com/en/issues/planning-and-tracking-with-projects/automating-your-project) and custom automation with [GitHub actions](https://github.com/features/actions)
* Supports filters on all standard and custom fields
* Covered under the existing Grants.gov ATO
* **Cons**
* Reporting is less robust than Zenhub and Jira
* Requires more initial setup to replicate some of Jira or Zenhub's out-of-the-box features
* Team has less experience working with GitHub projects
### Jira
Use Confluence's [Jira](https://www.atlassian.com/software/jira) platform for both ticket management and sprint planning.
* **Pros**
* Industry standard tool for ticket tracking and sprint planning
* Robust [reporting](https://www.atlassian.com/software/jira/features/reports) (e.g. burndown charts, velocity, etc.) out of the box
* Supports [custom fields](https://support.atlassian.com/jira-cloud-administration/docs/create-a-custom-field/) with multiple data types (e.g. numbers, drop downs, text fields, etc.)
* Supports multiple views of tickets (e.g. tabular, kanban board, roadmap)
* Supports filters on all standard and custom fields
* Supports built-in [workflow automation](https://www.atlassian.com/software/jira/features/automation)
* Supports custom ticket templates
* Third-party extensions are available to support public access to Jira boards
* Team has experience working with Jira
* **Cons**
* Licenses have a monthly fee (above 10 users)
* Sprint boards can't be viewed without Zenhub login
* Members of the public can't submit requests to a Jira board
* Requires tracking tickets and planning sprints on a different platform
### OpenProject
Use the open source project management tool [OpenProject](https://www.openproject.org/) for both ticket management and sprint planning.
* **Pros**
* Open source project with self-hosting option
* Robust reporting (e.g. burndown charts, velocity, etc.) with enterprise plan
* Supports multiple views of tickets (e.g. tabular, kanban board, roadmap)
* Supports key planning features like story points and epics out-of-the-box
* **Cons**
* Many basic features require enterprise license which has a per-user cost
* Higher investment of time to set up and maintain the project
* Sprint boards can't be viewed without logging in
* Members of the public can't submit requests to a Jira board
* Doesn't seem to support issue or ticket templates
* Doesn't seem to support workflow automation
* Team has less experience working with OpenProject
## Links
* [GitHub Issues](https://docs.github.com/en/issues)
* [GitHub Projects](https://docs.github.com/en/issues/planning-and-tracking-with-projects)
* [GitHub Projects Automation](https://docs.github.com/en/issues/planning-and-tracking-with-projects/automating-your-project)
* [GitHub Projects Actions](https://github.com/features/actions)
* [GitHub Projects Reporting](https://docs.github.com/en/issues/planning-and-tracking-with-projects/viewing-insights-from-your-project)
* [GitHub Projects Custom Views](https://www.google.com/search?q=https://docs.github.com/en/issues/planning-and-tracking-with-projects/customizing-your-project-views)
* [GitHub Projects Custom Fields](https://docs.github.com/en/issues/planning-and-tracking-with-projects/understanding-fields)
* [Zenhub](https://www.zenhub.com/)
* [Zenhub Epics](https://www.zenhub.com/blog-posts/working-with-epics-in-github)
* [Zenhub Story Points](https://www.zenhub.com/blog-posts/how-to-estimate-software-development-projects-with-story-points)
* [Zenhub Reporting](https://www.zenhub.com/reporting)
* [Jira](https://www.atlassian.com/software/jira)
* [Jira Reporting](https://www.atlassian.com/software/jira/features/reports)
* [Jira Automation](https://www.atlassian.com/software/jira/features/automation)
* [Jira Custom Fields](https://support.atlassian.com/jira-cloud-administration/docs/create-a-custom-field/)
* [Open Project](https://www.openproject.org/)
# Front-end Framework
* **Status:** Active
* **Last Modified:** 2023-07-14
* **Related Issue:** [#97](https://github.com/HHS/simpler-grants-gov/issues/97)
* **Deciders:** Lucas, Aaron, Billy, Sammy, Daphne, Andy, Sarah
## Context and Problem Statement
The project will need a web framework to build and manage the frontend. The goal of this ADR is to evaluate and select a frontend web framework.
## Decision Drivers
### Must Haves
* Active Maintenance: The web framework is actively maintained with patches and minor releases delivered on a regular basis
* Community of Users: The web framework has an active community of open source users and the framework is commonly used for frontend development
* Usability: The framework is relatively easy to learn for developers without prior experience in the particular framework, and there are plenty of resources and training materials available
* Static Site Generation: The framework can generate static pages (HTML/CSS + JavaScript) at build time that can be cached in a CDN for faster loading
* Server-Side Rendering: The framework can render some pages server-side with every request to get up-to-date information when the page loads
* Client-side Rendering: The framework also supports rendering or modifying content client-side based on user interaction with the page (e.g. filtering, searching, etc.)
### Nice to Have
* AuthN/AuthZ: The framework supports Authentication & Authorization routing natively or there are established extensions that provide this functionality
* Internationalization (i18n): The framework supports localized routing for different languages natively or there are established extensions that provide this functionality
* Middleware: The framework supports other types of middleware (i.e. functions or scripts that execute before a routing request is complete)
## Options Considered
* Create React App
* Next.js
* Vue.js or Nuxt.js
* Svelte or Sveltekit
## Decision Outcome
Chosen option: Next.js, because this option meets all our technical requirements, has a large community of support, is easy to learn with good documentation, and is well understood by Nava.
### Positive Consequences
* We can use the Nava Next.js template to get started quickly
### Negative Consequences
* We'll need to modularize our code so that if Next.js ever loses support, we can swap it out
## Pros and Cons of the Options
### Create React App
Create React App is a lightweight, client side, single page application framework for React, maintained by Facebook. While it is one of the most widely adopted react frameworks, it seems like Facebook has either stopped or will stop supporting it in the near future, as it has been [removed as a suggested framework from the react website](https://github.com/reactjs/react.dev/pull/5487).
* **Pros**
* One of the earliest react frameworks
* Widely used and understood
* **Cons**
* Either has or will stop receiving support
* No longer recommended by react
### Next.js
Next.js is a popular full stack framework for static and server‑rendered applications built with React and can [prerender pages it determines are static automagically](https://nextjs.org/docs/pages/building-your-application/rendering/automatic-static-optimization) alongside server rendered routes to improve performance. It includes styling and routing solutions out of the box, is optimized for performance and SEO, and provides great developer documentation and support. Next.js is maintained by Vercel, a PaaS for frontend hosting company.
* **Pros**
* Popular framework with dedicated support
* Supports static site generation, server side rendering, and client side rendering
* Easy to learn and use with good documentation
* **Cons**
* Very opinionated with routing which can significantly increase code complexity for non standard routes
### Vue.js or Nuxt.js
Vue.js is an open-source, JavaScript framework for building progressive user interfaces that also supports server side rendering. It was created by Evan You in 2014 and has grown in popularity, thanks to its reactive data binding and component-based architecture. Nuxt.js provides a set of conventions and tools for building Vue.js applications, including automatic code splitting, prefetching, and caching.
* **Pros**
* Static site generation is easy out of the box
* Code splitting helps reduce package sizes and makes caching easier
* **Cons**
* Small community of support
* Scalability
### Svelte or Sveltekit
Svelte is a JavaScript, front-end compiler that turns declarative and easy to understand JavaScript code into highly efficient JavaScript code optimized for the browser. In contrast to the React framework, SvelteKit, uses a "compiler-first" approach to add server side rendering capabilities to Svelte, eliminating the need for a virtual DOM, improving performance, and reducing bundle size.
* **Pros**
* Fast, performant, and very scalable
* Not opinionated and very flexible
* **Cons**
* Relatively new framework, with fewer resources and plugins than React
* Steeper learning curve, since many JavaScript developers are used to React
* Small community
* Limited documentation
## Links
* [React removing create react app from its recommendations](https://github.com/reactjs/react.dev/pull/5487)
# Front-end Code Quality Tools
## Front-end Code Quality Tools
* **Status:** Active
* **Last Modified:** 2023-07-17
* **Related Issue:** [#102](https://github.com/HHS/simpler-grants-gov/issues/102)
* **Deciders:** Aaron, Daphne, Sammy, Gina, Lucas, Billy
* **Tags:** ADR
### Context and Problem Statement
Front-end code quality tools should facilitate and efficiently enforce linting, auto-formatting, type-checking and security concerns. They should be easily configurable to suit our specific use-cases, uniformly applying conventions while alleviating the need for individual intervention.
### Decision Drivers
* **Ease of use and configurability:** Code quality tools should be configurable to meet the specific needs of our project and enforce standards uniformly. Running the tool should be simple, and feedback should be easily implementable and understandable.
* **Speed:** Code quality tools should be able to run in an timely manner, ideally in a pre-commit hook.
* **Documentation and resources:** Code quality tools should have robust and helpful documentation, specifically around usage and configuration.
* **Lean:** Selection of code quality tools should cover all use-cases with little overlap for ease of use and to minimize mental surface area required to understand the system. As in, each tool should have a specific, meaningful purpose.
* **Enforces security:** Tooling should assist us in identifying security concerns and vulnerabilities.
* **Well-maintained:** We have a preference towards libraries that are widely adopted and have active maintainers.
### Options Considered
1. Use a curated collection of JavaScript libraries from Next.js template repository (described below)
2. Use a different or modified set of JavaScript libraries
#### Dependency Management
[**npm**](https://www.npmjs.com/)**:** Package manager for Node.js.
* npm is a robust package manager that is pre-bundled with Node, simplifying installation steps
* Maintained by Github/Microsoft
#### Code Linting
[**ESLint**](https://eslint.org/)**:** Statically analyzes your code to quickly find problems.
* Supported by a huge variety of editors and easy to integrate into CI/CD pipeline
* Very long-standing project, hugely popular, and well-supported tool
#### Auto-formatting
[**Prettier**](https://prettier.io/)**:** Enforces code style and consistency while reducing the need for manual formatting or nitpicking.
* Top pick in many JavaScript repositories as a well-maintained formatting tool
#### Type Checking
[**TypeScript**](https://www.typescriptlang.org/)**:** Strongly typed programming language that builds on JavaScript
* Converts to JavaScript, therefore works everywhere JavaScript is compatible
* Static-type checking built into language
* Identifies problems early at compile time
* Open source and maintained by Microsoft
#### Dependency Checks
[**Renovate**](https://docs.renovatebot.com/)**:** Automated dependency updates.
* Free and open source
* Can batch, schedule, auto-assign and auto merge updates
* Multi-language and supports Python as well
*Note: Do we need the Safety package for our Python code as well or is this sufficient?*
#### License Checking[^1]
[**License Checker**](https://www.npmjs.com/package/license-checker)**:** Very aptly named tool to check licenses for dependencies. Can fail on specified input, a semicolon separated list.
### Decision Outcome
Option #1 is preferred. The tooling chosen in the template repository suits our needs, with the addition of License Checker.
## Other Options
**Dependency Management:**
[Yarn](https://yarnpkg.com/): Functionally similar to npm, however, requiring more steps to integrate into a project.
**Type Checking:**
[Flow](https://flow.org/): Static type checker maintained by Facebook, added to project as a dependency. Uses a special type syntax that is removed from code at compile time.
[Immutible](https://immutable-js.com/): Provides a set of persistent, immutible data structures. Preferred to start with TypeScript and see if that suits our needs as it comes with immutible functionality out of the box.
**Dependency Checks:**
[Dependabot](https://github.com/dependabot): Can easily switch to from Renovate at any time.
[^1]: Addition to the existing curated collection
# Front-end Testing & Coverage
* **Status:** Active
* **Last Modified:** 2023-07-18
* **Related Issue:** [183](https://github.com/HHS/simpler-grants-gov/issues/183)
* **Deciders:** Sammy, Lucas, Billy, Andy
* **Tags:** ADR
## Context and Problem Statement
Front-end testing is essential in maintaining a stable and healthy codebase, creating UI interfaces without regression, and an important part of the developer workflow.
## Decision Drivers
* **Robust:** Chosen testing frameworks should have features that offer diverse ways of verifying front-end codebase functionality, reducing the need for intensive manual testing
* **Well-maintained:** Accessible tooling is well-maintained by owners and keeps up with current ecosystems in which it will be integrated.
* **Ease of use:** Achieving high coverage should be attainable. ICs should be able to onboard with the tooling and execute in a reasonable time frame to maintain that coverage.
* **Fast:** Running tests shouldn't take ages.
## Options Considered
## Unit Testing
* Jest
* Mocha
### Pros and Cons of the Options
#### [Jest](https://jestjs.io/)
* **Pros**
* Lightweight, well-supported and documented testing solution
* Compatible with TypeScript, Node, React
* Snapshot capabilities to track UI changes
* Code coverage reporting that can be integrated into [Github Actions](https://github.com/marketplace/actions/jest-coverage-report)
* Already integrated in Next.js front-end template
* Built-in mocking & auto-mocking
* **Cons**
* Reports that snapshot capabilities are not for complicated UIs
* Auto-mocking reduces performance
#### [Mocha](https://mochajs.org/)
* **Pros**
* Has integration and end-to-end testing capabilities
* More robust in functionality, offering a wider array of capabilities and helpers
* Test coverage reporting available
* Well-documented and widely used in Node.js ecosystems
* **Cons**
* Not as lightweight as alternative, requiring integrating several other libraries
* Does not support snapshot testing
* Slower than Jest
## Visual, Interactive & Component Testing
* Storybook
* Cypress
### Pros and Cons of the Options
#### [Storybook](https://storybook.js.org)
* **Pro**
* Build reusable UI components and pages in isolation, mocking external dependencies and business logic
* Visually verify changes and debug on multiple device formats easily in the interface
* Stories can be imported into Jest or other testing frameworks
* Covers multiple testing types, including visual, interaction, component and accessibility
* Compatible with many helpful addons to assist in building front-ends faster
* Component testing addon uses Jest & Playwright under the hood for speed
* Snapshot capabilities to address DOM regressions
* Enhances collaboration between engineers and other team members such as designers
* **Cons**
* May not be compatible with all tools and eco-systems
* Initial ramp up for a conceptually new way of developing front-ends that ICs may not be accustomed with
* Hasn't been around as long as other testing tools
#### [Cypress](https://www.cypress.io/)
* **Pro**
* Stubbing and mocking is simple comparatively
* Well-documented with a long history of providing reliable end-to-end and component testing
* Allows for visual debugging
* Robust front-end end-to-end testing
* Interactive GUI for end-to-end and component testing
* **Cons**
* Functionality for our current use-case likely redundant with Storybook, need to maintain stories and component tests separately
* Can be very slow to run the full test suite
* Might easily get out of date as another tool and testing interface
* Uses command queuing instead of familiar async await syntax like Storybook
* Bundles many other third-party libraries
## Decision Outcome
### Unit Testing
**Jest**, because it is integrated into the Next.js template application, well-maintained, and lightweight. Importantly, documentation is thorough and helpful information for troubleshooting can be easily accessed.
### Visual, Interactive & Component Testing
**Storybook**, because it provides an effective methodology for developing robust front-ends quickly. It is already implemented in the Next.js template application. Integration allows multiple ways to test applications, including compatibility with Jest.
In the future, if we find that Storybook doesn't meet our testing needs, we can look into integration with Cypress. However, not recommended from the start because our use-cases are covered by Storybook and there's a strong preference to not have to integrate another tool or learn another syntax.
# Backend API Type
* **Status:** Accepted
* **Last Modified:** 2023-07-19
* **Related Issue:** [#186](https://github.com/HHS/simpler-grants-gov/issues/186)
* **Deciders:** Lucas, Aaron, Billy, Sammy, Daphne, Andy, Sarah
## Context and Problem Statement
The project will require a backend API. The goal of this ADR is to evaluate and select the API protocol our endpoints will adhere to.
## Decision Drivers
### Must Haves
* Well-Established Paradigm & Ease of Adoption: The API protocol needs to be well-established so that the consumer of the API is familiar with how to structure requests and process responses. The API should be simplistic, consistent, clear and backward compatibile
* Supports Common Uses Cases: The API needs to support our common use cases (e.g. querying a list of NOFOs that match certain search criteria), so that it "just works" out of the box without having to spend much time learning how to use it
* Easily Maintained and Scalable: The API needs to be relatively easy to maintain and scale, so that we can prioritize developing important product features over managing basic infrastructure. We need a flexible protocol that can satisfy both current and future needs of the project.
## Options Considered
* REST
* RPC
* SOAP
* GraphQL
## Decision Outcome
Chosen protocol: REST, because this option is widely opted, is highly scalable and can meet the demands of a large and active user base, and is flexible.
## Pros and Cons of the Options
### REST
REST (Representational State Transfer) is an architectural style for designing APIs. It uses HTTP verbs to represent different operations, such as GET to retrieve data, POST to create data, PUT to update data, and DELETE to delete data.
* **Pros**
* Simple and easy to understand
* Widely adopted
* Supports multiple data formats (flexible)
* Highly scalable
* **Cons**
* Can be verbose
* Not as efficient as some other protocols
* Not as well-suited for complex data structures
### SOAP
SOAP (Simple Object Access Protocol) is a protocol for exchanging information between applications. It uses XML to represent data and SOAP messages.
* **Pros**
* Well-defined and standardized
* Supports complex data structures
* Secure
* **Cons**
* Can be complex to implement
* Not as widely adopted as REST
* Not as efficient as some other protocols
### RPC
RPC (Remote Procedure Call) is a style of programming where a client application calls a procedure on a remote server.
* **Pros**
* Efficient
* Well-suited for complex data structures
* Secure
* **Cons**
* Not as widely adopted as REST or SOAP
* Can be complex to implement
* Not as flexible as REST
### GraphQL
GraphQL is a query language for APIs. It allows clients to request specific data from a server, rather than having to know what data is available.
* **Pros**
* Efficient
* Flexible
* Easy to use
* **Cons**
* Not as widely adopted as REST or SOAP
* Can be complex to implement
* Not as well-suited for complex data structures
## Links
* [REST vs. SOAP vs. GraphQL vs. RPC](https://www.altexsoft.com/blog/soap-vs-rest-vs-graphql-vs-rpc/)
* [Architectural Styles for APIs: SOAP, REST and RPC](https://medium.com/api-university/architectural-styles-for-apis-soap-rest-and-rpc-9f040aa270fa)
* [Different Types of APIs – SOAP vs REST vs GraphQL](https://www.freecodecamp.org/news/rest-vs-graphql-apis/)
* [SOAP vs REST - Difference Between API Technologies](https://aws.amazon.com/compare/the-difference-between-soap-rest/)
# Front-end Testing & Coverage
* **Status:** Accepted
* **Last Modified:** 2023-07-19
* **Related Issue:** [174](https://github.com/HHS/simpler-grants-gov/issues/174)
* **Deciders:** Sammy, Lucas, Billy, Andy, Daphne
* **Tags:** ADR
## Context and Problem Statement
Back-end testing is essential in maintaining a stable and healthy codebase, creating APIs without regression, and an important part of the developer workflow.
## Decision Drivers
* **Robust:** Chosen testing frameworks should have features that offer diverse ways of verifying back-end codebase functionality, reducing the need for intensive manual testing
* **Well-maintained:** Accessible tooling is well-maintained by owners and keeps up with current ecosystems in which it will be integrated.
* **Ease of use:** Achieving high coverage should be attainable. ICs should be able to onboard with the tooling and execute in a reasonable time frame to maintain that coverage.
* **Fast:** Running tests shouldn't take ages.
## Options Considered
## Unit Testing
* Pytest
## Test Coverage
* Coverage
### Pros and Cons of the Options
#### [Pytest](https://docs.pytest.org/)
The pytest framework makes it easy to write small, readable tests, and can scale to support complex functional testing for applications and libraries.
* **Pros**
* Lightweight, well-supported and documented testing solution
* Already integrated in Flask back-end template
* Modular fixtures for managing small or parametrized long-lived test resources
* Can run [unittest](https://docs.python.org/3/library/unittest.html) (including trial) and nose test suites out of the box
* Rich plugin architecture, with over 800+ external plugins and thriving community
* **Cons**
* Compatibility issues with other testing frameworks means it's difficult to swap out for other frameworks
#### [Coverage](https://coverage.readthedocs.io/)
Coverage.py is a tool for measuring code coverage of Python programs. It monitors your program, noting which parts of the code have been executed, then analyzes the source to identify code that could have been executed but was not.
* **Pros**
* Fully automated
* **Cons**
* Code coverage is only one piece of a stable and healthy testing approach
## Decision Outcome
### Unit Testing
**Pytest**, because it is integrated into the Nava Flask template application, well-maintained, and lightweight. Importantly, documentation is thorough and helpful information for troubleshooting can be easily accessed.
### Testing Coverage
**Coverage**, because it is integrated into the Nava Flask template application, well-maintained, and lightweight. Importantly, documentation is thorough and helpful information for troubleshooting can be easily accessed.
We added a code coverage threshold of %80 in `api/pyproject.toml`
# Deployment Strategy
* **Status:** Accepted
* **Last Modified:** 2023-07-20
* **Related Issue:** [187](https://github.com/HHS/simpler-grants-gov/issues/187)
* **Deciders:** Lucas, Daphne, Sammy, Billy
* **Tags:** ADR
## Context and Problem Statement
We need to choose a deployment strategy for the Grants.gov modernization effort that suits our core needs and will host our entire application ecosystem, both the API and frontend layers.
## Decision Drivers
* **Reliable:** The chosen deployment strategy should be able to offer at least 99.99% monthly application uptime with consistent results.
* **Scalable:** We should be able to scale our deployment to meet the demands of our containers while remaining lean.
* **Compatible with IaC:** The strategy should be compatible with Terraform as our chosen Infrastructure as Code solution.
* **Ease of use:** We prioritize ease of use as well as cost-efficiency. We understand that engineering time spent managing details of infrastructure is a trade off to less prescriptive strategies.
* **Growth oriented:** As our applications evolve, we need our chosen deployment strategy to grow with us or be easily swapped.
## Options Considered
* ECS with Fargate or EC2 launch type
* S3
* Lambda
## Decision Outcome
Chosen option: **ECS with Fargate launch type**, because it offers the most consistent and easy to use deployment strategy to host both the front-end and API layers of the Grants.gov modernization. Current template infrastructure integrates with ECS and the Fargate launch type.
## Pros and Cons of the Options
### ECS with Fargate or EC2 launch type
#### Fargate
Fargate is an AWS serverless compute tool for containers.
* **Pros**
* Run containers without having to manage or provision EC2 instances
* Removes operational overhead of scaling, patching, securing and managing servers
* Integrates with AWS Cloudwatch or other third party metrics tools
* Secure, running in dedicated runtime environments
* Scalable means pay for usage, not for reserved or wasted space
* Fargate Spot instances further reduce costs and are viable for our use-case
* **Cons**
* Offers less granular flexibility, favoring less DevOps overhead
* Possible that cost is higher than EC2 launch type ([Theoretical cost optimization by Amazon ECS launch type: Fargate vs EC2](https://aws.amazon.com/blogs/containers/theoretical-cost-optimization-by-amazon-ecs-launch-type-fargate-vs-ec2/))
* Nontrivial to gain direct access to a particular Fargate task
#### EC2
Secure and resizable compute capacity for virtually any workload.
* **Pros**
* Ability to granularly manage and provision resources of environment
* Can connect to an EC2 instance easily via SSH if necessary
* Many instance types to choose from to meet our needs
* Reliable, scalable and on-demand
* Compatible with many other tools in the AWS arsenal
* **Cons**
* Requires detailed management and provisioning of environment
* Mismanagement of environment can greatly increased costs
* Less friendly for engineering teams that are not DevOps dedicated
**A note on ECS:** It is possible to run EC2 instances to host our Docker containers without using ECS orchestration. However, since ECS is a free service provided by AWS and we would only pay for the underlying resources, forgoing ECS and an orchestration tool isn't an appealing strategy.
### S3
Object storage built to retrieve any amount of data from anywhere.
* **Pros**
* Suitable to host static websites, our current use-case
* Highly scalable with unlimited storage space
* Extremely cost-effective with pay-as-needed pricing model
* Highly durable with storage redundancy in multiple locations
* Easy to use interface with static website hosting options
* **Cons**
* Only suitable to host static websites, making it a difficult choice for a rapidly growing front-end in development
* Limited customization options
### Lambda
Run code without thinking about servers or clusters
* **Pros**
* Auto-scaling has limitations, and pay-per-request methodology is ultra lean
* No need for redundancy in multiple Availability Zones
* Run code without provisioning or managing any infrastructure
* Scalable to meet high demand
* **Cons**
* Ineffective for long-running processes, maximum duration of 15 minutes
* Deployment package maximum size is 250 MB
* Max Docker image size is 10 GB
* Deploying serverless applications might require project restructuring or additional tools and ramp up
* Less performant than alternatives with lack of dedicated resources
## Links
Interesting read on building a static React front-end hosted in S3 with Lambda back-end: [Server-side rendering for React in AWS Lambda](https://aws.amazon.com/blogs/compute/building-server-side-rendering-for-react-in-aws-lambda/)
# Use U.S. Web Design System for components and utility classes
* Status: Accepted
* Deciders: Loren, Rocket, Sawyer
## Context and Problem Statement
Projects should avoid reinventing the wheel where possible. A common place to do this is in the UI, by using a design system for frontend components and utility classes. This can help avoid inconsistencies in the UI, and can reduce barriers for new developers.
We want to use a design system that is:
* Section 508 compliant
* Open source
* Well maintained and documented
* Includes the typical components and design patterns needed for government websites
## Considered Options
* [U.S. Web Design System (USWDS)](https://designsystem.digital.gov/)
* [CMS Design System](https://design.cms.gov/)
## Decision Outcome
The template will provide U.S. Web Design System styling out of the box.
We will not follow their [install directions](https://designsystem.digital.gov/documentation/getting-started/developers), which suggests using Gulp as a task runner. Instead, to reduce the number of dependencies and configuration, we'll leverage Next.js's and Storybook's built-in Sass support. Copying the USWDS static assets into the project will be handled by a [`postinstall`](https://docs.npmjs.com/cli/v8/using-npm/scripts) script in `package.json`.
### Positive Consequences
* USWDS is the most popular design system for U.S. government websites and is maintained by GSA employees. It is the recommended way to meet the website standards detailed in the [21st Century Integrated Digital Experience Act](https://digital.gov/resources/21st-century-integrated-digital-experience-act/). [More key benefits can be read about here](https://designsystem.digital.gov/about/key-benefits/).
* [Project teams can theme the USWDS](https://www.navapbc.com/insights/us-web-design-system) if their project needs to match an existing brand.
### Negative Consequences
* Unlike the CMS Design System, USWDS doesn't provide React components. Project teams will need to create their own React components that output USWDS markup, or install a third-party library like [`react-uswds`](https://github.com/trussworks/react-uswds). In the future, [the template could include this library by default](https://github.com/navapbc/template-application-nextjs/issues/19).
* CMS projects may need to swap out USWDS for the CMS Design System, although the CMS Design System is based on USWDS, so this may not be necessary right away.
## Links
* [Previous research was done by Kalvin Wang and Shannon Alexander Navarro related to USWDS React libraries](https://docs.google.com/document/d/1KRWzH_wJUPKkFmBlxj6SM2yN3W7Or89Wa4TBVM3Ksog/edit)
# FE server rendering
### Context and Problem Statement
Next.js provides multiple ways of rendering a website, which have different commands: `next start` vs `next export`. For the platform, we need to determine which should be the default.
### Considered options
1. Server rendering (`next start`): Generates the full HTML markup for a page on the server in response to navigation. Initial page data can be fetched on the server.
2. Prerendering (`next export`): At compile time, a separate HTML file is generated for each URL. Only the initial state of the page is generated as static HTML. To display personalized data, client-side JS is required and the page's DOM is updated.
### Decision drivers
1. The selected option should represent what we consider the preferred approach for the types of websites we're typically building with Next.js: authenticated, personalized web applications (claimant portals, compare tools, case management systems, etc).
2. Reduce the need for third party dependencies or custom code, when a native option works just as well.
3. A (reasonable) increase in cloud costs is acceptable if it results in a system that is more maintainable for software teams and the government in the long term.
4. Prioritize end user experience above developer experience.
### Decision Outcome
Server rendering is the best option when the web application requires "live" data, such as the personalized sites we often build at Nava, like claimant portals or case management systems. Server rendering requires more upfront effort on the infra side, but it enables teams to achieve a clearer separation of concerns, and write less application code in the long run. This can translate to web applications that work well for a large spectrum of device and network conditions.
If a project team is building a site that renders the same content for every user, they can change their application to utilize [Next.js's static HTML export functionality](https://nextjs.org/docs/advanced-features/static-html-export) (prerendering). This flexibility to do either server rendering or prerendering within the same React framework is one benefit to using Next.js.
### Pros and Cons of the Options
#### Server rendering
Pros
* Data fetching occurs on the server. The browser natively handles the page's loading state. This means less overall code to write, test, and maintain. An uncaught error on the server will be louder (in a good way) than an uncaught error on the client (which could result in a never ending spinner).
* Makes it easier to implement a clearer separation of concerns:
* [Middleware](https://nextjs.org/docs/advanced-features/middleware) provide a place for enforcing auth, reading/setting secure cookies, setting HTTP headers, and redirects.
* [Loaders](https://nextjs.org/docs/basic-features/data-fetching/get-server-side-props) provide a place for fetching all data required for rendering the page.
* Running page logic and rendering on the server makes it possible to send lighter payloads to the client. This approach can work well for a large spectrum of device and network conditions. [You can make your server fast, but you can't control the user's device or network](https://remix.run/docs/en/v1/pages/philosophy%23serverclient-model).
* Data fetching on the server enables accessing authenticated APIs (e.g. using [TLS mutual auth](https://www.cloudflare.com/learning/access-management/what-is-mutual-tls/) to talk to fetch data from a protected third-party API).
* Low effort to implement [dynamic routes](https://nextjs.org/docs/routing/dynamic-routes) (e.g `/claim/:claim_id`)
* [API routes](https://nextjs.org/docs/api-routes/introduction) can be created to handle other types of HTTP requests (POST, PUT, etc).
* Nice side benefit: Server rendering is the only option for [Remix](https://remix.run/). It may be easier, from an infra standpoint and as a conceptual model, to migrate to Remix if the Next.js apps we're building were server rendered.
Cons
* Requires infra resources to run the containerized application, and all the things that come along with a server (rate limiting, auto scaling), such as [AWS App Runner](https://aws.amazon.com/apprunner/). This can have higher costs than a prerendered site.
* Higher operational and compliance burden due to the above. Requires more effort to create documentation for security approvals due to a larger attack surface.
#### Prerendering
Pros
* Great for mostly static sites, when the markup can be generated ahead of time.
* Minimal infrastructure is required. The prerendered HTML files can be served from a CDN, such as AWS CloudFront connected to an S3 bucket. Assuming other best practices are followed, like optimizing images and not loading MB's of client-side JS, this can translate to fast page loads and low costs.
* Lighter operational and compliance requirements. Security approval documentation is simpler due to a smaller attack surface, and on-call responsibilities are reduced.
Cons
* For sites with live/personalized data, pages would require client-side JS for data fetching. This has a few downsides:
* Client-side JS is required for rendering the loading, success, and error states (e.g. `fetch`, `isLoading`, `useEffect`, `catch`). Teams need to define their own code patterns to manage this (e.g hooks, higher-order components) or install third-party dependencies (e.g. [React Query](https://react-query-v3.tanstack.com/)). This increases the amount of code to be written and maintained, and can increase code complexity. More code and complexity provides more opportunity for introducing bugs.
* The prerendered HTML file is only a skeleton page in a pending state. Although the site might have a fast [First Paint](https://developer.chrome.com/docs/lighthouse/performance/first-contentful-paint), its [Time To Interactive](https://developer.chrome.com/en/docs/lighthouse/performance/interactive/) may still be slow.
* [Lacks support for Middleware, Internationalized Routing, API Routes, etc](https://nextjs.org/docs/advanced-features/static-html-export%23unsupported-features).
### Links
*
*
* 🔒 [PFML comparison of current static approach vs a possible server rendering approach](https://drive.google.com/file/d/1Wgpl4q3ceJGKE5uLFH3iXUhefPxJdHcw/view)
Backends for Frontends:
*
*
Web rendering:
*
*
*
*
# Use NPM over Yarn Architectural Decision Records
* Deciders: @aligg, @sawyerh, @lorenyu
* Date: 2022-09
## Context and Problem Statement
Initially this template repo used yarn for package management. We moved to npm because:
* npm is pre-bundled with node, so using npm removes an installation step
* some projects work on government furnished equipment and an additional package installation (e.g. installing yarn) is a significant and time-consuming step
* npm and yarn are comparable in function for the purposes of this template
## Considered Options
We considered the merits of yarn and npm only when making this decision.
## Decision Outcome
Chose npm to reduce installations and bureaucratic hurdles for folks using this template out of the box.
## Links
* [Original github issue for reference](https://github.com/navapbc/template-application-nextjs/issues/11)
# U.S. Web Design System in React
* Status: Accepted
* Deciders: @sawyerh, @aligg, @lorenyu, @rocketnova
* Date: 2022-12-05
Technical Story: #19
## Context and Problem Statement
* The U.S. Web Design System (USWDS) only provides HTML and CSS for its components. It includes a small bit of vanilla JS to add interactivity to some components like the date picker.
* It's common for projects to write their own React components to output the USWDS HTML, to reduce the amount of boilerplate needed to use the USWDS components.
* [Previous research by Kalvin and Shannon](https://docs.google.com/document/d/1KRWzH_wJUPKkFmBlxj6SM2yN3W7Or89Wa4TBVM3Ksog/edit) discovered that Nava engineers and designers universally agreed that being able to use a React USWDS component library when starting new projects would be valuable.
## Considered Options
* Use the existing open source [`react-uswds` library](https://github.com/trussworks/react-uswds)
* Create our own React USWDS component library
* Leave the responsibility to each project team
## Decision Outcome
Add [`react-uswds`](https://github.com/trussworks/react-uswds) as a template dependency, making it available to all teams who use the template. The primary reasons are to avoid reinventing the wheel and because it's overall a well-built and maintained library.
## Pros and Cons of the Options
### Use the existing open source [`react-uswds` library](https://github.com/trussworks/react-uswds)
`react-uswds` is maintained by Truss, another vendor in this space. [A Storybook for it can be found here](https://trussworks.github.io/react-uswds/). Truss also maintains a [USWDS Figma library](https://www.figma.com/community/file/836611771720754351) for designers.
#### Pros
* Includes React components for all USWDS components and patterns.
* Fairly well maintained.
* Intentionally does not include any non-USWDS components.
* Supports USWDS v3 (latest version)
* This was the recommended approach coming out of [Kalvin and Shannon's research](https://docs.google.com/document/d/1KRWzH_wJUPKkFmBlxj6SM2yN3W7Or89Wa4TBVM3Ksog/edit).
#### Cons
* They [pin the `@uswds/uswds` dependency version](https://github.com/trussworks/react-uswds/blob/a0558b69ec5b99903cfa8edddf2d8b058f5e296c/package.json#L52) to a specific version, which means that a project cannot use a newer version of USWDS until `react-uswds` updates it on their end. In practice, this could mean that a project may have delayed access to new component styles or CSS bug fixes that USWDS releases.
* Not necessarily a con, but just to call it out: We've only done a lightweight review of their technical implementation and hygiene — there's testing and linting, no reported a11y issues are open in GitHub or reported in Storybook, but we haven't done a comprehensive review of their code or a full accessibility audit. We're operating on trust in Truss's technical expertise, and an assumption that the outputted HTML markup is close to identical to what USWDS provides, so any a11y issues would likely be on USWDS's end.
### Create our own React USWDS component library
Nava could create our own React USWDS component library, similar to `react-uswds`.
#### Pros
* We'd have full control over the technical approach and wouldn't have a dependency on another vendor to incorporate changes or release new versions.
#### Cons
* Requires more time and effort than using an existing library. We'd have to build and maintain the library.
* Reinventing the wheel. We can always fork `react-uswds` if it no longer meets our needs.
### Leave the responsibility to each project team
This is the current approach. Each project team is responsible for creating their own React components for the USWDS components they need.
#### Pros
* No additional work required from the Platform team.
#### Cons
* Each project team has to spend time and effort building the components or making technical decisions related to how they'll integrate USWDS. Teams then have to write their own tests and fix their own bugs for these components. Overall a potential poor use of time and effort.
## Links
* [Decision to use the USWDS](/product/decisions/adr/2023-07-20-fe-design-system)
* [Kalvin and Shannon's research](https://docs.google.com/document/d/1KRWzH_wJUPKkFmBlxj6SM2yN3W7Or89Wa4TBVM3Ksog/edit)
* [Evaluation of `react-uswds`](https://docs.google.com/document/d/1T3eG4oRofDE_NkfL7-xEqS39ORlrXlI8bFYcjGaYoWs/edit)
# Communications Tooling: Video Conferencing
* **Status:** Accepted
* **Last Modified:** 2023-07-24
* **Related Issue:** [#99](https://github.com/HHS/simpler-grants-gov/issues/99)
* **Deciders:** Lucas, Billy, Sarah, Sumi
* **Tags:** communucations, open source, video conferencing
## Context and Problem Statement
The project aims to implement a reliable video conference platform to facilitate seamless communication for both internal collaborations involving HHS and contractors, as well as external engagements with members of the public. The primary objective of this ADR is to evaluate and ultimately choose a suitable video conference tool that aligns with the project's requirements. The selected platform should enable efficient virtual meetings, ensuring clear and effective communication among project stakeholders, and accommodating diverse use cases, from internal team discussions to public outreach and engagement.
## Decision Drivers
#### Must Have
* **Public Access**: If they have the right link, members of the public can join a video call without creating an account Waiting Room: Meeting organizers can configure their meetings to require that attendees must be admitted before they can join the call
* **Video Recording**: Meeting organizers can record a video call from within the platform
* **Screen Sharing**: Attendees can share their screen (if given the appropriate permissions by meeting organizer)
* **Chat**: Users can post comments and questions in a chat that are visible to other attendees
* **Phone Support**: Users can join by phone if they don't have access to a computer for video
* **Live Transcription**: The platform supports live transcription for attendees that may need closed captioning
* **Authority to Operate**: The platform should be covered under the existing ATO for Grants.gov
#### Nice to Have
* **Webinar**: The platform supports a webinar format, i.e. attendees who can join and post questions but not see one another or unmute without permission
* **Breakout Rooms**: A meeting organizer can split users out into virtual "breakout rooms" for small-group discussions
* **Open Source**: The code to run this platform is open source and offers a self-hosting option
* **Attendance Tracking**: The platform allows host to access attendees to make follow up and attendance tracking easier
* **Community & Support**: The platform has a strong community for ongoing support, updates, and bug fixes
* **Scalability**: The platform should be able to hadndle a growing number of participants and meetings wihtout performance issues
## Options Considered
* [Zoom](https://zoom.us/)
* [Microsoft Teams](https://www.microsoft.com/en-us/microsoft-teams/group-chat-software)
* [Google Meet](https://meet.google.com/)
* [Jitsu](https://meet.jit.si/)
* [meet.coop](https://www.meet.coop/)
## Decision Outcome
Zoom stands out as the optimal choice for our video conference tool. It fulfills all the key decision drivers we have outlined, including public access, waiting room feature, video recording capability, screen sharing, and interactive chat. Moreover, Zoom provides phone support, live transcription for closed captioning, and is FedRAMP approved. Its robust features and ease of use make it a reliable platform for both internal and external meetings, ensuring a seamless and inclusive communication experience for all stakeholders involved.
### Positive Consequences
* **Improved Communication**: Video conferencing enables real-tme face-to-face communication, fostering better understanding, transparency, and collaboration among team members, stakeholders, and the general public.
* **Enhanced Collaboration**: With screen sharing and chat features, participants can work together on documents, brainstorm ideas, and address issues efficiently, enhancing overall teamwork.
* **Extended Reach**: The tool allows you to connect with participants from different locations, making it easier to work with remote teams, partners, and the public.
* **Flexibility and Convenience**: Participants can join meetings from their preferred location, whether it's their office, home, or while on the go, promoting flexibility and convenience.
* **Accessibility**: Video conferencing tools often offer closed captioning and accessibility features, making meetings more inclusive for individuals with impairments.
### Negative Consequences
* **Technical Challenges**: Technical issues, such as poor internet connectivity, audio/video glitches, and compatibility problems, can disrupt meetings and lead to frustration.
* **Security Concerns**: Video conferencing tools may pose security risks, like unauthorized access, data breaches, or potential privacy violations if not properly configured and managed.
* **Fatigue and Burnout**: Frequent video meetings can lead to "Zoom fatigue" or virtual meeting burnout, affecting participant engagement and focus.
* **Distractions and Multitasking**: Participants may be prone to distractions or multitasking during virtual meetings, reducing attention and active engagement.
* **Learning Curve**: New users may find it challenging to navigate and fully utilize the features of the video conference tool, leading to a learning curve for some team members.
* **Bandwidth Consumption**: Video conferencing consumes significant internet bandwidth, which may impact other online activities if the network is not robust enough. This could also be limiting for individuals with low or slow broadband connection.
### Back-up Options
Jitsi serves as a strong backup option for our video conferencing needs. Jitsi aligns with several essential decision drivers, such as public access, waiting room functionality, and screen sharing capabilities. Additionally, its open-source nature allows for self-hosting, offering an added level of control and privacy. Our project could contribute code to functionality that may fall short. Jitsi could be a reliable and accessible alternative for hosting our meetings with both internal and external stakeholders.
## Comparison Matrix
* ✅ Feature available, meets requirement
* ❌ Feature not available, does not meet requirement
* 🔄 Partial feature, limited feature availability, feature in progress or undergoing improvements
* 1-3 Strength level
* ❓Unknown
| Factor | Zoom | Microsoft Teams | Google Meet | Jitsu | Meet.coop |
| --------------------- | :--: | :-------------: | :---------: | :---: | :-------: |
| Public Access | ✅ | ✅ | ✅ | ✅ | ✅ |
| Waiting Room | ✅ | ✅ | ✅ | ✅ | ❓ |
| Video Recording | ✅ | ✅ | ✅ | ✅ | ✅ |
| Screen Sharing | ✅ | ✅ | ✅ | ✅ | ✅ |
| Chat | ✅ | ✅ | ✅ | ✅ | ✅ |
| Phone Support | ✅ | ✅ | ✅ | ✅ | ✅ |
| Live Transcription | ✅ | ✅ | ✅ | ❌ | ❓ |
| Authority to Operate | ❓ | ✅ | ❓ | ❓ | ❓ |
| Webinar\* | ✅ | ✅ | 3 | 🔄 | ❓ |
| Breakout Rooms\* | ✅ | ✅ | ✅ | 🔄 | ✅ |
| Open Source\* | ❌ | ❌ | ❌ | ✅ | ✅ |
| Attendance Tracking\* | ✅ | ✅ | 🔄 | ❓ | ❓ |
| Community & Support\* | 3 | 3 | 3 | 3 | 2 |
| Scalability\* | ✅ | ✅ | ✅ | ✅ | 🔄 |
\*Nice to have
## Pros and cons of each option
### Zoom
Zoom is a widely used video conferencing platform known for its ease of use and comprehensive features. It offers virtual meetings, webinars, breakout rooms, screen sharing, chat functionality, and phone support.
#### Additional details and pricing
**Pricing**:
* Free with limitations like 40 minutes per meeting, 100 attendees per meeting, whiteboard basic, team chat, mail & calendar
* Pricing starts at $149.90/year/user and goes up from there.
* The Enterprise plan offers rooms and webinars but no pricing is available on the site. We would need to contact Zoom for specific pricing for our project.
* We do not need each every user to have access to our paid instance since anyone can join a Zoom session with the invite link.
* We would need around 15-30 users with access to the paid instance.
#### Pros
* User-friendly interface and easy to set up for both organizers and participants.
* Robust features like breakout rooms, screen sharing, and virtual backgrounds enhance collaboration.
* Offers a wide range of integrations with other apps and platforms.
* Availability of a free plan with limited meeting duration.
#### Cons
* Security concerns in the past, but the platform has made significant improvements.
* Free plan has limitations on meeting duration and participant count.
* Frequent updates may require users to adapt to changes.
### Microsoft Teams
Microsoft Teams is a collaboration platform integrated with Microsoft 365. It offers video conferencing, chat, file sharing, and integration with other Microsoft applications. It is ideal for organizations already using Microsoft's ecosystem, fostering seamless collaboration among team members.
#### Additional details and pricing
**Pricing**:
* Prices starting at $4 user / month
* Webinar hosting is only available in the Business Standard package which is $12.50/user / month
#### Pros
* Part of the Microsoft 365 suite, providing seamless integration with Office apps.
* Allows for large meeting participants and live event hosting.
* Extensive collaboration features, including document sharing and real-time editing.
* Supports third-party app integrations for enhanced functionality.
* Strong security features, making it suitable for enterprises.
#### Cons
* Initial setup and learning curve can be overwhelming for new users.
* Resource-intensive, leading to occasional performance issues on lower-end devices.
* Limited to Microsoft 365 users, which may restrict external collaboration.
* Frequent updates can lead to occasional disruptions.
* Potentially paying a premium for the Microsoft 365 suite when the video conferencing is all that is needed for the project.
### Google Meet
Google Meet is part of Google Workspace (formerly G Suite) and is well-suited for Google users. It provides straightforward video conferencing with high-quality audio and video. While it may lack some advanced features, it offers a user-friendly experience for those already using Google's tools.
#### Additional details and pricing
**Pricing**:
* Google Workspace pricing is for the whole suite, starting at $6 per user/ month. The Enterprise has more advanced options and features.
* We do not need each every user to have access to our paid instance since anyone can join a Google Meet session with the invite link.
* We would need around 15-30 users with access to the paid instance.
* **Live Transcription**: Yes but in English only
#### Pros
* Seamless integration with Google Workspace (formerly G Suite) apps for streamlined collaboration.
* High-quality video and audio, even in low-bandwidth conditions.
* No additional software installation required for users with Google accounts. Users can access simply through a browser.
* Good performance on various devices, including smartphones and tablets.
* Provides live captions for improved accessibility.
#### Cons
* Advanced features are limited compared to some other platforms.
* Some features may require a paid G Suite subscription.
* Limited customization options for meeting settings.
* Live transcription is in English only.
* Potentially paying a premium for the suite of Google products when the video conferencing is all that is needed for the project.
### Jitsu
Jitsi is an open-source video conferencing solution that stands out for its simplicity and self-hosting capabilities. It supports public access, open meetings, and offers easy setup.
#### Additional details and pricing
* **Pricing**: Free
* **Webinar**: Not a built-in feature (third-party integrations available)
* **Breakout Rooms**: Not a built-in feature (third-party integrations available)
* **Open Source**: Yes
* **Attendance Tracking**: Unknown if this exists. The team could potentially contribute and build this since Jitsu is open source.
#### Pros
* Open-source platform with a self-hosting option, providing greater control and privacy.
* Simple and lightweight, making it quick to set up and easy to use.
* No account creation required for participants, enhancing public access.
* Strong focus on security and privacy, allowing for end-to-end encryption.
* Availability of web-based access without software downloads.
#### Cons
* Fewer advanced features compared to more established platforms.
* Limited support and community compared to larger video conferencing tools.
* Self-hosting may require technical expertise and maintenance efforts.
* Video and audio quality may vary depending on server capacity and network conditions.
### Meet.coop
Meet.coop is a European based solution that provides online meeting and conferencing tools, powered by renewable energy, running on cooperatively owned infrastructure. Meet.coop emphasizes cooperative values, democratic governance, and shared decision-making. The platform's cooperative principles guide its development and operations. The community-driven project encourages open collaboration, knowledge sharing, and the creation of decentralized architecture
#### Additional details and pricing
**Pricing**:
* Prices starting at $12 / month for 10 participants and goes up to $115 / monthe for more users.
#### Pros
* Prides themselves as an ethical, sustainable, and community-driven solution which aligns with Simpler Grants values
* Data ownership is with the users
* Fosters a strong sense of community engagement and collaboration amongst its users, providing a platform for shared learning and support
#### Cons
* Users may not be as familiar with the Meet.coop and how to use the tool, however the tool does seem user-friendly
* Features and capabilities are not as mature as the alternatives
* Meet.coop may lack immediate and direct support options that come with commercial platforms
* There may be some risks in scalability to accommodate a larger number of users and meetings.
## Links
* [Zoom pricing](https://zoom.us/pricing)
* [Google Meet pricing](https://workspace.google.com/pricing.html)
* [Microsoft Teams pricing](https://www.microsoft.com/en-us/microsoft-teams/compare-microsoft-teams-options)
* [Jitsu](https://jitsi.org/jitsi-meet/)
* [Meet.coop](https://www.meet.coop/about/)
* [Meet.coop forum](https://forum.meet.coop/)
# Back-end Production Server
* **Status:** Accepted
* **Last Modified:** 2023-07-2i
* **Related Issue:** [#245](https://github.com/HHS/simpler-grants-gov/issues/245)
* **Deciders:** Daphne, Sammy, Billy, Lucas
* **Tags:** ADR
## Context and Problem Statement
The Flask development server is not meant for production use and only intended for local development. It is not secure, stable, efficient, or scaled for a production environment. In addition to choosing a production server, this ADR will specify a high level implementation option.
## Decision Drivers
* **Scalable:** The chosen solution should be configurable to scale and a multi-worker, multi-threaded production-ready, WSGI wrapper.
* **Ease of use:** The production server should be relatively simple to set up and start.
* **Well-maintained:** We have a preference towards a production server that is widely adopted and have active maintainers.
## Options Considered
### Production Server
* Gunicorn
* Waitress
### Implementation
1. API entrypoint responsible for conditional logic determining dev vs prod environment and starting corresponding server
2. Dockerfile executable command for the prod server is overridden in the IaC [task definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html), API by default starts dev server
3. Dockerfile executable command for the dev server is overridden in `docker-compose.yml`, API by default starts prod server
Note: Gunicorn can be set up either using their unique [configuration file](https://docs.gunicorn.org/en/latest/configure.html) or in our code using separate app entry points for dev and prod. We want to make sure we are scaling the appropriate number of workers based on CPU.
## Decision Outcome
### Production Server
Chosen option: Gunicorn, because it is the industry standard, well-supported and documented.
### Implementation
Chosen option: #3 Dockerfile executable command for the dev server is overridden in `docker-compose.yml`, API by default starts prod server. This is because it makes the most sense in our current development ecosystem and abstracts away the concept of environment in the API layer.
## Pros and Cons of the Options
### Production Server
#### [Gunicorn](https://gunicorn.org/)
* **Pros**
* Widely used, industry standard Python server
* Excellent ability to manage workers
* Simple and light on resources, written in C
* Highly compatible with most Python tooling
* **Cons**
* Does not run on Windows without WSL 🧐
#### [Waitress](https://github.com/Pylons/waitress)
* **Pros**
* Simple, lightweight
* Can run on Windows as well as UNIX systems
* No dependencies that aren't part of the standard Python library
* Purely Python
* **Cons**
* Runs on CPython and has "very acceptable performance"
### Implementation
#### #1 API entrypoint responsible for conditional logic determining dev vs prod environment and starting corresponding server
* **Pros**
* This is how the Flask app is already configured (small lift to modify)
* **Cons**
* Poor separation of concerns: apps remaining environment agnostic keeps them much simpler overall
* Can lead to confusing environment conditional logic
#### #2 Dockerfile executable command for the prod server is overridden in the IaC [task definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html), API by default starts dev server
* **Pros**
* Terraform handles our infrastructure and deployments, therefore conceptually makes sense to define Docker run commands in the task definition
* **Cons**
* Ignores `docker-compose.yml` for local development as a tool we have at our disposal
* Obscures prod server run command outside app ecosystem in IaC
* Implementation could potentially use separate app entry points for dev and prod
#### #3 Dockerfile executable command for the dev server is overridden in `docker-compose.yml`, API by default starts prod server
* **Pros**
* Local development is done via `docker-compose.yml` config, so it makes a ton of conceptual sense to pass a local Docker run command here
* **Cons**
* Implementation could potentially use separate app entry points for dev and prod
# Communications Tooling: Analytics Platform
* **Status:** Accepted
* **Last Modified:** 2023-08-01
* **Related Issue:** [#323](https://github.com/HHS/simpler-grants-gov/issues/323)
* **Deciders:** Lucas, Aaron, Billy, Sarah, Sumi
* **Tags:** communucations, open source, analytics
## Context and Problem Statement
The communications platform deliverable identifies a series of platforms through which the Grants API project needs to engage both internal and external stakeholders. One of these platforms is an analytics platform for tracking key metrics during the launch of the first version of simpler.grants.gov and as the project continues to grow. We will evaluate top 3-5 analytics tools, including analytics.gov and Google Analytics, to identify the best fit for our specific needs and objectives. The selected tool should provide valuable insights and data to help us measure and optimize the platform's performance.
## Decision Drivers
#### Must Have
* **Data tracking capabilities**: solution should be able to track the essential metrics and events relevant to the project's goals, such as page views, user interactions, and other custom events.
* **Data visualization and reporting**: The tool should offer clear and comprehensive data visualization and reporting features to present data in a way that is easy to understand and interpret for multiple audiences (public, internal HHS, etc.)
* **Integration with existing systems**: consider whether the analytics tool can seamlessly integrate with the project's existing infrastructure, and other tools used in the development process. We will have a content management system in the future and we should have forward thinking to ensure it could be compatible with various CMS options.
* **Real-time data processing**: Assess whether the analytics tool can provide real-time data processing capabilities, enabling timely responses to critical events.
* **Data privacy and security**: Ensure that the selected tool complies with data privacy regulations and provides robust security measures to protect sensitive information.
* **Scalability**: Consider whether the analytics tool can handle the expected growth in data volume and user traffic as the project expands.
* **Cost**: Evaluate the pricing structure of the analytics tool.
* **Customization and flexibility**: Determine the level of customization and flexibility offered by the tool to tailor it to the specific needs and requirements of the project.
* **Support and documentation**: Check the availability of technical support, documentation, and community forums to aid in implementation and troubleshooting.
* **User-friendly interface**: Consider the ease of use and intuitiveness of the tool's interface, enabling team members to efficiently navigate and extract insights.
#### Nice to Have
* **Open Source:** The tool used to manage and host the analytics should be open source, if possible.
## Options Considered
* [Analytics.usa.gov](https://analytics.usa.gov/)
* [Google Analytics](https://analytics.google.com/analytics/web/provision/#/provision)
* [Mixpanel](https://mixpanel.com/)
* [Adobe Analytics](https://business.adobe.com/products/analytics/adobe-analytics.html)
* [Matomo](https://matomo.org/)
## Decision Outcome
We recommend moving forward with analytics.usa.gov through the Digital Analytics Program for the public facing analytics. It is required for all public-facing government agency websites. The platform aligns well with our goals due to its robust data tracking capabilities, allowing us to monitor essential metrics and custom events critical to our project's success. Its comprehensive data visualization and reporting features ensure that data can be presented in an easily understandable format for various audiences, including the general public and internal stakeholders at HHS. Since it is run by the Digital Analytics Program, we have greater confidence in the data and security standards. It is free for government agencies. Additionally, it allows us to align with other government agencies and ensure transparency to public.
DAP cannot be used for authenticated pages. We recommend Google Analytics with any logged in or authenticated pages. Since DAP is powered by Google Analytics, it makes sense to have the same Analytics platform on those pages as well. An assumption is that we will be able to connect the user sessions between authenticated and the public site together. However, if it is not possible to stitch the user journey (unauthenticated to authenticated and back and forth) together, it is worth revisiting the analytics tool selection.
### Positive Consequences
* **Data-Driven Decision Making**: With an analytics platform, our project can gather valuable data and insights about usage and identify where there are shortfalls to our product. It can allow us to ensure we are building an accessible solution for the public. This data-driven approach enables informed decision-making and allows teams to identify trends, opportunities, and areas for improvement.
* **Improved Performance**: Analytics platforms help track key performance indicators (KPIs) and measure the success of various initiatives and deliverables. By monitoring these metrics, we can optimize our strategies and improve overall performance.
* **Enhanced User Experience**: Understanding user behavior through analytics helps tailor products, services, and content to meet users' needs and preferences leading to a better user experience.
### Negative Consequences
* **Data Privacy Concerns**: Collecting and analyzing user data can raise privacy concerns, especially with the increasing focus on data protection. We must handle data responsibly and set a high standard for compliance.
* **Data Overload**: Too much data without proper analysis can overwhelm teams and lead to decision paralysis. It's important to focus on relevant metrics and insights to continue moving forward.
* **Misinterpretation of Data**: Misinterpreting or misrepresenting data can lead to incorrect conclusions and misguided strategies. Proper data analysis and understanding are essential to draw accurate insights.
* **Cost and Resource Allocation**: Implementing and maintaining an analytics platform can be costly and resource-intensive that goes beyond just the cost of the tool.
* **Learning Curve**: Adopting new analytics platforms may require training and time for team members to become proficient, potentially affecting productivity during the initial stages.
## Comparison Matrix
* ✅ Feature available, meets requirement
* ❌ Feature not available, does not meet requirement
* 🔄 Partial feature, limited feature availability, feature in progress or undergoing improvements
* 1-4 Strength level
* ❓Unknown
| Factor | Analytics.usa.gov | Google Analytics | Mixpanel | Adobe Analytics | Matomo |
| --------------------------------- | :---------------: | :--------------: | :------: | :-------------: | :----: |
| Data tracking capabilities | ✅ | 3 | 3 | 3 | 3 |
| Data visualization and reporting | 1 | 3 | 2 | 3 | 1 |
| Integration with existing systems | ❓ | 3 | 2 | 1 | 2 |
| Real-time data processing | ✅ | 🔄 | ✅ | 🔄 | 🔄 |
| Data privacy and security | 3 | 3 | 3 | 3 | 4 |
| Scalability | 2 | 4 | 3 | 3 | 3 |
| Cost | Free (gov use) | Free + Premium | $ | $$ | Free |
| Customization and flexibility | 1 | 4 | 3 | 3 | 4 |
| Support and documentation | 2 | 4 | 3 | 4 | 3 |
| User-friendly interface | ✅ | ✅ | ✅ | ✅ | 🔄 |
| Authority to Operate | ✅ | ❓ | ❓ | ❓ | ❓ |
| Open Source\* | ❌ | ❌ | ❌ | ❌ | ✅ |
\*Nice to have
## Pros and Cons of the Options
### Analytics.usa.gov
The Digital Analytics Program (DAP) offers a web analytics tool, training, and support to federal agencies. The program is a shared service provided by the Technology Transformation Services (TTS) at the U.S. General Services Administration (GSA).
DAP provides federal agencies with:
* Free web analytics tools for public-facing federal websites that are comprehensive and easy-to-use
* Scalable infrastructure for measuring a broad range of .gov sites (large and small)
* Training on analytics tools and reporting
* View upcoming trainings and talks »
* View past recorded trainings »
* Ongoing help-desk support around implementation, data, and reporting
#### Details
* The public reporting page, analytics.usa.gov, is powered from the Google Analytics account that DAP manages. Currently, the DAP code snippet is implemented at grants.gov.
* Currently powered by Universal Analytics (UA), but we will have access to both UA and GA4. On July 1, 2024, the Digital Analytics Program (DAP) will replace Universal Analytics (UA) with Google Analytics 4 (GA4).
* DAP is required: On November 8, 2016, the Office of Management and Budget (OMB) released a memorandum on [Policies for Federal Agency Public Websites and Digital Services](https://www.whitehouse.gov/wp-content/uploads/legacy_drupal_files/omb/memoranda/2017/m-17-06.pdf) (PDF, 1.2 MB, 18 pages), which requires executive branch federal agencies to implement the DAP JavaScript code on all public facing federal websites.
* Details on the code are available at the [DAP Github Repo](https://github.com/digital-analytics-program/gov-wide-code). Under the Code Capabilities Summary there are details on the types of data that are collected.
* The DAP script should only be applied to public-facing pages. Public-facing web pages are defined as those that can be accessed without any authentication or login, and are not part of an otherwise “privileged session.”
* The DAP script tag should not be placed on pages visited during logged-in sessions. Notably, other seemingly “public” pages that can be accessed without authentication may also be part of privileged sessions; for example, a password reset page that is accessed by clicking a link in an email is not appropriate for DAP code because it assumes the visitor has the privilege of control over the email account used to provide the link.
#### Pros
* DAP is free for government agencies
* DAP provides insights across government agencies. It delivers an unprecedented, government-wide view of how the public interacts with federal websites.
* Follows the recommended standards by government
* Promotes transparency by sharing data on how citizens interact with various federal government websites. This transparency can foster accountability and open up insights into user behavior and government services usage.
* By providing real-time data and analytics, government agencies can make more informed decisions about website design, content, and functionality. This can lead to improved user experiences and more effective government services.
* The insights derived from Analytics.usa.gov can contribute to the optimization of government websites and services especially by our users and community as they will have a direct view, resulting in a more efficient and effective delivery of information and services to the public.
#### Cons
* DAP can only be on public facing systems, no logged in states should have DAP
### Google Analytics
Google Analytics is a widely used web and app analytics platform that provides in-depth insights into user behavior, traffic sources, and more. It offers a user-friendly interface, robust tracking capabilities, and integration with other Google services. Customization options, like creating custom reports and goals, allow tailored analysis.
#### Pros
* Robust tracking offering comprehensive website and app data providing insights into user behavior, traffic sources, and more.
* It integrates seamlessly with other Google services and tools, making it easy to connect and analyze data across platforms.
* It allows for advanced customization, including creating custom reports, segments, and goals.
#### Cons
* The collection of user data by Google Analytics raises privacy concerns, especially with stricter data protection regulations.
* Using Google Analytics means sharing data with Google, which can raise questions about data ownership and control.
* While Google Analytics offers real-time data, it's not as robust as some other real-time analytics tools.
* Will need to get ATO or FedRAMP approval
### Mixpanel
Mixpanel is a user-centric analytics tool primarily focused on app analytics. It excels in tracking specific events and user flows, providing valuable insights into user engagement. Its funnel analysis helps pinpoint where users drop off in processes. While it's strong for app analytics, its web tracking capabilities might be less comprehensive.
#### Pros
* Mixpanel focuses on user behavior, allowing for deep insights into how individual users interact with your app or website.
* It excels in tracking specific events, actions, and user flows, providing valuable insights for user engagement.
* Mixpanel offers powerful funnel analysis, helping you understand where users drop off in a specific process.
* Real-time tracking and reporting enable you to monitor user behavior as it happens.
#### Cons
* The extensive features can make Mixpanel more complex to set up and use, requiring a learning curve.
* While Mixpanel is strong for app analytics, its web tracking capabilities might not be as comprehensive as other platforms.
* Mixpanel's pricing can be higher compared to some other analytics tools, particularly as usage scales.
* Will need to get ATO or FedRAMP approval
### Adobe Analytics
Adobe Analytics is an enterprise-grade solution suitable for large organizations. It integrates well with other Adobe products and offers extensive customization options, allowing for tailor-made reports and dashboards.
#### Pros
* Adobe Analytics is a powerful enterprise-grade solution suitable for large organizations with complex analytics needs.
* It integrates well with other Adobe products, creating a seamless experience for marketing and data analysis.
* Adobe Analytics allows extensive customization and flexibility in creating reports, segments, and dashboards.
* Real-time data reporting is available, allowing for immediate insights into user behavior.
#### Cons
* Adobe Analytics can be complex to set up and manage, requiring expertise and dedicated resources.
* The enterprise-level features are more costly.
* Due to its advanced capabilities, Adobe Analytics might have a steeper learning curve for new users and the general public.
* Will need to get ATO or FedRAMP approval
### Matomo
Matomo, an open-source alternative, offers data ownership and control, making it attractive for privacy-conscious organizations. As self-hosted software, it can be customized and audited, addressing data privacy concerns.
#### Pros
* Matomo allows you to retain complete ownership and control of your data, addressing privacy concerns.
* As open-source software, Matomo's code can be audited, customized, and self-hosted.
* Matomo provides tools to help with GDPR compliance, making it suitable for businesses in regions with strict data protection laws.
* Matomo offers customization and flexibility similar to other analytics tools, allowing you to tailor reports and dashboards.
#### Cons
* There are downsides and costs to self-hosting
* The support ecosystem might not be as extensive as that of larger, commercial analytics platforms.
* Will need to get ATO or FedRAMP approval
## Links
* [Analytics.usa.gov](https://analytics.usa.gov/)
* [DAP](https://digital.gov/)
* [Google Analytics](https://analytics.google.com/analytics/web/provision/#/provision)
* [Adobe Analytics](https://business.adobe.com/products/analytics/adobe-analytics.html)
* [Matomo](https://matomo.org/)
# Commit and Branch Conventions and Release Workflow
* **Status:** Accepted.
* **Last Modified:** 2023-08-21
* **Related Issue:** [#185](https://github.com/HHS/simpler-grants-gov/issues/185)
* **Deciders:** Daphne, Sammy, Billy, Lucas
* **Tags:** process, workflow, change management
## Context and Problem Statement
This project needs standards for describing changes and introducing them into lower and production environments.
The items that encompass this are:
* Git commit conventions
* Branch naming conventions
* Branch merge strategy
* Branching model
* Release workflow and naming convention
## Decision Drivers
The standards should be:
* documented
* easy-to-use and understand
* support agile processes
* efficient and not create uneccessary work or friction
* faciliate deployments to lower and production environments
* facilitate continous integration and delivery
* make it easy to revert features
* facilitate both internal and external contributions
## Options Considered
### Branch merge strategy
#### [squash and merge](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges#squash-and-merge-your-commits)
* **Pros**
* A single commit for a PR makes it easier to see all changes introduced for an entire feature
* Squashing commits lowers the cognitive load for developers writing commits to a branch
* Github has a "squash and merge" feature which makes it easy to implement this strategy
* **Cons**
* [atomic commits](https://github.blog/2022-06-30-write-better-commits-build-better-projects/#the-solution) are seen by many as important method for creating a clear and revertable history
* Squashing, even in a small feature branch, means that each commit in the history does not have a minimal scope
* Reverting changes is harder because individual changes that comprise a feature cannot easily be pulled out in the history
#### [rebase and merging commits](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges#rebase-and-merge-your-commits)
The pros and cons for this strategy are described in reverse above. There is a higher congitive load for developers as each commit needs to be properly formatted, descriptive, and ideally atomic. However the history is more traversable if individual and tightly scoped commits are maintained.
### Git commit conventions
This encompasses the commit messages pushed to branches. This is tied to the decision to the branch merge strategy and assumes the use of the "squash and merge" strategy and github feature. The following options were considered:
#### No commit convention
* **Pros**
* Developers don't need to worry about the format of a commit while working on features
* Lower overhead for each commit for some contributors, promoting more frequent commits during development
* Less to read through when composing the "squash and merge" commit message if commits are shorter
* No tools or learning curve necessary
* Encourages necessary context be given in code comments or documentation instead of commit history
* **Cons**
* Individual messages in pull requests are harder to read as every developer might use a different format
* It is easier to accidentally include poorly worded or formatted messages using the "squash and merge" tool
* Developers need to consider how to word messages with each commit, instead of relying on a convention that describes how changes should be formatted
#### Conventional Commits
[Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) is a specification for adding human and machine readable meaning to commit messages.
* **Pros**
* Widely adopted
* Extreme precision in messages, reduced ambiguity
* Automatically generated CHANGELOGs
* Makes it easier for people to contribute to your projects, by allowing them to explore a more structured commit history
* CLI tools to facilitate adoption
* **Cons**
* Learning curve for adoption
* Less usfeul in a "squash and merge" strategy
* Not used by supporting organization
* While widely adopted, not universally embraced
#### "7 Rules" Convention
[The seven rules of a great Git commit message](https://cbea.ms/git-commit/#seven-rules) describes 7 widely adopted rules that are less presciptive than conventional commits, but still describe best practices.
* **Pros**
* Widely adopted
* Used by Github UI when editing or adding a file
* CLI tools to facilitate adoption
* Less prescriptive than conventional commits
* **Cons**
* Learning curve for adoption
* Less usfeul in a "squash and merge" strategy
* Not used by supporting organization
* While widely adopted, not universally embraced
### Branch naming conventions
This project will use `[github-username]/issue-[issue-number]-[feature-name]` convention for branch naming conventions.
### Branching model
#### Trunk Based Development
[Trunk Based Development](https://trunkbaseddevelopment.com/) is source-control branching model, where developers collaborate on code in a single branch called `main`.
* **Pros**
* Widely adopted in general and preferred by the supporting engineering organization
* Facilitates continuos integration
* Facilitates continuous delivery
* Facilitates lean experiments
* Simpler and more efficient than some other models
* **Cons**
* Doesn't facilitate releasing multiple versions of software, which is necessary in some cases (but not envisioned on this project)
#### Gitflow
[Gitflow](https://nvie.com/posts/a-successful-git-branching-model/) uses feature branches and multiple primary branches. [Some argue](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow) it has fallen in popularity in favor of trunk based development, and was not seriously considered for this project.
### Release workflow and Code Reviews
The following conventions will be adopted:
* Pull Requests (PRs) will be merged directly into `main`.
* PRs will be titled `[Issue N] Short description`
* PRs with no issue should use `[Fix] Short description`
* PRs will follow, with small changes, the [Code Change Lifecycle / Contributing ](https://docs.google.com/document/d/1EyLYuKCv8xjpY26zR8cODl6SEXy9Mx_yAuZ6vS8cPp8/edit?skip_itp2_check=true\&pli=1)guidelines and be documented in `CONTRIBUTING.md`
* Code reviews will follow, with small changes, the [Code Review Guidelines](https://docs.google.com/document/d/1pRtpg1ffLXcJn_yV_g90t1TlZjs7TLH2FfpS8DwW3_w/edit#heading=h.htn78a1hqoq)
### Release Workflow
* Changes `main` will be immediately deployed to a lower environment
* Releases will be made frequently and include a changelog of updates
* Releases will use [CalVer](https://calver.org/) versioning naming conventions
## Decision Outcome
This project will use:
* a trunk based development strategy with calendar versioned releases
* git commit conventions will not be enforced individual PR commits
* "squash and merge" stratey for merging PRs with a defined naming convention
* contribution and development practices will be documented in `CONTRIBUTING.md`, `READM.md`, and other documents in the repository
# Cloud Platform to Host the Project
* **Status:** Accepted
* **Last Modified:** 2023-07-14
* **Related Issue:** [#93](https://github.com/HHS/grants-api/issues/93)
* **Deciders:** Lucas, Aaron, Billy
* **Tags:** Hosting, Infrastructure, Security
## Context and Problem Statement
The project needs a hosting provider in order to operate. The hosting provider should have a competitive suite of tools, a proven history of working with Health and Human Services (HHS), and provide necessary security controls.
## Decision Drivers
* **Past Performance:** The provider should have a proven track record of providing services for HHS.
* **Security:** The provider should be [Fedramp Authorizedi](https://marketplace.fedramp.gov/products) as a Platform as a Service Provider.
* **Tools**: The service provider should offer a competitive suite of tools that can be used to host the project.
## Options Considered
* Amazon Web Services
* Gov Cloud
* Google Cloud Platform
## Decision Outcome
HHS has selected Amazon Web Services to provide hosting services for the project due to:
* HHS' existing relationship with AWS,
* the existing grants.gov infrastructure is on AWS,
* migrating data and systems is more diffictult between multiple service providers,
* AWS is Fedramp approved,
* AWS offers a competitive set of tools, and
* the current engineering team and supporting organization have years of experience on AWS.
The cumulative cost and risk associated with moving away from AWS and the fact that the other service providers do not offer markedly superior features or service models meant that a feature by feature comparison was not necessary for this decision.
### Positive Consequences
* AWS is well documented and supported, and is an industry standard, which will help the development team to perform efficiently.
* There are a large number of developers that are familiar with AWS, which will make it easier to get support and bring on more team member in the future.
* Migrating data is easier to configure, manage, secure, and is less costly than between multiple service providers.
* Simpler Grants team and existing Grants.gov teams will use the same cloud service provider.
### Negative Consequences
* The AWS tools and infrastructure are not open source, which has negative consequences like vendor lock-in and the inability for reuse and repurposing of publicly funded tools.
* Most of the hosting tools are AWS specific, which makes it more difficult for the project to move hosts in the future if desired and for the general public to use the open source contributions as the infrastructure code will be vendor-specific.
# Infrastructure as Code Tool
* **Status:** Accepted
* **Last Modified:** 2023-07-14
* **Related Issue:** [#93](https://github.com/HHS/grants-api/issues/93)
* **Deciders:** Lucas, Aaron, Billy
* **Tags:** Hosting, Infrastructure
## Context and Problem Statement
Infrastructure as Code (IaC) is the process of using code to manage hosting infrastructure. IaC is desirable because it produces more efficient, reproducable, readable, and interoperable deployment workflows.
IaC tools often have overlapping feature-sets and strategies, however they can be broadly characterized by provisioning versus configuration, mutable versus immutable, and procedural versus declarative. A tool or set of tools should should be selected for the project in order to best facilitate change management.
## Decision Drivers
* **Interoperability:** The tools should support multiple hosting providers.
* **Declarative:** The tools should support a declarative syntax.
* **Immutability:** The tools should support an immutable approach to infrastructure management.
* **Support and Documentation:** The tools should have excellent documentation and example use-cases.
* **Adoption:** The tools should have broad adoption to make it easier for the public to utilize the project code, get assistance from outside resources if necessary, and potentially grow the team.
## Options Considered
* Terraform with Docker
* CloudFormation with Docker
* Cloud Development Kit with Docker
* Chef with Ansible
## Decision Outcome
The project will use Terraform with Docker for provisioning infrastructure and creating and configuring images.
### Positive Consequences
* The provisioning and configuration of infrastracture supporting the project will be captured in code using widely adopted, open source, and well-documented tools.
* Deployments can be tested in lower environments using the same configuration as higher environments.
* IaC supports continuous and automated deployment for lower environments and production.
* History of changes to infratstructure is stored in version control which provides better auditability, an easy-to-read record of changes, and an easier path to reversion of changes.
* Infrastructure updates are faster and more reliable.
### Negative Consequences
* Infrastrcture takes longer to deploy initially.
## Pros and Cons of the Options
### Terraform with Docker
* **Pros**
* Terraform is interoperable, widely adopted (with [37K stars](https://github.com/hashicorp/terraform/stargazers) as example metric), has excellent documentation, and supports declarative and immutable strategies.
* Docker images offer a widely adopted and developer-friendly mechanism that captures most aspects of configuration management.
* Terraform with Docker best aligns with the each of the decision drivers.
* The engineering team has Terraform templates and modules that can be used on the project, experience using Terraform, and institutional support for the tool.
* **Cons**
* Terraform has changed its license from the [Mozilla Public License (v2.0) to the Business Source License (v1.1)](https://github.com/hashicorp/terraform/commit/b145fbcaadf0fa7d0e7040eac641d9aef2a26433) which could affect the community support for and adoption of the tool.
### CloudFormation with Docker
* **Pros**
* CloudFormation is created and supported by AWS.
* Tool is well-documented with many templates for projects and user interface tools.
* Tool is free with AWS account.
* **Cons**
* Tool is closed-source, procedural, and not interoperable.
* Tool is not modular which makes it harder to share recipes or configurations for similar tools or features.
* Testing is not well supported.
* Poor perceived developer experience.
### Cloud Development Kit with Docker
* **Pros**
* Cloud Development Kit is created and supported by AWS.
* Tool supports many of the decision drivers.
* Allows developers to use programming languages used in the API and front-end to manage cloud infrastructure.
* **Cons**
* Tool is not open source.
* Tool is not interoperable.
* Tool is relatively new and not widely adopted.
* Engineers on the team and supporting instutions have low level of experience with the tool.
### Chef with Ansible
* **Pros**
* Chef and Ansible are open source, interoperable, widely adopted, and well-documented.
* **Cons**
* Tools are procedural, which can lead to unexpected outcomes and fragile deployments.
* Docker captures most aspects of configuration management in a developer-friendly and immutable format.
# Data Replication Strategy & Tool
* **Status:** Active
* **Last Modified:** 2023-09-21
* **Related Issue:** [#322](https://github.com/HHS/simpler-grants-gov/issues/322)
* **Deciders:** Lucas, Billy, Sammy, Daphne, Aaron, Curtis, Lorenzo, Marwanl, Brandon
* **Tags:** Hosting, Infrastructure, Database
## Context and Problem Statement
The Simpler.Grants.Gov platform will need to consume live grants data securely and without impacting grants.gov performance. However, the production database was not planned to support additional load to the database from the beta api. The beta work will also want to test schema changes to the database to facilitate new queries and lifecycle tracking that will not be possible in the production database.
Additionally, the grants.gov database resides in another AWS account, which complicates access and security concerns. Any solution in production will need to comply with HHS security policies, be included on a security impact assessment (SIA), use tools and controls in or added to the System Security Plan (SSP), and not jeopardize the security of the environment with Approval to Operate (ATO).
## Decision Drivers
* Data source and destination compatibility: rep tool should support the data sources used in the project (db, file systems) and is compatible with the target destination (db, warehouses, cloud storage).
* Data volume and throughput: tool can handle the volume and throughput requirements of the data replication process efficiently.
* Data transformation capabilities: replication tool can handle data transformation during the replication process, including data format conversions and schema changes.
* Real-time vs. batch replication: determine whether real-time data replication or if batch replication at scheduled intervals is sufficient.
* Latency and performance: consider the latency and performance to ensure timely data updates and minimal impact on system performance.
* Security and encryption: replication tool provides adequate security features, including data encryption and secure data transmission.
* Monitoring and alerting: to promptly identify and address replication issues.
* Ease of use and configuration: Evaluate the tool's user-friendliness and ease of configuration, as complex setup processes can lead to inefficiencies.
* Scalability: Determine if the replication tool can scale to accommodate future growth and increased data demands.
* Cost: Consider the licensing and operational costs
* Support and community: Assess availability of support options and the size and activity of the tool's user community.
## Options Considered
### Data Replication
* Use Production Database
* Use AWS DMS (Database Migration Service)
* Create new data pipelines from data sources
* Import/Export DB snapshots weekly
### Data Traffic
* AWS VPC Pairing
* AWS PrivateLink
* Network Gateway with VPN
## Decision Outcome - Data Replication
Chosen option: Use AWS DMS, because it is the only option that allows us to deliver within our period of performance and doesn't impact the production database's ability to perform its existing role.
Additionally, [AWS DMS and AWS VPC Pairing are FedRAMP compliant](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/).
### Positive Consequences
* This solution will allow us to not only replicate the data, but transform it as well. This will allow us to pilot schema changes very quickly without having to spend the time creating new data pipelines from the original data sources
* This approach allows us to only replicate what we need when we need it, reducing the cost of replication, and limiting our security exposure.
* If we implement DMS with the intention of adding additional tables, or even replicating the entire database, this will be an agile tool to support us until we're able to deprecate the Oracle database.
* Data is extracted and replicated using a replica database as a source, rather than direct impact to the origin DB (production, test, etc.).
* Security is confined to East-2 rather than allowing connections to the primary site.
* Configurations are mostly made only against the replica database (some such as a new replication user will still be needed at the origin DB).
* Performance impacts of the solution replication are limited to the replica database.
### Negative Consequences
* When we want to eventually move away from the expensive Oracle database and it's unoptimized schema, this replica will need to be deprecated as well
* Connection issues between VPC will cause archived redo logs to backlog on the replica source until connections are re-established (could theoretically cause storage issues if it persists long enough).
* Replication monitoring is needed to ensure no backlogs, performance hits, etc.
* Specific configurations to the replica may need to be made as new objects are added to the replication (new tables, etc). DDL in some instances normally does not transfer via CDC replication without manual intervention.
* Additional maintenance times for all DB engine upgrades are needed to patch and update the replica.
## Decision Outcome - Data Traffic
Chosen option: Use AWS VPC Peering, because it is the most secure option for permitting traffic between two VPCs in AWS and is the AWS recommended tool for Multi-VPC DMS database replication. Aside from being FedRAMP compliant, and ensuring all traffic between VPCs is encrypted, it also keeps the traffic off the public internet which makes it harder for bad actors to capture that traffic.
In support of this decision, MicroHealth and Nava will need to work together to limit the traffic between VPCs to only the necessary services, over the necessary ports, using only the necessary protocols. This will need to be configured with security groups in both the MicroHealth and Nava AWS accounts.
### Positive Consequences
* If we require more services within the VPCs to talk with each other, we will already have a tool configured for that
### Negative Consequences
* This tool assumes MicroHealth and Nava will put security controls in place to limit the permitted traffic to only what's necessary, which will take some coordination between MicroHealth and Nava
## Security Implications
#### AWS DMS Service
The DMS service is a FedRAMP compliant service that sits in the target database's (beta team's) VPC, and is responsible for reading data from the source databases when there are changes and writing them to the target database. All traffic through DMS is [encrypted in transit and at rest](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.DataProtection.html). In addition to FedRAMP compliance, DMS is in compliance with many other compliance programs demonstrating its high security standards, including: SOC, PCI, ISO, FedRAMP, DoD CC SRG, HIPAA BAA, MTCS, CS, K-ISMS, ENS High, OSPAR, and HITRUST CSF. You can read more [DMS Compliance information, including FedRAMP, on its dedicated page](https://docs.aws.amazon.com/dms/latest/userguide/dms-compliance.html).
Tools are of course only as secure as the way they are configured and used. For our purposes, since we are effectively looking to create a read-only replica of the database, we only require SELECT permissions for the DMS instance role on the source database, as described in this [aws guide on configuring OracleDB as a source database](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html).
Between the limited scope of the DMS instance user access to the MicroHealth database, and the extensive security testing and controls of DMS as a service, DMS seems like a secure solution to replicating the data from the grants.gov AWS account to the beta AWS account.
#### VPC Peering
When permitting network traffic between two VPCs in AWS, [AWS provides several solutions and guidance on others](https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/amazon-vpc-to-amazon-vpc-connectivity-options.html). Several of the options are discussed in the Pros and Cons section below, however VPC Peering seems to be the best fit for our use case and as secure or more secure than the other options. [VPC Peering is FedRAMP compliant](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/), keeps traffic off the public internet, encrypts traffic in transit, and provides tools to manually secure the connection between the VPCs even further. Some of the manual controls include restricting VPC Peering to a specific CIDR block, or subnet, and using security groups to limit traffic protocol, origin, and destination port. In practice, we can limit traffic through the VPC Peering connection, which is all encrypted and off the public internet, to come from only the beta DMS instance and go to only the grants.gov database subnet or load balancer, using a specific protocol, and targeting a specific port. All other traffic will be denied.
AWS PrivateLink is not FedRAMP compliant. Notice it is not included on the [List of FedRAMP compliant AWS services](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/), however it is included on the [List of Canadian Center for Cyber Security compliant services](https://aws.amazon.com/compliance/services-in-scope/CCCS/), which shows its exclusion from the FedRAMP list isn't an oversight. Additionally, AWS PrivateLink does not encrypt traffic in transit as stated explicitly under the Security and Filtering section of [AWS PrivateLink FAQ](https://aws.amazon.com/privatelink/faqs/#Security_and_filtering). AWS PrivateLink is more expensive than VPC Peering, assuming the VPCs are in the same AZ, which is our plan, [data transfer is free with VPC Peering](https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/), but AWS PrivateLink has a [service fee and data transfer bandwidth fee](https://aws.amazon.com/privatelink/pricing/).
AWS Transit Gateway has a similar security posture to VPC Peering as [the underlying technology is the same](https://docs.aws.amazon.com/whitepapers/latest/building-scalable-secure-multi-vpc-network-infrastructure/transit-gateway.html), however, that solution is optimized for a hub and spoke VPC architecture with thousands of connected VPCs and is overly complicated for two VPCs to connect to each other.
Non AWS solutions require traffic to leave the AWS network and [traverse the public internet via internet gateways](https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/software-vpn-1.html). While that traffic can be encrypted with a VPN, that is inherently less secure than keeping the traffic within the AWS boundary.
## Implementation Guide
After reviewing the AWS documentation on configuring DMS where the target and source database reside in different VPCs, the recommended approach is to use AWS DMS with VPC Peering to permit the necessary traffic. The following details the division of tasks between Nava and MicroHealth to set up the necessary services.
VPC Peering must be configured before DMS can complete, however in order to limit VPC Peering traffic to only DMS, the DMS instance and subnet must be created with its CIDR block and IP address to share with MicroHealth for updating their route tables.
#### VPC Peering
**Nava**
* Confirm there are no overlapping IPv4 or IPv6 CIDR blocks
* if there are overlapping CIDR blocks create a new VPC with non overlapping CIDR blocks and migrate resources
* Info to share with MicroHealth
* DMS security group ID
* DMS instance subnet CIDR block
* DMS instance IP address
* [Create vpc peering connection](https://docs.aws.amazon.com/vpc/latest/peering/create-vpc-peering-connection.html)
* [Update your security groups to reference peer security groups](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-security-groups.html)
* * [Update route tables for peering connection to the db subnet](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-routing.html) or to a [specific IP address](https://docs.aws.amazon.com/vpc/latest/peering/peering-configurations-partial-access.html)
**MicroHealth**
* Confirm there are no overlapping IPv4 or IPv6 CIDR blocks
* Share information with Nava:
* Region information for the VPC
* AZ info for the DB
* Account ID
* VPC ID
* Database or new data replication security group ID
* [Accept VPC Peering connection request](https://docs.aws.amazon.com/vpc/latest/peering/accept-vpc-peering-connection.html)
* [Update route tables for peering connection to the db subnet](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-routing.html)
* [Update route tables for peering connection to the db subnet](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-routing.html) or to a [specific IP address](https://docs.aws.amazon.com/vpc/latest/peering/peering-configurations-partial-access.html)
#### AWS DMS Service
**Nava**
* Create a user with AWS Identity and Access Management (IAM) credentials that allows you to launch Amazon RDS and AWS Database Migration Service (AWS DMS) instances in your AWS Region.
* Size your target PostgreSQL database host based on the current db host load profile.
* Create the schemas in the target database
* Create the AWS DMS user to connect to your target database, and substitute our own username and password:
* ```
CREATE USER WITH PASSWORD '';
ALTER USER WITH SUPERUSER;
```
* Create a user for AWS SCT.
* ```
CREATE USER WITH PASSWORD '';
GRANT CONNECT ON DATABASE database_name TO ;
GRANT USAGE ON SCHEMA schema_name TO ;
GRANT SELECT ON ALL TABLES IN SCHEMA schema_name TO ;
GRANT ALL ON ALL SEQUENCES IN SCHEMA schema_name TO ;
```
* [Convert the Oracle Schema to PostgreSQL](https://docs.aws.amazon.com/dms/latest/sbs/chap-rdsoracle2postgresql.steps.convertschema.html)
* [Create an AWS DMS Replication Instance](https://docs.aws.amazon.com/dms/latest/sbs/chap-rdsoracle2postgresql.steps.createreplicationinstance.html) using terraform
* [Create AWS DMS Source and Target Endpoints](https://docs.aws.amazon.com/dms/latest/sbs/chap-rdsoracle2postgresql.steps.createsourcetargetendpoints.html)
* [Create and Run Your AWS DMS Migration Task](https://docs.aws.amazon.com/dms/latest/sbs/chap-rdsoracle2postgresql.steps.createmigrationtask.html)
**MicroHealth**
* Communicate the load profile of the current source Oracle database host. Consider CPU, memory, and IOPS.
* ensure that ARCHIVELOG MODE is on to provide information to LogMiner. AWS DMS uses LogMiner to read information from the archive logs so that AWS DMS can capture changes.
* Retaining archive logs for 24 hours is usually sufficient
* supplemental logging to be enabled on your source database
* identification key logging be enabled
* You can set this option at the database or table level
* Create or configure a database account to be used by AWS DMS
* [Instructions Guide](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html)
* AWS DMS requires the following privileges (note, there is one create for session, the rest are select)
* ```
GRANT CREATE SESSION TO ;
GRANT SELECT ANY TRANSACTION TO ;
GRANT SELECT ON V_$ARCHIVED_LOG TO ;
GRANT SELECT ON V_$LOG TO ;
GRANT SELECT ON V_$LOGFILE TO ;
GRANT SELECT ON V_$LOGMNR_LOGS TO ;
GRANT SELECT ON V_$LOGMNR_CONTENTS TO ;
GRANT SELECT ON V_$DATABASE TO ;
GRANT SELECT ON V_$THREAD TO ;
GRANT SELECT ON V_$PARAMETER TO ;
GRANT SELECT ON V_$NLS_PARAMETERS TO ;
GRANT SELECT ON V_$TIMEZONE_NAMES TO ;
GRANT SELECT ON V_$TRANSACTION TO ;
GRANT SELECT ON V_$CONTAINERS TO ;
GRANT SELECT ON ALL_INDEXES TO ;
GRANT SELECT ON ALL_OBJECTS TO ;
GRANT SELECT ON ALL_TABLES TO ;
GRANT SELECT ON ALL_USERS TO ;
GRANT SELECT ON ALL_CATALOG TO ;
GRANT SELECT ON ALL_CONSTRAINTS TO ;
GRANT SELECT ON ALL_CONS_COLUMNS TO ;
GRANT SELECT ON ALL_TAB_COLS TO ;
GRANT SELECT ON ALL_IND_COLUMNS TO ;
GRANT SELECT ON ALL_ENCRYPTED_COLUMNS TO ;
GRANT SELECT ON ALL_LOG_GROUPS TO ;
GRANT SELECT ON ALL_TAB_PARTITIONS TO ;
GRANT SELECT ON SYS.DBA_REGISTRY TO ;
GRANT SELECT ON SYS.OBJ$ TO ;
GRANT SELECT ON DBA_TABLESPACES TO ;
GRANT SELECT ON DBA_OBJECTS TO ; -– Required if the Oracle version is earlier than 11.2.0.3.
GRANT SELECT ON SYS.ENC$ TO ; -– Required if transparent data encryption (TDE) is enabled. For more information on using Oracle TDE with AWS DMS, see Supported encryption methods for
using Oracle as a source for AWS DMS.
GRANT SELECT ON GV_$TRANSACTION TO ; -– Required if the source database is Oracle RAC in AWS DMS versions 3.4.6 and higher.
GRANT SELECT ON V_$DATAGUARD_STATS TO ; -- Required if the source database is Oracle Data Guard and Oracle Standby is used in the latest release of DMS version 3.4.6, version 3.4.7, and higher.
GRANT EXECUTE on DBMS_LOGMNR to ;
GRANT SELECT on V_$LOGMNR_LOGS to ;
GRANT SELECT on V_$LOGMNR_CONTENTS to ;
GRANT LOGMINING to ; -– Required only if the Oracle version is 12c or higher.
```
* Add the exposeViews=true extra connection attribute to your source endpoint
* Provide the username and password for the DMS db account to Nava
* Run the following command in RDS to ensure that logs are retained: `exec rdsadmin.rdsadmin_util.set_configuration('archivelog retention hours',24);`
* Run the following db command: `ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;`
* Run the following command in RDS: `exec rdsadmin.rdsadmin_util.alter_supplemental_logging('ADD');`
* Run the following db command: `ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;`
* Run the following command in RDS: `exec rdsadmin.rdsadmin_util.alter_supplemental_logging('ADD','PRIMARY KEY');`
* Create a user for AWS SCT:
* ```
CREATE USER IDENTIFIED BY password;
GRANT CONNECT TO ;
GRANT SELECT_CATALOG_ROLE TO ;
GRANT SELECT ANY DICTIONARY TO ;
```
## Preparing for Production
Currently the beta AWS account is designated as a lower environment and therefore will only connect to the grants.gov lower environment. However, when we are ready, our plan is to create a second AWS account for our production environment which will then need to peer with the grants.gov production environment. Therefore, our strategy is to implement these tools and study the security impact in action to determine any security risk we need to address in production. It is also our assumption that the production environment will meet all HHS ITS security constraints, just like the grants.gov production environment. Nava will work collaboratively with MicroHealth to determine additional security measures that are necessary to ensure production environments and production data meet the necessary security standards.
## Pros and Cons of the Options - Data Replication
### Use MicroHealth Database
Connect to the Microhealth lower environment replica database, that contains only fixture data, for the lower environment.
* **Pros**
* No additional cost for data storage
* Easiest to set up
* **Cons**
* Additional load and db connections could degrade performance of critical grants.gov operations
* No data transformation possible
* Significantly increases traffic between VPCs
* beta application and availability will be dependant on grant.gov's database availability without necessary alarms or troubleshooting access
### Use AWS DMS
Create a new Postgres Database in the Beta lower and production environments and configure AWS DMS to replicate select tables from the MH lower environment database for the beta lower environment and the production database for the beta production environment. This solution requires MH configure VPC Peering to allow DMS traffic between our VPCs.
For this solution we will only replicate opportunities data at first to limit the cost of storage and restrict our environment to publicly accessible data. However, we will build it with the intention of making it easy to add tables to the replication, or even replicating the entire database when that becomes necessary.
* **Pros**
* DMS is AWS's best practice tool for our use case
* Negligible impact to source database, even with replicating ongoing changes
* Replicating only public data reduces our security criticality
* Ability to transform data is part of the DMS tool and well documented
* Ensures that simpler.grants.gov service remains available even if grants.gov has unexpected or planned downtime
* Limits the cross VPC traffic to just DMS
* **Cons**
* Additional Cost
* Networking support and coordination required from MH
### Create new data pipelines from data sources
Create new data pipelines from the source of truth similar to the production database. Instead of copying the production schema, design a new database schema that incorporates all the lessons learned from running the current production database as well as designing the new schema for additional requirements that the current schema is not optimized or able to meet.
* **Pros**
* No impact to production database
* Facilitates moving off expensive Oracle database
* Can optimize database schema for current and future requirements
* **Cons**
* We do not have clear requirements for current and existing APIs to design the schema around and will have to work on that first
* Very long time to deliver
* Team is not currently staffed to support this work
### Import/Export DB snapshots weekly
MicroHealth will export a database snapshot on a weekly basis that we will use to update our database on a weekly basis. The exports will be done during times of low database usage so as to have negligible impact on production operations. However, the data will be up to seven days old.
* **Pros**
* Negligible impact to production database
* Simple to do manually and also to automate
* **Cons**
* Data will be up to 7 days old
## Pros and Cons of the Options - Data Traffic
### AWS VPC Peering
Configure AWS VPC Peering on both the Nava and MicroHealth AWS VPCs to allow traffic between the two VPCs. For security, lock down the VPC Peering to only allow traffic between the DMS instance in the Nava account and the database instance or database load balancer in the Microhealth account. All traffic between VPCs using VPC Peering is encrypted. Additionally, the traffic between VPCs stays within the AWS Global Backbone and never makes its way to the public internet. Finally, AWS VPC Peering is a FedRAMP compliant AWS feature.
* **Pros**
* Many layers of security: encryption in transit, traffic stays off public internet, additional manual controls
* AWS best practice for multi-VPC DMS configuration
* FedRAMP compliant
* Free to operate
* **Cons**
* Requires configuration on both Nava and MicroHealth sides
* cost for bandwidth
### AWS PrivateLink
AWS PrivateLink provides private connectivity between virtual private clouds (VPCs), supported AWS services, and on-premises networks without exposing traffic to the public internet. Using AWS PrivateLink a VPC can expose interface VPC endpoints, similar to APIs, for others to query. This is a one way connection through a VPC barrier, instead of the two way connection provided by VPC Peering. Additionally, other security tools can be leveraged to enhance security of AWS PrivateLink, like security groups and VPC endpoint policies, which is similar to VPC Peering. AWS PrivateLink is compatible with DMS across VPCs, however it is not FedRAMP compliant.
* **Pros**
* One way connection
* Traffic stays off the public internet
* **Cons**
* Does not provide encryption
* Anyone can connect
* Not FedRAMP compliant
* There is a cost to operate and cost for bandwidth
### AWS Transit Gateway
The AWS Transit service consolidates the AWS VPC routing configuration for a region with a hub-and-spoke architecture. This service uses the same technology as VPC Peering, but instead of connecting VPC directly, they connect through another service called a Transit Gateway. This solution is recommended if many VPCs need to connect to each other across regions as VPC Peering gets significantly more complicated when more than a few VPCs are involved.
* **Pros**
* Similar to VPC Peering
* **Cons**
* More complicated to implement than VPC Peering
* More expensive than VPC Peering
## Links
* [AWS DMS](https://aws.amazon.com/dms/)
* [AWS DMS Cross VPC Config](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.VPC.html#CHAP_ReplicationInstance.VPC.Configurations.ScenarioVPCPeer)
* [What is VPC Peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html)
* [List of FedRAMP compliant AWS services](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/)
* [AWS VPC to AWS VPC Connectivity Options](https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/amazon-vpc-to-amazon-vpc-connectivity-options.html)
* [Multi-VPC Network Infrastructure Whitepaper](https://docs.aws.amazon.com/whitepapers/latest/building-scalable-secure-multi-vpc-network-infrastructure/welcome.html)
# HHS Communications Site
* **Status:** Active
* **Last Modified:** 2023-09-22
* **Related Issue:** [#498](https://github.com/HHS/simpler-grants-gov/issues/498)
* **Deciders:** Lucas, Elizabeth, Julius, Billy
* **Tags:** topic: comms
## Context and Problem Statement
HHS needs a way to distribute information about the Simpler Grants workstreams that is tailored to internal stakeholders. This internal site should prioritize hosting and broadcasting read-only content about these workstreams in a user-friendly format, but also provide a mechanism for stakeholders to pose questions or provide feedback.
## Decision Drivers
* **Deployment Timeline:** We can deploy a site with this tool in less than a month
* **User Experience (UX):** The site is easy to navigate with a modern-looking user interface
* **Hosting Cost:** It is not prohibitively expensive to extend access to all of HHS
* **Access Control:** We can control who can read and edit the site with minimal overhead
* **Content Management:** It is easy for HHS staff and contractors to update the site content
* **Feedback Mechanism:** We can solicit feedback and questions from site visitors
## Options Considered
* GitBook publishing options
* Guest Access
* Visitor Authentication
* “Private” Links
* SharePoint hosting options
* HHS-owned site
* Contractor-owned site
* Options for custom-built sites
* Page on HHS intranet site
* Contractor-built static site
## Decision Outcome
Use **GitBook** to create and manage content publishing to the site via **“private” link**. The link to this site will be added to the HHS intranet and/or OG Resource Center so that HHS staff can find it.
In parallel we'll actively work on developing a **visitor authentication option** to restrict access to the site in the long-term. Once this visitor authentication option is built and undergoes security review and approval, we'll change the publishing strategy for GitBook from "private" link to visitor authentication.
### Positive Consequences
* Allows us to deploy and publish the site in time for the kickoff of the initiative
* Enables HHS staff and contractors to manage the content directly in a easy to use editor
* Provides a modern user interface for visitors to the site.
### Negative Consequences
* During the period in which we are using the "private" link publishing strategy we'll need to be avoid of including any sensitive information on the site. The strategy to address this is to keep sensitive information in an access controlled environment and to simply link to where that information is stored from GitBook.
* We'll need to allocate some technical resources in the upcoming period of performance to work on implementing the visitor authentication option for GitBook.
* We may need to have a period of downtime when we switch from the "private" link publishing strategy to the visitor authentication strategy.
## Comparison Matrix
### Table Legend
* ✅ - Meets decision criteria
* ❌ - Does not meet decision criteria
* 🟡 - Partially meets criteria or requires more info
### Table
| Option | Timeline | UX | Cost | Access | Content | Feedback |
| ------------------------------- | :------: | :-: | :--: | :----: | :-----: | :------: |
| GitBook: Guest Access | ✅ | ✅ | ❌ | 🟡 | ✅ | ✅ |
| GitBook: Visitor Authentication | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Gitbook: “Private” Link | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |
| SharePoint: HHS-owned | 🟡 | 🟡 | ✅ | ✅ | 🟡 | ✅ |
| SharePoint: Contractor-owned | 🟡 | 🟡 | ❌ | 🟡 | ✅ | ✅ |
| Custom: HHS Intranet Page | 🟡 | ✅ | ✅ | 🟡 | ❌ | 🟡 |
| Custom: Contractor-built Site | ❌ | ✅ | ✅ | 🟡 | ✅ | ✅ |
## Pros and Cons of the Options
### GitBook: Guest Access
Create and maintain the site contents in **GitBook** and extend view access to through **GitBook-managed licenses**.
**Bottom Line:** GitBook is likely the best platform, but this is not a viable publishing option because it would be too costly and difficult to administer
* **Pros**
* **Timeline:** Site can be deployed in 1-2 weeks
* **Usability:** Clean and customizable user interface
* **Access:** Can control both edit and view access
* **Content:** HHS staff and contractors can edit content
* **Feedback:** Supports embedding Microsoft forms
* **Cons**
* **Cost:** More than $25k per month for all HHS users
* **Access:** View access needs to be managed per user
### GitBook: Visitor Authentication
Create and maintain the site contents in **GitBook** and extend view access to through an **HHS-managed authentication solution**.
**Bottom Line:** GitBook is likely the best platform, and this publishing option meets most of the key criteria except our desired timeline for deployment
* **Pros**
* **Usability:** Clean and customizable user interface
* **Cost:** Only pay for editors, no per-viewer costs
* **Access:** Can control both edit and view access
* **Content:** HHS staff and contractors can edit content
* **Feedback:** Supports embedding Microsoft forms
* **Cons**
* **Timeline:** May take several months to set up authentication (due to security review)
### GitBook: Visitor Authentication
Create and maintain the site contents in **GitBook** and extend view access through a **shareable link with a secret token**.
**Bottom Line:** GitBook is likely the best platform, and this publishing option meets most of the key criteria except our desired desired access controls
* **Pros**
* **Timeline:** Site can be deployed in 1-2 weeks
* **Usability:** Clean and customizable user interface
* **Cost:** Only pay for editors, no per-viewer costs
* **Content:** HHS staff and contractors can edit content
* **Feedback:** Supports embedding Microsoft forms
* **Cons**
* **Access:** Anyone with the link can view the site
### SharePoint: HHS-owned
Create and maintain the site contents in an **HHS-owned SharePoint** instance. Use a **communications site template** (if available) for a modern look and feel.
**Bottom Line:** SharePoint could be a viable option, but there are still open questions and challenges getting access for contractors
* **Pros**
* **Usability:** Modern UI (with correct SharePoint version)
* **Cost:** No cost for HHS staff with O365 licenses
* **Access:** Supports fine-tuned access controls
* **Feedback:** Supports integration with Microsoft forms
* **Cons**
* **Usability:** Current version may not support modern UI
* **Timeline:** Timeline for creating a new site is unclear
* **Content:** Contractors may not have edit access
### SharePoint: Contractor-owned
Create and maintain the site contents in a **contractor-owned SharePoint** instance. Use communications site template (if available) for a modern look and feel.
**Bottom Line:** SharePoint could be a viable option, but using a contractor-owned site would likely be too costly and difficult to administer
* **Pros**
* **Usability:** Modern UI (with correct SharePoint version)
* **Timeline:** Likely faster to create than HHS-owned site
* **Content:** Likely easier to give contractors edit access
* **Feedback:** Supports integration with Microsoft forms
* **Cons**
* **Usability:** We may not have access to required version
* **Cost:** May have to pay per user for HHS access
* **Access:** May have to provision access per user
### Custom: HHS Intranet Site
Work with **HHS intranet site** admins to create a custom page. Manage updates and content through their existing request process.
**Bottom Line:** A page on the HHS intranet is a safe fallback option, but may have limits on timeline and the ability to manage and update content
* **Pros**
* **Usability:** Intranet site uses modern USWDS branding
* **Cost:** No additional per-user cost to host site
* **Access:** HHS staff already have access to the site
* **Cons**
* **Timeline:** Timeline for creating a new page is unclear
* **Access:** Contractors don’t have view or edit access
* **Content:** Changes may need to go through approval
* **Feedback:** Options for collecting feedback are unclear
### Custom: Contractor-built Site
Add a **custom-built site** to the scope for the next period of performance with MicroHealth/Nava. Use the **Storyblock CMS** or another solution to manage content updates.
**Bottom Line:** A contractor-built site is a strong back-up option, but it would delay the timeline and it may divert resources from other priorities
* **Pros**
* **Usability:** Gives us more control over the look and feel
* **Cost:** No additional per-user cost to host site
* **Access:** We can control both read and write access
* **Content:** Content edits could be made by contractors
* **Feedback:** We can build our own custom forms
* **Cons**
* **Timeline:** May take several months to set up authentication (due to security review)
* **Content:** Fewer staff may be able to manage content
* **Access:** Some technical details to work out
## Links
* [GitBook visitor authentication](https://docs.gitbook.com/publishing/visitor-authentication)
* [GitBook private links](https://docs.gitbook.com/publishing/share/share-links)
* [HHS comms site options slide deck](https://docs.google.com/presentation/d/1vo7GvTqQKxHcXX65sXyTIJziEXfrPeLh/edit#slide=id.g281eab1baba_1_111)
# Communications Tooling: Email Marketing
* **Status:** Active
* **Last Modified:** 2023-12-04
* **Related Issue:** [#590](https://github.com/HHS/simpler-grants-gov/issues/590)
* **Deciders:** Lucas, Aaron, Billy, Sarah, Sumi
* **Tags:** communucations, open source, email marketing
## Context and Problem Statement
An email marketing tool is primarily used for creating, sending, and tracking email campaigns to a list of subscribers. It allows for us to engage with our audience, push notifications and newsletters to our users, share information, and build customer relationships through email.
## Decision Drivers
#### Must Have
* **Usability:** Non-technical users should be able to access and create content with minimal training or guidance.
* **Content Review:** Collaborators should be able to review and edit draft content before emails are sent out.
* **Comments:** Reviewers should be able to leave in-line comments on content that they are reviewing.
* **Internationalization (i18n):** The solution should provide support for displaying content in multiple languages.
* **Analytics:** The platform should provide support for tracking open rates, click-through-rate and other other web analytics.
* **Onboarding Costs:** Onboarding new members to the platform should be relatively inexpensive, both in terms of staff time/resources and direct costs (e.g. licensing fees).
* **Maintenance Costs:** It should not be prohibitively expensive to maintain the email marketing tool, both in terms of staff time/resources and direct costs (e.g. hosting fees).
* **Authority to Operate (ATO):** The tool should be FedRAMPed or have an Authority to Operate as it will have names and email addresses of public users.
#### Nice to Have
* **Open Source:** The tool used to manage the email marketing content should be open source, if possible.
* **Community & Support**: The platform has a strong community for ongoing support, updates, and bug fixes
* **Scalability**: The platform should be able to hadndle a growing number of subscribers without performance issues
## Options Considered
* [Sendy](https://sendy.co/)
* [Salesforce Marketing](https://www.salesforce.com/products/engagement-marketing/)
* [MailChimp](https://mailchimp.com/)
* [Hubspot](https://www.hubspot.com/)
* [Mautic](https://www.mautic.org/)
* [Adobe Campaign](https://business.adobe.com/products/campaign/adobe-campaign.html)
## Decision Outcome
The suggested approach is to utilize Sendy and create a campaign for Simpler Grants.gov within the existing main account for the near-term, leveraging existing knowledge and allowing for faster use. This allows for a faster initiation of the process. One of the most important criteria given by users of the tool, the Grants.gov Communications team, is the need to be able to send an email to the current 1M current subscribers quickly which Sendy cannot do. There are other limitations of this solution so we recommend that we select a back-up option email marketing solution for our long-term needs.
There are limitations to Sendy and the recommendation is to move to another email marketing tool that is a better long-term solution that works for HHS and the larger communications strategy for Simpler Grants.gov, current Grants.gov, and NOFO. This makes it easier and consistent to implement and manage one tool.
### Open Questions
1. Can we transfer the Simpler Grants.gov email list from Sendy to a different tool in the future?
Yes, we are able to transfer the list created in Sendy to a new tool in the future.
2. What is the long-term tool that we should use?
As we're determining the brand identity and communications strategy and how the different HHS teams will work together, we will determine the long-term solution at a later time.
3. Can we send to individual lists?
We can use Sendy to send to individuals and also for a separate subscription lists which is called a "Brand" in Sendy.
5. How can we integrate with the current implementation of Sendy?
A Brand can be created under the existing Sendy account which allows users who subscribe through that form to be added to a separate list specific to Simpler.Grants.gov. This list will be distinct from the other lists within the Sendy account.
6. For the long term solution, there are certain requirements and considerations that we should consider?
* Currently when users unsubscribe, a script needs to be run to remove them from the list. We should consider a system that automatically updates the system.
* The new platform should have an API that is robust enough to allow users to subscribe, unsubscribe, and check for existing users.
## Comparison Matrix
* ✅ Feature available, meets requirement
* ❌ Feature not available, does not meet requirement
* 🔄 Partial feature, limited feature availability, feature in progress or undergoing improvements
* 1-3 Strength level
* ❓Unknown
| Factor | Sendy | Salesforce | MailChimp | Hubspot | Mautic | Adobe |
| ------------------------------- | :---: | :--------: | :-------: | :-----: | :----: | :---: |
| Usability | 2 | 1 | 3 | 3 | 1 | 3 |
| Content Review | 🔄 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Comments | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ |
| Multi-Media | ✅ | ✅ | ❌ | ✅ | ✅ | 🔄 |
| Analytics | 1 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Internationalization | ❓ | 2 | 3 | 2 | 1 | 1 |
| Onboarding costs | 1 | 3 | 1 | 2 | 1 | 2 |
| Send 1M emails quickly | ❌ | ❓ | ❓ | ❓ | ❓ | ❓ |
| Authority to Operate or FedRAMP | ✅ | ❓ | ❌ | ❌ | ❌ | ✅ |
| Open source\* | 🔄 | ❌ | ❌ | ❌ | ✅ | ❌ |
| Community and Support\* | 1 | 3 | 3 | 3 | 3 | 3 |
| Scalability\* | 🔄 | ✅ | ✅ | ✅ | ✅ | ❌ |
\[\*] Nice to have
## Pros and Cons of the Options
### Sendy
#### Details
Sendy is a self-hosted email marketing application that allows users to send newsletters, manage subscribers, and track campaign performance. It is known for its cost-effectiveness and integration with Amazon SES for efficient email delivery.
#### Pros
* Sendy is currently used by HHS and it would be simple for existing users to start using the tool right away
* It provides decent metrics and analytics
* HHS is able to send a few campaigns simultaneously
* Sendy is self-hosted which means we have control over the email list and the data. Sendy is a self hosted application that runs on our own web server. Pay once and it's yours, there's no recurring fee.
* It is a cost-effective option as it is self-hosted.
* Self-hosting provides control over data and infrastructure
#### Cons
* Campaigns with larger audiences or number of emails take a long time to send and a user cannot use the tool while an email is being sent. Currently, we have 1M subscribers and it takes a long time to send to that many subscribers. There are methods that could be done to improve this.
* Lacks some advanced features present in other platforms.
* It requires technical knowledge for setup and maintenance.
* Limited customer support compared to premium services.
### Salesforce Marketing
#### Details
Salesforce Marketing Cloud is a comprehensive marketing automation platform that enables organizations to create and manage personalized customer journeys. It includes features for email marketing, social media advertising, customer segmentation, and analytics.
#### Pros
* Salesforce Marketing is a comprehensive solution that offers a wide range of features and tools for marketing automation
* If we use any other Salesforce tools in the future, it is an easier integration
* It provides robust analytics and reporting capabilities
#### Cons
* Salesforce Marketing can be expensive, especially for smaller businesses
* There is learning curve as it may be more complex for non-technical users
* It may have more features than needed
### MailChimp
#### Details
Mailchimp is a widely used email marketing platform that offers a user-friendly interface for designing and sending emails, managing subscriber lists, and analyzing campaign performance. It also provides marketing automation features.
#### Pros
* MailChimp is user-friendly for non-technical users with a simple interface
* MailChimp does offer a free plan for small-scale needs
* There are automation features for targeted campaigns
#### Cons
* Pricing can become high as subscriber lists grow
* Some users may find limitations for advanced customization
* Free plan users have limited access to customer support
### Hubspot
#### Details
HubSpot is an all-in-one inbound marketing, sales, and customer service platform. It provides tools for content marketing, social media, lead generation, and customer relationship management (CRM) to help businesses attract, engage, and delight customers.
#### Pros
* Hubspot combines marketing, sales, and customer service tools
* Intuitive interface for non-technical users
* Strong automation and personalization features
#### Cons
* Can be expensive, especially for additional features
* Some users may find it complex initially
* More features th an needed for some smaller organizations
### Mautic
#### Details
Mautic is an open-source marketing automation platform that allows users to automate marketing tasks, personalize communication, and track the behavior of leads. It is known for its flexibility, open-source nature, and strong community support.
#### Pros
* Offers flexibility for customization
* Strong community support for ongoing development
* Can handle growing subscriber lists
#### Cons
* Moderate usability, some users may find it less user-friendly
* May require additional development for certain integrations
* Non-technical users might face a learning curve
### Adobe Campaign
#### Details
Adobe Campaign is a marketing automation tool that forms part of the Adobe Experience Cloud. It enables users to create, execute, and measure multi-channel marketing campaigns, including email, mobile, social, and web, while integrating with other Adobe products.
#### Pros
* Adobe Campaign is FedRAMP certified
* Adobe Campaign is part of the broader Adobe Experience Cloud which is a suite of products and could be beneficial if we use more Adobe products for marketing and analytics tools
* Enables the creation and execution of multi-channel marketing campaigns, including email, mobile, social, and web
* Offers robust features for personalizing content and creating dynamic, targeted campaigns
#### Cons
* The platform may have a learning curve, especially for users new to advanced marketing automation tools
* Adobe Campaign can be relatively expensive, making it more suitable for larger enterprises with substantial marketing budgets
* Implementation and maintenance may require dedicated resources and expertise
* The extensive feature set may be more than what smaller businesses need, leading to potentially unnecessary complexity
## Links
* [Sendy](https://sendy.co/)
* [Salesforce Marketing](https://www.salesforce.com/products/engagement-marketing/)
* [MailChimp](https://mailchimp.com/)
* [Hubspot](https://www.hubspot.com/)
* [Mautic](https://www.mautic.org/)
* [Adobe Campaign](https://business.adobe.com/products/campaign/adobe-campaign.html)
# Communications Tooling: Listserv
* **Status:** Accepted
* **Last Modified:** 2023-10-16
* **Related Issue:** [#378](https://github.com/HHS/simpler-grants-gov/issues/378)
* **Deciders:** Lucas, Andy, Billy, Sarah, Sumi
* **Tags:** communucations, open source, listserv
## Context and Problem Statement
In order to foster an open and transparent communication with the public through newsletters and other proactive engagement, the project needs to select an efficient listserv tool. The challenge lies in identifying a platform that ensures accessibility, ease of participation, and comprehensive message archiving to engage the public effectively in project discussions. This decision must facilitate making it vital to strike a balance between features, user-friendliness, and the potential for creating a vibrant and inclusive community for the project.
## Decision Drivers
#### Must Have
* **Ease of use**: this will be used by HHS, contractors, and the general public and should be user-friendly and easy for both administrators and users to navigate, set up, and manage.
* **Security**: The solution must prioritize strong security measures, consider encryption, user authentication, and protection against spam and phishing
* **Cost**: Evaluate the cost implications as this tool will be used by HHS, contractors, and the general public.
* **Features**: Consider features such as email list management, subscription options, message archiving, and customizable templates to meet the communication needs of your project.
* **Scalability**: Ensure the solution can handle potential growth in subscribers and messages as your project expands without sacrificing performance.
Nice to have
* **Community engagement**: Opt for a solution that encourages active participation, discussion, and collaboration within the open-source community.
* **Open source**: Consider a solution that aligns with the open-source philosophy of your project, allowing for transparency, customization, and community contributions.
* **Customization**: Choose a platform that allows you to customize the appearance and functionality of the listserves to match your project's branding and communication needs.
## Decision Outcome
We selected Google Groups as it excels in ease of use, ensuring that both HHS staff, contractors, and the general public can seamlessly navigate and manage communications. The platform's commitment to security, encompassing encryption, authentication, and anti-phishing measures, meets our high standards. Moreover, Google Groups offers cost-effective solutions suitable for the diverse user groups of our project. Its rich features, including email list management, subscription flexibility, and message archiving, cater to our project's unique communication needs. With scalability to accommodate growth, it's well-equipped to support our expanding community. While not mandatory, the platform's capacity for community engagement aligns with our desire for active participation and collaboration. It's a highly favorable choice that complements the open-source philosophy of our project, permitting customization for a cohesive branding and an inclusive communication environment. The decision to opt for Google Groups assures that we have an efficient and versatile listserv platform to meet our project's diverse and evolving requirements.
### Decision Considerations and Implications
We intend the primary focus of **Google Groups** to be as a channel to create discussion forums and online communities. Users can utilize it to post questions and exchange information and ideas. The channel should be used by users to post questions and exchange ideas and information.
If users have private questions, the **Contact Us** email at is the best method of communication.
The email marketing tool that we will select in the ADR: [#590](https://github.com/HHS/simpler-grants-gov/issues/590) will be used for individuals who wish to subscribe to Simpler Grants.gov newsletters. These newsletters are tailored for targeted messaging and notifications, including updates about the Simpler Grants.gov project.
Regarding Google Groups, it's important to note that if we plan to use individual emails from our Google Group to send newsletters, we must obtain explicit permission from users to send them newsletter notifications. Additionally, we should explore how we can import or integrate individuals who join Google Groups and wish to receive newsletters.
When implementing and using Google Groups (listserv), the 'Contact Us' option, and an email marketing tool, it's important that we clearly communicate the purpose of each tool and guide users on the best way to reach out to us, as there are distinct differences in their usage.
### Positive Consequences
* **User-Friendly Experience:** Google Groups provides an intuitive and user-friendly interface, ensuring that both administrators and members can easily navigate and manage communications.
* **Robust Security:** The platform prioritizes security, offering features like encryption, user authentication, and protection against spam and phishing.
* **Cost-Effective:** Google Groups offers cost-effective solutions, aligning with the budget constraints of the project.
* **Feature-Rich:** Google Groups comes with a suite of features, including email list management, subscription options, message archiving, and customizable templates.
* **Scalability:** The platform can accommodate potential growth in subscribers and messages as the project expands without compromising performance.
* **Community Engagement:** While not a core requirement, Google Groups encourages community engagement, discussion, and collaboration within the open-source community.
* **Open Source Alignment:** The platform aligns with the open-source philosophy of the project, promoting transparency, customization, and community contributions.
### Negative Consequences
* **Dependence on External Platform:** Choosing an external platform like Google Groups may raise concerns about dependence on a third-party service. This dependency could potentially result in disruptions if the service experiences downtime or policy changes.
* **Privacy Concerns:** Storing project communications on an external platform may raise privacy concerns, particularly for sensitive or confidential information.
* **Limited Control:** While Google Groups allows some customization, it may not offer the same level of control as an internally hosted solution.
* **Data Migration Challenges:** Transferring data to and from Google Groups may present challenges
* **User Account Requirements:** To participate in Google Groups, users typically need Google accounts. This requirement might exclude potential participants who do not wish to create Google accounts or who have restrictions on using Google services.
* **Advertisement Display:** Google Groups may display advertisements to users, which could be distracting or raise concerns about privacy and data usage.
### Back-up Options
* **Mailman:**
* Pros: Mailman is an open-source email list management software. It provides a highly customizable and self-hosted solution. It aligns with the open-source philosophy, promotes community contributions, and offers robust security.
* Cons: Setting up and maintaining Mailman may require more technical expertise. The user interface may not be as user-friendly as some commercial alternatives.
* **Microsoft 365 Groups:**
* Pros: Microsoft 365 Groups offers a comprehensive collaboration platform and is potentially used on the HHS side with other Microsoft products. It includes email distribution lists, calendars, shared documents, and more.
* Cons: It is a commercial solution, and some features may be beyond the project's budget. Compatibility with non-Microsoft users could be a concern.
* **Groups.io:**
* Pros: Groups.io is a platform designed to replace the functionality of Yahoo Groups. It offers a free plan and provides features for managing email lists and discussions.
* Cons: It may not be as feature-rich as Google Groups, and the free plan may have limitations.
* **Hosted Email Lists:**
* Pros: Some web hosting providers offer email list management tools with hosting plans. These can provide full control and customization.
* Cons: The availability of such services can vary among hosting providers. It may require technical knowledge to set up and maintain.
### Open Questions
* How will Google Groups integrate or be used with the email marketing tool that we will use in the future? This should be considered in the [Email Marketing Tool ADR #590](https://github.com/HHS/simpler-grants-gov/issues/590).
# Use Ethnio for design research
# Uptime Monitoring
* **Status:** Active
* **Last Modified:** 2023-11-22
* **Related Issue:** [#656](https://github.com/HHS/simpler-grants-gov/issues/656)
* **Deciders:** Lucas, Billy, Sammy, Daphne, Aaron
* **Tags:** Infrastructure, Notifications, Reliability
## Context and Problem Statement
We need a tool for external uptime monitoring of the website and API. We have [internal monitoring](https://github.com/HHS/simpler-grants-gov/blob/main/infra/modules/monitoring/main.tf) setup, but not external. This would be useful for cases in which a load balancer or CDN (if we adopt one) are not operating correctly, or there is a DNS issue with the site.
## Decision Drivers
* Ease of setup: This tool should be easy to configure and update as needed over time.
* Notifications: This tool must be able to notify the engineering team in the case of downtime.
* Cost: This tool should be as cost effective as possible.
* Process: This tool should fit into our existing processes and procedures for review and discussion.
* Dashboard (Optional): This tool should provide a dashboard for non engineers to see uptime metrics in real time.
## Options Considered
* AWS Cloudwatch Synthetic Canary
* Pingdom
* New Relic
## Decision Outcome
Chosen option: AWS Cloudwatch Synthetic Canary, because it satisfies the availability monitoring requirement without adding much overhead to our existing toolset. This service can be configured in terraform so we can document exactly what it does with code and review it via our normal code review and approval process. Canaries will also need to be configured to send SNS notifications to an email group for outages. Any additional queries that could be supported by a third party tool will need to be done in google analytics.
### Positive Consequences
* We will have uptime monitoring for our site from the perspective of public users.
* We will be notified in the event of an outage or error with our DNS or networking configuration.
### Negative Consequences
* We will not have a dashboard for people without aws access to review uptime.
## Pros and Cons of the Options
### AWS Cloudwatch Synthetic Canary
Amazon CloudWatch Synthetics uses "canaries", configurable scripts that run on a schedule, to monitor endpoints and APIs to follow the same routes and perform the same actions as a user. Canaries scripts can be written in Node.js or Python to create Lambda functions in your account and work over both HTTP and HTTPS protocols. Canaries offer programmatic access to a headless Google Chrome Browser via Puppeteer or Selenium Webdriver. Canaries check the availability and latency of your endpoints and can store load time data and screenshots of the UI. They monitor your REST APIs, URLs, and website content, and they can check for unauthorized changes from phishing, code injection and cross-site scripting. Priced at $0.0012 per canary run.
* **Pros**
* Uses tools of existing ecosystem
* Configurable as infrastructure as code in terraform
* FedRAMP compliant
* **Cons**
* Dashboard only accessible within AWS
### Pingdom
Pingdom offers uptime monitoring from over 100+ locations worldwide, page speed analysis, and transaction monitoring ( test simple or highly complex transactions, such as: new user registrations, user login, search, shopping cart checkout, URL hijacking, etc.). Creating uptime checks and alerts is easy in the third party dashboard. Priced starting at 10$/month for 10 uptime check configurations. PCI, HIPAA, and EU data protection certified.
* **Pros**
* Quick and easy to configure
* Standalone dashboard
* **Cons**
* Additional tool/dashboard to keep track of
* Outside existing codebase, build, review, deployment process
* Not FedRAMP Compliant
### New Relic
New Relic is an observability platform that provides monitoring of infrastructure, application performance, and availability. With convenient automated configuration tools, including for node.js and python applications, as well as aws accounts, it is easy to get started with a sophisticated and holistic monitoring system. New Relic is FedRAMP certified. New Relic charges by data ingest ($0.30 or $0.50 per GB after the first 100GB) as well as for user licenses from $50/user/month to $658/user/month.
* **Pros**
* Easy to set up by having it scan the codebase or aws tenant
* Easy to build, access, and share dashboards
* Nava experience from other projects
* FedRAMP compliant
* **Cons**
* Complex tool that is redundant with some of our other tools
* Outside existing codebase, build, review, deployment process
# Database Migrations
* **Status:** Accepted
* **Last Modified:** 2023-12-06
* **Related Issue:** [#779](https://github.com/HHS/simpler-grants-gov/issues/779)
* **Deciders:** Lucas, Aaron, Billy, Sammy, Michael
* **Tags:** Database, Backend
## Context and Problem Statement
We need a tool that can help manage the schema of our database, as well as make updates to the schema as we add more to it.
NOTE: At the time of writing this, the API already uses Alembic, and this document is more to describe why we use it, rather than to make a decision.
## Decision Drivers
* Ease of use: Creating new migrations should be easy and intuitive
* Maintenance effort: The tool should be easy to maintain as we add more to our database schema
* Minimizing risk: The tool should gracefully handle migration failures
* Cost: This tool should be cost effective
## Options Considered
* Alembic
* AWS Database Migration Service
* Django
* Flyway
* Liquibase
## Decision Outcome
Chosen option: Alembic, because it handles our known use cases, has minimal overhead, and is the go-to framework for the ORM we also use.
Any other option requires us to create migrations through a process that requires significantly more work, either by manually making those files, or generating them with another tool. Alembic just reads the ORM models we already are writing and generates the migrations for us.
### Positive Consequences
* *Ease of use*: Migrations are generated directly from our SQLAlchemy ORM models
* *Maintenance effort*: Migrations are generally uneventful to add, and just require running a single command to automatically generate
* *Minimizing risk*: Migrations are run in transactions, if any error occurs, they automatically roll back. If an issue occurs after the migration happens, a downgrade is generated for you as well to revert.
* *Cost*: Alembic is open source and free
### Negative Consequences
* *Maintenance effort*: Some advanced migration features may be more difficult to execute
## Pros and Cons of the Options
### Alembic
[Alembic](https://alembic.sqlalchemy.org/en/latest/) is a database migration tool that can generate database migrations from our [SQLAlchemy ORM](https://www.sqlalchemy.org/) schema. It is capable of [automatically detecting](https://alembic.sqlalchemy.org/en/latest/autogenerate.html#what-does-autogenerate-detect-and-what-does-it-not-detect) most common database modifications including table, column, and index additions.
The migration files Alembic generates each receive a unique identifier, and the only information it needs to track is the current one. When you run the database migrations, it finds the file with the current id, and sees if any migration is derived from it, similar to a linked list. If more than one file references the same migration (most commonly caused by two developers merging changes in quick succession), then Alembic will error when attempting to run the migrations, requiring someone to create a new merge migration and define the order.
When Alembic migrations fail, they do not commit any changes to the database.
See [database-management.md](https://github.com/HHS/simpler-grants-gov/blob/main/documentation/api/database/database-management.md) for some details on how we use Alembic in this environment
* **Pros**
* Built specifically to work with SQLAlchemy, the ORM layer that we already use
* Capable of automatically generating migrations for most common schema changes from our SQLAlchemy models, making most migrations zero effort
* Migrations are generated locally by a developer, and can be run against your local database to test the changes
* No cost, is an open source extension of SQLAlchemy which is also free
* **Cons**
* There are some scenarios where Alembic cannot detect database changes like renaming a table or column - instead a developer would need to manually modify the migration to do the rename
* Certain repeatable migrations (like updating functions, views, or triggers) aren't detected by default, but a [library](https://github.com/olirice/alembic_utils) exists which can detect and automate this for you.
### AWS Database Migration Service
[AWS Database Migration Service](https://aws.amazon.com/dms/) (DMS) is a tool for copying and converting your database.
* **Pros**
* We already will be using DMS to copy data from the existing Oracle DB to our new Postgres DB which will require schema configuration to work
* **Cons**
* Eventually we will no longer be copying data from the legacy Oracle DB to our DB, at which point we'd need to find a new migration approach
* Local development would not be able to setup a database using AWS DMS, local testing of schema changes would require a separate local-only process
* AWS DMS is opinionated on database types in ways that conflict with Postgres documentation. For example, DMS largely prefers using VARCHAR for any string column despite [Postgres docs](https://www.postgresql.org/docs/current/datatype-character.html) saying TEXT and VARCHAR are the same performance-wise. While you can convert with AWS DMS, this adds additional overhead to a very common column type.
### Django
The Django framework comes with its own [migration process](https://docs.djangoproject.com/en/4.2/topics/migrations/).
* **Pros**
* Provides a way to [squash migrations](https://docs.djangoproject.com/en/5.0/topics/migrations/#squashing-migrations) as they build up over time
* Thoroughly integrated with the Django framework with lots of customization possible
* **Cons**
* We would need to switch our API stack to use Django, see [API Framework](/product/decisions/adr/2023-07-07-api-framework) for further details on this decision process
### Flyway
[Flyway](https://flywaydb.org/) is a Java-based application that manages running SQL migrations.
* **Pros**
* [Supports](https://documentation.red-gate.com/flyway/flyway-desktop/database-devops-practices) a variety of deployment approaches, and version controls
* **Cons**
* Creating new migrations requires using their Flyway Desktop application to generate the migration files which is another tool we would need to learn
* Has cost tiers depending on your organization size
### Liquibase
[Liquibase](https://www.liquibase.com/) is Java-based application that manages running SQL migrations.
* **Pros**
* Allows you to configure your migrations in XML, JSON, or YAML, not just raw SQL
* Thorough [documentation](https://docs.liquibase.com/start/home.html)
* **Cons**
* While it has a free open-source version, many key features like rollbacks look to be locked behind the Pro version
# 30k ft deliverable reporting strategy
* **Status:** Active
* **Last Modified:** 2023-12-18
* **Related Issue:** [#854](https://github.com/HHS/simpler-grants-gov/issues/854)
* **Deciders:**
* Lucas
* Sarah
* Sumi
* Aaron
* Esther
* Billy
* **Tags:** analytics, github
## Context and Problem Statement
In order to report on the progress we're making toward our 30,000 foot deliverables (30k deliverables), we need to be able to connect those deliverables to the ground-level tasks that need to be completed for delivery. Being able to reliably identify which tasks are required for a given deliverable not only enables us to report key metrics like burnup or percent completion for each deliverable, it also allows us to understand which deliverables are being worked on within a given sprint.
Currently, however, we do not have a consistent strategy for indicating which 30k deliverable a given issue is associated with. For custom reporting, we have been using the milestone that an issue is assigned to as a proxy for its 30k deliverable, and for our sprint board, we've been using issue labels to drive filtering and reporting. The fact that we represent this hierarchy differently across reporting channels creates an additional maintenance overhead and increases the likelihood of presenting conflicting information between reports.
Additionally, as the number of 30k deliverables defined for the project grows, we'll need to adjust our strategy for including them in project reporting. By default, we include all 30k deliverables in our reports, but this results in a large chart where in which many deliverables have no points or issues assigned to them. In order to more accurately reflect the status of our deliverables, we need to determine when and how to include them in reporting.
In light of these needs, the goal of this ADR is to:
1. Recommend a consistent strategy for assigning issues to a parent 30k deliverable across reporting channels
2. Recommend an approach to limiting the number of 30k deliverables that appear in our reports
## Decision Drivers
### Assigning issues to deliverables
Our recommended strategy for assigning issues to a given 30k deliverable should consider the following criteria:
* Users assigning an issue to a 30k deliverable should only need to update that value in one place
* Users should be able to search or filter for the issues assigned to a given 30k deliverable
* Users should be able to group by 30k deliverables in the sprint board and in [GitHub insights reporting](https://docs.github.com/en/issues/planning-and-tracking-with-projects/viewing-insights-from-your-project/about-insights-for-projects)
* The relationship between issues and 30k deliverables should be consistently represented across all reports
### Limiting the 30k deliverables included in reporting
Our recommended approach to limiting the deliverables that are included in reporting should consider the following criteria:
* The logic for including a 30k deliverable in reporting should be clear and easy to understand
* Users should be able to add or remove a given 30k deliverable from reports without having to change the underlying report logic or make changes to individual GitHub Issues
* If necessary, users should be able to change the reporting logic without rewriting significant sections of the source code
## Options considered
### Assigning issues to deliverables
* **Milestones per 10k:** Add the issue to a milestone that represents a 10k deliverable and tag the 30k deliverable in the body of the milestone
* **Milestones per 30k:** Add the issue to a milestone that represents a 30k deliverable
* **Labels:** Tag the issue with a label that represents a 30k deliverable
* **Deliverable column:** Add a custom "deliverable" column to the GitHub projects with values representing each 30k deliverable
* **Reserved phrase:** Use a reserved phrase to tag a 30k deliverable in the body of the issue
### Limiting the 30k deliverables included in reporting
* **Label:** Use a label to indicate when a deliverable should be included in reporting
* **Status:** Use the status of the deliverable in the provisional roadmap project
* **GitHub action:** Explicitly list the deliverables to include in the GitHub action for reporting
## Decision outcomes
### Assigning issues to deliverables
We've decided to use a "deliverable" column to assign issues to a parent 30k deliverable. This involves:
* creating a single select column in both the product roadmap and the sprint board
* populating that column with options that represent each 30k deliverables in our roadmap
* using that column to indicate the 30k deliverable that an issue is assigned to
* **Positive outcomes**
* We can filter, group, and sort issues by deliverable in GitHub projects and GitHub's automated insight reporting.
* The engineering team can continue to use GitHub milestones to organize issues into units of work that are smaller than a 30k deliverable. **Note:** While enabling the engineering team to use milestones to organize issues into smaller units of work, we want to avoid situations in which a given milestone contains issues from multiple 30k deliverables -- we can enforce this with a linter as well as a view in the GitHub project.
* The logic for grouping issues by deliverable will be consistent between GitHub insight reports and our custom-built reports.
* **Negative outcomes**
* We won't be able to filter for issues assigned to a given 30k deliverable within the GitHub repository.
* Updating a field on a project can be done from the issue page itself, but the issue first needs to be added to the project before the field is updated. This requires an extra step compared to assigning a label or milestone, making it more prone to be skipped.
* We'll have to make sure the list of options in the deliverable column is consistent across GitHub projects in order to join issues from different projects correctly in our custom reporting.
* We'll have to update the current logic in our custom reporting so that issues are joined to their parent deliverable using the value of this column.
* When creating a new Issue, it's not sufficient to add that Issue to a Milestone; it must also be added to the 30k deliverable for that Milestone for reporting purposes. We will need to monitor for Issues that do not have a 30k and triage them regularly.
> \[!NOTE] If the team finds a consistent need to filter the list of issues by deliverable within the repository, we may revisit this decision and choose to use ***both*** a label and a deliverable column to assign an issue to a 30k deliverable. We've decided **not** to use both options for the time being to avoid having multiple (potentially conflicting) ways of assigning an issue to a given deliverable.
### Limiting the 30k deliverables included in reporting
We've decided to use the status of the deliverable in the product roadmap board to determine when deliverables are included or excluded from reporting.
* **Positive outcomes**
* Keeps our reporting strategy closed aligned with how we're monitoring our delivery progress.
* Avoids creating *another* field or label that needs to be maintained separately.
* **Negative outcomes**
* Tightly couples reporting logic with delivery management, which may introduce challenges if there are instances in which we want to report on some but not all deliverables in a given status.
* If stakeholders aren't familiar with the relationship between deliverable status and reporting, it may be harder for them to understand why certain deliverables appear in a given report but others do not.
## Evaluation - Assigning issues to deliverables
### Comparison matrix
| Factor | Milestone per 10k | Milestone per 30k | Label | Deliverable column | Reserved phrase |
| ------------------------------------- | :---------------: | :---------------: | :---: | :----------------: | :-------------: |
| Supports filtering in repo | ❌ | ✅ | ✅ | ❌ | ❌ |
| Supports filtering in GH project | ❌ | ✅ | ✅ | ✅ | ❌ |
| Supports grouping in GH project | ❌ | ✅ | ❌ | ✅ | ❌ |
| Supports grouping in GH reporting | ❌ | ✅ | ❌ | ✅ | ❌ |
| Supports custom sorting in GH project | ❌ | ✅ | ❌ | ✅ | ❌ |
| Supports custom reporting | ✅ | ✅ | ✅ | ✅ | ✅ |
| Enables engineers to organize issues | ✅ | ❌ | ✅ | ✅ | ✅ |
| # of steps to create a deliverable | 5 | 5 | 5 | 6 | 3 |
| # of steps to rename a deliverable | 1 | 2 | 2 | 3 | 1 |
### Milestones per 10k deliverable
This option involves creating GitHub milestones that loosely map to the 10k deliverables under a given 30k deliverable. Each milestone then uses a key phrase `maps to 30k ft deliverable: #{issue number}` to indicate which 30k deliverable it rolls up to, and each issue assigned to that milestone is counted toward the 30k deliverable that is tagged.
> \[!TIP] **Bottom line:** This option works for custom reporting, but is probably not the best option moving forward because it doesn't allow us to easily group, search, and filter issues by 30k deliverable in GitHub projects or reporting.
#### Examples
* [Milestone mapped to 10k deliverable](https://github.com/widal001/project-demo/milestone/11)
* [Issue assigned to this milestone](https://github.com/widal001/project-demo/issues/59) (see milestone section on the right side)
#### Steps to create a 30k deliverable and assign an issue to it
1. Create a 30k deliverable issue
2. Create a milestone that corresponds to a 10k deliverable.
3. Tag the number of the 30k deliverable it rolls up to in the body of the milestone using the phrase "Maps to 30k ft deliverable: #{issue number}"
4. Create an issue representing a task that is required for a given 30k deliverable
5. Assign that issue to the same milestone
#### Steps to rename a 30k deliverable
1. Rename the 30k deliverable issue
#### Pros and cons
* **Pros**
* Most closely aligns with our current strategy for assigning issues to 30k deliverables.
* Only requires users to associate milestones to 30k deliverables, instead of each individual issue.
* **Cons**
* Does not support filtering or searching by 30k deliverable within the GitHub repository.
* Does not support filtering or searching by 30k deliverable within GitHub projects (e.g. roadmap or sprint board).
* Does not support grouping by 30k deliverable within GitHub projects or GitHub insight reporting.
### Milestone per 30k deliverable
This option involves creating a milestone for each 30k deliverable and assigning issues to that milestone if they are required for delivery of that 30k.
> \[!TIP] **Bottom line:** This is the best option if we:
>
> * want a consistent way to filter and group issues by deliverable across all GitHub projects and reporting as well as within the repository
> * but are okay with preventing the engineering team from using milestones to organize issues into smaller units of work
#### Examples
* [Milestone mapped to 30k deliverable](https://github.com/widal001/project-demo/milestone/10)
* [Issue assigned to this milestone](https://github.com/widal001/project-demo/issues/70)
* [Searching for issues with this milestone in the repo](https://github.com/widal001/project-demo/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22API+soft+launch%22)
* [Searching for issues with this milestone in the GitHub project](https://github.com/users/widal001/projects/3/views/1?filterQuery=milestone%3A%22API+soft+launch%22)
* [Grouping issues by milestone in the GitHub project](https://github.com/users/widal001/projects/3/views/6)
* [Grouping issues by milestone in GitHub insight reporting](https://github.com/users/widal001/projects/3/insights/3)
#### Steps to create a 30k deliverable and assign an issue to it
1. Create a 30k deliverable issue
2. Create a milestone that corresponds to this 30k deliverable
3. Assign the 30k deliverable issue to that milestone
4. Create an issue representing a task that is required for a given 30k deliverable
5. Assign that issue to the same milestone
#### Steps to rename a 30k deliverable
1. Rename the 30k deliverable issue
2. Rename the milestone that corresponds to the 30k deliverable
#### Pros and cons
* **Pros**
* Ensures that there is just one source of truth for the list of issues associated with a given 30k deliverable.
* Supports filtering both within the repository and within GitHub projects (e.g. roadmap and sprint board).
* Supports grouping both within the GitHub projects and within GitHub insight reporting.
* **Cons**
* Prevents the engineering team from using GitHub milestones to organize and group related issues outside of the 30k deliverable framework.
* Milestones that represent 30k deliverables will have a *lot* of tickets unless we work on downscoping 30k deliverables.
* Does not support customizing the order of the groups on the sprint board when grouping by milestone -- milestones can only be sorted alphabetically.
### Label
This option involves creating a label for each 30k deliverable with a consistent prefix (e.g. `30k: Public launch`) and then applying this label to each of the issues required for delivery of that 30k.
> \[!TIP] **Bottom line:** This is the best option if:
>
> * we want a consistent way to filter issues by deliverable in the repository and across GitHub projects
> * but can compromise on being able to **group** by deliverable in GitHub projects or reporting
#### Examples
* [Issue with this label](https://github.com/widal001/project-demo/issues/75)
* [Searching for issues with this label in the repo](https://github.com/widal001/project-demo/issues?q=is%3Aissue+is%3Aopen+label%3A%2230k%3A+API+soft+launch%22)
* [Searching for issues with this label in the GitHub project](https://github.com/users/widal001/projects/3/views/1?filterQuery=label%3A%2230k%3A+API+soft+launch%22)
* [Attempting to group by label in GitHub insight reporting](https://github.com/users/widal001/projects/3/insights/5)
#### Steps to create a 30k deliverable and assign an issue to it
1. Create a 30k deliverable issue
2. Create a label that represents this 30k deliverable
3. Apply that label to the 30k deliverable issue
4. Create an issue representing a task that is required for a given 30k deliverable
5. Apply that label to the task-level issue
#### Steps to rename a 30k deliverable
1. Rename the 30k deliverable issue
2. Rename the label that corresponds to the 30k deliverable
#### Pros and cons
* **Pros**
* Allows the engineering team to continue to use GitHub milestones to organize issues into units of work that make sense to them.
* Supports filtering both within the repository and within GitHub projects (e.g. roadmap and sprint board).
* **Cons**
* Does not easily support grouping in GitHub projects or GitHub insights reporting.
* Will result in a large number of labels if 30k labels aren't deleted once a 30k deliverable is complete. And if we delete old deliverable labels, it will impact our ability to report on past deliverables.
### Deliverable column
This option involves creating a single select "deliverable" column in the GitHub projects, with values for each 30k deliverable, and then selecting the corresponding "deliverable" value for each issue added to the project.
> \[!TIP] **Bottom line:** This is the best option if:
>
> * we want a consistent way of filtering, grouping, and sorting issues by deliverable in GitHub projects and reporting
> * but can compromise on being able to filter by deliverable in the **repository**
#### Examples
* [Issue with this deliverable value](https://github.com/widal001/project-demo/issues/65) (Click to expand the down arrow to expand the "Demo sprint board" project details)
* [Searching for issues with this deliverable in the GitHub project](https://github.com/users/widal001/projects/3/views/1?filterQuery=deliverable%3A%22Static+site+launch%22)
* [Grouping issues by deliverable in GitHub project](https://github.com/users/widal001/projects/3/views/7)
* [Grouping issues by deliverable in GitHub insight reporting](https://github.com/users/widal001/projects/3/insights/2)
* [Slicing by deliverable and grouping by milestone](https://github.com/users/widal001/projects/3/views/8?sliceBy%5Bvalue%5D=Open+source+group+kickoff)
#### Steps to create a 30k deliverable and assign an issue to it
1. Create a 30k deliverable issue
2. Add a new value for this 30k deliverable to the deliverable columns in:
* The product roadmap GitHub project
* The sprint planning GitHub project
3. Choose that deliverable value for the 30k deliverable issue in the product roadmap
4. Create an issue representing a task that is required for a given 30k deliverable
5. Choose that deliverable value for the task-level issue in the product roadmap
#### Steps to rename a 30k deliverable
1. Rename the 30k deliverable issue
2. Rename the deliverable value that corresponds to the 30k deliverable in:
* The product roadmap GitHub project
* The sprint planning GitHub project
#### Pros and cons
* **Pros**
* Allows the engineering team to continue to use GitHub milestones to organize issues into units of work that make sense to them.
* Supports filtering within GitHub projects (e.g. roadmap and sprint board).
* Supports grouping within GitHub projects and within GitHub insight reporting.
* Supports customizing the order of groups on the sprint board when grouping by deliverable -- for example we can make "Static site public launch" appear above "GET Opportunities" even though GET opportunities would be first alphabetically.
* **Cons**
* Does not support filtering or searching within the GitHub repository.
* Will result in a large number of values for the deliverable column if not regularly maintained. And if we delete old deliverable values it will impact our ability to report on past deliverables.
* List of deliverable column values will have to be maintained separately across projects -- inconsistencies between these lists may introduce bugs in custom reporting. Though we could address this by creating a linter and the discrepancy would be easy to notice the next time the report is run.
### Reserved phrase
This option involves tagging a 30k deliverable from the body of each issue using a reserved phrase (e.g. "30k deliverable: #123"). It would function much like \[linking a pull request to an issue]\[linking pull requests] does in GitHub.
> \[!TIP] **Bottom line:** Probably not the best option unless we only cared about grouping issues by deliverable in custom reporting.
#### Examples
* [Issue with reserved phrase](https://github.com/widal001/project-demo/issues/73)
#### Steps to create a 30k deliverable and assign an issue to it
1. Create a 30k deliverable issue
2. Create an issue representing a task that is required for a given 30k deliverable
3. Tag that 30k deliverable in the body of the task using a reserved phrase
#### Steps to rename a 30k deliverable
1. Rename the 30k deliverable issue
#### Pros and cons
* **Pros**
* Allows the engineering team to continue to use GitHub milestones to organize issues into units of work that make sense to them.
* Easy to parse in custom reporting.
* **Cons**
* Does not support filtering or searching within the GitHub repository.
* Does not support filtering or searching within GitHub projects (e.g. roadmap or sprint board).
* Does not support grouping within GitHub projects or GitHub insight reporting.
* Very difficult to audit or maintain programmatically.
## Evaluation - Limiting 30k deliverables in reporting
### Label
This option involves creating a label that indicates when a deliverable should be included in reporting (e.g. "add to report") and then filtering for this label in our custom reporting.
> \[!NOTE] The code should be configured to accept an arbitrary set of labels as filters, so that if the label changes, we only need to change the top-level configuration and not several lines of the source code.
> \[!TIP] **Bottom line:** This is the best option if we:
>
> * want an easy way to explicitly indicate when a deliverable should be included or excluded from reporting
> * and are okay with decoupling the reporting logic from the business logic around deliverable status
* **Pros**
* Makes the choice to include a deliverable in a report explicit and easy to understand.
* Enables users to add or remove deliverables from reporting without changing the logic or the source code.
* Makes it easy to change the logic that determines which issue labels are used to include/exclude issues from reporting.
* **Cons**
* Decouples reporting logic from the status of ongoing deliverables. **Note:** this could be a pro or a con depending on how tightly coupled we want reporting and delivery management to be.
* Adds to the already long list of labels in the repository.
### Status
This option involves filtering the deliverables that are included in reporting based on their status in the provisional roadmap project. For example, we may only want to report on deliverables that have the status "In progress" or "Planning".
> \[!NOTE] The code should be configured to accept an arbitrary set of statuses as filters, so that if we want to change which statuses are included, we only need to change the top-level configuration and not several lines of the source code.
> \[!TIP] **Bottom line:** This is the best option if we:
>
> * want deliverables to automatically be included in reporting once they enter a specific status (e.g. "in progress")
> * and we don't anticipate edge cases that require including or excluding deliverables from reporting despite their status
* **Pros**
* Enables users to add or remove deliverables from reporting without changing the logic or the source code.
* Makes it easy to change the logic that determines which statuses are used to include/exclude issues from reporting.
* Keeps the reporting and delivery management aligned by automatically updating our reports to reflect the deliverables we're currently working on.
* **Cons**
* Logic behind which deliverables are included in reporting is not as explicit as having a dedicated label.
* Requires the reporting logic to be tightly coupled with the rules for assigning deliverable status.
### GitHub action
This option involves specifying the deliverables we want to include in the reporting within the GitHub action itself, either by name or by issue number.
> \[!NOTE] The code should be configured to accept an arbitrary list of issue titles or numbers, so that if we want to change which deliverables are included, we only need to change the top-level configuration and not several lines of the source code.
> \[!TIP] **Bottom line:** This is probably not a good option unless we want to version control when we add or remove deliverables from reporting.
* **Pros**
* Makes the choice to include a deliverable in a report explicit and easy to understand.
* Prevents someone from accidentally removing a deliverable from reporting by changing a label or status.
* **Cons**
* Decouples reporting logic from the status of ongoing deliverables. **Note:** this could be a pro or a con depending on how tightly coupled we want reporting and delivery management to be.
* Does not allow users to add or remove deliverables from reporting without changing a configuration file.
* Prone to error if users enter the wrong issue title or number.
## Links
* [Linking pull requests to issues](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue)
* [GitHub insights reporting](https://docs.github.com/en/issues/planning-and-tracking-with-projects/viewing-insights-from-your-project/about-insights-for-projects)
# Public measurement dashboard architecture
* **Status:** Active
* **Last Modified:** 2023-12-18
* **Related Issue:** [#845](https://github.com/HHS/simpler-grants-gov/issues/845)
* **Deciders:**
* Lucas
* Aaron
* Michael
* Billy
* **Tags:** analytics, data
## Context and Problem Statement
One of the goals for this project is to make all of the Simpler.Grants.gov operational metrics publicly available and to publish as much non-sensitive data about the grants pipeline as possible. In order to help internal and external stakeholders make sense of this data, we'll need to create and maintain a set of public measurement dashboards that summarize and contextualize the key metrics we track in the project.
There are a number of architectural patterns we can follow to build and publish a set of dashboards, and the pattern we choose will influence how we set up our data engineering pipeline, as well as our infrastructure for analytics and business intelligence (ABI).
The goal of this ADR is to evaluate and recommend an architectural approach for our initial public measurement dashboard deliverable, which will focus on sprint and delivery metrics. However, this recommendation should also create a flexible yet robust foundation for future dashboards as well.
## Decision Drivers
#### Must have
* Members of the public can access the dashboard without a login
* Multiple related dashboards can be combined into a single "application" for external users
* New charts and dashboards are easy to implement
* Users can export or download the data behind a given dashboard
* Open source contributors can host a local version of our dashboards using exported data
#### Nice to have
* The platform used to build dashboards can also be used to answer ad-hoc questions about the data
* System-to-system users can programmatically access the data behind a given dashboard via API
* Dashboards can include narrative text that explains the metrics we publish
## Options Considered
* S3 bucket + dashboard user interface
* Analytics API + dashboard user interface
* Custom dashboard application (e.g. [Dash](https://dash.plotly.com/) or [R Shiny](https://shiny.posit.co/))
* Open source dashboard solution (e.g. [Metabase](https://www.metabase.com/) or [Redash](https://redash.io/))
* SaaS dashboard solution (e.g. [PowerBI](https://www.microsoft.com/en-us/power-platform/products/power-bi), [Tableau](https://www.tableau.com/trial/tableau-software), [Looker](https://cloud.google.com/looker), [Amazon Quicksight](https://aws.amazon.com/quicksight/), or [Domo](https://www.domo.com/business-intelligence))
> \[!NOTE] This ADR does not aim to recommend a *specific* platform or tool, but rather a general architectural approach to building and publishing dashboards. Once we select an approach from the list above (e.g. open source dashboard solution), we'll need to create a separate ADR that evaluates the options available within that approach (e.g. Metabase vs Redash).
## Decision Outcome
### Long-term
Our recommendation for a long-term dashboard architecture is to combine **analytics API + dashboard UI** with an **open source dashboard solution** in a way that allows us to quickly iterate on and test the value of new metrics or charts, then "promote" the ones we find most helpful to an official public dashboard.
The steps for this promotion strategy would likely include:
1. **ETL:** Write the data needed to calculate operational and program metrics to a data warehouse via an ETL pipeline.
2. **Ad hoc report:** Prototype a new metric by building an ad hoc SQL report in an open source dashboard solution (e.g. Metabase or Redash) connected to that data warehouse.
3. **Temporary dashboard:** If that ad hoc report is useful to stakeholders, then incorporate it into a temporary dashboard within the open source dashboard tool.
4. **Public dashboard and endpoint:** Once we get feedback on this temporary dashboard, formalize the metric in a new (or modified) API endpoint that is consumed by a new (or modified) page in the public-facing dashboard application.
### Short-term
Because setting up the infrastructure for the long-term dashboard architecture will take a significant amount of time and resources, in the short-term, we recommend following the **s3 bucket + dashboard UI** option. This approach reduces the upfront infrastructure investment needed to publish an initial dashboard, while still laying the foundation for parts of the long-term approach.
The steps for publishing a dashboard using this short-term strategy would likely include:
1. **ETL:** Use a simple ETL pipeline (e.g. scheduled GitHub action) to:
1. Extract data needed to calculate operational and program metrics from external sources.
2. Calculate the metrics we want to add to a dashboard.
3. Load both the source data and the metrics results to s3 buckets.
2. **Dashboard UI:** Load the metrics from s3 and visualize them in a static UI (e.g. Jupyter notebook, static site, GitBook page) that gets refreshed with new data after the ETL pipeline runs.
### Positive Consequences
* Minimizes the upfront investment in infrastructure needed to publish our first dashboard publicly.
* Enables us to experiment with different ETL and dashboard UI options that will help narrow the decision for the long-term architecture.
* In the long-term, provides a pipeline for testing and getting feedback on a new metric before "promoting" it to our public dashboard.
### Negative Consequences
* In the short-term, requires a higher level technical expertise to build even basic dashboards.
* In the short-term, does not easily support ad hoc reporting that is accessible to non-engineers.
* In the short-term, because s3 does not provide fine-tuned access control to publicly accessible buckets, we'll have to limit our metrics to non-sensitive data or post only the aggregated results of metrics calculated from sensitive data.
* Once the long-term approach has been adopted, it may require reimplementing some ETL pipelines and/or dashboards that were built in the short-term.
## Pros and Cons of the Options
### S3 bucket + user interface
This option involves running an analytics pipeline on a regular basis (likely once per day) then writing the results of the analysis to an s3 bucket as a static file (e.g. JSON, csv, etc.). That same pipeline, or a separate pipeline, could also refresh a standalone dashboard UI (e.g. Jupyter notebook, static site, etc.) which reads data from that s3 bucket and visualizes the results in a series of charts.
> \[!TIP] **Bottom line:** This option is best if:
>
> * we want an easy-to-implement solution that gives us control over the look and feel of the dashboard *and* enables users to access the underlying data,
> * but we are willing to wait to adopt a more sustainable long-term solution that provides support for adhoc reporting and/or a more robust analytics API.
* **Pros**
* Requires the least amount of upfront investment in infrastructure to build and publish a dashboard.
* Provides more fine-grained control over the look and feel of the dashboard.
* Enables S2S users to access underlying analytics data from the S3 bucket.
* Allows us to include narrative text that explains the metrics we publish.
* Enables open source contributors to host local versions of our dashboard or easily create their own dashboards.
* Aligns with the open source values and approach of the project.
* Creates a foundation for other options, such as the Analytics API + dashboard UI.
* **Cons**
* Building or modifying dashboards is still quite technical, not something a business analyst could do on their own.
* Harder to implement new charts or dashboards than SaaS or open source dashboard solution.
* Sharing data via s3 bucket does not provide as good a developer experience as sharing data via a well-designed analytics API.
* Does not easily support adhoc reports or dashboards in the same tool.
* Not a sustainable option for a long-term data and analytics platform.
### Analytics API + user interface
This option involves building both a custom analytics API that will serve the data behind our key project metrics and a separate user interface to consume from that API and render these metrics as a dashboard for end users. Following this approach, we could choose to re-use our existing front-end and backend infrastructure or make slightly different tooling choices based on our needs.
> \[!TIP] **Bottom line:** This option is best if:
>
> * we want to follow the existing architecture pattern of Simpler.Grants.gov *and* have maximum control over the structure and design of the public dashboard,
> * but we are willing to dedicate more engineering resources to building and hosting this dashboard *and* don't need to update the analytics endpoints or dashboards frequently.
* **Pros**
* Provides the most fine-grained control over the look and feel of the dashboard application.
* Enables us to combine multiple dashboards into a single application.
* Enables S2S users to access underlying analytics data via API.
* Allows us to include narrative text that explains the metrics we publish.
* Enables open source contributors to host local versions of our dashboard or easily create their own dashboards.
* Aligns with the open source values and approach of the project.
* Data endpoints are simple and easy to understand.
* **Cons**
* Building or modifying dashboards is quite technical, not something a business analyst could do on their own.
* Publishing new charts or dashboards requires making changes both to the API and to the dashboard UI separately.
* Hardest to maintain of all of the options considered.
* Does not easily support adhoc reports or dashboards without using a separate tool.
### Custom dashboard app
This option involves building a custom dashboard app using an existing framework, such as [Dash](https://dash.plotly.com/) (python) or [R Shiny](https://shiny.posit.co/). This custom built application would be hosted as its own application and be slightly more integrated than a separate analytics API + dashboard UI, but it would be more customizable than using an open source or proprietary dashboard solution.
> \[!TIP] **Bottom line:** This option would be best if:
>
> * we want to retain control over the look and feel of the dashboard *and* to manage the backend and frontend of the dashboard together,
> * but we are willing to commit more engineering resources to building and hosting it *and* don't need to expose analytics data via API.
* **Pros**
* Provides more fine-grained control over the look and feel of the dashboard application, on par with option 1.
* Allows a single engineer familiar with Python (or R if we use Shiny) to manage both the backend (e.g. analytics) and frontend (visualization and design of the dashboard).
* Faster to implement new charts and dashboards than an analytics API and separate frontend.
* Enables us to combine multiple dashboards into a single application.
* Allows open source contributors to host local versions of our dashboards.
* Aligns with the open source values and approach of the project.
* **Cons**
* Building or modifying dashboards is still quite technical, not something a business analyst could do on their own.
* Harder to implement new charts or dashboards than SaaS or open source dashboard solution.
* Does not easily support adhoc reports or dashboards in the same tool.
* Does not easily support exposing analytics data to S2S users via API.
### Open source dashboard solution
This option involves selecting and hosting an open source dashboard solution, such as [Metabase](https://www.metabase.com/) or [Redash](https://redash.io/). This solution would most likely be self-hosted and connect directly to our data warehouse, and enable business analysts to build adhoc reporting and dashboards with SQL and a drag-and-drop interface. Individual dashboards can then be configured for broader publication to external stakeholders.
> \[!TIP] **Bottom line:** This option would be best if:
>
> * we want to adopt an open source solution that enables business analysts to build *and* host dashboards with minimal support from engineers,
> * but we are willing to commit the upfront resources needed to set up the infrastructure *and* can compromise on the amount of control we have over the look and feel of those dashboards.
* **Pros**
* Enables a business analyst or engineer with basic SQL experience to build and manage dashboards.
* Faster and easier to implement new charts and dashboards than options 1 or 2.
* Supports adhoc reports and dashboards in the same tool.
* Allows open source contributors to host local versions of our dashboards (depending on the tool we choose).
* Aligns with the open source values and approach of the project.
* No per user cost to grant edit access to self-hosted version of dashboard solution.
* **Cons**
* Harder to control the look and feel of the resulting dashboards.
* Harder to combine multiple dashboards into a single "application".
* Some solutions offer API access to underlying data, but endpoints are harder to understand and may not be publicly available. For example, here's the [Redash API docs](https://redash.io/help/user-guide/integrations-and-api/api) and the [Metabase API docs](https://www.metabase.com/docs/latest/api-documentation#about-the-metabase-api).
* Still somewhat harder to host and maintain than a fully SaaS dashboard solution.
### SaaS dashboard solution
This solution involves adopting a Software-as-a-Service dashboard solution, such as Tableau or PowerBI, and using this solution to enable business analysts to build adhoc reporting and dashboards with a drag-and-drop interface. Individual dashboards can then be configured for broader publication for external stakeholders.
> \[!TIP] **Bottom line:** This option would be best if:
>
> * we want to adopt an externally hosted solution that enables business analysts to build dashboards with no direct support from engineers,
> * but we are willing to accept a higher per-user cost, closed-source tool, and fewer options for customization.
* **Pros**
* Enables a business analyst to easily build and maintain dashboards, even with minimal SQL or programming experience.
* Faster and easier to implement new charts and dashboards than options 1 or 2.
* Supports adhoc reports and dashboards in the same tool.
* Easier to combine multiple dashboards into a single "application" than open source dashboard solution.
* **Cons**
* Harder to control the look and feel of the resulting dashboards (depending on the tool).
* Harder to version control the structure and content of dashboards (depending on the tool).
* Some solutions offer API access to underlying data, but endpoints are harder to understand and may not be publicly available. For example, here's the [Tableau API docs](https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref.htm) and the [Looker API docs](https://developers.looker.com/api/explorer/4.0/).
* Does not allow open source contributors to host local versions of our dashboards.
* Higher direct costs than all of the other options.
## Links
* [Dash](https://dash.plotly.com/)
* [Shiny](https://shiny.posit.co/)
* [Metabase](https://www.metabase.com/)
* [Redash](https://redash.io/)
* [Tableau](https://www.tableau.com/trial/tableau-software)
* [PowerBI](https://www.microsoft.com/en-us/power-platform/products/power-bi)
* [Looker](https://cloud.google.com/looker)
* [Amazon Quicksight](https://aws.amazon.com/quicksight/)
* [Domo](https://www.domo.com/business-intelligence)
# Method and technology for "Contact Us" CTA
* **Status:** {Active}
* **Last Modified:** 2023-12-20
* **Related Issue:** [#682](https://github.com/HHS/simpler-grants-gov/issues/682)
* **Deciders:** Lucas, Andy, Sumi
* **Tags:**
## Context and Problem Statement
How might we give users an alternative to public comments (GitHub, Google Groups, listserv, etc.) so that they might submit questions or feedback more privately? Is email the appropriate method? And how might we manage this feedback channel?
## Decision Drivers
* The "Contact Us" method should not be (or feel) like a black box.
* Incoming messages should be easy/simple to triage, respond to, and escalate to the appropriate channels.
* Users should have clarity on how their messages are received, processed, and addressed.
* To maintain transparency, it should be possible to implement a feedback loop that keeps users informed of the status of their inquiries (e.g. providing confirmation of receipt, updates on the progress, and resolutions).
## Options Considered
* Self-hosted .gov email address:
*
*
* [Google collaborative inbox](https://support.google.com/a/users/answer/167430?hl=en)
* Help Desk tool (such as):
* [Loop](https://www.intheloop.io/)
* [Front](https://front.com/)
* [Odoo CRM](https://www.odoo.com/app/crm)
## Decision Outcome:
Chosen option: ****, because it is simple and accessible for end users, can be easily managed by comms staff, and matches the Simpler.Grants.gov domain.
### Positive Consequences
* Simple email communication is a technology that most all users are fimiliar and comfortable with.
* Since it does not require creation of accounts, or using 3rd-party ticketing dashboards, email is the most straightforward method by which users can submit their feedback.
* Provisioning an email address is fast/inexpensive and does not require procuring a service subscription.
* echoes the project domain
* @grants.gov is an official/trusted top-level domain (TLD)
* This inbox can be easily triaged by the Simpler.Grants.gov project team via the "[Send emails to Slack](https://slack.com/help/articles/206819278-Send-emails-to-Slack)" feature.
### Negative Consequences
* We do not yet know the volume of submissions that will come through this feedback channel. A more robust Help-Desk-type of solution may be warranted in the future. We should revisit this decision if/when managing this inbox becomes unwieldy.
* There are no built-in, convenient ways to run analyses on an email inbox, or tag and filter messages by categories. It may be difficult to track metrics on common/recurring themes. This may become more of an issue as the number of submissions grows.
## Pros and cons of other options
###
This option has similar benefits as the chosen option. However, we decided that it's better that the TLD of the email match that of the site.
* **Pros**
* (See above chosen option)
* **Cons**
* (See above chosen option)
* Difficult to read
* User may be confused as to why HHS is the TLD
* This @hhs.gov email address is not easily accessible to comms staff
### Google collaborative inbox
Again, this option has similar benefits as the chosen option. However, we decided that it's better to use an official .gov TLD.
* **Pros**
* (See above chosen option)
* **Cons**
* (See above chosen option)
* Not an official/trusted .gov email address
### Help Desk tool
Tools like [Loop](https://www.intheloop.io/), [Front](https://front.com/), [Odoo CRM](https://www.odoo.com/app/crm), and others were given cursory consideration. There are many various benefits. However, it is not yet know whether their functionalities are necessary. These options should be reevaluated if managing the email inbox becomes unwieldy.
* **Pros**
* Automated workflows
* Assigning, tracking, status
* Labels, filters
* Reporting and insights
* Metrics on user satisfaction
* **Cons**
* Additional costs and/or maintenance
* Ticketing system are typically used by large support teams and can sometimes feel like a black box or impersonal to users
# E2E / Integration Testing Framework
* **Status:** Accepted
* **Last Modified:** 2024-02-26
* **Related Issue:** [#1337](https://github.com/HHS/simpler-grants-gov/issues/1337)
* **Deciders:** Billy, Ryan
* **Tags:** e2e, integration, testing
## Context and Problem Statement
An end-to-end (E2E) testing framework can be used to programmatically test specific flows against the entire application. The goal of E2E testing is to simulate scenarios users will encounter in as close to a production-like environment as possible. This ADR is meant to to evaluate E2E frameworks based on the factors outlined below.
## Decision Drivers
* **Speed:** Tests should be able to be run efficiently regardless of where they will be run (e.g. as part of a continuous integration workflow or during local development).
* **Developer experience:** The E2E framework should provide robust documentation and resources, including thorough configuration instructions and debugging tooling to easily identify how tests are failing or are potentially flakey.
* **Well-maintained:** The E2E framework is well-maintained by owners and keeps up with current ecosystems in which it will be integrated.
* **Ease of use**: Individual contributors should be able to quickly and effectively write new tests to verify functionality for specific user flows.
## Considered Options
* [Cypress](https://www.cypress.io/)
* [Playwright](https://playwright.dev/)
## Decision Outcome
Playwright's modern syntax, variety of tracing tools, and speed makes for an ideal developer experience, which increases in the case that a developer's IDE of choice is VS Code. It also supports a variety of browsers and platforms and efficiently spins up new browser contexts for each test. Cypress has considerable documentation and an extensive commmunity, which more than makes up for how it handles async/sync code. If there isn't a specific need that warrants E2E tests be run in browser or adversion to a Microsoft-backed open-source project, then the performance gains that Playwright offers is reason enough to choose it as the E2E framework.
## Pros and Cons of the Options
### Cypress
#### Pros
* Runs in browser, which means test code is evaluated with JavaScript as opposed to Node or another server-side language and there's native access to whatever portion of the application is being tested
* Extensive support for reusability, including creating shortcuts to recreating specific application states
* Plenty of examples and tutorials to refer to as this framework has been in existence for many years
#### Cons
* Runs in browser, which means interacting with a DB or other backend service requires extra work. For instance, methods would need to be exposed in order to seed a DB.
* Cypress handles chains of commands such that mixing async and sync code is difficult to read and understand without familiarity of its syntax (e.g. promise chaining as opposed to a more readable async/await approach)
* Limited iFrame, multiple tabs, and hover support
* Testing on Safari browsers is experimental and relies on Playwright
### Playwright
#### Pros
* Tests are run in parallel by default and benchmark speeds outpace Cypress
* Can run tests in headless, headed, or UI mode depending on use case and on Windows, Linux, and macOS with support for Chromium, WebKit and Firefox
* Supports native mobile emulation of Google Chrome for Android and Mobile Safari
* Backed by Microsoft and has a powerful VS Code extension to debug tests within the IDE
* Uses async/await syntax and locator methods return elements
#### Cons
* Many of the debugging and ease-of-use functionality leverages VS Code, which means developers who prefer another IDE might be less effective at debugging tests
## Links
* [Next.js Testing Overivew](https://nextjs.org/docs/app/building-your-application/testing)
* [Cypress vs Selenium vs Playwright vs Puppeteer speed comparison](https://www.checklyhq.com/blog/cypress-vs-selenium-vs-playwright-vs-puppeteer-speed-comparison/)
* [Playwright vs. Cypress](https://www.qawolf.com/blog/why-qa-wolf-chose-playwright-over-cypress)
# Logging and Monitoring Platform
* **Status:** Active
* **Last Modified:** 2024-03-04
* **Related Issue:** [#630](https://github.com/HHS/simpler-grants-gov/issues/630)
* **Deciders:** Lucas and/or Billy
## Context and Problem Statement
We want to decide on our long-term logging and monitoring platform. The platform should meet a wide variety of needs, and be highly usable when meeting those needs.
## Decision Drivers
* Platform UX: we want a platform that is easy to use and learn, for a variety of roles in the organization
* Capabilities: we want a platform that satisfies a variety of production operations needs
* Cost: we want a cost-effective platform. Note that this ADR does not attempt to calculate the prices of various platforms directly.
* (...?)
## Options
* Cloudwatch
* Sentry
* Datadog
* New Relic
* Splunk
* Grafana
### Cloudwatch
Cloudwatch is a built-in monitoring platform that comes for free with AWS. As such, it wins on the "cost-effectiveness" decision driver. Unfortunately, Cloudwatch is only an ideal solution if you are significantly cost-constrained. As a free platform, and as a part of AWS's massive product offering, there isn't much motivation to keep Cloudwatch's feature set competitive with the market. Cloudwatch gets the worst score in the realm of usability, it is challenging to find what you need in Cloudwatch, which makes it an inefficient production operations platform. By contrast, Cloudwatch's generous free tier, and the fact that it's active in AWS by default, make it a good security and compliance platform. The recommendation for Cloudwatch is to use it exclusively for security and compliance purposes - and leverage its cost-effectiveness to invest in another platform that will be more effective at actual production operations.
* **Decision Status**: Not Recommended
* **Pros**
* Generous free tier
* **Cons**
* Abysmal UX
Links:
* the first 3 social links for "Cloudwatch UX" are about how bad it is [1](https://news.ycombinator.com/item?id=18584679), [2](https://www.reddit.com/r/aws/comments/nmsapj/this_cloudwatch_ui_sucks/), [3](https://news.ycombinator.com/item?id=18550722)
* [Pricing](https://aws.amazon.com/cloudwatch/pricing/)
### Sentry
Sentry is a frontend-focused APM (application performance monitoring) platform that has not (yet) expanded to become a logging and metrics platform. It is specifically focused on frontend exception and error handling, as well as performance. As simpler.grants.gov is an API-driven platform, Sentry is a non-ideal choice for our first production operations platform. That said, Sentry fulfills its role very effectively, and with fantastic UX. So Sentry might be a good platform to consider if we decide down the line that we need additional monitoring coverage on the frontend. Sentry is a fairly popular product but is not yet listed by [FedRAMP](https://marketplace.fedramp.gov/products).
* **Decision Status**: Not Recommended (at this time)
* **Pros**
* Great UX
* **Cons**
* Not a fully featured platform
### Datadog
Datadog is a fully featured platform with support for logging, metrics dashboards, and APM. Several members of our team have prior experience with Datadog. Datadog excels in the market, is listed on [FedRAMP](https://marketplace.fedramp.gov/products), and would be an excellent choice of platform. It has fantastic UX, for both backend and frontend applications. The only "flaw" is that, at a high level, Datadog's product offering is hard to distinguish from New Relic's. The bulk of this paragraph is copied into the New Relic description, to emphasize that point.
One large advantage of Datadog is that it was built around its metrics and dashboard capabilities, and was built for API-driven applications. As such, Datadog has best-in-class dashboard functionality. This gives it a slight leg-up relative to New Relic. Datadog additionally can make dashboards public, which is a functionality that we may want to leverage to expose our API status (4XX / 5XX rate, etc) to the general public.
Grants.gov used Datadog in the past and then decided to move to Cloudwatch, due to cost.
* **Decision Status**: Top Choice
* **Pros**
* Well-known to our team
* Fully featured platform (logs, metrics, APM)
* Strong metrics product
* Best for API-driven applications
* **Cons**
* Likely more expensive than its closest competitor
Links:
* [Pricing](https://www.datadoghq.com/pricing/)
### New Relic
New Relic is a fully featured platform with support for logging, metrics dashboards, and APM. Several members of our team have prior experience with New Relic. New Relic excels in the market, is listed on [FedRAMP](https://marketplace.fedramp.gov/products), and would be an excellent choice of platform. It has fantastic UX, for both backend and frontend applications. The only "flaw" is that, at a high level, New Relic's product offering is hard to distinguish from Datadog's. The bulk of this paragraph is copied into the Datadog description, to emphasize that point.
New Relic has a free tier, and their pricing is less complex than Datadog's.
New Relic was built around its APM capability, so it will have better capabilities for debugging performance issues and hunting down esoteric production bugs.
* **Decision Status**: Top Choice
* **Pros**
* Well-known to our team
* Fully featured platform (logs, metrics, APM)
* Free tier, simpler pricing than the closest competitor
* Strong APM product
Links:
* [Pricing](https://newrelic.com/pricing)
### Splunk
Splunk is a fully featured platform with support for logging, metrics dashboards, and APM. At least one member of our current team has used Splunk, and would not recommend it. Splunk is listed on [FedRAMP](https://marketplace.fedramp.gov/products), and would likely be a good choice of platform. There are some notable high-level differences between Splunk and \[New Relic or Datadog]. Splunk is a fully featured platform yes, but at its core, it's an enterprise data-driven platform. Splunk is best at helping understand data flow in large and complex applications. Simpler.grants.gov is not a data-driven platform and is not looking to reach an "enterprise application" scale. Therefore, Splunk's core product offering would likely be slightly mismatched for our needs, despite being outwardly very similar to \[New Relic or Datadog].
* **Decision Status**: Viable Option
* **Pros**
* Fully featured platform (logs, metrics, APM)
* **Cons**
* Platform features not designed for our use case
### Grafana
Grafana is a fully open-source metrics platform that comes with other services that handle functionalities like logging and APM. Deploying the full suite of Grafana Labs tooling can provide you with a FOSS platform that is feature-competitive with all the other closed-source platforms on this ADR. The open-source nature comes with high costs to both UX and deployment, though. The UX of Grafana is somewhere halfway between Cloudwatch and Datadog. Grafana has managed options, but ultimately it's a platform built for self-hosting. As such, Grafana as a platform works best for companies that have an infrastructure team that can perform an on-call rotation to support it. At the time of writing, Simpler.grants.gov has no on-call rotation, which is a strong point against our use of a platform like Grafana.
* **Decision Status**: Not Recommended (at this time)
* **Pros**
* Open-source
* **Cons**
* High maintenance and training burden
* Mediocre UX
## Decision Status
Having collected all this information, it is the opinion of the author that both New Relic and Datadog would be strong choices. New Relic is the safer option due to its less complex pricing scheme. Datadog is more likely to have niche features that we find incredibly valuable (like public dashboards). We should move forward from here with a New Relic trial, and consider an additional Datadog trial if New Relic turns out non-ideal in some way.
Here are some blog posts comparing Datadog with New Relic. This ADR is generally aligned with the contents of these posts. [1](https://betterstack.com/community/comparisons/datadog-vs-newrelic/) [2](https://sematext.com/blog/datadog-vs-new-relic/) [3](https://www.upguard.com/blog/datadog-vs-newrelic) [4](https://signoz.io/blog/datadog-vs-newrelic/)
# Dashboard Data Storage
* **Status:** Active
* **Last Modified:** 2024-03-19
* **Related Issue:** [#1506](https://github.com/HHS/simpler-grants-gov/issues/1506)
* **Deciders:** Aaron, Billy
## Context and Problem Statement
We want to determine the data storage solution for our upcoming business intelligence dashboard project.
This data storage solution will be our canonical source of truth for many types of imported data. These data types will range from thousands of records a month (ex. sprint data), up to potentially millions of records of records (ex. Google Analytics) once this project reaches maturity. The chosen solution should be easy to use, cost-effective, and performant for all of our in-scope data types. The types of data we will be importing include:
* Google Analytics
* Grants program data, eg.
* Grants opportunities from the database
* Grant applications from S3
* USA spending data CSVs
* Communication platform stats, eg.
* Slack stats
* Google Group stats
* GitHub stats
* API application metrics
* API infrastructure metrics from Cloudwatch
We will not be importing all of these types of data immediately. On the 0 - 6 month timeframe, we will only likely only be importing the smaller datasets (thousands of records), with the exception of the infrastructure metrics. By 2 - 5 years we will be importing all of these types of data, and our data size will be quite large (many millions of records). The desired solutions have different cost/performance characteristics in those time ranges, and we will need to evaluate those differences.
That said, with respect to data size, there is still a large outstanding question. For those sources of data, do we want to ingest point level data (e.g. individual page views, clicks, and API calls) or do we want to do some level of aggregation before loading it into our data warehouse. Choosing to pre-aggregate the data would decrease the total size of our data across every time range.
## Decision Drivers
* cost
* ease of use
* performance
All of these options should be evaluated for 0 - 6 months, 6 months - 2 years, 2 years - 5 years
## Options Considered
* S3
* Redshift
* Postgres
* Snowflake
## Pros and Cons of the Options
### S3
AWS S3 is a file storage system, traditionally used to store data in the form of individual documents. S3 is best for storing data types formatted as individual documents, or data types where all of the relevant data can be stored within a single file. Due to that, S3 becomes non-ideal for anything in the millions of records, particularly when those records are split up across multiple files. S3 is best in a business intelligence context when used with small datasets, such as our GitHub data. Large data sets require a query layer to be integrated into S3, a query layer like AWS Athena or AWS Redshift. For large enough data sizes, performance starts to become the key issue with S3.
[Data is hosted in S3 at $0.023 per GB-month](https://aws.amazon.com/s3/pricing/).
At 0 - 6 months, S3 is a reasonable choice due to our small data sizes. Past that point, performance issues with large data sizes make S3 a non-ideal choice.
At 2 - 5 years, S3's performance issues require the introduction of another query / compute layer like AWS Athena or AWS Redshift.
### Redshift
AWS Redshift is a Postgres-based data warehouse that you can use to store large data sets. This ADR assumes we are going to be using Redshift Serverless, rather than statically provisioned Redshift. We assume that serverless technologies will work in this case because this is a data warehouse backed by an ELT process, so it's data is not being accessed most of the time. Redshift is optimized for OLAP queries, which makes it an ideal choice for analyzing business analytics data. To query CSV data in Redshift, you must first upload it to S3 and then into Redshift, which makes Redshift feel like a "query layer" for S3.
[Data is hosted in Redshift at $0.024 per GB-month](https://aws.amazon.com/redshift/pricing/), essentially the same price as S3.
At 0 - 6 months, Redshift loses to S3 due to S3 being vastly easier to set up and configure.
At 6 months - 2 years, Redshift starts to beat S3 as the initial setup cost has already been paid, and Redshift will start to show performance advantages relative to S3.
At 2 - 5 years, performance bottle-necks with S3 mean that Redshift easily wins any comparison.
### Postgres
Postgres, as hosted by AWS RDS, is an open-source SQL database that you can use to store large data sets. Many of the positive characteristics that apply to Redshift apply to Postgres, with the caveat that Postgres is not built for business intelligence use cases. That said, Postgres's open-source nature gives it a distinct advantage over Redshift, especially when thinking about collaboration with open-source contributors. We have an existing Postgres database, but we would not want to re-use it for our data analytics purposes. We would however be able to re-use its terraform configuration.
[Data is hosted in Postgres at $0.115 per GB-month](https://aws.amazon.com/rds/postgresql/pricing/), higher than S3 and Redshift.
At 0 - 6 months, Postgres beats Redshift due to us already having a functional and secure Postgres configuration. Additionally, for small data sizes, Postgres is optimized to return queries more quickly than Redshift.
At 6 months - 2 years, Redshift and Postgres are essentially identical solutions, given that we have already paid down the initial setup cost.
In the 2 - 5 year range, Postgres becomes less appealing due to its higher GB-month hosting cost. There is however evidence that Redshift's OLAP advantages don't kick in until the data grows to terabytes in size. It is unclear if our data will ever grow to that size.
### Snowflake
Snowflake is a data warehouse that is traditionally used to store big data for processing and analytics. Snowflake is a third-party data hosting platform and does not have an AWS-native deployment solution. Deploying Snowflake would require us to either self-host the database or build the infrastructure to connect our systems with an externally managed service. Similarly to Redshift, Snowflake is an OLAP optimized data warehouse, that is well-designed for business analytics queries.
At 0 - 6 months, Snowflake loses to Redshift and Postgres due to the AWS managed databases being vastly easier to set up and configure.
At 6 months - 2 years, Snowflake Redshift and Postgres are essentially identical solutions, given that we have already paid down the initial setup cost.
At 2 - 5 years, Snowflake may start to show cost and performance advantages relative to Redshift or Postgres. Hard evidence to back this up is hard to come by, though.
## Decision Outcome
At present this ADR has the most evidence in favor of **Postgres**, in large part due to the fact that we are already using it. Additional evidence that we will have "big data" (eg 1 TB or larger) would shift this ADR in favor of **Redshift**, but we do not have evidence of that at this time.
# Dashboard Data Tool
* **Status:** Active
* **Last Modified:** 2024-04-10
* **Related Issue:** [#1507](https://github.com/HHS/simpler-grants-gov/issues/1507)
* **Deciders:** Aaron, Billy
## Context and Problem Statement
We are looking to implement a BI (Business Intelligence) tool for Simpler. The BI tool will be the centerpiece of our "Delivery Dashboard" work. A BI tool is software designed to analyze, process, and visualize large volumes of data to help organizations make informed decisions. These tools gather data from various sources, including databases, spreadsheets, and cloud services, and transform it into actionable insights through reports, dashboards, and interactive visualizations. BI tools often include features such as data querying, data mining, statistical analysis, and predictive modeling to uncover trends, patterns, and correlations within the data.
Adopting a BI tool will be instrumental in optimizing decision-making processes and enhancing our delivery practices. BI tools enable agencies to analyze vast amounts of data efficiently, helping to identify trends, patterns, and areas for improvement. By harnessing the power of BI, we can improve resource allocation, monitor program effectiveness, and ensure transparency and accountability in our operations. Furthermore, BI tools facilitate evidence-based decision making by providing us with timely and accurate insights into our needs and trends. Leveraging BI will empower Simpler to better serve citizens, drive efficiencies, and achieve our project goals.
## Desired Solution
We will evaluate the BI tool based on the following capabilities and attributes:
* Ability to share public dashboards
* Ability to show private dashboards to selected users
* Ability to connect to common data sources (S3, Redshift, Postgres)
* Allows technical users to create ad hoc queries to create graphs
* Easy-to-use UI for non-coders
* Replicable for users outside of the project
* Cost of ownership
* Ease of deployment
* Account configuration
## Solution Options
The possible solution space here is quite large, but we have narrowed it down to 5 to options total, only 2 of which are evaluated in this ADR. Only 2 options were thoroughly evaluated in the interest of time. The 5 total options we evaluated are listed below.
* AWS QuickSight - evaluated below
* Metabase - evaluated below
* Tableau
* Redash
* Apache Superset
### AWS QuickSight
> AWS QuickSight is a cloud-based Business Intelligence (BI) service provided by Amazon Web Services (AWS). It enables users to easily create and share interactive dashboards and visualizations from various data sources, including AWS services, databases, and third-party applications. QuickSight offers features such as ad-hoc analysis, machine learning-powered insights, and seamless integration with AWS services like Amazon Redshift, Amazon RDS, and Amazon S3. It provides users with the ability to explore data through drag-and-drop interfaces, create custom visualizations, and perform advanced analytics without requiring extensive technical expertise. With pay-as-you-go pricing and scalability, QuickSight offers an accessible and cost-effective solution for organizations looking to harness the power of BI in the cloud.
Here's how QuickSight evaluates against our criteria:
* ✅ Ability to share public dashboards - [AWS QuickSight supports public dashboards](https://docs.aws.amazon.com/quicksight/latest/user/embedded-analytics-1-click-public.html)
* ✅ Ability to show private dashboards to selected users - [AWS QuickSight supports access controlled dashboards](https://docs.aws.amazon.com/quicksight/latest/user/sharing-a-dashboard.html)
* ✅ Ability to connect to common data sources (S3, Redshift, Postgres) - [AWS QuickSight supports common data sources](https://docs.aws.amazon.com/quicksight/latest/user/supported-data-sources.html)
* ✅ Allows technical users to create ad hoc queries to create graphs - [AWS QuickSight supports creating a variety of visual types](https://docs.aws.amazon.com/quicksight/latest/user/working-with-visual-types.html)
* ✅ Easy-to-use UI for non-coders - Subjectively, the AWS QuickSight UI was found to be easy to use.
* ❌ Replicable for users outside of the project - AWS QuickSight is not open source, so its results can only replicated by having access to our AWS account
* Cost of ownership - A rough estimate puts AWS QuickSight at about \~$300/month for our quantity of users. [Pricing page.](https://aws.amazon.com/quicksight/pricing/)
* ✅✅ Ease of deployment - [AWS QuickSight can be deploy via Terraform](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/quicksight_account_subscription). The entire deployment would be AWS managed, we do not need to manage the deployment in any way.
* ✅ Account configuration - [AWS QuickSight users must be deployed via Terraform or the AWS console. These users require an associated IAM user to be created.](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/quicksight_user) These users can also be configured with AWS SSO and MFA.
### Metabase
> Metabase is an open-source Business Intelligence (BI) tool that enables users to easily query, visualize, and share insights from their data. It offers a user-friendly interface that allows users to create and customize dashboards and visualizations without the need for advanced technical skills. Metabase supports various data sources, including SQL databases like MySQL, PostgreSQL, and MongoDB, as well as cloud services like Google BigQuery and Amazon Redshift. With features such as SQL querying, interactive dashboards, and natural language querying, Metabase empowers users to explore and understand their data in a flexible and intuitive way. Additionally, being open-source, Metabase allows for community contributions and customization, making it a popular choice for organizations seeking a cost-effective and customizable BI solution.
Here's how Metabase evaluates against our criteria:
* ✅ Ability to share public dashboards - [Metabase supports public dashboards](https://www.metabase.com/docs/latest/questions/sharing/public-links)
* ✅ Ability to show private dashboards to selected users - [Metabase supports access controlled dashboards](https://www.metabase.com/learn/administration/guide-to-sharing-data)
* ✅ Ability to connect to common data sources (S3, Redshift, Postgres) - [Metabase supports common data sources](https://www.metabase.com/data_sources/)
* ✅ Allows technical users to create ad hoc queries to create graphs - [Metabase supports creating a variety of visual types](https://www.metabase.com/learn/visualization/)
* ✅ Easy-to-use UI for non-coders - Subjectively, the Metabase UI was found to be easy to use.
* ✅ Replicable for users outside of the project - Metabase is open-source and could be replicated by people outside the project by giving them access to a copy of our analytics database.
* Cost of ownership - The cost of running Metabase is the cost of running an appropriately sized AWS Fargate task 24/7. That cost works out to about \~$100/month.
* ✅ Ease of deployment - [Metabase provides an official docker image that we can run on AWS ECS](https://www.metabase.com/docs/latest/installation-and-operation/running-metabase-on-docker). This ECS service would be managed by us, so we would be responsible for managing upgrades to the service.
* ✅ Account configuration - [Metabase uses Google SSO as its secure account configuration option](https://www.metabase.com/docs/latest/people-and-groups/google-and-ldap). This works across multiple domains and multiple Google Workspaces. As it uses Google SSO, it also supports MFA.
### QuickSight and Metabase compared
Metabase's UX, and open-source nature, make it slightly beat out AWS QuickSight. That said, they both unambiguously satisfy the majority of our decision criteria. Either tool would be a good choice to implement.
## Decision
This ADR supports Metabase as our chosen BI tool.
## Links
* [Best BI tools for startups: How to choose a BI tool](https://www.airops.com/blog/best-bi-tools-for-startups-how-to-choose-a-bi-tool)
* [Metabase vs QuickSight Comparison](https://www.restack.io/docs/metabase-knowledge-metabase-vs-quicksight-comparison)
* [Amazon Quicksight vs Metabase](https://stackshare.io/stackups/amazon-quicksight-vs-metabase)
---
[Next Page](/llms-full.txt/1)