Add Architectural Decision Records to US Notify (#284)

This changeset introduces Architectural Decision Records (ADRs) to the US Notify project.  This initial commit includes the following:

- Updates to the main project README.md that mentions ADRs
- A new adr/ folder under the docs/ folder
- An ADR template to copy and follow for the future
- Our first ADR, ADR-0001 - Establishing ADRs
- A README.md specific to ADRs that has more information and instructions
- ADR-0001 finalized and accepted

A huge thank you to @stvnrlly for the help in making this come together!

Signed-off-by: Carlo Costino <carlo.costino@gsa.gov>
Co-authored-by: Steven Reilly <stvnrlly@users.noreply.github.com>

README.md

@@ -21,6 +21,7 @@ Our other repositories are:
 
 - [Roadmap](https://notifications-admin.app.cloud.gov/features/roadmap)
 - [Using the API](./docs/api-usage.md)
+- [Architectural Decision Records](./docs/adrs/)
 
 ### Infrastructure
 

docs/adrs/0001-establishing-adrs-for-us-notify.md

@@ -0,0 +1,153 @@
+# TITLE:  Establishing ADRs for US Notify
+
+
+| CREATED DATE | LAST UPDATED | STATUS | AUTHOR | STAKEHOLDERS |
+| :---: | :---: | :---: | :---: | :---: |
+| 06/01/2023 | 06/05/2023 | Accepted | @ccostino | @stvnrlly |
+
+
+## CONTEXT AND PROBLEM STATEMENT
+
+**OPEN ISSUE:** https://github.com/GSA/notifications-api/issues/282
+
+As a developer of the system, I'd like to be able to keep track of system
+architecture decisions and understand why they were made, including what
+trade-offs and alternatives might have existed at the time. I'd also like to
+keep track of these in a place that I can refer back to later for historical
+context.
+
+
+## DECISION DRIVERS
+
+These are the key considerations for creating ADRs for US Notify:
+
+- We'd like to establish a decision-making framework for future proposals to
+  improve or change the product/service.
+
+- We'd like to document the outcome of our decisions and include the rationale
+  behind them to know what we've already considered previously.
+
+- In the spirit of open source and collaboration, we'd like to make our
+  decisions as open as possible, but recognize there are times when we cannot;
+  in those cases, we'll follow the same process but in a private location.
+
+- We need to make sure we're accounting for any security compliance concerns
+  and considerations ahead of time, while we're actively thinking about how to
+  architect and implement a thing instead of after the fact.
+
+
+### SECURITY COMPLIANCE CONSIDERATIONS
+
+- Documenting architectural details in the open
+  - We should err on the side of documenting in the open whenever possible, but
+    some details we will not be able to share.  We should create issues for
+    those cases to note the work happening in a private space.
+
+- Sensitive information must not be shared
+  - We need to be judicious in not documenting any sensitive bits of information
+    like account credentials or passwords, environment variable values, etc.
+
+
+## CONSIDERED OPTIONS
+
+- **Architectural Decision Records:**  A common document format for capturing
+  architectural decisions that many development teams have adopted in recent
+  years, including at large technology companies such as
+  [GitHub](https://adr.github.io/) and [Amazon Web Services](https://docs.aws.amazon.com/prescriptive-guidance/latest/architectural-decision-records/welcome.html) and TTS' own [18F](https://18f.gsa.gov/2021/07/06/architecture_decision_records_helpful_now_invaluable_later/).
+
+  - Pros:
+    - Well-known format that has many example templates to choose from
+    - Can be as simple or complex as desired
+    - Additional tooling exists to help manage ADRs
+    - ADRs are committed to and live with the code itself
+    - Easy to link to other parts of the repo
+
+  - Cons:
+    - Takes a bit of prep to setup; best supported with a template to copy from
+    - Setting up additional tooling takes time and requires additional
+      maintenance
+    - Requires some training for folks not already familiar with ADRs
+    - Easy to go overboard with
+    - Requires being mindful of what can live in the public space vs. not
+
+- **Google Docs in Google Drive:**  Writing up notes docs in Google Drive with
+  Google Docs.
+
+  - Pros:
+    - Simple and lightweight to do
+    - Possible to setup a doc template to create from, if desired
+    - Ability for team members to collaborate in real-time with each other
+    - Useful for documenting things that cannot be public
+    - Access to tools/features like a spellchecker
+
+  - Cons:
+    - Google Drive organization is difficult; keeping track of documents
+      can become hard
+    - Easy to not follow a standard and agreed upon format
+    - Not open to the public for things that can be shared publicly
+    - Documentation does not live directly with the code
+
+- **GitHub Issues and/or Wiki:**  Writing up notes and decisions directly in
+  GitHub issues and/or the wiki associated with a repo.
+
+  - Pros:
+    - Simple and lightweight to do
+    - Possible to configure an issue template to create from, if desired
+    - Easy to link to related issues, wiki pages, etc.
+
+  - Cons:
+    - Documentation lives in a GitHub itself, not the code repository directly;
+      therefore, it's not portable
+    - Easy to not follow a standard and agreed upon format if no template is
+      provided
+    - Requires being mindful of what can live in the public space vs. not
+
+
+## PROPOSED OPTION:  Architectural Decision Records
+
+I am proposing that our team adopts using Architectural Decision Records going
+forward for any decisions that need to be proposed or discussed that will have a
+significant impact on the platform.
+
+By documenting our changes in this fashion, it will improve our team's
+development practices and software quality in a few ways:
+
+- Encourage us to slow down and think through a new change, especially anything
+  of significance
+- Hold us accountable to each other in soliciting feedback for our work and
+  engaging in discussions earlier in the process of building something
+- Provide a mechanism to propose ideas for changes and improvements to the
+  system that is also archived with the code itself
+- Bake security compliance considerations into our development process from the
+  start, ensuring they are not just after-thoughts once something is completed
+
+ADRs have a wealth of material and support to draw from, other teams across TTS
+are already using them (e.g., cloud.gov, a variety of 18F projects, and others),
+and other large organizations, including GitHub and Amazon, have also adopted
+them.  Some example material to reference:
+
+- [How to Create ADRs - and How Not To](https://www.ozimmer.ch/practices/2023/04/03/ADRCreation.html)
+- [The Markdown ADR (MADR) Template Explained and Distilled](https://www.ozimmer.ch/practices/2022/11/22/MADRTemplatePrimer.html)
+- [The Ultimate Guide to Architectural Decision Records](https://betterprogramming.pub/the-ultimate-guide-to-architectural-decision-records-6d74fd3850ee)
+
+
+### Consequences
+
+- Positive
+  - Formal decision documentation and history
+  - Proactive security compliance considerations in the decision-making process
+  - Accepted means of proposing new ideas for the future
+
+- Negative
+  - A bit of a learning curve in making sure all team members are aware and
+    brought up to speed of what ADRs are
+  - Some configuration and set up required; mainly new templates, though one is
+    provided with this proposal
+
+
+## VALIDATION AND NEXT STEPS
+
+@stvnrlly and I went over this proposal and have worked together to get it in
+the shape it needs to be for the team to work off of.  The corresponding ADR
+README.md that was a part of the original pull request was also refined to make
+sure it contains all relevant information and instructions.

docs/adrs/README.md

@@ -0,0 +1,151 @@
+# US Notify Architectural Decision Records (ADRs)
+
+This sub-folder in the US Notify API project contains the bulk of our
+Architectural Decision Records (henceforth referred to as ADRs) for the overall
+product and service.
+
+
+## What are ADRs?
+
+ADRs serve a few purposes for our team:
+
+- Document important decisions related to the behavior, architecture, and/or
+  dependencies of the platform
+- Capture the decision making process behind a change, from its initial proposal
+  all the way through to its decided outcomes
+- Identify alternative approaches and note why they were ultimately not chosen
+- Denote who is the decision maker(s) for a change within the team
+
+The collection of ADRs in this repository make up our architectural decision log
+(ADL).  An index of the log is maintained right here in this README just below.
+
+For more information, you can see the details in
+[our first ADR](./0001-establishing-adrs-for-us-notify.md) that establishes
+everything!
+
+
+## When should we write an ADR?
+
+An ADR should be written when the team is discussing any significant change to
+the system that will alter its behavior, infrastructure, and/or dependencies.
+
+We should also consider writing an ADR when we're ready to propose something
+that is new to the system, e.g., adding a new feature, leveraging a new cloud
+service for additional capabilities, etc.  An ADR is a great format to write a
+proposal and then share it with the rest of the team to discuss it and decide
+whether or not to move forward, with or without any changes.
+
+
+## How are ADRs created, reviewed, and maintained?
+
+First, we have an ADR template that folks can use to work off of.  The template
+exists as both a GitHub issue template and a standalone Markdown file that can
+be copied as needed if folks prefer to work locally first.
+
+By following the template, we ensure that our ADRs are consistent in language
+and structure.  This allows us to easily review the documentions and discuss
+them as a team.  It also guarantees that the ADR has all of the required
+information.
+
+**ADRs are intended to be living documents.**  As such, it is not uncommon to
+see multiple pull requests (PRs) filed to update them, especially during an
+active discussion and research taking place.  This is also why there is a
+*status* marker on them as a part of their metadata.
+
+Once an ADR has been reviewed and is ready to be finalized (either as accepted,
+rejected, or some other status), some final edits are made to update the ADR
+with decision details and next steps.  After this, future PRs can be opened to
+make additional updates, especially if an ADR becomes deprecated or superceded
+by another one.
+
+
+### Creating an ADR
+
+To create a new ADR in this repository, you can do one of two things:
+
+- Open a new GitHub issue and select the Architecture Decision Record issue type
+- Clone the repo locally, create a new branch for yourself, and make a copy of
+  the Markdown template
+
+In either scenario, check to see what the latest ADR filename is, because they
+always start with a number (e.g., `0001`).  Name your ADR with a number one
+after the last ADR written; if the latest ADR starts with `0021-`, your ADR
+should start with `0022-`.
+
+At this point, it is a matter of filling in the details outlined in the template
+that are relevant to the ADR.
+
+
+### Reviewing an ADR
+
+Once an ADR is created, it's time for review and discussion!  This could happen
+a few ways:
+
+- Asynchronously via comments on the pull request itself
+- Synchronously with a scheduled meeting(s) and a facilitator
+- A combination of these, depending on the nature of the ADR and needs of the
+  team
+
+Whichever way is chosen, the review process should allow the team to dig into
+the proposal and talk through its merits, address anything needing
+clarification, discuss any potential alternatives, and develop an understanding
+of the trade-offs in deciding to move forward with the proposal or not.
+
+If it turns out that one of the alternatives proves to be a better solution, the
+ADR should be updated to reflect that and a follow-up discussion and/or review
+should be held to make sure everything is accurate and up-to-date.
+
+**Please note:** Similar to sprint retrospectives, these review sessions *must*
+ensure a healthy and open dialog within the team; therefore, we actively work
+to promote psychological safety so that everyone and their contributions are
+welcomed and respected.
+
+As a reminder, we can reference these statements, just as we would in a sprint
+retrospective:
+
+>We are here to improve our team and our way of working incrementally over time.
+>This is a safe space, where we can openly discuss anything related to the team
+>or project in a [blameless manner](https://opensource.com/article/19/4/psychology-behind-blameless-retrospective).
+
+[Retrospective Prime Directive](https://retrospectivewiki.org/index.php?title=The_Prime_Directive):
+
+>“Regardless of what we discover, we understand and truly believe that everyone
+>did the best job they could, given what they knew at the time, their skills and
+>abilities, the resources available, and the situation at hand.”
+
+*– Norm Kerth, Project Retrospectives:  A Handbook for Team Review*
+
+An approach we can take during the discussions is to use the principles of
+[The Art of Alignment](https://drive.google.com/file/d/1pPIzJG1kcnudR1HjZiB5UZgwYJ1dyetS/view?usp=share_link).
+There are also other frameworks and tools for sharing proposals and achieving
+consensus within a team.
+
+
+### Maintaining an ADR
+
+If an ADR requires some updates or is ready to be accepted or rejected, you can
+either edit the file directly in GitHub or create a new branch in the repo on
+your local machine and make the changes necessary.
+
+In either scenario, you'll create a pull request (PR) with your changes that
+will then be ready for review from others on the team.
+
+ADR statuses can be one of the following:
+
+- Proposed
+- Accepted
+- Rejected
+- Deprecated
+- Superseded By (new ADR number and link)
+
+Once the ADR itself is updated, this README also needs to be updated so that the
+ADR is listed in the Architecture Decision Log just below.  This lists all of
+our ADRs in reverse chronological order so we have a convenient index of them.
+
+
+## Architecture Decision Log
+
+This is the log of all of our ADRs in reverse chronological order (newest is up
+top!).
+
+- [ADR-0001](./0001-establishing-adrs-for-us-notify.md) - Establishing ADRs for US Notify

docs/adrs/adr-template.md

@@ -0,0 +1,116 @@
+# TITLE:  ADR Title Here
+
+
+| CREATED DATE | LAST UPDATED | STATUS | AUTHOR | STAKEHOLDERS |
+| :---: | :---: | :---: | :---: | :---: |
+| Date when ADR was created - MM/DD/YYYY format | Date when ADR was last updated - MM/DD/YYYY format - or N/A | Current ADR status - one of Proposed / Accepted / Rejected / Deprecated / Superceded By ADR - link to ADR | GitHub username(s) of author(s) | GitHub username(s) or team name(s) of other folks involved |
+
+
+## CONTEXT AND PROBLEM STATEMENT
+
+**OPEN ISSUE(S):** Link(s) to related issue(s) here, if applicable.
+
+Explain the nature of the problem, challenge, or decision to be made here. Be as
+succinct as possible, but don't leave out any details.  State things matter of
+factly and as objectively as you can.
+
+
+## DECISION DRIVERS
+
+List anything that plays a major role in making a decision here.  These could
+be one or more of the following:
+
+- Desired qualities
+- Desired outcomes
+- Primary concerns
+- Constraints
+- ... (anything else pertinent to the decision-making process)
+
+
+### SECURITY COMPLIANCE CONSIDERATIONS
+
+Because we work in a regulated space with many compliance requirements, we need
+to make sure we're accounting for any security concerns and adhering to all
+security compliance requirements.  List them in this section along with any
+relevant details:
+
+- Security concern 1
+  - Concern detail 1
+  - Concern detail 2
+  - Concern detail ...
+
+- Security concern 2
+  - Concern detail 1
+  - Concern detail 2
+  - Concern detail ...
+
+
+## CONSIDERED OPTIONS
+
+List all options that have either been discussed or thought of as a potential
+solution to the context and problem statement.  Include any pros and cons with
+each option, like so:
+
+- **Name of first option:**  A brief summary of the option.
+  - Pros:
+    - Pro 1
+    - Pro 2
+    - Pro ...
+
+  - Cons:
+    - Con 1
+    - Con 2
+    - Con ...
+
+- **Name of second option:**  A brief summary of the option.
+  - Pros:
+    - Pro 1
+    - Pro 2
+    - Pro ...
+
+  - Cons:
+    - Con 1
+    - Con 2
+    - Con ...
+
+
+## PROPOSED OR CHOSEN OPTION:  Proposed/Chosen Option Title Here
+
+Summarize the decision for the proposed/chosen option here.  Be as concise and
+objective as possible while including all relevant details so that a clear
+justification is provided.  Include a list of consequences for choosing this
+option, both positive and negative:
+
+
+### Consequences
+
+- Positive
+  - Positive consequence 1
+  - Positive consequence 2
+  - Positive consequence ...
+
+- Negative
+  - Negative consequence 1
+  - Negative consequence 2
+  - Negative consequence ...
+
+
+## VALIDATION AND NEXT STEPS
+
+This section likely won't be filled out until a decision has been made by the
+team and the ADR is accepted.  If this comes to pass, then write up the criteria
+that would ensure this ADR is both implemented and tested correctly and
+adequately.  This could be a short summary and/or a list of things:
+
+- **Criterion name 1:**  Description of criterion 1
+  - Requirement or action 1
+  - Requirement or action 2
+  - Requirement or action ...
+
+- **Criterion name 2:**  Description of criterion 2
+  - Requirement or action 1
+  - Requirement or action 2
+  - Requirement or action ...
+
+Lastly, include a link(s) to an issue(s) that represent the work that will
+take place as follow-ups to this ADR.