From 4e63a601e97833cf115f0570decfdbdd5c322a0d Mon Sep 17 00:00:00 2001 From: Jonathan Bobel Date: Wed, 31 Jan 2024 11:10:46 -0500 Subject: [PATCH 1/9] Bug - very small content update on pricing page --- app/templates/views/pricing/index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/templates/views/pricing/index.html b/app/templates/views/pricing/index.html index 436da7735..73c67770e 100644 --- a/app/templates/views/pricing/index.html +++ b/app/templates/views/pricing/index.html @@ -30,7 +30,7 @@ more parts towards the allowance if you:

Long text messages

-

If a text message is longer than 160 characters (including spaces), it counts as more than one message.

+

If a text message is longer than 160 characters (including spaces), it counts as more than one message part.

{% call mapping_table( From e71351d94938779145a1478c6683e3eafcc5a327 Mon Sep 17 00:00:00 2001 From: Carlo Costino Date: Fri, 23 Feb 2024 16:02:16 -0500 Subject: [PATCH 2/9] Add E2E documentation and clean E2E tests This changeset adds additional documentation for how to write new E2E tests and cleans up the two existing tests slightly to make better use of the fixtures that are defined. Signed-off-by: Carlo Costino --- docs/end_to_end_tests.md | 99 +++++++++++++++++++ tests/end_to_end/conftest.py | 21 +++- tests/end_to_end/test_accounts_page.py | 4 - .../test_landing_and_sign_in_pages.py | 9 +- 4 files changed, 118 insertions(+), 15 deletions(-) diff --git a/docs/end_to_end_tests.md b/docs/end_to_end_tests.md index 02eec2e8b..557bdf327 100644 --- a/docs/end_to_end_tests.md +++ b/docs/end_to_end_tests.md @@ -106,6 +106,105 @@ All of the E2E tests are found in the `tests/end_to_end` folder and are written as `pytest` scripts using [Playwright's Python Framework](https://playwright.dev/python/docs/writing-tests). +Inside the `tests/end_to_end` folder you'll see a `conftest.py` file, +which is similar to the one found in the root `tests` folder but is +specific to the E2E tests. + +There a few fixtures defined in here, but the three most important at +this time are these: + +- `end_to_end_context`: A Playwright context object needed to interact + with a browser instance. +- `authenticated_page`: A Playwright page object that has gone through + the sign in process the E2E user is authenticated. +- `unauthenticated_page`: A Playwright page object that has only loaded + the home page; no authentication done. + +In short, if you're starting a test from scratch and testing pages that +do not require authentication, you'll start with the +`unauthenticated_page` fixture and work from there. + +Any test that requires you to be authenticated, you'll start with the +`authenticated_page` object as that'll have taken care of getting +everything set for you and logged into the site with the E2E test user. + +The `end_to_end_context` fixture is there more for the two page than for +direct use, but there may be instances where it's easier to get data +or manipulate tests in ways that are better done with the context object +instead of working back from the page object. + + +### Creating a new test file + +If you want to create a new test file to help organize tests (a great +idea!), it will be handy to import the Playwright `expect` and set the +base URL/URI for yourself, like this: + +```python +from playwright.sync_api import expect + +E2E_TEST_URI = os.getenv("NOTIFY_E2E_TEST_URI") +``` + +By importing Playwright's `expect` object for tests and setting +something like `E2E_TEST_URI` for yourself, it will make writing tests +much easier. + + +### Using the fixtures + +To use the `authenticated_page` or `unauthenticated_page` fixtures, you +start by defining a test function and then passing in the fixture you +need as a positional argument. This works the same as the other +functions defined to create a test for pytest. + +For example, the test for the landing page starts with this: + +```python +def test_landing_page(unauthenticated_page): + page = unauthenticated_page + ... +``` + +Note the passing in of the `unauthenticated_page` fixture - there is no +need to import this or anything, just pass it into the function. pytest +takes care of everything else for you. + +The second line that defines a `page` variable is a convenience, since +you'll be referencing the page object a lot. This is recommended to +help keep tests readable while keeping fixture names descriptive. + +If you need to test an authenticate page, such as the accounts page, +use the `authenticated_page` fixture instead, like so: + +```python +def test_add_new_service_workflow(authenticated_page): + page = authenticated_page + ... +``` + +Again, it's helpful to assign the fixture to a `page` variable for easy +reference throughout the test. + +Lastly, if you need want access to the Playwright context object that is +used behind the page fixtures, you can reference it directly as well: + +```python +def test_add_new_service_workflow(authenticated_page, end_to_end_context): + page = authenticated_page + + # Prepare for adding a new service later in the test. + current_date_time = datetime.datetime.now() + new_service_name = "E2E Federal Test Service {now} - {browser_type}".format( + now=current_date_time.strftime("%m/%d/%Y %H:%M:%S"), + browser_type=end_to_end_context.browser.browser_type.name, + ) + ... +``` + +In this example, I've used the context to get to the browser object +itself to get the name of the browser for test data. + ## Maintaining E2E Tests with GitHub diff --git a/tests/end_to_end/conftest.py b/tests/end_to_end/conftest.py index 48d1bcd35..4819ec007 100644 --- a/tests/end_to_end/conftest.py +++ b/tests/end_to_end/conftest.py @@ -89,17 +89,30 @@ def end_to_end_authenticated_context(browser): @pytest.fixture(scope="session") -def authenticated_page(end_to_end_context): - # Open a new page and go to the staging site. +def unauthenticated_page(end_to_end_context): page = end_to_end_context.new_page() - page.goto(f"{E2E_TEST_URI}/") - sign_in_button = page.get_by_role("link", name="Sign in") + # Wait for the next page to fully load. + page.wait_for_load_state("domcontentloaded") + + return page + + +@pytest.fixture(scope="session") +def authenticated_page(end_to_end_context): + # Open a new page and go to the site. + page = end_to_end_context.new_page() + page.goto(f"{E2E_TEST_URI}/") + + # Wait for the next page to fully load. + page.wait_for_load_state("domcontentloaded") # Sign in to the site - E2E test accounts are set to flow through. + sign_in_button = page.get_by_role("link", name="Sign in") sign_in_button.click() # Wait for the next page to fully load. page.wait_for_load_state("domcontentloaded") + return page diff --git a/tests/end_to_end/test_accounts_page.py b/tests/end_to_end/test_accounts_page.py index 411728e33..b6fe8c5ac 100644 --- a/tests/end_to_end/test_accounts_page.py +++ b/tests/end_to_end/test_accounts_page.py @@ -9,10 +9,6 @@ E2E_TEST_URI = os.getenv("NOTIFY_E2E_TEST_URI") def test_add_new_service_workflow(authenticated_page, end_to_end_context): page = authenticated_page - page.goto(f"{E2E_TEST_URI}/") - - # Wait for the next page to fully load. - page.wait_for_load_state("domcontentloaded") # Prepare for adding a new service later in the test. current_date_time = datetime.datetime.now() diff --git a/tests/end_to_end/test_landing_and_sign_in_pages.py b/tests/end_to_end/test_landing_and_sign_in_pages.py index dd6a70beb..8b0f04f86 100644 --- a/tests/end_to_end/test_landing_and_sign_in_pages.py +++ b/tests/end_to_end/test_landing_and_sign_in_pages.py @@ -6,13 +6,8 @@ from playwright.sync_api import expect E2E_TEST_URI = os.getenv("NOTIFY_E2E_TEST_URI") -def test_landing_page(end_to_end_context): - # Open a new page and go to the staging site. - page = end_to_end_context.browser.new_page() - page.goto(f"{E2E_TEST_URI}/") - - # Check to make sure that we've arrived at the next page. - page.wait_for_load_state("domcontentloaded") +def test_landing_page(unauthenticated_page): + page = unauthenticated_page # Check the page title exists and matches what we expect. expect(page).to_have_title(re.compile("Notify.gov")) From 34d20c4d65cb6a23d10b7bf5df02adb3122e8ff0 Mon Sep 17 00:00:00 2001 From: Carlo Costino Date: Fri, 23 Feb 2024 16:53:39 -0500 Subject: [PATCH 3/9] Trying to get landing page test working again Signed-off-by: Carlo Costino --- tests/end_to_end/test_landing_and_sign_in_pages.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/end_to_end/test_landing_and_sign_in_pages.py b/tests/end_to_end/test_landing_and_sign_in_pages.py index 8b0f04f86..ab1f885b1 100644 --- a/tests/end_to_end/test_landing_and_sign_in_pages.py +++ b/tests/end_to_end/test_landing_and_sign_in_pages.py @@ -7,7 +7,12 @@ E2E_TEST_URI = os.getenv("NOTIFY_E2E_TEST_URI") def test_landing_page(unauthenticated_page): + # Open a new page and go to the staging site. page = unauthenticated_page + page.goto(f"{E2E_TEST_URI}/") + + # Check to make sure that we've arrived at the next page. + page.wait_for_load_state("domcontentloaded") # Check the page title exists and matches what we expect. expect(page).to_have_title(re.compile("Notify.gov")) From c891bc1b3f800c60dc5bef7b9392dbdf394e256d Mon Sep 17 00:00:00 2001 From: Carlo Costino Date: Fri, 23 Feb 2024 17:02:18 -0500 Subject: [PATCH 4/9] Another attempt at fixing the landing page test Signed-off-by: Carlo Costino --- tests/end_to_end/conftest.py | 5 ----- 1 file changed, 5 deletions(-) diff --git a/tests/end_to_end/conftest.py b/tests/end_to_end/conftest.py index 4819ec007..91e52ac40 100644 --- a/tests/end_to_end/conftest.py +++ b/tests/end_to_end/conftest.py @@ -91,11 +91,6 @@ def end_to_end_authenticated_context(browser): @pytest.fixture(scope="session") def unauthenticated_page(end_to_end_context): page = end_to_end_context.new_page() - page.goto(f"{E2E_TEST_URI}/") - - # Wait for the next page to fully load. - page.wait_for_load_state("domcontentloaded") - return page From 5c6a23c87342f5ea50b4c438a92db9bdb2d2ae34 Mon Sep 17 00:00:00 2001 From: Carlo Costino Date: Fri, 23 Feb 2024 17:10:41 -0500 Subject: [PATCH 5/9] Undo changes to the landing page test Signed-off-by: Carlo Costino --- tests/end_to_end/conftest.py | 11 +++-------- tests/end_to_end/test_landing_and_sign_in_pages.py | 6 +++--- 2 files changed, 6 insertions(+), 11 deletions(-) diff --git a/tests/end_to_end/conftest.py b/tests/end_to_end/conftest.py index 91e52ac40..f21fbe0fc 100644 --- a/tests/end_to_end/conftest.py +++ b/tests/end_to_end/conftest.py @@ -68,11 +68,6 @@ def login_for_end_to_end_testing(browser): context.storage_state(path=auth_state_path) -@pytest.fixture(scope="session") -def end_to_end_context(browser): - context = browser.new_context() - return context - @pytest.fixture(scope="session") def end_to_end_authenticated_context(browser): @@ -89,9 +84,9 @@ def end_to_end_authenticated_context(browser): @pytest.fixture(scope="session") -def unauthenticated_page(end_to_end_context): - page = end_to_end_context.new_page() - return page +def end_to_end_context(browser): + context = browser.new_context() + return context @pytest.fixture(scope="session") diff --git a/tests/end_to_end/test_landing_and_sign_in_pages.py b/tests/end_to_end/test_landing_and_sign_in_pages.py index ab1f885b1..a9148cb31 100644 --- a/tests/end_to_end/test_landing_and_sign_in_pages.py +++ b/tests/end_to_end/test_landing_and_sign_in_pages.py @@ -6,9 +6,9 @@ from playwright.sync_api import expect E2E_TEST_URI = os.getenv("NOTIFY_E2E_TEST_URI") -def test_landing_page(unauthenticated_page): - # Open a new page and go to the staging site. - page = unauthenticated_page +def test_landing_page(end_to_end_context): + # Open a new page and go to the site. + page = end_to_end_context.browser.new_page() page.goto(f"{E2E_TEST_URI}/") # Check to make sure that we've arrived at the next page. From 564fc7352120829ad28320eb9897736efaaa3f3e Mon Sep 17 00:00:00 2001 From: Carlo Costino Date: Fri, 23 Feb 2024 17:38:47 -0500 Subject: [PATCH 6/9] Fixed up documentation to match current state; blackened formatting Signed-off-by: Carlo Costino --- docs/end_to_end_tests.md | 29 +++++++++---------- tests/app/main/views/test_manage_users.py | 6 ++-- tests/app/main/views/test_template_folders.py | 8 +++-- tests/conftest.py | 14 +++++---- tests/end_to_end/conftest.py | 1 - 5 files changed, 30 insertions(+), 28 deletions(-) diff --git a/docs/end_to_end_tests.md b/docs/end_to_end_tests.md index 557bdf327..11758a16e 100644 --- a/docs/end_to_end_tests.md +++ b/docs/end_to_end_tests.md @@ -110,29 +110,22 @@ Inside the `tests/end_to_end` folder you'll see a `conftest.py` file, which is similar to the one found in the root `tests` folder but is specific to the E2E tests. -There a few fixtures defined in here, but the three most important at -this time are these: +There a few fixtures defined in here, but the two most important at this +time are these: - `end_to_end_context`: A Playwright context object needed to interact with a browser instance. - `authenticated_page`: A Playwright page object that has gone through the sign in process the E2E user is authenticated. -- `unauthenticated_page`: A Playwright page object that has only loaded - the home page; no authentication done. In short, if you're starting a test from scratch and testing pages that do not require authentication, you'll start with the -`unauthenticated_page` fixture and work from there. +`end_to_end_context` fixture and work from there. Any test that requires you to be authenticated, you'll start with the `authenticated_page` object as that'll have taken care of getting everything set for you and logged into the site with the E2E test user. -The `end_to_end_context` fixture is there more for the two page than for -direct use, but there may be instances where it's easier to get data -or manipulate tests in ways that are better done with the context object -instead of working back from the page object. - ### Creating a new test file @@ -153,7 +146,7 @@ much easier. ### Using the fixtures -To use the `authenticated_page` or `unauthenticated_page` fixtures, you +To use the `authenticated_page` or `end_to_end_context` fixtures, you start by defining a test function and then passing in the fixture you need as a positional argument. This works the same as the other functions defined to create a test for pytest. @@ -161,12 +154,17 @@ functions defined to create a test for pytest. For example, the test for the landing page starts with this: ```python -def test_landing_page(unauthenticated_page): - page = unauthenticated_page +def test_landing_page(end_to_end_context): + # Open a new page and go to the site. + page = end_to_end_context.browser.new_page() + page.goto(f"{E2E_TEST_URI}/") + + # Check to make sure that we've arrived at the next page. + page.wait_for_load_state("domcontentloaded") ... ``` -Note the passing in of the `unauthenticated_page` fixture - there is no +Note the passing in of the `end_to_end_context` fixture - there is no need to import this or anything, just pass it into the function. pytest takes care of everything else for you. @@ -187,7 +185,8 @@ Again, it's helpful to assign the fixture to a `page` variable for easy reference throughout the test. Lastly, if you need want access to the Playwright context object that is -used behind the page fixtures, you can reference it directly as well: +used behind the page fixtures, you can reference it directly as well +using the `end_to_end_context` fixture: ```python def test_add_new_service_workflow(authenticated_page, end_to_end_context): diff --git a/tests/app/main/views/test_manage_users.py b/tests/app/main/views/test_manage_users.py index 9bdabf925..c3d128155 100644 --- a/tests/app/main/views/test_manage_users.py +++ b/tests/app/main/views/test_manage_users.py @@ -862,9 +862,9 @@ def test_should_show_page_if_prefilled_user_is_already_invited( mock_get_invites_for_service, platform_admin_user, ): - active_user_with_permission_to_other_service[ - "email_address" - ] = "user_1@testnotify.gsa.gov" + active_user_with_permission_to_other_service["email_address"] = ( + "user_1@testnotify.gsa.gov" + ) client_request.login(platform_admin_user) mocker.patch( "app.models.user.user_api_client.get_user", diff --git a/tests/app/main/views/test_template_folders.py b/tests/app/main/views/test_template_folders.py index fae020e58..0bf2fcb08 100644 --- a/tests/app/main/views/test_template_folders.py +++ b/tests/app/main/views/test_template_folders.py @@ -31,9 +31,11 @@ def _folder(name, folder_id=None, parent=None, users_with_permission=None): "name": name, "id": folder_id or str(uuid.uuid4()), "parent_id": parent, - "users_with_permission": users_with_permission - if users_with_permission is not None - else [sample_uuid()], + "users_with_permission": ( + users_with_permission + if users_with_permission is not None + else [sample_uuid()] + ), } diff --git a/tests/conftest.py b/tests/conftest.py index 6c4f94b4f..4c2ceeea9 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -904,9 +904,11 @@ def create_service_templates(service_id, number_of_templates=4): "{}_template_{}".format(template_type, template_number), template_type, "{} template {} content".format(template_type, template_number), - subject="{} template {} subject".format(template_type, template_number) - if template_type == "email" - else None, + subject=( + "{} template {} subject".format(template_type, template_number) + if template_type == "email" + else None + ), ) ) @@ -1101,9 +1103,9 @@ def active_user_with_permission_to_other_service( active_user_with_permission_to_two_services["permissions"].pop(SERVICE_ONE_ID) active_user_with_permission_to_two_services["services"].pop(0) active_user_with_permission_to_two_services["name"] = "Service Two User" - active_user_with_permission_to_two_services[ - "email_address" - ] = "service-two-user@test.gsa.gov" + active_user_with_permission_to_two_services["email_address"] = ( + "service-two-user@test.gsa.gov" + ) return active_user_with_permission_to_two_services diff --git a/tests/end_to_end/conftest.py b/tests/end_to_end/conftest.py index f21fbe0fc..16940d4e0 100644 --- a/tests/end_to_end/conftest.py +++ b/tests/end_to_end/conftest.py @@ -68,7 +68,6 @@ def login_for_end_to_end_testing(browser): context.storage_state(path=auth_state_path) - @pytest.fixture(scope="session") def end_to_end_authenticated_context(browser): # Create and load a previously authenticated context for Playwright E2E From 66909e96249a56385b8454c25505a31a731a7762 Mon Sep 17 00:00:00 2001 From: Beverly Nguyen Date: Fri, 23 Feb 2024 17:50:25 -0800 Subject: [PATCH 7/9] added conditional --- app/main/views/dashboard.py | 1 + 1 file changed, 1 insertion(+) diff --git a/app/main/views/dashboard.py b/app/main/views/dashboard.py index 24b3e0490..4d5aca14d 100644 --- a/app/main/views/dashboard.py +++ b/app/main/views/dashboard.py @@ -76,6 +76,7 @@ def service_dashboard(service_id): "notifications": aggregate_notifications_by_job.get(job["id"], []), } for job in job_response + if aggregate_notifications_by_job.get(job["id"], []) ] return render_template( "views/dashboard/dashboard.html", From 2c6e02137bbdcdf1ccceeee49877c743dc518fce Mon Sep 17 00:00:00 2001 From: Jonathan Bobel Date: Thu, 1 Feb 2024 12:09:43 -0500 Subject: [PATCH 8/9] Updates per Meghan --- app/templates/views/guidance/index.html | 4 +-- app/templates/views/roadmap.html | 4 +-- app/templates/views/security.html | 37 ++----------------------- app/templates/views/support/index.html | 6 ++-- 4 files changed, 9 insertions(+), 42 deletions(-) diff --git a/app/templates/views/guidance/index.html b/app/templates/views/guidance/index.html index e754ef245..b7d6ffa1a 100644 --- a/app/templates/views/guidance/index.html +++ b/app/templates/views/guidance/index.html @@ -41,7 +41,7 @@

To create and format your message

  1. All messages start from a template
    2. -
  2. Click “Send Messages”. You’ll see existing templates.
    3. +
  3. Click “Send Messages”. You’ll see existing templates.
  4. Add a new template or choose an existing template and select Edit.
@@ -120,7 +120,7 @@ {# Identify your program #} -

Identify your program

+

Identify your program

You can help your recipients identify your texts as legitimate by customizing your messages to clearly state who they are from. Consider using the program or benefit name that is most familiar to your recipients.

diff --git a/app/templates/views/roadmap.html b/app/templates/views/roadmap.html index 6ea2e7cfb..58c9ad4d1 100644 --- a/app/templates/views/roadmap.html +++ b/app/templates/views/roadmap.html @@ -50,9 +50,9 @@
  • Message send/failure analytics
    • - Next +

    Next

    -

    If the pilot is successful, we hope to recruit additional high-impact partners to improve outcomes for low-income individuals and families.

    +

    If the pilot is successful, we hope to recruit additional partners to improve outcomes for low-income individuals and families.

    Goals during this stage:

    diff --git a/app/templates/views/security.html b/app/templates/views/security.html index d676b37bf..35abece07 100644 --- a/app/templates/views/security.html +++ b/app/templates/views/security.html @@ -65,9 +65,9 @@

    Protect sensitive information

    Some messages include sensitive information like security codes or password reset links.

    If you’re sending a message with sensitive information, you can choose to hide those details on the Notify dashboard once the message has been sent. This means that only the message recipient will be able to see that information.

    + Screenshot of a teat message in review with the link to 'hide personalization after sending' circled. -

    User permissions and signing in

    -

    You can set different user permissions in Notify. This lets you control who in your team has access to certain parts of the service.

    Two-factor authentication

    To sign in to Notify, you’ll need to enter:

      @@ -76,11 +76,6 @@

    If signing in with a text message is a problem for your team, contact us to find out about using an email link instead.

    - Screenshot of a teat message in review with the link to 'hide personalization after sending' circled. - -

    How to hide PII after sending a message

    -

    User permissions and signing in

    You can set different user permissions in Notify. This lets you control who in your team has access to certain parts of the service.

    @@ -93,32 +88,4 @@

    If signing in with a text message is a problem for your team, contact us to find out about using an email link instead.

    - - - - - - {% endblock %} diff --git a/app/templates/views/support/index.html b/app/templates/views/support/index.html index d9d6abd53..dd5ad9c10 100644 --- a/app/templates/views/support/index.html +++ b/app/templates/views/support/index.html @@ -13,9 +13,9 @@

    Contact us

    Notify is designed to be easy to use.

      -
    • For information on personalization and data preparation, see Guidance.
      • -
    • For help interpreting delivery reports, see Delivery Status.
      • -
    • For details on pricing and what counts as a message part, see Pricing.
      • +
    • For information on personalization and data preparation, see Guidance.
      • +
    • For help interpreting delivery reports, see Delivery Status.
      • +
    • For details on pricing and what counts as a message part, see Pricing.

    If you have other questions, we are available at notify-support@gsa.gov.

    From e9f96332870e9e0f59769dbbcf1cce9c5386938a Mon Sep 17 00:00:00 2001 From: Carlo Costino Date: Tue, 27 Feb 2024 11:01:37 -0500 Subject: [PATCH 9/9] Fix typo --- app/templates/views/security.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/templates/views/security.html b/app/templates/views/security.html index 35abece07..6d0b3e67d 100644 --- a/app/templates/views/security.html +++ b/app/templates/views/security.html @@ -66,7 +66,7 @@

    Some messages include sensitive information like security codes or password reset links.

    If you’re sending a message with sensitive information, you can choose to hide those details on the Notify dashboard once the message has been sent. This means that only the message recipient will be able to see that information.

    Screenshot of a teat message in review with the link to 'hide personalization after sending' circled. + alt="Screenshot of a test message in review with the link to 'hide personalization after sending' circled.">

    Two-factor authentication

    To sign in to Notify, you’ll need to enter: