Commit Graph

6426 Commits

Author SHA1 Message Date
Chris Hill-Scott
0955403a3b Slim down the /organisations response
At the moment this response returns a list of service IDs for hundreds
of organisations.

The admin app doesn’t use this information, but having to wait for it to
be serialized and sent across the network slows it down all the same.
2019-06-18 10:48:41 +01:00
Chris Hill-Scott
4dd245ca87 Remove deprecated fields from orgs and services
Once the admin app has started using the new fields returned in
https://github.com/alphagov/notifications-api/pull/2539 these fields
won’t be needed any more.
2019-06-18 10:48:41 +01:00
Chris Hill-Scott
175f56da90 Merge pull request #2542 from alphagov/add-more-agreement-signed-fields
Add columns for on whose behalf agreement signed
2019-06-18 10:48:02 +01:00
Chris Hill-Scott
8977f5193e Add columns for on whose behalf agreement signed
This is changing because we’re going to introduce accepting contracts
and MoUs online.

Previously
---
We had one column for who signed the agreement, which is foreign keyed
to the user table. This is still relevant, because there will always be
a user who is clicking the button.

Now
---
We add two new fields for the name and email address of the person on
whose behalf the agreement is being accepted. This person:
- is different from the one signing the agreement
- won’t necessarily have a Notify account
2019-06-18 10:41:22 +01:00
Chris Hill-Scott
3cde623ce2 Merge pull request #2539 from alphagov/optimise-orgs-services-for-user
Return all required information about a user’s organisations and services
2019-06-18 10:28:20 +01:00
Chris Hill-Scott
a123b02081 Merge pull request #2533 from alphagov/perf-plat-manual-run
allow you to pass in date to send perf stats
2019-06-18 09:56:12 +01:00
Chris Hill-Scott
64242506f0 Merge pull request #2544 from alphagov/put-firetext-url-in-manifest
AUTO: Explicitly include FIRETEXT_URL in manifest
2019-06-18 09:56:03 +01:00
Andy Paine
57705fd6fe AUTO: Explicitly include FIRETEXT_URL in manifest
- We are explicit about MMG_URL but not FIRETEXT_URL
- credentials has already been updated (checked by doing make
  generate-manifest for all envs)
2019-06-14 15:22:18 +01:00
Chris Hill-Scott
d974ab3b86 Return count of live services on organisations too
This makes it consistent, so the admin app can always rely on that
property being available.
2019-06-12 15:24:11 +01:00
Chris Hill-Scott
b6fdb269e4 Return all required org and services info for user
The admin app now needs to know a few extra things about orgs and
services in order to list them. At the moment it does this by making
multiple API calls.

This commit adds extra fields to the existing response. Once the admin
app is using this fields we’ll be able to remove:
- `reponse['services_without_organisations']`
- `reponse['organisations']['services']`
2019-06-12 15:24:11 +01:00
Leo Hemsted
5045590d75 allow you to pass in date to send perf stats
make it easier to replay sending data for a day if it failed the first
time round
2019-06-11 13:57:17 +01:00
Andy Paine
d19dbe5b61 Merge pull request #2532 from alphagov/enable-statsd-on-staging
AUTO: Enable statsd exporter on staging
2019-06-10 11:40:26 +01:00
Andy Paine
1010b511ee Merge pull request #2531 from alphagov/get-gunicorn-stats
AUTO: Get gunicorn stats
2019-06-10 11:40:11 +01:00
Andy Paine
2d17827780 AUTO: Enable statsd exporter on staging
- We want to do some load testing so we want to use the Prometheus
  metrics for observing the system
- Roll out the statsd exporter work to staging too
2019-06-10 11:12:44 +01:00
Andy Paine
ed9839a8ad AUTO: Get gunicorn stats
- Gunicorn can give us some interestings stats about what it is doing
  that might prove useful when trying to work out the
gunicorn/eventlet/sqlalchemy kind of bottlenecking issues
- We already ship these env vars to the app so this is a pretty easy
  change to make
- Uses statsd which is UDP so it shouldn't have any performance impact
  on gunicorn
2019-06-07 15:33:26 +01:00
Rebecca Law
c8196b0631 Merge pull request #2529 from alphagov/add-expiry-date-to-constraint
Allow api_keys names to be reused
2019-06-07 13:19:28 +01:00
Katie Smith
87cb6f2597 Merge pull request #2513 from alphagov/archive-user
Add endpoint and DAO function to archive user
2019-06-06 09:41:17 +01:00
Katie Smith
ffcd731a68 Add endpoint to archive a user
This archives a user if their state is 'active'.
2019-06-05 14:54:52 +01:00
Katie Smith
bef24408d0 Add DAO function to archive a user
For a user to be able to be archived, each service that they are a
member of must have at least one other user who is active and who has
the 'manage-settings' permission.

To archive a user we remove them from all their services and
organisations, remove all permissions that they have and change some of
their details:
- email_address will start with '_archived_<date>'
- the current_session_id is changed (to sign them out of their current
session)
- mobile_number is removed (so we also need to switch their auth type to
email_auth)
- password is changed to a random password
- state is changed to 'inactive'

If any of the steps fail, we rollback all changes.
2019-06-05 14:54:52 +01:00
Katie Smith
844f22bcd0 Remove unused fixture and add test function to create permissions 2019-06-05 14:54:52 +01:00
Leo Hemsted
aab8b99605 Merge pull request #2524 from alphagov/free-allowance-report
Free sms allowance in billing report
2019-06-05 14:34:55 +01:00
Rebecca Law
a1f6a5f78a Update the unique constraint for the name of an api_key so that it only looks at api_keys that are not expired (or expiry_date is null).
This will allow clients to create a new api key with the same name.
2019-06-04 15:30:27 +01:00
Leo Hemsted
26f6af988b Merge pull request #2527 from alphagov/bump-utils
Bump utils to 32.0.1
2019-06-04 14:32:19 +01:00
Leo Hemsted
202055a88b Merge pull request #2528 from alphagov/delete-count
Fix delete count
2019-06-04 14:06:49 +01:00
Leo Hemsted
7db2b031a2 add free_sms_fragment_limit to platform admin billing report
a little complicated because the free_sms_fragment_limit comes from
the annual_billing table. This relies on there always being at least
one row for every service on annual billing - I checked on prod and
that is true.

Join to the annual billing table, then join to a subquery getting the
latest year for that service to extract only the most recent year.
2019-06-04 11:34:10 +01:00
Leo Hemsted
968d94d350 clean up dao_fetch_live_services
a bit of DRY - use the column definitions to determine what goes into
the dict, and use a `next` iterator rather than a while loop to find
the existing service row. Take advantage of dict mutability to avoid
needing to refer to the list by index.

Also change the tests so if there's an error, the diff is slightly
more readable. But not much
2019-06-04 11:33:19 +01:00
Leo Hemsted
208abfc722 remove unused function 2019-06-04 11:33:19 +01:00
Leo Hemsted
4fc7cc6f2e clean up tests
* remove unnecessary parametrisation where we're not testing
type-specific things
* remove some unused fixtures
* replace len(x.all()) with x.count()
* don't deleted notificationhistory now that we know it's not written
* remove some pre-test function asserts
2019-06-04 11:27:28 +01:00
Andy Paine
226b6e46a2 Merge pull request #2523 from alphagov/point-preview-statsd-at-tools
AUTO-413: Point preview statsd at tools
2019-06-04 10:59:08 +01:00
Leo Hemsted
2ac2cbbd37 don't pass running totals in to functions
or you can easily end up double-counting things. (the test written
previously returned 6)
2019-06-03 17:47:42 +01:00
Leo Hemsted
67f171f2f8 refactor tests
re-order notification dao delete notifications test to move "fixtures"
to the top of the file

changed create_service_data_retention to take an ORM object, not an id.
brings it in line with other db.py test functions
2019-06-03 17:27:08 +01:00
Katie Smith
06d82161cc Bump utils to 32.0.1
This updates requirements and brings in the change to preserve trailing semicolons
on URLs.
2019-06-03 16:15:46 +01:00
Rebecca Law
396149ddde Merge pull request #2525 from alphagov/dont-insert-notification-history
Don't insert notification history
2019-06-03 15:28:48 +01:00
Rebecca Law
cfd42a2eb9 Update subquery to be more efficient.
Update subquery to run again but for test keys. Test data is never inserted in Notifications so they need to be deleted separately now given the join to NotificationHistory.
2019-06-03 15:16:46 +01:00
Leo Hemsted
fb05b5ee80 Merge pull request #2526 from alphagov/cronitor
ensure cronitor decorator is inside the notify_task wrapper
2019-06-03 12:57:48 +01:00
Rebecca Law
b8399b8b9b Add a where clause to join to NotificationHistory, this is some extra assurance that the Notification will not be deleted unless the history exists. 2019-06-03 11:47:02 +01:00
Leo Hemsted
09888f7479 ensure cronitor decorator is inside the notify_task wrapper
the celery decorator should always be on the outside so that all other
decorators will be captured within the celery task. We had problems
with cronitor not reporting, and only for this task.
2019-06-03 11:46:07 +01:00
Rebecca Law
c23ae15f32 Remove insert to NotificationHistory
Fix all test failures
2019-05-31 16:52:22 +01:00
Andy Paine
e61619f3e0 AUTO-413: Point preview statsd at tools
- We are running a statsd exporter on tools to collect all our statsd
  metrics for scraping by Prometheus
- Update preview to point there instead of at the local one which has
  issues with redeployment and DNS changing
2019-05-30 17:03:08 +01:00
Andy Paine
b0baf10229 Merge pull request #2522 from alphagov/use-port-health-checks-for-api
BAU: Use port health checks for API
2019-05-30 14:58:27 +01:00
Rebecca Law
a57648d789 Merge pull request #2521 from alphagov/fix-reference-bug
Add missing reference to the update statement.
2019-05-30 12:16:12 +01:00
Andy Paine
adf81ef689 BAU: Use port health checks for API
- We've been seeing an issue when traffic spikes of the http health
  checks taking over 1s and PaaS killing the app
- Port health checks won't care about being stuck in a queue so should
  continue to work even at high loads
- We have functional tests to catch if a deployment brings up the app
  (and so passes port health check) but then doesn't work
2019-05-30 11:56:19 +01:00
Rebecca Law
4154251970 Addd missing reference to the update statement. 2019-05-30 10:54:47 +01:00
Pea (Malgorzata Tyczynska)
82e27c4064 Merge pull request #2511 from alphagov/verify_new_reply_to_address
Verify new reply to address
2019-05-28 16:16:06 +01:00
Pea Tyczynska
c4d20667a6 Add notification deletion to downgrade to respect foreign key constraints 2019-05-28 15:25:06 +01:00
Pea Tyczynska
5f1f688c7b Create template to verify service email reply-to addresses
So that template with the same ID is present on all environments
2019-05-28 15:14:09 +01:00
Rebecca Law
1490c8e377 Merge pull request #2520 from alphagov/fix-bug
Fix a bug introduced when refactoring some code.
2019-05-23 17:12:21 +01:00
Rebecca Law
2844fa530b Fix a bug introduced when refactoring some code. The notification status happened in the wrong order - this resolved that.
This meant notifications sent with a test key never got a 'delivered' status.
2019-05-23 17:04:55 +01:00
Pea Tyczynska
5692a8596d Code refactors and corrections, details below:
Change method name to be more relevant

Check if verification notification we send is a correct one

pass in notify db session for tests instead of sample_user

Refactor tests by using admin_request instead of client

Refactor all tests for affected reply-to endpoints for good measure

Allow overwriting reply-to address with the same address

Skip checking for duplicates if it's an reply-to email update

Fix refactored tests

Verify duplicates exception not needed
2019-05-23 15:36:09 +01:00
Pea Tyczynska
3c3dde635b Prevent service from adding duplicate reply-to addresses
Check for duplicate reply-to email address has been added on:
-verification endpoint, so we do not send the verifying notification
needlessly
- add reply-to email address and update reply-to email address
endpoints, as those can be hit multiple times after the email address
has been verified (so the same email address could end up being added
multiple times). EDIT: this has now been prevented on admin app,
but it's better to retain double-check for safety.
2019-05-23 15:36:09 +01:00