Commit Graph

3606 Commits

Author SHA1 Message Date
Chris Hill-Scott
57473ef65d Rename template_content to content
`template_content` is a template that hasn't had its placeholders filled
in whereas content is where it would have (which is what has happened
above)
2020-10-08 11:39:17 +01:00
Chris Hill-Scott
80352bafe1 Update test to check personalisation
While the API still has a column and field for personalisation I think
it makes sense for it to consider the personalisation when serialising
the broadcast, so we should have a test for this.

This way the API still works as a coherent whole, and the admin app just
happens to be a client of the API which doesn’t implement the
personalisation feature.

If we want to remove personalisation from the API at another time we
should do it wholesale.
2020-10-07 10:52:31 +01:00
Chris Hill-Scott
7da0d46767 Serialise content in broadcast message response
This will let us show the content of the broadcast message in places we
can’t at the moment.
2020-10-02 16:39:24 +01:00
David McDonald
cfe32cf459 Merge pull request #2975 from alphagov/rename-redis-template-keys
Step 2 of renaming cache keys for templates
2020-10-02 14:26:03 +01:00
Chris Hill-Scott
f2314333b5 Merge pull request #2982 from alphagov/improve-efficiency-of-process-missing-rows
Improve efficiency of process missing rows task
2020-10-02 11:23:35 +01:00
Chris Hill-Scott
19963b9b58 Merge pull request #2984 from alphagov/add-stats-endpoint-for-scheduled-jobs
Add an endpoint for stats about scheduled jobs
2020-09-29 10:24:09 +01:00
Leo Hemsted
59839d21b8 Merge pull request #2974 from alphagov/make-letter-folder-name-code-readable
Make letter folder name code readable
2020-09-28 13:58:08 +01:00
Chris Hill-Scott
4cee47da68 Add more tests for scheduled job stats
Ensures we don’t count jobs that are in a different status, or belong to
another service.
2020-09-28 13:15:39 +01:00
Rebecca Law
e4e61b1e4b Merge pull request #2978 from alphagov/add-reference-to-msg
Add reference to log message
2020-09-28 12:04:59 +01:00
Chris Hill-Scott
2fcde009ac Add an endpoint for stats about scheduled jobs
At the moment we display the count of scheduled jobs on the dashboard
by sending all the scheduled jobs to the admin app and letting it work
out the stats.

This is inefficient and, because the get jobs response has a page size
of 50, becomes incorrect if a service schedules more than 50 jobs.

This commit adds a separate endpoint which gives the admin app the stats
it needs directly and correctly.
2020-09-28 09:57:32 +01:00
Chris Hill-Scott
1d50bfaafc Remove unused column from query 2020-09-26 12:11:15 +01:00
Chris Hill-Scott
aace1bdd8a Allow 20 minutes before checking for missing rows
Since we’ve doubled the number of rows in a job, jobs can take twice as
long to insert all the notifications. We don’t check for missing rows
until we’re pretty confident that the original tasks have finished
processing. This means we need to double the time we wait to still be
as sure.
2020-09-26 11:38:38 +01:00
Rebecca Law
d7e53cdf50 Adding reference to message for "precompiled letters have been pending-virus-check for over 90 minutes"
This saves having to go to the db to get it.
2020-09-24 14:16:16 +01:00
David McDonald
5b0cc8819a Use new redis key for templates
Equivalent of https://github.com/alphagov/notifications-admin/pull/3639
but for the API.

This is the only place in the API we appear to be getting setting
templates or broadcast messages.
2020-09-21 17:40:54 +01:00
Leo Hemsted
1e928a926a rename sending_date to created_at
we don't name letters based on the day we send them on, rather, the day
we create them on. If we process a letter for a second time for whatever
reason, even if it's a couple of days later, it'll still go in a folder
based on the created_at timestamp. There's still a slight confusion,
however - if the timestamp is after 5:30pm, the folder will be for the
day after. However, still the day after creation, so I think created_at
still makes the most sense.

Remove the term `sending_date` to try and make this relationship more
apparent.
2020-09-21 14:40:22 +01:00
Leo Hemsted
4f1e5ee5a3 remove ignore_folder from get_folder_name
it doesn't make sense to get the folder name if you know you aren't
going to use it.
2020-09-21 14:33:38 +01:00
Leo Hemsted
bb33927b3d rename letter get_folder_name args
`_now`? why would we ever use a different _now? instead say created_at,
because that's what it'll always be set to, even if we're replaying old
letters. We always set the folder name to when the letter was
created_at, or we might not know where to look to find it.

`dont_use_sending_date` doesn't really tell us what might happen if we
don't use it - the answer is we return an empty string. we ignore the
folder entirely. so lets call it that.

Also, remove use of freeze_gun in the tests, to prove that we don't use
the current time in any calculations. Also add an assert to a mock in
the get_pdf_for_templated_letter test, because we were mocking but not
asserting before, so the tests didn't fail when the function signature
changed.
2020-09-21 14:32:57 +01:00
David McDonald
120b760cae Merge pull request #2972 from alphagov/broadcast-invite-template
Broadcast invite template
2020-09-16 14:34:31 +01:00
David McDonald
f9911c7965 Send broadcast invite email for broadcast service invites
This means the copy is more accurate and mentions sending emergency
alerts rather than previous copy about sending emails texts and letters.
2020-09-15 16:47:41 +01:00
David McDonald
23ac7bcf56 Use sample broadcast service instead of sample service for tests
We only want to use a service that has permissions for broadcasts to
make our tests more realistic
2020-09-15 16:47:41 +01:00
Katie Smith
aee7887c14 Fix the filenames for international precompiled letters
We were determing the filename for precompiled letters before we had
checked if the letters were international. This meant that a letter
could have a filename indicating it was 2nd class, but once we had
sanitised the letter and checked the address we were setting the
notification to international.

This stopped these letters from being picked up to be sent to the DVLA,
since the filename and postage of the letter did not match.

We now regenerate the filename after the letter has been sanitised (and when
we know the postage) and use the updated filename when moving the letter
into the live PDF letters bucket.
2020-09-15 16:17:33 +01:00
David McDonald
5b2dee5ddb Bump utils to 42.0.0
Requires unit test updating as we now expect broadcast event areas to
be a dict containing a list of areas and simple polygons.
2020-09-14 15:21:55 +01:00
David McDonald
d352c99142 Remove unused send_broadcast_message task
We only call send_broadcast_event now
2020-09-14 15:16:59 +01:00
David McDonald
5aee6269e0 Fix format of simple_polygons
It is not of the form
[[lat, long][lat, long]] as this would only hold a single polygon. It
instead needs to handle multiple polygons so instead is of the form
[[[lat, long][lat, long]]].
2020-09-14 14:55:01 +01:00
Chris Hill-Scott
92af5b8d67 Merge pull request #2968 from alphagov/cancel-international-letters
Allow international letters to be cancelled
2020-09-09 14:10:23 +01:00
Chris Hill-Scott
cfda289746 Allow international letters to be cancelled
Our code was assuming that any notifications with `international` set to
`True` were text messages. It was then trying to look up delivery
information for a notification which wasn’t sent to a phone number,
causing an exception.
2020-09-09 10:55:55 +01:00
Rebecca Law
795a035fac When the organisation updates the crown attribute it should update all the services associated with that organisation too. 2020-09-09 10:43:16 +01:00
Pea Tyczynska
b3ff172780 Validate that both polygons and areas have to be updated
Because areas and polygons for broadcast messages should
be in sync.
2020-09-07 15:54:31 +01:00
Pea Tyczynska
5927aae019 Serialise simple_polygons attribute of BroadcastMessage 2020-09-07 15:54:31 +01:00
Pea Tyczynska
2f6b96dafc Ensure broadcast with no areas cannot be approved
This is already checked by admin app, but we want to have a
fallback safety measure as a precaution.
2020-09-07 15:54:30 +01:00
Pea Tyczynska
5cf6e1cf72 Persist simple polygons in the db.
They are being sent over from admin, and persisted
in the db so we can send them on to the broadcast
provider later on.
2020-09-07 15:52:14 +01:00
Rebecca Law
93475912ba Merge pull request #2950 from alphagov/international-letters-for-all
Default international_letters for service permissions.
2020-09-07 07:39:52 +01:00
Katie Smith
b30701d7e1 Set 'international' for letters in ft_billing
`international` for letters in `ft_billing` was always False. Now that
letters can be international, this changes the column value to the value
of `international` for the notification.
2020-08-21 09:19:27 +01:00
Katie Smith
8dcc65d11c Add endpoints to get invited users
We want to display flash messages in admin when invites have been
cancelled. This message needs to display the user's email address, so
this commit adds endpoints to GET a single invited service user and org
user so that we can look up the email address of a cancelled user.
2020-08-19 08:49:33 +01:00
David McDonald
db0655472a Merge pull request #2957 from alphagov/missing-template-version
Fix bug with missing template version param
2020-08-17 17:33:29 +01:00
David McDonald
7b5f4ad156 Fix bug with missing template version param
We were asking for the latest version of a letter template rather than
the version that the notification was sent with. This mean that if you
previewed a letter and had made edits to the template since it was sent
you would be shown an incorrect preview.
2020-08-17 15:30:16 +01:00
Leo Hemsted
bdf2253298 send broadcast events rather than messages
use the new endpoint from cbc proxy. create a new task that just
serializes the event and sends it across rather than sending a template
and the broadcast message.

some changes to serialize to make it json friendly etc. it also expects
sent_at and transmitted_finishes_at to always be set (we set them in the
code but don't enforce it n the DB right now), as they're required by
utils template. not sure whether we'll update db constraints to be more
strict or utils template to be more permissive just yet, wait until we
find out more about the requirements of the CBCs we integrate with.
2020-08-14 17:41:44 +01:00
Leo Hemsted
6fda3707a3 save broadcast events when sending a message, also send cancel messages 2020-08-14 10:47:28 +01:00
David McDonald
36614e5492 Log warning for SES send rate throttling rather than exception
We have hit throttling limits from SES approximately once a week during
a spike of traffic from GOV.UK. The rate limiting usually only lasts a
couple of minutes but generates enough exceptions to cause a p1 but with
no potential action for the responder.

Therefore we downgrade the warning for this case to a warning and assume
traffic will level back out such that the problem resolves itself.

Note, we will still get exceptions if we go over our daily limit, rather
than our per minute sending limit, which does require immediate action
by someone responding.

If we were to continually go over our per second sending rate for a long
continous period of time, then there is a chance we may not be aware but
given the risk of this happening is low I think it's an acceptable risk
for the moment.
2020-08-13 17:51:09 +01:00
Katie Smith
f47e9116d6 Delete test for Redis config in sandbox
We don't use separate config per environment for Redis now.
2020-08-13 09:54:03 +01:00
Katie Smith
fb36404b6c Fix error message when invitation has expired
The error message for when an invitation to Notify had expired was
displaying in admin with square brackets round it because admin is not
expecting the message to be a list
(a85134ee22/app/models/user.py (L500))
2020-08-12 18:30:00 +01:00
Rebecca Law
d9fd541ab7 Add international letters as a default permission when creating a new service 2020-08-11 15:59:09 +01:00
Rebecca Law
89a8d8912a Set postage and international for letters uploaded with a CSV
If the letter is outside of the United Kingdom, set the postage and international flag.
2020-08-10 09:58:49 +01:00
Rebecca Law
725ce38a06 Merge pull request #2943 from alphagov/international-postage
Set international postage and international flag for international letters
2020-08-10 08:22:52 +01:00
Leo Hemsted
05791f22a6 allow trial mode service users to approve their own broadcasts
we won't let trial mode services send real broadcasts, and it's helpful
for users to see the flow of messages without having to have a second
person with them
2020-08-06 11:15:49 +01:00
Rebecca Law
4a9f9e4b17 Remove the template_postage parameter for persist_notification
It was confusing to have 2 differnt postage parameters.
2020-08-06 07:35:13 +01:00
Chris Hill-Scott
929e1fbbdc Remove whitelist from URLs
The admin app is now[1] using the newer, `…/guest-list` URLs, so we can
remove the older, deprecated, `…/whitelist` ones.

***

1. https://github.com/alphagov/notifications-admin/pull/3536
2020-08-03 17:13:26 +01:00
Chris Hill-Scott
32f5f454de Merge pull request #2928 from alphagov/serve-on-slash-guest-list
Rename API URLs for guest list to guest list
2020-08-03 16:44:41 +01:00
Rebecca Law
10fe7d9fe8 Add postage for send-one-off letters.
The postage is set to europe or rest-of-world for international letters, otherwise the template postage is used.

Also set international for letters.
2020-08-03 14:01:59 +01:00
Rebecca Law
ed5e73d548 Set postage for templated letters when the address is not from the united-kingdom.
If the address is from the united-kingdom use the postage from the template.
2020-08-03 12:03:35 +01:00