Commit Graph

7236 Commits

Author SHA1 Message Date
Katie Smith
19bf68b567 Delete unused function 2020-03-19 13:41:14 +00:00
Pea M. Tyczynska
900664c172 Merge pull request #2748 from alphagov/format-postcode-for-csvs
Format postcode for CSV letter rows
2020-03-19 11:07:08 +00:00
Rebecca Law
67339d7fd9 Merge pull request #2760 from alphagov/add-try-catch
Add a try/except around the code to get the files.
2020-03-19 09:28:10 +00:00
Rebecca Law
7459a4f6f6 Add a try/except around the code to get the files.
The idea is to log the exception but keep going. That way the "good" files still get sent and we can investigate why a file failed.
2020-03-19 09:15:38 +00:00
Rebecca Law
4ebfce6b8d Merge pull request #2756 from alphagov/stop_checking-daily_limit
Remove the check for daily limits
2020-03-17 16:03:15 +00:00
Rebecca Law
852cf478f8 Remove the check for daily limits 2020-03-17 15:51:21 +00:00
Rebecca Law
ac07ea3e3f Merge pull request #2755 from alphagov/fix-serialization-error
Fix serialisation error
2020-03-17 10:44:47 +00:00
Chris Hill-Scott
90783c0f23 Merge pull request #2754 from alphagov/update-jobs-contact_list_id
Update jobs contact list
2020-03-17 10:23:49 +00:00
Chris Hill-Scott
6667c04cf2 Fix unused import 2020-03-17 10:07:47 +00:00
Rebecca Law
51f43563d3 Fix serialisation error when creating the create_letters_pdf in resend_created_notifications_older_than 2020-03-17 08:48:02 +00:00
Rebecca Law
95c2dabaca Add service_contact_list id to the JobSchema. 2020-03-17 08:20:01 +00:00
Rebecca Law
b30deaa989 Increase the queue pool size to 30.
This should add an extra 400 connections maximum, which will not tip us over the allowable 5000 db connections. And it may help with the queue pool connection errors.
2020-03-16 16:46:19 +00:00
Rebecca Law
8545b097f9 [WIP] 2020-03-16 16:45:34 +00:00
Rebecca Law
fa72eb651f Merge pull request #2750 from alphagov/get-single-contact-list
Endpoint to return a ServiceContactList for a given id.
2020-03-16 09:22:24 +00:00
Pea Tyczynska
bf1cab76a9 Bump utils 2020-03-13 17:35:47 +00:00
Pea Tyczynska
1fb040dc61 Format postcode for CSV letter rows
We are formatting the postcode here, because if we did it in template
preview, that could break flows like API and admin one-off, since
we are not validating postcode there yet, and format_postcode
needs a nice validated postcode.

We are not doing it in admin, as then we would have to either
rewrite the CSV file or pass data differently to API. First would
be nasty, second is a lot of overhead.

In the long run we might want to move postcode formatting to
template preview so that the postcode in letter preview looks the same
before and after user sends it, but now to get it out quickly it's better
to format the postcode here in the task.
2020-03-13 17:35:36 +00:00
Rebecca Law
3bf18d0ac3 Endpoint to return a ServiceContactList for a given id. 2020-03-13 17:21:59 +00:00
Rebecca Law
069bc1cdb6 Merge pull request #2749 from alphagov/emergency-contact-list-tbl
New table and endpoints for service contact lists
2020-03-13 12:21:32 +00:00
Rebecca Law
654e6fc657 New table and endpoints for service contact lists.
- Table to store meta data for the emergency contact list for a service.
- Endpoint for fetching contact lists for service
- Endpoint for saving contact list for service.

The list will be stored in S3. The service will then be able to send emergency announcements to staff.
2020-03-13 12:11:16 +00:00
Chris Hill-Scott
7636c4225d Merge pull request #2727 from alphagov/ship-uploads
Give existing services upload letters permission
2020-03-11 16:22:15 +00:00
Chris Hill-Scott
e12829e56e Give existing services upload letters permission
This gives the feature to all existing users.
2020-03-11 15:31:53 +00:00
David McDonald
f9751d6ebd Merge pull request #2746 from alphagov/fix-non-content-outside-errors
Undo bug for previewing precompiled letters
2020-03-11 15:29:10 +00:00
Chris Hill-Scott
87428685fb Merge pull request #2740 from alphagov/invited-by-notify
Let the Notify team invite people to an organisation
2020-03-11 14:52:05 +00:00
Chris Hill-Scott
4374ed6220 Merge pull request #2744 from alphagov/find_jobs_within_retention
Find only uploads within service data retention period
2020-03-11 14:51:56 +00:00
David McDonald
483c00b3a2 Undo bug for previewing precompiled letters
https://github.com/alphagov/notifications-api/pull/2742/files#diff-d9a6761afaff4a491e60b64f6063654fL265
introduced a change in behaviour which causes template preview to 500
for a precompiled letter that is invalid but not because the content is
outside the printable area.

The changes it back to the previous behaviour, where we b64encode and
utf-8 decode for a page that is invalid for reasons that are not content
outside the printable area, for example a non portrait a4 letter.

I didn't end up writing a unit test for this because I really can't
figure out the existing behaviour we expect for this use case. This
method is too big and complex but I'm confident by looking at the change
in the commit mentioned above that this puts behaviour back to how it
was. Manual testing also confirms the fix.

I've also taken this opportunity to improve two test names whilst I was
looking at things.
2020-03-11 14:34:12 +00:00
Pea Tyczynska
8b60e69157 Finding only jobs and letters within data retention works and tested 2020-03-10 15:53:56 +00:00
Rebecca Law
5902960516 Merge pull request #2741 from alphagov/make-sms-char-count-consistent
Update validators to use is_message_too_long()
2020-03-10 14:59:28 +00:00
David McDonald
0ba0bdca15 Merge pull request #2742 from alphagov/overlay-sometimes
Only add overlay for pages which are invalid for precompiled letters uploaded via the API
2020-03-10 13:54:22 +00:00
David McDonald
34d571405e Refactor of logic to make code more readable
Functionality remains as it was
2020-03-10 13:38:20 +00:00
Katie Smith
d0cb5db19b Merge pull request #2743 from alphagov/international-letters-permission
Add international_letters service permission
2020-03-10 10:12:54 +00:00
Rebecca Law
a994e8fb6e Update validators to use is_message_too_long()
- update check_sms_content_char_count to use the SMSTemplate.is_message_too_long function, and updated the error message to align with the message returned by the admin app.
- Update the the code used by version 1 of the api to use the validate_template method.
  - I did find a couple of services still using the old api, however, this change should not affect them as I checked the messages being sent and they are not too long.
  - We will be sending a message to them to see if they can upgrade.
- Update the log message for authenication to include the URL - makes it easier to track if a service is using version 1 of the api.
2020-03-10 09:38:16 +00:00
Katie Smith
a9abc63297 Add international_letters service permission 2020-03-09 15:42:07 +00:00
David McDonald
08ec06295a Merge pull request #2738 from alphagov/pyup-scheduled-update-2020-03-04
Scheduled weekly dependency update for week 09
2020-03-09 15:41:29 +00:00
David McDonald
a8f2aa2022 run make freeze-requirements 2020-03-09 15:29:26 +00:00
pyup-bot
6d22c8fe14 Update marshmallow-sqlalchemy from 0.22.2 to 0.22.3 2020-03-09 15:29:26 +00:00
David McDonald
d9243b8b8a When requesting a png for a single page, show overlay as appropriate
If the png page is invalid then show the overlay for that page
If the png page is not invalid, even if other pages are invalid, then
don't shown the overlay for that page

These keeps the behaviour were if you get the pdf for a pdf which has
invalid pages then the whole PDF is requested (not just a single page)
and all of the pages include the overlay

Tests have been improved to be more explicit about what we are testing
2020-03-09 14:54:58 +00:00
David McDonald
9eda30f4fd Refactor tests to be a bit more readable
Make arguments more verbose and have pytest params as inputs at the
start and expected things at the end
2020-03-09 14:48:33 +00:00
David McDonald
d168dd106e Add uncovered test cases
We want to know that the overlay is added regardless of which page
number you are requesting when a pdf contains pages that go beyond the
print area
2020-03-09 14:47:31 +00:00
David McDonald
296a8b1ed7 Merge pull request #2736 from alphagov/tidy-makefile
Tidy makefile
2020-03-09 11:58:16 +00:00
Chris Hill-Scott
851435701f Merge pull request #2737 from alphagov/returned-letter-statistics
Add an endpoint to return returned letter stats
2020-03-05 16:11:09 +00:00
Chris Hill-Scott
6fd92c4931 Test for letters not returned recently
If you have some returned letters, but none in the last 7 days, we don’t
count how many you have in the last 7 days. But we should test to make
sure we’re not going to the database again.
2020-03-05 14:32:04 +00:00
David McDonald
2a4b1a58f8 Merge pull request #2739 from alphagov/statsd-prefix
Remove unused statsd_prefix variable
2020-03-05 13:43:47 +00:00
Chris Hill-Scott
bdfeb08055 Let the Notify team invite people to an organisation
We want to start granting access to the org page. But it will be a bit
weird if the invites come from us personally, since the people we’re
inviting don’t know us.

It makes more sense, and sounds more official if the invites appear to
come from the ‘GOV.UK Notify team’ instead.
2020-03-05 12:34:17 +00:00
Chris Hill-Scott
02c824a980 Test for scenario when no returned letters exist 2020-03-05 11:10:32 +00:00
Chris Hill-Scott
70b2afa124 Rename method to be clear it’s recent-only 2020-03-05 11:10:32 +00:00
David McDonald
f56795655e Remove unused STATSD_PREFIX variable
We moved from sending statsd metrics to hosted graphite to sending to
one that is running on the paas. Therefore we no longer need to send
statsd metrics to a particular prefix at the statsd app as it is only
receiving statsd metrics from our apps (not other users like would have
been the case with HostedGraphite).

This should change no behaviour as the only place the environment
variable was being used was in the gunicorn config and it was an empty
string which is the default behaviour anyway as per:
https://docs.gunicorn.org/en/stable/settings.html#statsd-prefix
2020-03-05 10:41:26 +00:00
Chris Hill-Scott
8c47d07845 Optimise method to prevent redundant counting
If we know that the most recently returned letter was reported more than
7 days ago then we know, without having to go to the database again,
that the count of returned letters in the last 7 days is 0.
2020-03-05 10:00:30 +00:00
Chris Hill-Scott
9c03438a53 Add an endpoint to return returned letter stats
Currently the dashboard in the admin app pull the entire returned letter
summary for a service to calculate how many letters have been returned
in the last seven days.

Adding a separate endpoint for this purpose is better because:
- it’s a more efficient query
- it’s less data to send down the pipe
- it gives us a place to return the complete datetime, so the dashboard
  can be more precise about when the most recent report was
2020-03-03 17:16:54 +00:00
Chris Hill-Scott
b952b35714 Merge pull request #2726 from alphagov/launch-uploads
Give new services the upload letters permission
2020-03-03 14:38:32 +00:00
Chris Hill-Scott
223ec1360c Merge pull request #2735 from alphagov/dont-check-upload-letter-permission
Remove check on permission to upload letters
2020-03-03 12:04:01 +00:00