Commit Graph

7420 Commits

Author SHA1 Message Date
Chris Hill-Scott
b19451c7c6 Rename DAO file
To reflect new name of feature
2020-07-28 12:56:40 +01:00
Chris Hill-Scott
65346852ed Rename variables and functions in tests
To reflect the new name of the feature.
2020-07-28 12:56:32 +01:00
Chris Hill-Scott
716eb67bfd Re-label error messages
To reflect the new name of the feature.
2020-07-28 12:56:23 +01:00
Chris Hill-Scott
e41022214f Rename backref to service model
To reflect the new name. Appears this is only used by the tests.
2020-07-28 12:56:14 +01:00
Chris Hill-Scott
083573e4dc Rename model
Reflects the new name of the feature.

Note that the name of the underlying table hasn’t changed because it’s
explicitly set to `service_whitelist`. Changing this will be a more
involved process.
2020-07-28 12:56:04 +01:00
Chris Hill-Scott
4d896aa642 Rename function in service utils
To reflect the new name of the feature.

squash! Rename function in service utils

Rename function, variable and argument names in service utils
2020-07-28 12:55:55 +01:00
Chris Hill-Scott
6384b9ef4f Rename whitelist DAO functions
To reflect the new name of the feature.
2020-07-28 11:24:34 +01:00
Chris Hill-Scott
7d09599bc5 Rename methods and variables in rest.py
To reflect the new name of this feature.
2020-07-28 10:23:56 +01:00
Chris Hill-Scott
e9fed12a1e Rename API URLs for guest list to guest list
This is a small part of removing the term ‘white list’ from the API.

Once the admin app is pointed at these new URLs, we can remove the old
ones.
2020-07-28 10:23:56 +01:00
Chris Hill-Scott
bc0cfed468 Merge pull request #2855 from alphagov/contact-list-used-property
Add property to contact lists to say if they’ve ever been used
2020-07-27 11:42:54 +01:00
Rebecca Law
80e4dc8af8 Merge pull request #2935 from alphagov/remove-jobs-incomplete-error
Stop raising error in scheduled task
2020-07-24 07:49:35 +01:00
Pea M. Tyczynska
a2f911b253 Merge pull request #2924 from alphagov/add-contact-block-to-serialize-template
Add contact block to serialize template for v2
2020-07-23 15:42:17 +01:00
Rebecca Law
dd126df122 We have a scheduled task to check that all the jobs have completed, this will catch if an app is shut down and the job is complete yet, we only wait 10 seconds before forcing the app to shut down.
The task was raising a JobIncompleteError, yet it's not an error the task is performing it's task correctly and calling the appropriate task to restart the job.
Also used apply_sync to create the task instead of send_task.
2020-07-22 17:00:20 +01:00
Toby Lorne
ec5eeac0aa Merge pull request #2933 from alphagov/stub-cbc-urls
Isolated CBC stubs
2020-07-21 16:55:13 +01:00
Chris Hill-Scott
313b69f95c Make property methods into proper properties
Since these methods only take `self` as an argument they can be
properties. And this means we don’t need to follow the semi-convention
we have of indicating an attribute is a method by starting its name with
`get_`.

The other advantage of using `@property` to indicate a getter is that it
will raise an exception if someone tries to set the attribute, eg by
doing `contact_list.has_jobs = False`. This is because we (rightly)
haven’t defined a setter.
2020-07-21 15:12:44 +01:00
Chris Hill-Scott
dbbff3ba64 Add property to contact lists to say if they’ve ever been used
At the moment we return a count of recent jobs for contact lists, where
recent is defined as being within the service’s data retention period.

This lets us write nice bits of UI copy like ‘used 3 times in the last
7 days’. But it’s hard to write the copy for when the count is 0,
because this could be for one of two reasons:
- the contact list has never been used
- the contact list has been used, but not within the data retention
  period for that channel

At the moment we can’t know which of those reasons is the case, so we
can’t write nice clear content like ‘never been used’.

This commit adds a property to contact lists which says whether they’ve
ever been used.

It also renames the existing, as-yet-unused property to make clear that
it’s only counting within the data retention (so can still be 0 even if
`has_jobs` is `True`).
2020-07-21 15:08:05 +01:00
Toby Lorne
fd389d5c76 cbc: use one stub cbc per environment
so we can break preview without breaking staging or production

related https://github.com/alphagov/notifications-cbc-proxy/pull/7

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-07-21 13:39:10 +01:00
Katie Smith
5b401eaf8d Merge pull request #2929 from alphagov/int-letter-rate-fix
Fix international letter rate start dates
2020-07-16 10:35:42 +01:00
Rebecca Law
c9831b0a44 Merge pull request #2930 from alphagov/change-mmg-firetext-ratio
Change MMG Firetext ratio
2020-07-15 14:23:58 +01:00
Rebecca Law
8bba04e1ce Update the MMG/Firetext ratio to 70/30 as requested. 2020-07-15 14:07:19 +01:00
Katie Smith
7c12c6c8f0 Fix international letter rate start dates
These were not taking timezones into account before.
2020-07-15 10:23:54 +01:00
Katie Smith
3fa25272b6 Merge pull request #2888 from alphagov/international-letter-rates
Add international letter rates
2020-07-15 08:36:59 +01:00
Katie Smith
da8eaaed44 Update letter data for usage-for-all-services report
Usage for all services is a platform admin report that groups letters by
postage. We want it to show `europe` and `rest-of-world` letters under a
single category of `international`, so this updates the query to do
that and to order appropriately.
2020-07-14 10:22:30 +01:00
Katie Smith
5d47e1cf06 Migration to add international letter rates
New rows giving the prices of letters with a post_class of `europe` and
`rest-of-world` have been added to the `letter_rates` table. All rates
are currently the same for international letters.
2020-07-14 10:22:30 +01:00
Pea M. Tyczynska
fbdfa6416f Merge pull request #2921 from alphagov/remove-statsd-http-api-decorators
Remove statsd http api decorators and turn statsd back on for celery apps
2020-07-14 10:16:44 +01:00
Pea Tyczynska
f6f88750b5 Add contact block to serialize template for v2
This attribute has been requested by a user, and we have not been
serializing it so far.
2020-07-13 17:04:01 +01:00
David McDonald
e6578e80d0 Merge pull request #2926 from alphagov/more-logging
Add better logging to starting collate-letter-pdfs-to-be-sent
2020-07-10 11:46:42 +01:00
David McDonald
ef551247b5 Add better logging to starting collate-letter-pdfs-to-be-sent
This will help us better understand how far through the task has got if
it gets interrupted halfway (as was the case this morning and we
struggled to understand).
2020-07-10 11:32:03 +01:00
David McDonald
247b926bde Merge pull request #2920 from alphagov/bump-gds-metrics
Fix risk of uncaught exceptions due to gds metrics
2020-07-10 10:45:52 +01:00
David McDonald
34538bcad8 Fix risk of uncaught exceptions due to gds metrics
Similar to https://github.com/alphagov/notifications-admin/pull/3510

Because of https://github.com/alphagov/gds_metrics_python/pull/8
2020-07-10 10:37:09 +01:00
Leo Hemsted
4de75b8c56 Merge pull request #2925 from alphagov/send-broadcast
Send broadcast
2020-07-10 09:22:21 +01:00
Leo Hemsted
8b4a954f2b move schema import in to function level
solves `AttributeError: 'DummySession' object has no attribute 'query'`

if you don't do this you get really hard to diagnose errors in unrelated
tests, due to strange import order problems or something
2020-07-09 20:10:34 +01:00
Leo Hemsted
403885722e trigger send_broadcast_message task when user approves
lets leave the cancellation can of worms alone for now
2020-07-09 18:23:30 +01:00
Leo Hemsted
eca37d0853 add send_broadcast_message task
task takes a brodcast_message_id, and makes a post to the cbc-proxy
for now, hardcode the url to the notify stub. the stub requires template
as the admin/api get it, so use the marshmallow schema to json dump it.
Note - this also required us to tweak the BroadcastMessage.serialize
function so that it converts uuids in to ids - flask's jsonify function
does that for free but requests.post doesn't sadly.

if the request fails (either 4xx or 5xx) just raise an exception and let
it bubble up for now - in the future we'll add retry logic
2020-07-09 18:23:30 +01:00
Leo Hemsted
fb64cbe621 Merge pull request #2923 from alphagov/fix-quick-typo-bug
set finishes_at correctly
2020-07-09 15:58:43 +01:00
Leo Hemsted
6b5e2af497 set finishes_at correctly
whoops
2020-07-09 15:36:08 +01:00
Leo Hemsted
a71530fe06 Merge pull request #2917 from alphagov/broadcast-crud
Broadcast Message CRUD endpoints
2020-07-09 15:16:57 +01:00
Leo Hemsted
efa2e75e56 add broadcast message tests
sorry for big unhelpful test commit
2020-07-09 14:19:58 +01:00
Leo Hemsted
61a5730596 add more friendly datetime validator to jsonschema
add `datetime` format (note, not the built-in `date-time`) to our json
schemas. this uses the iso8601 library to try and parse the string.

also, move `strict-rfc3339` and `rfc3987` (used by jsonschema to
validate `date-time` and `uri` formats respectively from test
requirements to regular requirements. if they're not installed,
validation silently succeeds, so validation wouldnt reject anything bad
on prod, only in unit tests.
2020-07-09 14:19:58 +01:00
Leo Hemsted
67f6dcae45 add broadcast message crud
new blueprint `/service/<id>/broadcast-message` with the following
endpoints:

* GET / - get all broadcast messages for a service
* GET /<id> - get a single broadcast message
* POST / - create a new broadcast message
* POST /<id> - update an existing broadcast message's data
* POST /<id>/status - move a broadcast message to a new status

I've kept the regular data update (eg personalisation, start and end
times) separate from the status update, just to keep separation of
concerns a bit more rigid, especially around who can update.

I've included schemas for the three POSTs, they're pretty
straightforward.
2020-07-09 14:19:56 +01:00
Pea Tyczynska
fff004da2b Turn statsd back on, but not for api main app, only for delivery
workers.
2020-07-09 10:10:24 +01:00
Rebecca Law
40f747d8ef Merge pull request #2911 from alphagov/clean-up
Clean up
2020-07-09 07:56:22 +01:00
Pea M. Tyczynska
5538c9697b Merge pull request #2905 from alphagov/dont-fail-check-migration
always run migrations if app is down
2020-07-08 11:49:32 +01:00
Pea M. Tyczynska
9186083ea7 Merge pull request #2796 from alphagov/split-letters-into-zips-based-on-postage
Split letters into zips based on postage
2020-07-08 11:49:21 +01:00
Leo Hemsted
d2007674a9 Merge pull request #2918 from alphagov/serialised-template-fix
remove broadcast_data from SerialisedTemplate.ALLOWED_PROPERTIES
2020-07-08 09:55:24 +01:00
Pea Tyczynska
9c4205c7c6 Remove statsd decorators from dao functions
This done so that we do not use statsd on our http endpoint.
We decided we do not need metrics that this gave us. If we
change our minds, we will add Prometheus-friendly decorators
instead in the future.
2020-07-07 18:02:24 +01:00
Leo Hemsted
9c25b7dfd5 remove broadcast_data from SerialisedTemplate.ALLOWED_PROPERTIES
ALLOWED_PROPERTIES is a list containing fields that it will attempt to
deserialize into the returned object. If a field isn't present on the
underlying data source (whether that's a dump from a marshmallow schema,
a blob retrieved from redis, or anything else), then SerialisedModel
will raise an exception. However, SerialisedModel isn't involved when
setting the cache, so with that said the correct flow for adding a
column to a cached database model:

PR #1

* add new column to DB
* add new column to model, and to template_schema (because that schema
  is used to create the dict that goes in to redis). New redis keys
  start getting populated.

Deploy that through, and then clear redis

PR #2

* add new column to SerialisedTemplate.ALLOWED_PROPERTIES. this means
  that it'll start reading that value from redis
* now instances of the app will always have the new field in their
  template objects, whether they came from database directly or from
  the cache

This commit removes the field from ALLOWED_PROPERTIES. After it's
deployed we'll be able to clear redis, observe redis being populated
with the new field, and then we'll be able to re-add it to
ALLOWED_PROPERTIES, ready to use.
2020-07-07 17:00:44 +01:00
Leo Hemsted
c3dadde505 Merge pull request #2914 from alphagov/broadcast-models
Broadcast DB models
2020-07-07 15:25:00 +01:00
Leo Hemsted
3486355354 remove any broadcast templates in downgrade step 2020-07-06 17:43:44 +01:00
Leo Hemsted
0282a76bf7 rename template.serialize to serialize_for_v2
make it clear that this is for the public api, and we shouldn't add
fields to it without considering impacts

also add the broadcast_messages relationship on service and template to
the exclude from the marshmallow schemas, so it's not included elsewhere
2020-07-06 16:42:31 +01:00