Commit Graph

7716 Commits

Author SHA1 Message Date
Richard Baker
4dd37acecb Set cbc proxy message_format to cap
The CBC proxy lambda expects the message_format parameter to be one of `cap` or `ibag`.

Signed-off-by: Richard Baker <richard.baker@digital.cabinet-office.gov.uk>
2020-12-09 17:10:40 +00:00
Pea M. Tyczynska
a70b7c521e Merge pull request #3053 from alphagov/ibag-message-number
Add sequential message number to broadcast provider messages
2020-12-09 13:02:25 +00:00
Pea Tyczynska
ac3f56f4ed Delete unneeded code form migration 2020-12-09 12:31:50 +00:00
Pea Tyczynska
def7a16765 Establish relation between provider message and message number
this is so we can access brodcast_provider_message_number from
BroadcastProviderMessage object
2020-12-09 11:41:22 +00:00
Pea Tyczynska
8af4b27fd6 Separate functions for cbc clients
Also move message_format to the clients.
2020-12-09 11:13:50 +00:00
Pea Tyczynska
553565bc91 Send message format to CBC
Either cap or ibag
2020-12-08 11:15:26 +00:00
Pea Tyczynska
9e4176ac50 Add Vodafone client to list of allowed CBCs 2020-12-08 09:51:21 +00:00
Leo Hemsted
f5ef4cdf99 Merge pull request #3055 from alphagov/put-manifest-in-tmp-folder
put manifest in tmp folder
2020-12-07 18:39:33 +00:00
Leo Hemsted
9e5920472f put manifest in tmp folder
for reasons unknown, using a file descriptor no longer works on
concourse (or in that docker container generally). It might be a change
within cf-cli v7 vs v6.

Either way, it does't work, so use a temporary file. Clean up the
temporary file afterwards.

If the command fails, the temporary file will still stick around, so
I've added the file to the /tmp/ folder instead. it's full of secret
keys and things so if you do have a deployment error while running
locally, you should make sure you clean up the file (make cf-rollback
and make clean will both do this for you).
2020-12-07 18:27:22 +00:00
Leo Hemsted
6b01546c20 Merge pull request #3054 from alphagov/utils-bump
bump utils
2020-12-07 16:07:44 +00:00
Leo Hemsted
9502f17d84 flake8 fixes
a stricter flake8 bump. mostly things around f strings and format
strings, but a couple of bad placeholder names in loops
2020-12-07 15:24:02 +00:00
Leo Hemsted
d6555d887c provide a location for create_bucket
this is required when the boto3.resource itself is given a region
2020-12-07 15:03:41 +00:00
Leo Hemsted
45cc74bcc0 bump test requirements 2020-12-07 14:51:05 +00:00
Leo Hemsted
dc881b262e bump utils
this is to fix pip errors due to bad requirements
2020-12-07 14:35:06 +00:00
Leo Hemsted
db7759a9d8 Merge pull request #3052 from alphagov/v7-makefile
use cf-cli@v7 in makefile
2020-12-07 13:51:33 +00:00
Pea Tyczynska
2952b70930 Only create sequential numbers for Vodafone messages 2020-12-07 13:13:13 +00:00
Pea Tyczynska
932a09fe5b Pass message_number to proxy clients 2020-12-07 13:13:12 +00:00
Pea Tyczynska
e95dc9450e Include message number in send_broadcast_provider_message 2020-12-07 13:13:12 +00:00
Pea Tyczynska
a186d2d296 Format sequential number into an 8 char long hex
As per Vodafone spec for ibag format message number
2020-12-07 13:13:11 +00:00
Pea Tyczynska
b34bffaae6 Sends sequential number to Vodafone as link test 2020-12-07 13:13:11 +00:00
Pea Tyczynska
e6824dc3ff Broadcast provider message created with a sequential number
This is for the IBAG format (similar to CAP format, but proprietary)
used in the XMLs that we exchange with broadcast providers (specifically
Vodafone).
2020-12-07 13:13:10 +00:00
Pea Tyczynska
2a04148ea1 Add sequential numbering for broadcast messages
We need that to send broadcast messages using proprietary IBAG
format that Vodafone currently uses.
2020-12-07 13:13:10 +00:00
Leo Hemsted
b019419a5d use cf-cli@v7 in makefile
a bug in cf-cli v6 caused us to get rate limited, one solution to this
is to bump the version of cf-cli we're using to version 7. This has a
few syntax changes as the old v3 commands become mainline.

To upgrade locally, grab the latest version from brew.

```sh
brew install cloudfoundry/tap/cf-cli@7
```
2020-12-07 11:54:12 +00:00
Leo Hemsted
6405c1695f Merge pull request #3051 from alphagov/provider-per-service
Restrict providers per service
2020-12-04 09:59:24 +00:00
Leo Hemsted
fd335e3d8b move available provider logic to the service model
make sure it's in an accessible place so we don't end up duplicating our
work
2020-12-03 22:50:50 +00:00
Leo Hemsted
72f8a15d4f respect service broadcast provider restrictions when sending 2020-12-03 13:39:09 +00:00
Leo Hemsted
0ef063ab14 return allowed_broadcast_provider via get by service id 2020-12-03 12:38:31 +00:00
Leo Hemsted
0bbd00d2a5 return service restrictions from the service endpoint 2020-12-03 12:38:04 +00:00
Leo Hemsted
1a083134fa add service broadcast provider restriction table
some services only send to one provider. This is a platform admin
setting to allow us to test integrations and providers manually without
affecting other broadcasts from different services.

one-to-one - a service can either send to all as normal, or send to only
one provider.
2020-12-03 12:38:04 +00:00
Chris Hill-Scott
3bf936b782 Merge pull request #3049 from alphagov/jobs-from-contact-lists-uploads
Don’t return jobs sent from contact lists (uploads page edition)
2020-12-01 16:56:30 +00:00
Chris Hill-Scott
682cbc5130 Don’t return jobs sent from contact lists
Now that we’re grouping jobs sent from contact lists within their
parent, they shouldn’t also be listed on the jobs page at the top level.

The jobs page uses the uploads API, not the jobs API, so this commit
makes sure that filtering is happening in the proper place.
2020-12-01 15:26:36 +00:00
Chris Hill-Scott
10e1fe6902 Revert "Don’t return jobs sent from contact lists"
This reverts commit 061c0a0050.
2020-12-01 15:18:32 +00:00
Chris Hill-Scott
5bc8d8609b Merge pull request #2842 from alphagov/dont-return-jobs-from-contact-list
Don’t return jobs sent from contact lists
2020-12-01 14:48:04 +00:00
David McDonald
cf0d424910 Merge pull request #3048 from alphagov/uuid-to-string
Fix comparison of uuid to string
2020-12-01 12:21:20 +00:00
Chris Hill-Scott
061c0a0050 Don’t return jobs sent from contact lists
Now that we’re grouping jobs sent from contact lists within their
parent, they shouldn’t also be listed on the jobs page at the top level.
2020-12-01 11:56:34 +00:00
David McDonald
bb6e671bd1 Fix comparison of uuid to string
`service.id` is a uuid so will not be matched to anything in
`current_app.config.get('HIGH_VOLUME_SERVICE')` because that is a list
of strings.

This is why we are never falling into the first if statement and having
any metrics for high volume services on our dashboards at the moment.

Note, I had taken the existing line from the `post_notification`
endpoint, but that is using a serialised service which already has the
UUID converted to a string.
2020-12-01 11:16:15 +00:00
David McDonald
6340fed02a Merge pull request #3047 from alphagov/sending-times-broken-down
Tweak sending time metrics to only include live notifications
2020-11-30 15:31:11 +00:00
David McDonald
988c3a335a Remove old metric for delivery sending times
We no longer need a metric that covers both test and live keys as this
is not useful
2020-11-30 15:12:13 +00:00
David McDonald
b1336c97a4 Tweak sending time metrics to only include live notifications
Changes the high volume and not high volume metrics to both only include
non test notifications. This is because when looking at the grafana
metrics, it was impossible to tell what affect the high volume/non high
volume effect was having vs the test/live notification effect.

This leaves us with no break down of high volume/not high volume sending
times for test notifications but I don't think we really need that.
2020-11-30 15:05:40 +00:00
David McDonald
b8b99b607b Merge pull request #3046 from alphagov/sending-times-broken-down
Break down how long it takes to send a notification
2020-11-30 13:35:17 +00:00
David McDonald
2110bc3eef Break down how long it takes to send a notification
We currently measure the sending time for all. This commit then breaks
it down into
- test keys and non test keys
- high volume services and non high volume services

Breaking it down into test keys and non test keys is important because
we don't care as much about sending test notifications within 10
seconds, only non test keys so we don't want our graphs to reflect poor
performance if it's just test keys affecting this

Breaking it down into high volume and non high volume will allow us to
easily debug issues with slow sending if they are high volume or non
high volume issues
2020-11-30 12:07:32 +00:00
Chris Hill-Scott
09a26ea4a9 Merge pull request #3045 from alphagov/bump-utils-43.5.2
Bump utils to 43.5.2
2020-11-30 11:31:47 +00:00
Chris Hill-Scott
a497788740 Bump utils to 43.5.2
Brings in:
- [x] https://github.com/alphagov/notifications-utils/pull/808/files

Changes:
- https://github.com/alphagov/notifications-utils/compare/43.5.1...43.5.2
2020-11-30 10:52:48 +00:00
Leo Hemsted
a6426e7785 Merge pull request #3044 from alphagov/cbc-proxy-enabled-flag
add CBC_PROXY_ENABLED config flag to control if tasks are triggered
2020-11-26 11:56:41 +00:00
Leo Hemsted
e2fa0116a0 add CBC_PROXY_ENABLED config flag to control if tasks are triggered
previously we made some incorrect assumptions about set-up on staging
and prod - they currently don't have any cbc_proxy aws creds at all.

We shoudn't be attempting canaries or link tests when there's no AWS
infrastructure to connect to.

We also shouldn't bother writing a row into the database at all for the
broadcast_provider_message since we're not even attempting to send, and
we shouldn't get confused between messages that failed and messages we
never wanted to send at all.
2020-11-26 10:16:22 +00:00
Leo Hemsted
54fecf2182 Merge pull request #3035 from alphagov/broadcast-event-response
Send broadcast events per provider
2020-11-25 10:16:30 +00:00
David McDonald
928b62e079 Merge pull request #3042 from alphagov/revert-3041-turn-on-sms-email-stubs-staging
Revert "Turn on SMS and email stubs on staging"
2020-11-24 14:30:40 +00:00
David McDonald
09759e1fe4 Revert "Turn on SMS and email stubs on staging" 2020-11-24 12:12:53 +00:00
Pea M. Tyczynska
fefcb009d5 Merge pull request #3041 from alphagov/turn-on-sms-email-stubs-staging
Turn on SMS and email stubs on staging
2020-11-23 12:02:15 +00:00
Pea Tyczynska
d57b99e307 Turn on SMS and email stubs on staging
This is done because we will be load testing on staging.
2020-11-23 11:49:16 +00:00