Martyn Inglis
25c8f71f2c
Reduced memory footprint of the API apps.
...
Staging and prod now default to 768M of RAM, down from a 1G saves 512M per instance type
Preview down to 256M per app
2017-05-31 11:47:52 +01:00
minglis
f7e1ccea8b
Merge pull request #987 from alphagov/redo-queue-visibitlity-timeout
...
Redo queue visibitlity timeout
2017-05-30 10:36:53 +01:00
Martyn Inglis
75c5e5e5d5
Merge branch 'master' into redo-queue-visibitlity-timeout
...
Conflicts:
app/notifications/process_notifications.py
app/v2/notifications/post_notifications.py
2017-05-30 10:18:18 +01:00
kentsanggds
c59746a4a7
Merge pull request #992 from alphagov/ken-refactor-service-schema
...
Refactor schema to remove redundant bits
2017-05-26 17:40:09 +01:00
Ken Tsang
112c673593
Removed a few lines from schema
2017-05-26 17:23:01 +01:00
Ken Tsang
18b8382d6e
Refactor schema and improve tests
2017-05-26 17:17:15 +01:00
Ken Tsang
56e9faab2e
Refactor schema
2017-05-26 16:21:22 +01:00
kentsanggds
263adac805
Merge pull request #984 from alphagov/ken-update-api-service-permissions-handling
...
Ken update api service permissions handling
2017-05-26 14:39:27 +01:00
Imdad Ahad
6b27091edf
Merge pull request #985 from alphagov/imdad-feat-search-normalised-notifications
...
Search notifications against normalised address
2017-05-26 12:39:18 +01:00
Ken Tsang
7aca3d8f43
Remove flags process in service_dao.create_service
2017-05-26 12:04:16 +01:00
Imdad Ahad
77b82305f4
Search normalised in get_notifications_for_service:
...
* Use dao method to search against normalised(recipient)
* Add filter to accept one or more statuses
2017-05-26 11:58:50 +01:00
Imdad Ahad
78c10b7d30
Search notification against normalised recipient with filter for status
2017-05-26 11:45:56 +01:00
Imdad Ahad
cfe08a4d8b
Store the normalised number on the notification
2017-05-26 11:36:24 +01:00
Imdad Ahad
44badf38cc
Merge pull request #981 from alphagov/imdad-feat-persist-normalised-recipient
...
Persist normalised recipient to `Notification`
2017-05-26 10:40:11 +01:00
Imdad Ahad
6c4377bd44
Persist normalised email
2017-05-26 10:26:07 +01:00
Imdad Ahad
cbc92a6173
Store the normalised number on the notification
2017-05-26 10:26:07 +01:00
Ken Tsang
1375bbe400
Refactor schema to improve error response
2017-05-25 17:55:24 +01:00
Ken Tsang
54d85fd8df
Add tests for services with no permissions
2017-05-25 17:48:09 +01:00
Ken Tsang
8488895612
Refactored tests
2017-05-25 17:48:09 +01:00
Ken Tsang
f7a18f77cf
Update model to cascade permissions assoc proxy
2017-05-25 17:48:09 +01:00
Ken Tsang
234312ece0
Update service permissions to ensure state in sync
2017-05-25 17:48:09 +01:00
Ken Tsang
2f626fa6fc
Add permissions validation in service schema
2017-05-25 17:47:21 +01:00
Ken Tsang
e59a1ab104
Update tests for existing flags to set service permissions
2017-05-25 17:47:21 +01:00
Ken Tsang
8e3e31faaf
Updated service DAO and API end points
2017-05-25 17:47:21 +01:00
Rebecca Law
389515ce02
Merge pull request #990 from alphagov/schedule-api-notification
...
Add iso8601 to requirements
2017-05-25 16:26:53 +01:00
Rebecca Law
1f591d3490
Add iso8601 to requirements
2017-05-25 16:13:33 +01:00
Rebecca Law
80119e2c84
Merge pull request #975 from alphagov/schedule-api-notification
...
Schedule api notification
2017-05-25 15:51:56 +01:00
minglis
0b82516951
Merge pull request #989 from alphagov/imdad-feat-add-sqs-queue-delete-script
...
Add script to delete SQS queues
2017-05-25 15:12:37 +01:00
Rebecca Law
6ad36b274b
Fix merge conflict with migration file
2017-05-25 13:46:03 +01:00
Martyn Inglis
29b6a918e8
Ensure dev and test builds will read from old and new queues
2017-05-25 13:43:31 +01:00
Rebecca Law
b745e13a28
Merge branch 'master' into schedule-api-notification
2017-05-25 13:37:04 +01:00
Imdad Ahad
586c83a2c7
Add a script to delete sqs queues:
...
* Uses boto to retrieve/delete queues
* Additional functions to output/read from csv
2017-05-25 13:33:26 +01:00
Leo Hemsted
d23a92fbc0
Merge pull request #952 from alphagov/international-delivery-receipts
...
update delivery receipts for countries that return them
2017-05-25 13:13:49 +01:00
Rebecca Law
f555c7a73b
Refactor tests to use the create_notication in tests.app.db
2017-05-25 11:41:07 +01:00
Martyn Inglis
58af4e5ee9
timeout slightly longer than retry
2017-05-25 11:20:23 +01:00
Martyn Inglis
753d58f138
Adds new queues to the PaaS workers
2017-05-25 11:15:28 +01:00
Martyn Inglis
4768f0b9fd
Change retries policy.
...
Before we had a long back off, now we have more, but shorter backoffs.
- PREVIOUS
When we had an error talking to a provider we retried quickly and if we still got errors we backed off more and more. Maximum attempts was 5, max delay 4hours. This was to allow us time to ship a build if that was required.
- NOW
Backing off 48 times of 5 minutes each. This gives us the same total backoff, but many more tries in that period.
- WHY
Having the long back off meant messages could be delayed 4 hours. This was happening more and more, as PaaS deploys can place things into the "inflight" state in SQS. The inflight state MUST have an expiry time LONGER than the maximum retry back off. This meant that messages would be delayed 4 hours, even when there was no app error.
By doing this we can reduce this delay to 5 minutes. Whilst still giving us time to fix issues.
2017-05-25 11:12:40 +01:00
Leo Hemsted
57b7cb8d24
Merge pull request #986 from alphagov/create-service-bug
...
separate service deserialization from validation
2017-05-25 10:52:38 +01:00
Martyn Inglis
2591d3a1df
This massive set of changes uses the new queue names object throughout the app and tests.
...
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
2017-05-25 10:51:49 +01:00
Martyn Inglis
21586c917c
Created an object to hold queue names.
...
- This is to be used throughout the app.
- Allows us to manage queue names centrally
- All queues renamed to allow us to change the retry processing/visibility timeout
2017-05-25 10:50:55 +01:00
Leo Hemsted
4b8b6ca91e
add test to ensure that updating other things doesnt affect sms sender
2017-05-25 10:37:17 +01:00
Rebecca Law
9f6c037530
Use iso8601 to validate scheduled_for datetime.
...
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
2017-05-24 16:27:15 +01:00
Leo Hemsted
554a193cff
separate service deserialization from validation
...
Marshmallow validates and deserialises - BUT, when it deserialises,
it explicitly sets `sms_sender=None`, even when you haven't passed
sms_sender in. This is problematic, because we wanted to take advantage
of sqlalchemy's default value to set sms_sender to `GOVUK` when the
actual DB commit happens.
Instead, still use marshmallow for validating, but manually carry out
the json deserialisation in the model class.
This fixes a bug that only manifested when the database was upgraded,
but the code hadn't updated. 🎉
2017-05-24 16:27:12 +01:00
Imdad Ahad
c986b88e21
Merge pull request #979 from alphagov/imdad-migration-add-normalised-to-notification
...
Add normalised_to field to notification
2017-05-24 16:21:15 +01:00
Martyn Inglis
07b527bb1b
All queues now managed in the same way:
...
- TEST and DEV builds set up their own queue lists for reading
- PROD/STAGE/PREVIEW use the separate worker process with the -Q flag
This enables us to rename queues in due course
2017-05-24 15:47:20 +01:00
Rebecca Law
383dee3bb2
Updated the serialization of Notification.scheduled_for to include minutes.
2017-05-24 14:52:32 +01:00
Rebecca Law
96b0e42959
Merge branch 'master' into schedule-api-notification
...
Conflicts:
app/celery/scheduled_tasks.py
tests/app/celery/test_scheduled_tasks.py
2017-05-24 13:21:22 +01:00
minglis
8a2715e3e1
Merge pull request #980 from alphagov/change-delete-jobs-to-delete-by-type
...
Change delete jobs to delete by type
2017-05-24 12:35:04 +01:00
Rebecca Law
180a212853
Merge pull request #970 from alphagov/server-command-for-db-update
...
Create a server command to run custom db scripts.
2017-05-24 09:16:44 +01:00
Leo Hemsted
c5816ee412
Merge pull request #982 from alphagov/revert-977-non-null-sender
...
Revert "Non null sender"
2017-05-23 16:33:18 +01:00
