Chris Hill-Scott
e443dc7fed
Output contact blocks with placeholders replaced
...
Brings in:
- [ ] https://github.com/alphagov/notifications-utils/pull/161
Also brings in:
https://github.com/alphagov/notifications-utils/compare/16.1.3...17.0.2
2017-05-11 16:50:36 +01:00
Martyn Inglis
2643a891fc
Renumber the sql migrations.
2017-05-11 15:43:52 +01:00
Martyn Inglis
d324d72926
Merge branch 'master' into async-job-stats
2017-05-11 15:41:20 +01:00
Rebecca Law
76a267b896
Script to update international flag 10,000 rows at a time.
2017-05-11 15:40:56 +01:00
Imdad Ahad
048861b968
Add dao to get active users for service
2017-05-11 15:39:15 +01:00
Martyn Inglis
e382d699a6
using the periodic worker to process the stats tasks. May need it's own worker in due course, but right now piggybacking.
2017-05-11 15:24:10 +01:00
Martyn Inglis
02f7abcc48
Wire in the new task. Runs at 5am after the other tasks.
2017-05-11 15:23:45 +01:00
Martyn Inglis
f0e47029f2
"Created at" on the model
2017-05-11 15:23:18 +01:00
Martyn Inglis
8818bd5dba
Scheduled task to call the timeout function
2017-05-11 15:22:57 +01:00
Martyn Inglis
b519321798
Adds a query to timeout the job counts.
...
After three days we timeout the notifications that we have not received a receipt for.
In the same way we bump the failed count to match the job count if there is a descepenecy. We do this after the same period that we do for notifications.
2017-05-11 15:22:43 +01:00
Martyn Inglis
e993a91a11
Added a created at column to the job stats table.
2017-05-11 15:21:27 +01:00
Leo Hemsted
4d0b90bbe2
Merge pull request #943 from alphagov/schema-validation
...
fix v2 schema phone/email validation when non-str passed in
2017-05-11 14:42:22 +01:00
Imdad Ahad
8e9c811665
Merge pull request #945 from alphagov/imdad-migration-add-go-live-template
...
Add go live template (Migration)
2017-05-11 13:58:42 +01:00
Imdad Ahad
373c2b029f
Add go live template
2017-05-11 12:38:32 +01:00
Rebecca Law
ae85bcf45a
Merge pull request #947 from alphagov/revert-944-update-notification-international
...
Revert "Update notification international"
2017-05-11 12:12:46 +01:00
Rebecca Law
c7479b6fbb
Revert "Update notification international"
2017-05-11 12:12:27 +01:00
Martyn Inglis
76494a285e
New logic to:
...
- increment sent only if the notifications is in a NON-finished state
- Increment outcome only if notification is in a finished state
2017-05-11 12:10:46 +01:00
Martyn Inglis
22a47106af
Refactored the DAO to be clearer, and wrote tests for the bug whereby different types of inserts/updates caused others to reset.
2017-05-11 12:09:57 +01:00
Rebecca Law
75e1d922e3
Merge pull request #944 from alphagov/update-notification-international
...
Update notification international
2017-05-11 11:27:01 +01:00
Leo Hemsted
7032e426c2
Merge pull request #946 from alphagov/schema-fix
...
marshmallow schemas no longer return _status_enum column
2017-05-11 09:36:04 +01:00
Leo Hemsted
466c833c39
marshmallow schemas no longer return _status_enum column
...
now return `status`, as they should
2017-05-10 17:30:09 +01:00
Rebecca Law
ed4a8fba6c
Script to update the international column to be not null
2017-05-10 16:14:30 +01:00
Rebecca Law
d1e9586fbb
Update the script to set the international flag to do the notifications and notification_history in separate loops.
...
It takes about 1.5 minutes to update 27,000 notifications and 27,000 notification_history. The update is a row level lock so will only affect updates to the same row.
This is unlikely as the data being updated should be older than 3 days.
The second scripts updates the table to set international as not null, to make the model.
2017-05-10 14:38:21 +01:00
Rebecca Law
3473bc8965
Merge branch 'master' into update-notification-international
2017-05-10 14:04:18 +01:00
Rebecca Law
f0a5851d73
The script updates 10,000 rows at a time, committing after each one. This should limit the distrubtion to the environment as only the rows being updated will be locked.
...
It is likely those rows will not have a conflicting update at the same time since the records are older than 3 days.
2017-05-10 14:02:11 +01:00
Leo Hemsted
348183d93b
Merge pull request #942 from alphagov/status-as-table
...
notification status as table
2017-05-10 13:54:49 +01:00
Leo Hemsted
5ec4829d00
fix v2 schema phone/email validation when non-str passed in
...
jsonschema states:
> A format attribute can generally only validate a given set of
> instance types. If the type of the instance to validate is not in
> this set, validation for this format attribute and instance SHOULD
> succeed.
We were not checking for the type of the input, and our validators were
behaving in unexpected manners (throwing TypeErrors etc etc). Despite
us declaring that the phone_number field is of type `str`, we still
need to make sure the validator passes gracefully, so that the inbuilt
type check can be the bit that catches if someone passes in a non-str
value. We've seen this with people passing in integers instead of strs
for phone numbers. This'll make them receive a nice 400 error
(e.g. "phone_number 12345 is not of type string"), rather than us
having a 500 internal server error
2017-05-10 11:04:12 +01:00
Martyn Inglis
19c14982a2
Call the task wrapper outcome function in the statistics_tasks file. This wraps the logic around which tasks to creates and simplifies the logic in the Callback classes.
2017-05-09 22:03:57 +01:00
Martyn Inglis
a4549a2e26
Put the statistics tasks into a new queue. This currently has no worker in PaaS so blocks the deployment until new PaaS worker is configured.
2017-05-09 22:03:13 +01:00
Martyn Inglis
caed193647
Use the new task wrapper methods rather than creating a task directly
2017-05-09 18:17:55 +01:00
Martyn Inglis
f2a47044a4
Add a wrapper method around calling the tasks to manage when to call stats tasks
2017-05-09 18:17:26 +01:00
Martyn Inglis
70bc468da0
Ensure date set on creation
2017-05-09 18:16:44 +01:00
Leo Hemsted
c10d4a36e7
populate notification_status_types table, and add foreign keys
2017-05-09 15:46:58 +01:00
Martyn Inglis
b4bf332ea0
Added in tasks for created initial job stats and for updating following an outcome.
2017-05-09 15:24:05 +01:00
Martyn Inglis
2cbf8d29fd
Removed white space for pep8
2017-05-09 14:46:15 +01:00
Martyn Inglis
4faafe01d9
Tightened the error checking.
...
- Looks now for a specific Integrity error, not a generic SQLAlchemy error when checking for an insert failure
- If the update fails post an insert error, then there is an issue so raise an exception so tasks can be retried.
2017-05-09 14:41:29 +01:00
Martyn Inglis
15065c4bc7
Added DAO methods to add and update the stats table for JOBS
...
- create_or_update_job_sending_statistics
This will try and update an existing row. if this fails as it hasn't been created, then it will insert a row. If this fails as another process has got there first then it should try and update again.
This is a code version of upset
- update_job_stats_outcome_count
Will update the outcome states. Uses the NOTIFICATION_STATUS_TYPES_FAILED to determine if the notification failed.
Any thing in DELIVERED will be marked as delivered. Statues not in the FAILED array or delivered provoke no update to the table.
2017-05-09 14:06:27 +01:00
Martyn Inglis
a5caed5ac8
Changed job_stats table to have an update at time stamp
2017-05-09 14:04:19 +01:00
Martyn Inglis
9c3ea95dcd
Changed job_stats table to have an update at time stamp
2017-05-09 14:04:07 +01:00
Martyn Inglis
b669c4930a
Building the methods around async stats work
2017-05-09 12:30:31 +01:00
Martyn Inglis
9bedcbc1b3
removed the reference to internation numbers from migration script
2017-05-09 11:22:45 +01:00
Martyn Inglis
d647640b65
Added job statistics table
2017-05-09 11:22:05 +01:00
Leo Hemsted
4ff4717fdf
make sure we update both statuses when updating notification history
2017-05-09 11:09:16 +01:00
Rebecca Law
aa19c727e1
Merge branch 'master' into update-notification-international
2017-05-09 10:12:14 +01:00
Rebecca Law
f5beb62c9f
Merge pull request #940 from alphagov/search-by-number-or-email
...
Search for notification by number or email
2017-05-09 10:11:28 +01:00
Rebecca Law
a564b9aeb0
Update /service/{}/notifications to look for a "to" query string, if it exists call the search notifications_by_to_field method.
2017-05-08 17:20:21 +01:00
Martyn Inglis
6e888260fc
Fix import
2017-05-08 16:46:44 +01:00
Rebecca Law
3513670408
Script to update the international flag in the notifications and notification_history tables.
...
This script may move where it is not run on db migration but as a one-off script from a server command.
2017-05-08 16:40:56 +01:00
Martyn Inglis
0e38259cb4
Remove all DAO get service calls, replaced with reference to new service object in context.
2017-05-05 15:23:06 +01:00
Martyn Inglis
064b7436db
Extra call to explicitly load API keys on service load - avoids extra calls later.
2017-05-05 15:22:21 +01:00
