Commit Graph

421 Commits

Author SHA1 Message Date
Ken Tsang
7011b90bd4 Refactor is_precompiled_letter to model 2018-03-07 23:03:03 +00:00
Ken Tsang
564504bf97 Add template hidden field in response 2018-03-06 12:46:06 +00:00
Rebecca Law
c474b2312b Process responses for letters even after the notification has been deleted.
This will continue to update the notification history for letter notifications.
We currently have an issue where the responses to letters from the provider is taking a long time.
This is due to the manual nature of their process.
Updating the status of the letter will still work if the notification has been purged.

Also turned back on the purge letter notification scheduled task.
2018-03-02 11:29:22 +00:00
Rebecca Law
d6c929d127 Another unused method to delete 2018-03-02 11:05:06 +00:00
Rebecca Law
bffc4863db Remove all methods no longer used now that we only send pdf files to DVLA. 2018-03-02 11:05:05 +00:00
Leo Hemsted
34e98bc14c send letters for services without letters_for_pdf
also make sure that the service isn't in research mode
2018-02-21 15:59:52 +00:00
Richard Chapman
d855b4e4ec Removed statsd from the api and use the statsd in the utils library.
The statsd code was added to the utils library a while ago, uses the
statsd from the util library and therefore consolidates the code into
once place.
2018-02-06 09:52:15 +00:00
Leo Hemsted
3d389472b8 fix 500 when receiving delivery receipt from some international nums
if the international_billing_rates.yml has `dlr: null`, that means we
don't know what delivery receipts they provide - they might not provide
any. So if we do get an update, we don't know for sure that the message
was actually delivered - lets not update it.
2018-01-30 12:08:18 +00:00
Katie Smith
84e25d6b98 Compare letter page count with billable units in DVLA response file
We compare the page_count in the response file we receive from the DVLA
with the billable_units of the letter. If these don't match, we log an
error.
2018-01-23 08:59:01 +00:00
Ken Tsang
8103540261 Renamed run-letter-pdfs to trigger-letter-pdfs-for-day
- also set optional date_to_process argument for dao_get_count_of_letters_to_process_for_date to None, so it's set in the code instead
2017-12-19 13:23:55 +00:00
Ken Tsang
441651bbd1 Add get_count_of_letters_to_process to notifications_dao
- will get the letter notifications from day before >= letter processing deadline (17:30)
- letters_as_pdf permission is required in the service
2017-12-19 13:23:55 +00:00
Ken Tsang
da93ba296e Refactored notifications_dao
- Introduce a `_raise` flag for `get_notification_by_id` so that sql alchemy will raise the NoResults error rather than the app
- Refactor `dao_set_created_live_letter_api_notifications_to_pending` to use a join for getting services that don't have `letters_as_pdf` as marginally faster.
2017-12-12 12:07:55 +00:00
Ken Tsang
f42df8af73 Ignore services that have letters_as_pdf permission
- We don't want to process the letters as a build file for api calls when the service is generating letters as pdf.
2017-12-12 12:07:55 +00:00
venusbb
46226a25b4 modified dao_timeout_notifications
added send delivery status in scheduled timeout sending email/sms tasks
2017-12-07 16:44:08 +00:00
venusbb
5482ee4fe7 - wrap apply_async parameter notification_id in a str() argument
- check if service_callback_api exist before putting tasks on queue
- create_service_callback_api in tests before asserting if send_delivery_status_to_service has been called.
2017-12-04 17:58:38 +00:00
venusbb
771ce9e2bc fixed code style 2017-12-01 22:24:53 +00:00
venusbb
a0fe5c6971 created dao to get notifications by references 2017-12-01 09:31:03 +00:00
Rebecca Law
e9a1cc5f83 Stop deleting the mapping tables. 2017-11-30 11:27:08 +00:00
Rebecca Law
8aa94af5d7 Stop populating the notification to sender mapping table when creating a notificaiton. 2017-11-29 16:47:23 +00:00
Alexey Bezhan
a5ca66a01b Update notification queries with new TemplateHistory join
Changes the queries to use `Notification.template` relationship instead
of `.template_history` and TemplateHistory model insteaad of Template.
2017-11-10 15:12:07 +00:00
Rebecca Law
f5e79302cd Remove NotificationStatistics
NotificationStatistics was added as a spike but didn't work out as expected. This is finally removing all that unused code.
I'll drop the table in the next PR
2017-11-01 15:02:50 +00:00
Rebecca Law
d671adc1eb Fix codestyle 2017-10-30 16:51:27 +00:00
Rebecca Law
db6668eb61 Added the notification_to_sms_sender mapping table to the purge notifications query 2017-10-30 15:17:01 +00:00
Rebecca Law
0887910b1b Get the sms sender from the notificaiton_sms_sender mapping table if that does not exist get the default sms sender to pass on to the sms provider. 2017-10-30 14:55:44 +00:00
Rebecca Law
4eec11b633 Added an optional parameter in the form for POST /v2/notifications/sms and /service/<service_id>/send-notification to pass in the SMS sender id.
The send_sms_to_provider still needs to use the SMS sender being passed in to the POST.

As part of https://www.pivotaltracker.com/story/show/152106587
2017-10-30 13:36:49 +00:00
Rebecca Law
76c8b15f59 Update the billable units with the page count from the response file for letter notifications. 2017-10-27 13:53:55 +01:00
Richard Chapman
543535e2af Process Incomplete Jobs Updates
Comments are PR review. Updated code style in a few places to make it
more consistent with other code, added tests for letters and emails
so they are testedt, refactored some database queries to dao file

- Fixed code style
- Refactored database queries to dao code
- Added tests for emails and sms.
2017-10-17 11:07:36 +01:00
Ken Tsang
1eaf6e089b Refactor code to make it clearer
- Added code in `delete_notifications_created_more_than_a_week_ago_by_type` to remove notifications to email_reply_to older than 7 days
- Added transactional to `delete_notifications_created_more_than_a_week_ago_by_type`
2017-10-11 12:26:58 +01:00
Ken Tsang
2c46f201b5 Purge noti to email reply to older than 7 days 2017-10-09 14:51:27 +01:00
Richard Chapman
068ecd80a1 Changes after review. Style changes to ensure consistency with the rest of the code base to ensure good maintainability in future. 2017-10-05 16:50:18 +01:00
Richard Chapman
7974e59eb7 Changes after review. Updates to style and import order to be more consistent with other code.
Added clarification to an error message to give better debugging information.
Removed using dao_get_reply_to_by_service_id in tests to be more consistent with other code and use the test db functions or remove the need for a call altogether making the code less complex.
2017-10-05 16:50:18 +01:00
Richard Chapman
d2168b7985 Added the mapping between notification and reply to email to the database and persisted the mapping when the request is received by the end point. the end point also checks if the reply to email id exists and if not returns an error. Also added tests to test the functionality. 2017-10-05 16:50:17 +01:00
Leo Hemsted
aaadf09562 add tests for new sched task 2017-09-26 12:03:06 +01:00
Leo Hemsted
b1928b928c update process_letter tests 2017-09-26 11:28:54 +01:00
Leo Hemsted
f3db920c71 remove jobs from letter api calls
we now no longer create a job. At the end of the post there is no
action, as we don't have any tasks to queue immediately - if it's a
real notification it'll get picked up in the evening scheduled task.

If it's a test notification, we create it with an initial status of
sending so that we can be sure it'll never get picked up - and then we
trigger the update-letter-notifications-to-sent-to-dvla task to sent
the sent-at/by.
2017-09-26 09:57:36 +01:00
Leo Hemsted
17ba8db97f remove jobs from letter api and make success/error ftp callback tasks
1. No longer create jobs when creating letters from api 🎉
2. Bulk update notifications based on the notification references after
   we send them to DVLA - either as success or as error
2017-09-26 09:57:35 +01:00
Leo Hemsted
7dd3c1df5a set letter notifications to pending while notify-ftp does its stuff
this means that if the task is accidentally ran twice (eg we autoscale
notify-celery-worker-beat to 2), it won't send letters twice.

Additionally, update some function names and config variables to make
it clear that they are referring to letter jobs, rather than all letter
content
2017-09-26 09:57:35 +01:00
Imdad Ahad
72de309b26 Make perf platform processing stats query the NotificationHistory table 2017-08-31 12:52:59 +01:00
Katie Smith
49a6bfc06b Send '0', not 'null', to perf platform if no notifications are sent 2017-08-30 16:02:30 +01:00
Katie Smith
132d65bc75 Add query to get processing time stats for performance platform
We are only interested in API notifications, not including test
messages. Letters are not included.
2017-08-30 14:40:10 +01:00
Leo Hemsted
ae683cad6e Merge pull request #1192 from alphagov/deps
update dependencies
2017-08-22 15:39:49 +01:00
Chris Hill-Scott
bdc935a8b1 Exclude test keys when searching by recipient
The activity page doesn’t show notifications sent with a test key.

However it _does_ when you search by recipient. This is confusing
and inconsistent.
2017-08-21 17:38:21 +01:00
Leo Hemsted
c2152f9cb8 remove billable-units endpoint
it wasn't used anywhere, and the return value changed when flask was upgraded
2017-08-21 10:57:15 +01:00
Imdad Ahad
f5e38a896c Update the last template usage query to check Notification table:
* Don't check the NotificationHistory table (this can cause a timeout)
* Check template actually exists first
2017-08-07 10:19:21 +01:00
Leo Hemsted
13917c9c57 give test letter api notifications a different filename
so they can be distinguished on the frontend.

Also, some related cleanup:

* don't show test api letters on the frontpage
* make sure the subject is returned from the API for letters
* make sure the letter's address is returned for letters
2017-08-01 18:23:29 +01:00
Imdad Ahad
c8c47f44a9 Don't update older status in timeout notifications task 2017-07-10 14:09:30 +01:00
Imdad Ahad
f0ffbad77d Fix syncing issue with Notification statuses:
The timeout_notifications job runs daily and does not correctly
update both the status fields to keep them in sync.
2017-07-06 12:30:08 +01:00
Martyn Inglis
e3644bae96 Merge branch 'master' into template-query-to-not-limit-by-days 2017-06-16 14:06:56 +01:00
Martyn Inglis
1bc8073a83 Tried to make this clearer 2017-06-08 15:11:41 +01:00
Martyn Inglis
3702ebdd93 This is a bit gnarly. When querying the notifications table for template usage, we don't apply the days limit if we're trying to get exactly 7 days.
This is because 7days should be the amount of data in that table so don't restrict.

Note not all queries do this in the same way and a pivotal bug has been raised to align this. This is a bug fix as right now the numbers are out in prod.
2017-06-08 13:42:41 +01:00