darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2026-09-10 18:23:01 -04:00
Code Issues Packages Projects Releases Wiki Activity
11,974 Commits 122 Branches 63 Tags
API-1391_Restructure_Celery_Retry_Logic
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Leo Hemsted 91200a2088 Merge pull request #3502 from alphagov/provider-report 
add new daily sms provider volume report
 2022-04-12 15:48:24 +01:00
Leo Hemsted a2cbe20325 fix rediss ssl eventlet sslerror bug 
eventlet works by monkey-patching core IO libraries (such as ssl) to be
non-blocking. However, there's currently a bug: In the normal socket
library it may throw a timeout error as a `socket.timeout` exception.
However eventlet.green.ssl's patch raises an ssl.SSLError('timed out',)
instead. redispy handles socket.timeout but not ssl.SSLError, so we
solve this by monkey patching the monkey patching code to raise the
correct exception type 😱

Note: This code should _only_ be called when we're using eventlets, or
we'll run into issues with regular code failing with max recursion
errors. With that in mind we put this code in gunicorn_config, as that
isn't imported when we run celery or run flask locally.
 2022-04-12 14:50:36 +01:00
Leo Hemsted 2fbe9e85ac Merge pull request #3479 from alphagov/auto-retry-stuck-av-letters 
automatically retry letters stuck in pending-virus-scan
 2022-03-15 11:43:42 +00:00
Leo Hemsted 9e8df8b623 remove "letters stuck pending av" runbook 
there's not anything we know we need to do now that we resolve stuck
letters automatically. Letters couuld still get into this state, so it's
worth alerting us. However, we don't have anything concrete that we know
how to fix these letters, so we should just remove the runbook entirely.
 2022-03-10 14:10:01 +00:00
Leo Hemsted 00259893f1 automatically retry letters stuck in pending-virus-scan 
Since sept 2019 we've had to log on to production around once every
twenty days to restart the virus scan task for a letter. Most of the
time this is just a case of making sure the file is in the scan bucket,
and then triggering the task. If the file isn't in the scan bucket we'd
need to do some more manual investigation to find out exactly where the
file got stuck, but I can only remember times when it's been in the scan
bucket.

So if the file is in the scan bucket, we can just check that with code
and kick the task off automatically.
 2022-03-07 18:31:46 +00:00
Leo Hemsted e497cbbec6 Merge pull request #3474 from alphagov/fix-returned-letters 
split returned letters tasks into a max count of returned letters
 2022-03-02 11:49:12 +00:00
Leo Hemsted b1636b7a1a split returned letters tasks into a max count of returned letters 
if we have too many returned letters, we'll exceed SQS's max task size
of 256kb. Cap it to 5000 - this is probably a bit conservative but
follows the initial values we used when implementing this for the
collate-letters-task[^1]. Also follow the pattern of compressing the
sqs payload just to reduce it a little more.

[^1]: https://github.com/alphagov/notifications-api/pull/1536
 2022-03-02 10:51:08 +00:00
Leo Hemsted dc8d077e73 Merge pull request #3466 from alphagov/gunicorn-pin 
Bump gunicorn
 2022-03-01 10:40:39 +00:00
Leo Hemsted d34c58925e bump cffi 
v1.15 adds arm64 support for macbooks with an m1 chip
 2022-02-23 16:43:21 +00:00
Leo Hemsted ba2479b6e4 pin gunicorn to git commit; bump eventlet 
gunicorn doesn't pin eventlet, but functionally, gunicorn==20.1.0
depends on eventlet<=0.30.2 due to a change in eventlet. Gunicorn have
fixed this compat issue, however, haven't released it. By pinning to a
git commit, we're able to bump eventlet up to 0.33, thus solving a
security advisory. (Note that the security advisory didn't actually
impact us as it only affects websockets, however, it was noisy and
distracting).

Note - pip may have cached the old version of gunicorn. You may need to
run `pip install -r requirements.txt --no-cache-dir` to get the updated
version of gunicorn locally.
 2022-02-23 16:43:21 +00:00
Leo Hemsted 21071b2908 Revert "AUTO-413: Use an internal app for statsd preview" 2019-05-22 17:06:43 +01:00
Leo Hemsted 5d05a55857 Revert "testing build failure" 2019-05-16 17:06:34 +01:00
Leo Hemsted 7a711cf314 Revert "Zero downtime deploy" 2019-05-15 13:48:40 +01:00
Leo Hemsted fda3f4b41a Revert "Stub out SMS providers on staging for the perf tests" 2019-05-07 15:32:35 +01:00
Leo Hemsted 09decfe3a8 Merge pull request #2460 from alphagov/forgot-about-beat 
add delivery-celery-beat to paas_app_wrapper
 2019-04-12 11:25:02 +01:00
Leo Hemsted 5d26724022 Merge pull request #2456 from alphagov/remove-multi-app-manifests 
Remove multi app manifests
 2019-04-12 10:34:49 +01:00
Leo Hemsted c654bc1e5e Merge pull request #2433 from alphagov/billing-fix 
fix create_nightly_billing date bug
 2019-04-03 15:59:54 +01:00
Leo Hemsted fcc25abf18 Merge pull request #2430 from alphagov/nightly-fix 
fix nightly ft stats tables task to respect BST
 2019-04-02 15:49:43 +01:00
Leo Hemsted 1250e47cc7 Merge pull request #2425 from alphagov/remove-inbound-get 
remove get for inbound sms by phone number
 2019-04-01 16:43:06 +01:00
Leo Hemsted 29df5730cb Merge pull request #2423 from alphagov/more-retention 
combine post + get inbound, and make them respect data retention
 2019-03-28 15:58:12 +00:00
Leo Hemsted 329fa9ba0d Merge pull request #2421 from alphagov/inbound-retention 
make inbound sms page respect data retention
 2019-03-28 14:28:54 +00:00
Leo Hemsted b4deedd8a9 Merge pull request #2414 from alphagov/unsorted-case 
ignore case in the cost_threshold in dvla response files
 2019-03-22 14:12:46 +00:00
Leo Hemsted ba49b84677 Merge pull request #2412 from alphagov/hash-filename 
add a hash of letter filenames to the dvla zip file name
 2019-03-21 15:49:59 +00:00
Leo Hemsted 31d4bfed84 Merge pull request #2405 from alphagov/dvla-filename 
pass upload filename to notify-ftp
 2019-03-21 11:45:37 +00:00
Leo Hemsted 6f4b415426 Merge pull request #2397 from alphagov/slow-provider-switch 
lower provider switch threshold from 20% to 30%
 2019-03-14 16:22:52 +00:00
Leo Hemsted 3a746dd4a2 Merge pull request #2396 from alphagov/service-error-handler 
fix constraint name check in service error handler
 2019-03-14 16:15:12 +00:00
Leo Hemsted 26243cd2b0 Merge pull request #2377 from alphagov/skip-antivirus 
stub out antivirus in dev
 2019-02-27 13:38:01 +00:00
Leo Hemsted 26d42b5ea3 Merge pull request #2373 from alphagov/slow-provider-threshold 
move slow sms provider threshold from 10% to 20%
 2019-02-25 15:22:42 +00:00
Leo Hemsted ae3505d692 Merge pull request #2369 from alphagov/provider-switch-fix 
allow pending notifications to influence switchover.
 2019-02-22 16:57:10 +00:00
Leo Hemsted eeebdd3e0c Merge pull request #2367 from alphagov/reqs 
bump requirements
 2019-02-21 16:40:08 +00:00
Leo Hemsted f8f7d245b3 Merge pull request #2352 from alphagov/remove-dvla-part-2 
remove dvla_organisation from database
 2019-02-15 09:15:23 +00:00
Leo Hemsted e890e5cf32 Merge pull request #2351 from alphagov/remove-dvla-part-1 
remove dvla_organisation
 2019-02-14 14:27:12 +00:00
Leo Hemsted 5037293724 Merge pull request #2349 from alphagov/stop-using-dvla-organisation 
Don't fallback to dvla_organisation if letter branding unset
 2019-02-13 14:32:24 +00:00
Leo Hemsted b5b1056c02 Merge pull request #2343 from alphagov/reqs 
reqs bump
 2019-02-13 10:20:09 +00:00
Leo Hemsted a6660b7d91 Merge pull request #2345 from alphagov/wigan 
fix new services being assigned to wrong letter branding
 2019-02-11 13:44:14 +00:00
Leo Hemsted a0e33f0d75 Merge pull request #2320 from alphagov/letter_branding_data_migration 
Letter branding data migration
 2019-02-07 14:36:35 +00:00
Leo Hemsted 26a90f72fa Merge pull request #2326 from alphagov/error-downgrade 
downgrade error to info for notification not found for callback
 2019-02-05 14:51:12 +00:00
Leo Hemsted 208af8a508 Merge pull request #2318 from alphagov/letter-branding 
Letter branding
 2019-01-29 10:59:55 +00:00
Leo Hemsted bd4b125ac3 Merge pull request #2315 from alphagov/nightly-task-typo 
remove unnecessary job_types arg from remove_csv_files celery tasks
 2019-01-22 11:16:07 +00:00
Leo Hemsted f316c1d02b Merge pull request #2312 from alphagov/cronitor 
create cronitor decorator that alerts if tasks fail
 2019-01-21 12:50:28 +00:00
Leo Hemsted 5e7bff08aa Merge pull request #2287 from alphagov/redis-bump 
bump utils (inc redis bump)
 2019-01-11 10:59:46 +00:00
Leo Hemsted a970318586 Merge pull request #2294 from alphagov/remove-unused-functions 
remove unused usage functions
 2019-01-11 10:59:17 +00:00
Leo Hemsted d4570281fa Merge pull request #2288 from alphagov/plat-adm-fix 
Platform admin fix
 2019-01-09 14:04:30 +00:00
Leo Hemsted fe498f9a79 Merge pull request #2280 from alphagov/redis-pin 
pin redis to 2.x
 2019-01-07 11:30:34 +00:00
Leo Hemsted 63e16ea0ba Merge pull request #2276 from alphagov/utils-bump 
bump utils
 2019-01-04 16:37:09 +00:00
Leo Hemsted ea3070d645 Merge pull request #2272 from alphagov/delivery-receipt-logging 
log more info when we receive multiple delivery callbacks
 2019-01-04 09:52:54 +00:00
Leo Hemsted 016421a6b2 Merge pull request #2274 from alphagov/multi-worker-exit-script 
fix multi worker exit script
 2019-01-02 15:50:23 +00:00
Leo Hemsted 3cf5902ca4 Merge pull request #2265 from alphagov/job-proccess-start 
base job start of processing_started rather than created_at
 2018-12-17 16:03:57 +00:00
Leo Hemsted 2bd4f74ad0 Merge pull request #2258 from alphagov/dashboard-jobs-speedup 
stop dashboard reading notification_history
 2018-12-17 14:52:06 +00:00
Leo Hemsted 2d9609b1be Merge pull request #2253 from alphagov/folder-warning-msgs 
template folder error messages
 2018-12-10 15:51:44 +00:00
Leo Hemsted 3033015754 Merge pull request #2238 from alphagov/letter-500 
look in correct bucket for letters sent after 5:30pm
 2018-11-27 13:42:43 +00:00
Leo Hemsted 6d8534a8fb Merge pull request #2232 from alphagov/rename-staging-csv-uploads-bucket 
Rename staging CSV uploads bucket to match other environments
 2018-11-21 14:43:00 +00:00
Leo Hemsted e4ff771ab7 Merge pull request #2214 from alphagov/reqs 
bump requirements
 2018-11-07 14:46:15 +00:00
Leo Hemsted 51a7a0add7 Merge pull request #2199 from alphagov/template-folder-crud 
add template folder CRUD
 2018-11-01 10:44:48 +00:00
Leo Hemsted f0c61c8f29 Merge pull request #2190 from alphagov/template-folders 
add TemplateFolder model
 2018-10-29 15:00:49 +00:00
Leo Hemsted 08caaec0f8 Merge pull request #2179 from alphagov/on-second-thoughts-lets-go-alone 
pin itsdangerous to v0.24
 2018-10-22 17:07:52 +01:00
Leo Hemsted 6dd255e474 Merge pull request #2178 from alphagov/reqs-bumps 
bump requirements
 2018-10-22 15:50:20 +01:00
Leo Hemsted 340db1dec6 Merge pull request #2177 from alphagov/err-logs 
don't log exception info for retries
 2018-10-22 13:32:03 +01:00
Leo Hemsted 4dbedd02a3 Merge pull request #2167 from alphagov/compare-to-uuid 
compare service id to UUID
 2018-10-18 13:20:15 +01:00
Leo Hemsted e0056aad3f Merge pull request #2164 from alphagov/fix-bucket-names 
make sure validation_failed notifications still return pdf
 2018-10-17 17:04:25 +01:00
Leo Hemsted 126092948b Merge pull request #2076 from alphagov/use_sanitised_pdf 
Use sanitised pdfs for sending and handle invalid pdfs
 2018-10-17 13:31:55 +01:00
Leo Hemsted f55b4ef594 Merge pull request #2142 from alphagov/revert-2131-celery4 
Revert "Celery 4"
 2018-10-09 14:28:07 +01:00
Leo Hemsted 2ed50e760f Revert "Celery 4" 2018-10-09 13:27:49 +01:00
Leo Hemsted ba7cf10393 Merge pull request #2131 from alphagov/celery4 
Celery 4
 2018-10-05 16:03:44 +01:00
Leo Hemsted 0843503b7a Merge pull request #2123 from alphagov/estimate-deliv-date 
letter estimated delivery now takes postage into account
 2018-10-03 11:44:52 +01:00
Leo Hemsted 6aa4bfa426 Merge pull request #2118 from alphagov/fix-awscli 
fix awscli botocore dependency
 2018-09-28 11:37:29 +01:00
Leo Hemsted f0f54c0dff Merge pull request #2113 from alphagov/fix-noti-migration 
separate notification postage constraint into three separate commits
 2018-09-27 15:48:44 +01:00
Leo Hemsted f73d9ef852 Merge pull request #2108 from alphagov/postage-constraint 
Postage constraint
 2018-09-27 13:45:49 +01:00
Leo Hemsted ecbba69f53 Merge pull request #2103 from alphagov/populate-notification-postage 
populate historic notification postage data
 2018-09-24 11:13:07 +01:00
Leo Hemsted 7c48dd2c4e Merge pull request #2100 from alphagov/notification-postage 
Notification postage
 2018-09-20 13:25:34 +01:00
Leo Hemsted 889fd796fb Merge pull request #2057 from alphagov/sanitise-precompiled 
Sanitise precompiled pdfs after virus scan
 2018-08-29 12:53:27 +01:00
Leo Hemsted 0e284701b9 Merge pull request #2028 from alphagov/requirements 
bump requirements
 2018-08-09 16:56:26 +01:00
Leo Hemsted 1d2e086109 Merge pull request #1994 from alphagov/fix-billing-update 
make rebuild-ft-billing-for-day remove unused rows
 2018-07-30 09:58:29 +01:00
Leo Hemsted d61300accb Merge pull request #1944 from alphagov/research-letter-time 
create fake letter response files with variables timestamps
 2018-07-20 12:22:44 +01:00
Leo Hemsted d88003d143 Merge pull request #1940 from alphagov/dashboard-day-limit 
Sort out the dashboard day limits
 2018-07-11 16:07:31 +01:00
Leo Hemsted 6705f9d209 Merge pull request #1933 from alphagov/delete-service-stuff 
Delete service apis (callback + inbound)
 2018-07-06 15:19:01 +01:00
Leo Hemsted 732106119a Merge pull request #1930 from alphagov/monthly-statuses-page 
Monthly statuses page from ft_notification_status
 2018-07-04 13:36:56 +01:00
Leo Hemsted 3b5eefdd3d Merge pull request #1923 from alphagov/pyup-update-sqlalchemy-1.2.8-to-1.2.9 
Update sqlalchemy to 1.2.9
 2018-07-02 14:30:42 +01:00
Leo Hemsted 956822ee94 Merge pull request #1925 from alphagov/bump-utils-for-email-validation 
Updated notifications-util to the latest version
 2018-07-02 14:30:31 +01:00
Leo Hemsted cb8b17e7d3 Merge pull request #1926 from alphagov/letter-timezone-bug 
fix letters being put in the wrong bucket when near midnight cut-off
 2018-07-02 14:28:59 +01:00
Leo Hemsted 639a3f42b2 Merge pull request #1911 from alphagov/pyup-update-pytest-3.6.1-to-3.6.2 
Update pytest to 3.6.2
 2018-06-26 16:07:22 +01:00
Leo Hemsted 1f2f12238c Merge pull request #1909 from alphagov/pyup-update-flask-migrate-2.2.0-to-2.2.1 
Update flask-migrate to 2.2.1
 2018-06-26 16:07:10 +01:00
Leo Hemsted 01fbafd41e Merge pull request #1908 from alphagov/pyup-update-psycopg2-binary-2.7.4-to-2.7.5 
Update psycopg2-binary to 2.7.5
 2018-06-26 16:06:00 +01:00
Leo Hemsted 9b7cd15586 Merge branch 'master' into pyup-update-monotonic-1.4-to-1.5 2018-05-23 09:59:00 +01:00
Leo Hemsted a9bd189a24 Merge pull request #1868 from alphagov/pyup-update-requests-mock-1.4.0-to-1.5.0 
Update requests-mock to 1.5.0
 2018-05-11 14:07:28 +01:00
Leo Hemsted 9ac511e725 Merge pull request #1871 from alphagov/revert-1854-pyup-update-gunicorn-19.7.1-to-19.8.1 
Revert "Update gunicorn to 19.8.1"
 2018-05-11 11:29:32 +01:00
Leo Hemsted 307fd24072 Revert "Update gunicorn to 19.8.1" 2018-05-11 10:57:04 +01:00
Leo Hemsted 95475d130b Merge pull request #1865 from alphagov/separate-stats-endpoint 
create new stats endpoint
 2018-05-09 14:28:46 +01:00
Leo Hemsted 238e85ea9e Merge pull request #1833 from alphagov/read-redis 
Read template usage stats from new redis keys
 2018-04-30 15:15:16 +01:00
Leo Hemsted 901383777b Merge pull request #1844 from alphagov/zendesk 
zendesk instead of deskpro
 2018-04-27 16:59:08 +01:00
Leo Hemsted 46021f4604 Merge pull request #1826 from alphagov/pyup-update-pytest-mock-1.7.1-to-1.8.0 
Update pytest-mock to 1.8.0
 2018-04-09 15:17:23 +01:00
Leo Hemsted aa7a33c859 Merge pull request #1815 from alphagov/pyup-update-sqlalchemy-1.2.5-to-1.2.6 
Update sqlalchemy to 1.2.6
 2018-04-09 15:17:14 +01:00
Leo Hemsted 41e3a150d8 Merge pull request #1824 from alphagov/redis-populate-command-fix 
make sure redis command filters dates properly
 2018-04-09 15:17:07 +01:00
Leo Hemsted 730eff7e97 Merge pull request #1818 from alphagov/redis-templates 
add new redis template usage per day key
 2018-04-04 16:25:39 +01:00
Leo Hemsted f637d49871 Merge pull request #1757 from alphagov/process-job-fix 
Process job fix
 2018-03-09 17:28:21 +00:00
Leo Hemsted a777e13e4d Merge pull request #1754 from alphagov/revert-service-callback-eventlets 
Revert service callback eventlets
 2018-03-08 14:16:40 +00:00
Leo Hemsted a87373571c Merge pull request #1714 from alphagov/reqs 
bump requirements
 2018-02-27 13:46:14 +00:00
Leo Hemsted c52ca3e7bb Merge pull request #1681 from alphagov/fix-test-db 
make sure tests always run in test db
 2018-02-22 16:54:57 +00:00
Leo Hemsted 25d6b45ea2 Merge pull request #1692 from alphagov/service-name-change 
don't return or accept sms_sender in update service anymore
 2018-02-22 14:37:47 +00:00
Leo Hemsted 01ed4e1e56 Merge pull request #1679 from alphagov/pyup-update-eventlet-0.22.0-to-0.22.1 
Update eventlet to 0.22.1
 2018-02-21 16:26:27 +00:00