it doesn't match a string 😩
I couldn't think of a good way to test this in a unit test, since
it involves changing the service id on all of the components of a
service.
Added the filename of a service's letter logo to the service schema. We want
this in the schema so that it is possible to call
`current_service.letter_logo_filename` from notifications-admin and to pass this value
through to template-preview.
the move from virus scan to validation failed function was called with
the wrong variables, and had some internal logic that was slightly
wrong.
Also, Don't use `update_notification_by_id` for notifications if they
are not in `created`, `sending`, `pending`, or `sent`. It silently
doesn't update them. I didn't want to do a deeper investigation into
the reasons behind this terrifying state machine as part of this commit
so I just changed the functions to call `dao_update_notification`
manually
- pass new, sanitised pdf for sending
- move invalid pdfs to a newly created bucket
- set status fro notifications that failed pdf validation to a new status validation-failed
- adjust existing tests
Added a filename column to the dvla_organisation table and populated it
with the filenames that are currently hard-coded in template-preview.
The filenames for letter logos are going to be stored in the database,
instead of in template-preview.
remove pip-accel - it's not been updated in two years, and pins our
version of pip to a version that is several breaking changes old.
make sure commands work if you're already in a venv - mostly by
checking for presence of $VIRTUAL_ENV, and ensuring we use the correct
pip to install packages. Also clean up the commands a bit.
you need to `pip install celery[sqs]` to get the additional
dependencies that celery needs to use SQS queues - there are two libs -
boto3 and pycurl.
pycurl is a bunch of python handles around curl, so needs to be
installed from source so it can link to your curl/ssl libs. On paas and
in docker this works fine (needed to add `libcurl4-openssl-dev` to the
docker container), but on macos it can't find openssl. We need to pass
a couple of flags in:
* set the environment variable PYCURL_SSL_LIBRARY=openssl
* pass in the global options `build_ext` and `-I{openssl_headers_path}`.
As shown here:
https://github.com/pycurl/pycurl/issues/530#issuecomment-395403253
Env var is no biggie, but using any install-option flags disables
wheels for the whole pip install run. (See
https://github.com/pypa/pip/issues/2677 and
https://github.com/pypa/pip/issues/4118 for more context on the
install-options flags). A whole bunch of our dependencies don't
install nicely from source (but do from wheel), so this commit installs
pycurl separately as an initial step, with the requisite flags, and
then installs the rest of the requirements as before.
I've updated the makefile and bootstrap.sh files to reflect this, but
if you run `pip install -r requirements.txt` from scratch you will run
into issues.
previously, we were confusing things by appending to CELERY_QUEUES in
both dev and test configs - these are executed at import time, so the
list contained all queues twice, regardless of what config you're
actually using.
Fortunately, the -Q command that we supply the workers with overrides
this config option, so other environments weren't affected. Given that,
we can tidy up this code by just declaring it in the base config every
time
We were already returning the month, notification_type, billing_units
and rate from the /monthly-usage billing endpoint. This adds in the
postage too so that we can display postage details on the usage page of
admin.
* Changed update_fact_billing DAO function to update the table with the
real data for postage instead of hard-coding in 'second'.
* Added a test for the create nightly billing task to test that rows
with different postage are being inserted correctly.
Removed the occasionally failing test to check how ft_billing upserts
postage data. This test will be re-added once the postage column has been
added to the primary key.
* Updated the 'fetch_billing_data_for_day' DAO function to take postage into
account
* Updated the 'update_fact_billing' DAO function to insert postage for
new rows. When updating rows which are identical apart from the postage, the
original row will be kept. (This behaviour will change once postage is
added to the primary key - at this point, upserting will add a new row.)
* Also changed some fixtures / test set up functions to take postage
into account
Updated the 'migrate-data-to-ft-billing' command to populate the new
postage column of ft_billing. This will be populated with the
postage of the notification for letters, or 'none' for email or sms. We
need to ensure there are no null values in postage so that the postage
column can become part of the primary key later.
Also updated the query to get the right rate letter rate now that we are
updating rates in the letter_rates table.
The FactBilling model and the ft_billing database table have diverged
slightly - this makes some minor changes to the model columns so that
the model matches the table (which appears to be the correct version).
The ft_billing table is currently like this:
Column | Type | Modifiers | Storage | Stats target | Description
--------------------+-----------------------------+-----------+----------+--------------+-------------
bst_date | date | not null | plain | |
template_id | uuid | not null | plain | |
service_id | uuid | not null | plain | |
notification_type | text | not null | extended | |
provider | text | not null | extended | |
rate_multiplier | integer | not null | plain | |
international | boolean | not null | plain | |
rate | numeric | not null | main | |
billable_units | integer | | plain | |
notifications_sent | integer | | plain | |
updated_at | timestamp without time zone | | plain | |
created_at | timestamp without time zone | not null | plain | |
Indexes:
"ft_billing_pkey" PRIMARY KEY, btree (bst_date, template_id, service_id, rate_multiplier, provider, notification_type, international, rate)
"ix_ft_billing_bst_date" btree (bst_date)
"ix_ft_billing_service_id" btree (service_id)
"ix_ft_billing_template_id" btree (template_id)
