This will allow us to store details of which channel a service should be
sending to.
See the comment about how all broadcast services can have a row in the
table but may not at the moment. This has been done for speed as it's
the quickest way to let us set up different services to send to
different channels for some needed testing with the mobile handset
providers in the coming week.
It's clear that we need a way to track updates to a broadcast message.
It's also clear that we'll need some kind of audit log that captures
exactly what was sent out in a message.
This commit adds a new database table, `broadcast_event`, which maps 1:1
with CAP XML sent to the CBCs. We'll create one of these just before
sending out.
The main driver for this was that cancel and update messages need to
contain a list of references of all previous messages that they're
amending. This is of format `{sender},{identifier},{sent_timestamp}`,
and the identifier itself needs to be unique for each message.
test_config manipulates os.environ. os.environ is an `environ` object,
which acts like a dict but isn't in some subtle unknowable ways. The
`reload_config` fixture would create a dict copy of the env, and then
just call `os.environ = old_env` afterwards.
Boto3 would then complain that it couldn't load credentials (despite us
using the mock_s3 fixture and also not having creds in the environment
in the first place). Not entirely sure why this happens, but it does.
For some reason, it being a `dict` instead of an `environ` object causes
the mocking of boto3 to fail.
The solution is to not overwrite os.environ entirely, rather, use the
standard dictionary setitem syntax to update the values to their
previous values. Use `clear` to empty the environment too.
we randomly choose between sms providers now - this means that tests may
sometimes send firetext and sometimes mmg, so we'd need to patch out
different HTTP calls, expect different values in sent_by, etc etc.
To ensure tests are consistent, add a new fixture that is always used by
notify_db_session, which sets the priorities of the sms providers to
100% mmg 0% firetext. if you need to test other values, then you should
set the values manually in the test file
since there'll be a bunch of threads running functional test tasks at
the same time, there's no point always trying to start from the same
second and then stepping back to the same one-second-back file each
time. Also, this leads us to an increased risk of race conditions.
This change takes the same thirty second range, but shuffles it. The
tests, since they're no longer deterministic, now use a new Matcher
object (w/ credit to alexey) to match any filename from within that
thirty second range
Added the letter_rate table to the list of tables which does not get
deleted after each test run and changed the tests to use the real letter
rates.
Also removed the letter rate DAO since this was only being used in
tests, so was no longer needed.
Sometimes, when a test using one of the set_config[_values] context managers
failed or raised an exception it would cause the context to not be able
to revert its config changes, resulting in a 'spooky action at a
distance' where random tests would start to fail for non-obvious reasons.
previously we didn't do this because the tests all used the same DB
(test_notifications_api), however @minglis shared a snippet that simply
creates one test db per thread.
in tests, we were replacing os.environ with a basic dict so that
we didn't overwrite the contents of the real environment during tests.
However, os.environ doesn't accept non-str values, so this commit
changes the fixture so that it asserts all values set are strings.
We needed to change how we store ip whitelist stuff in the env because
of this.
We now have a new column in the database, but it isn't being
populated. The first step is to make sure we update this column,
while still keeping the old enum based column up to date as well.
A couple of changes have had to happen to support this - one irritating
thing is that if we're ever querying columns individually, including
`Notification.status`, then we'll need to give that column a label,
since under the hood it translates to `Notification._status_enum`.
Accessing status through the ORM (i.e., my_noti.status = 'sending' or
similar) will work fine.
Once we have the new columns in notifications table, the query will need to include the rate multiplier and if the number is international.
The monthly billing query will be built next.
If the template is marked as priority the notification will be sent using the `notify` queue.
The `notify` queue is a low volume queue, messages here will not be queue behind a large job and should be delivered with in a more consistent time frame.
- Added templates.process_type and templates_history.process_type column.
- Added a template_process_type table to handle the enum for templates.process_type, initial values are normal and priority
https://www.pivotaltracker.com/story/show/135429147
the provider details tests were previously very stateful - they
would update a value, and then because provider_details is a "static"
table that is not wiped by the notify_db_session fixture, the tests
were forced to make a second update that reverted that change. if the
test fails for whatever reason, the provider_details table ends up
permanently modified, playing havoc on tests further down the line.
this commit adds the fixture `restore_provider_details` to conftest.
this fixture stores a copy of the contents of ProviderDetails and
ProviderDetailsHistory tables outside of the session, runs the test,
and then clears the table and puts those old values in
this means that the tests have been cleaned up so that they do not
do operations twice in reverse. they've also been cleaned up
generally, including fixture optimisations and such
replaces previous notify_api fixture with the age old:
```
with notify_api.test_request_context():
with notify_api.test_client() as client:
```
just runs those two context managers in a yield fixture (new pytest 3.0 feature)
requirements should be kept up to date to ensure we get bug fixes and new features as they come - particularly py.test, which we were running an 18 month old version for, and missing out on some useful xfail and fixture enhancements, among other things
- ensured statues not deleted on test runs
- returns in API call
Merge branch 'add-new-column-to-jobs-for-delayed-sending' into scheduled-delivery-of-jobs
Conflicts:
app/models.py
a service now has branding and organisation_id columns, and two new
tables have been aded to reflect these:
* branding is a static types table referring to how a service wants
their emails to be branded:
* 'govuk' for GOV UK branding (default)
* 'org' for organisational branding only
* 'both' for co-branded output with both
* organisation is a table defining an organisation's branding. this
contains three entries, all of which are nullable
* colour - a hex code for a coloured bar on the logo's left
* logo - relative path for that org's logo image
* name - the name to display on the right of the logo
if passed in, returns the service object with additional statistics
dictionary, which will be used in the admin app to populate dashboard
components. A new schema has been created for this to avoid clashing/
causing confusion with the existing schema, which is already used
for PUT/POST as well, and this schema can be easily tailored to
reduce ambiguity and lazy-loading
* single-column static data table that currently contains two types: 'normal' and 'team'
* key_type foreign-keyed from api_keys
- must be not null
- existing rows set to 'normal'
* key_type foreign-keyed from notifications
- nullable
- existing rows set to null
* api_key foreign-keyed from notifications
- nullable
- existing rows set to null
We were dropping all tables at the end of the test run, however the alembic_version table is not part of the metadata so was being persisted. Alembic then doesn't upgrade the database next test run, since the version appears up to date, so we were, in the notify_db_session fixture, recreating from MetaData (sqlalchemy models). This involves quite a costly comparison of the postgres system tables and the tables in models.py, which was adding half a second to each test that uses the notify_db_session fixture (virtually all of them).
