Commit Graph

6205 Commits

Author SHA1 Message Date
Leo Hemsted
66ca98fbfb create manifest from jinja template
newer versions of cf api don't allow you to have multiple apps per
manifest file. So, instead of our current inheritance based model, move
to the newer doc-dl/antivirus/template-preview approved jinja based
model.

the new single manifest.yml.j2 file sets a bunch of variables based on
the CF_APP variable - things like NOTIFY_APP_NAME, default instances,
etc. Then the manifest is built up to define all of the app options
based on these defaults. Things default to sensible values, which can
vary based on environment.

When adding new environment variables, you'll need to add them to the
manifest file. If they're json encoded lists, you'll need to pass them
back to the `tojson` filter, or jinja2 will print them as python lists,
with single quotes around strings.
2019-04-10 15:15:48 +01:00
Leo Hemsted
fe77d4f654 move commands from manifest to procfile
cf v3 commands don't appear to support commands in manifest files. They
say that they do, but in practice they fail on cf v3-zdt-push with an
error message "No process types returned from stager". This can be
solved by moving the command from the manifest to a Procfile.

However, the Procfile is part of the source code, and as such is the
same for each app. To get around this, make the Procfile command invoke
a new wrapper script, which checks the NOTIFY_APP_NAME env var and then
calls the correct command
2019-04-09 17:05:56 +01:00
Rebecca Law
193ff8548c Merge pull request #2450 from alphagov/email-branding-unique-constraint
Update EmailBranding.name to be not null and unique.
2019-04-09 14:47:09 +01:00
Rebecca Law
e5663ceec9 Merge pull request #2448 from alphagov/fix-alert-for-ack-file
The rstrip() was not working for all file names.
2019-04-09 14:36:22 +01:00
Rebecca Law
64428368d1 Update EmailBranding.name to be not null and unique.
Needed to update old migration scripts so that the email_branding name is not null when creating the test dbs.
This should no affect the migrations elsewhere.
2019-04-09 14:33:38 +01:00
Rebecca Law
4ce2b9eaba The rstrip was not working for all file names so this changes it to a replace. 2019-04-08 12:04:14 +01:00
Rebecca Law
1ce044011c Merge pull request #2447 from alphagov/associate-all-services
The active flag in the query was excluding some services.
2019-04-08 11:37:24 +01:00
Rebecca Law
aca2817184 The active flag in the query was excluding some services.
The active flag wasn't always used, plus a service should always be active on the first version of the service.
2019-04-08 11:24:45 +01:00
Rebecca Law
0b4030183c Merge pull request #2446 from alphagov/fix-command
Ignore services where an organisation was not found.
2019-04-05 17:13:47 +01:00
Rebecca Law
7cb51840b5 Ignore services where an organisation was not found. 2019-04-05 17:03:11 +01:00
Chris Hill-Scott
ab22a3d88c Merge pull request #2440 from alphagov/set-default-domains
Set default branding for all organisations
2019-04-05 16:07:30 +01:00
Chris Hill-Scott
f26abe87b2 Merge pull request #2443 from alphagov/ignore-gsi-when-search-for-domain
Ignore gsi in email address when searching for organisation
2019-04-05 16:07:05 +01:00
Rebecca Law
36b4a344fc Update the dao_get_organisation_by_email_address to ignore .gsi in the email address. 2019-04-05 15:47:16 +01:00
Chris Hill-Scott
2ef9618e70 Set default branding for all organisations
Currently when someone creates a service we match them to an
organisation. Then if the organisation has a default branding set, their
service gets that branding.

However none of the organisations yet have a default branding set up.
This commit migrates the data about the relationship between an
organisation and its branding from being inferred from the `domain`
field on the branding, to being a proper database relationship.
2019-04-05 15:40:20 +01:00
Rebecca Law
905251fbb8 Merge pull request #2441 from alphagov/associate-services-to-organisations
Command to associate services to organisations.
2019-04-05 15:12:57 +01:00
Rebecca Law
1848c358f3 Command to associate services to organisations.
The association is based on the email address of the user that created the service, meaning we need to look at version 1 of the service_history.
Then use the existing methods to find the organisation by email address, then add the association.
2019-04-05 15:02:58 +01:00
Chris Hill-Scott
c13d5f0d6d Merge pull request #2434 from alphagov/get-org-by-domain
Add a method to guess organisation from a domain
2019-04-05 14:51:40 +01:00
Rebecca Law
4a085a1e31 Merge pull request #2438 from alphagov/collate-letter-pdfs-cronitor
Add Cronitor decorator collate-letter-pdfs-for-day
2019-04-05 13:57:45 +01:00
Rebecca Law
8990b70eaa Merge pull request #2439 from alphagov/fix-boolean-columns
Fix boolean columns for organisation migration
2019-04-05 13:57:25 +01:00
Rebecca Law
38b8a00344 Fix boolean columns 2019-04-05 12:09:57 +01:00
Toby Lorne
0022923bd0 Add Cronitor decorator collate-letter-pdfs-for-day
This celery task was not decorated with the cronitor decorator so never
checked in with cronitor.

Adding the decorator will ensure this task is monitored.

The requisite cronitor key is in the credentials repository already.

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2019-04-05 10:26:18 +01:00
Katie Smith
46c2f6a4ac Merge pull request #2436 from alphagov/command-to-get-zips-sent-details
Add command to get notification details from 'zips sent' files
2019-04-05 10:24:19 +01:00
Katie Smith
1f1c94be9d Add command to get notification details from 'zips sent' files
Added a command to get more detailed information about the letters that
are listed in the files of zip files sent that are stored on S3. This takes
one or more file paths as arguments and creates a CSV file with a row for each
letter.
2019-04-04 16:21:58 +01:00
Rebecca Law
7feb2f128d Merge pull request #2435 from alphagov/populate-org-data-from-file
Populate org data from file
2019-04-04 16:20:12 +01:00
Rebecca Law
2b36678a26 Populate data from file for organisation data
Finished the command to populate organistions and domains from a spreadsheet.
Should be a one time use thing.

fix codestyle
2019-04-04 16:04:18 +01:00
Chris Hill-Scott
f62ed714f8 Add a method to guess organisation from a domain
We need this in the admin app while we still have pages that:
- talk about the data sharing and financial agreement
- but aren’t within a service (so can’t look at the service’s
  organisation)

This is a get, but it deliberately won’t work if you pass it an email
address, in order not to put personally identifying information in our
logs.
2019-04-04 10:13:19 +01:00
Leo Hemsted
c654bc1e5e Merge pull request #2433 from alphagov/billing-fix
fix create_nightly_billing date bug
2019-04-03 15:59:54 +01:00
Leo Hemsted
1c6b291a42 we cant reason about the order of sets when iterated
split out assert into three parts
2019-04-03 15:47:53 +01:00
Leo Hemsted
28bff28786 remove letter rates between tests
we only use them in ft_billing so no reason not to delete them. makes
the tests read better as it's obvious how they work too.
2019-04-03 15:34:02 +01:00
Leo Hemsted
9f1f858997 update fact_billing_dao::get_rate to use dates not datetimes
update unit tests too
2019-04-03 14:52:41 +01:00
Leo Hemsted
6f41f6c7d7 use db models instead of tuples when referring to rate objects
makes it less confusing
2019-04-03 13:08:53 +01:00
Rebecca Law
b17cfd50b7 Merge pull request #2431 from alphagov/change-file-format-for-ack-files
Update alert for new DVLA file format
2019-04-03 12:09:59 +01:00
Rebecca Law
dc8159104e Update letter_raise_alert_if_no_ack_file_for_zip for new DVLA file format
When we send a zip file of letters to DVLA we expect them to send back an acknowledgement of those files.
Previously they named the files like NOTIFY.20180202091254.ACK.TXT and the contents would contain the name of the zip file we sent with a date of when they got it.
They have updated this format to mirror the format of the zip file because there was an instance where they sent 2 files of the same name so the later overwrote the first.
Since the name matches our name, there is no need to get the file from S3 but just compare file names.
2019-04-03 11:03:42 +01:00
Leo Hemsted
fcc25abf18 Merge pull request #2430 from alphagov/nightly-fix
fix nightly ft stats tables task to respect BST
2019-04-02 15:49:43 +01:00
Leo Hemsted
1dc084be54 fix nightly ft stats tables task to respect BST
the create_nightly_notification_status task runs at 00:30am UK time,
however this means that in summer datetime.today() will return the
wrong date as the server (which runs on UTC) will run the task at
23:30 (populating the wrong row in the table).

fix this to use nice tz aware functions
2019-04-02 15:15:07 +01:00
Rebecca Law
8cb0d30cec Merge pull request #2429 from alphagov/perf-platform-date
Fix performance platform date bug
2019-04-02 12:50:25 +01:00
Leo Hemsted
f1b3ae553f add tests for get_london_midnight_in_utc with a date object
we sometimes call it with a timestamp, but sometimes with just a date
object. It works with both, lets add the tests to prove that
2019-04-02 11:52:37 +01:00
Leo Hemsted
3739d9055d clean up usage of dates/datetimes in performance platform tasks
* call variables unambiguous things like `start_time` or `bst_date` to
  reduce risk of passing in the wrong thing
* simplify the count_dict object - remove nested dict and start_date
  fields as superfluous
* use static datetime objects in tests rather than calculating them
  each time
2019-04-02 11:49:20 +01:00
Toby Lorne
4abbb7137a Fix sending of performance platform
The pp client converts to UTC using the convert_utc_to_bst notify util.

This requires a datatime not a date, pass it a datetime, and add an
assertion in an existing test.

I didn't want to use the midnight conversion util in the test.

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2019-04-02 09:06:39 +01:00
Leo Hemsted
1250e47cc7 Merge pull request #2425 from alphagov/remove-inbound-get
remove get for inbound sms by phone number
2019-04-01 16:43:06 +01:00
Rebecca Law
020eb76f73 Merge pull request #2427 from alphagov/fix-date-bug-for-performance-platform
Fix date bug for performance platform
2019-04-01 12:16:31 +01:00
Rebecca Law
fb398876ee Merge branch 'master' into fix-date-bug-for-performance-platform 2019-04-01 12:05:48 +01:00
Rebecca Law
1456aa7789 Fix for performance platform updates.
Changed the query to get the performance platform stats from ft_notification_status. But the date used for the query needed to be a date, not datetime so the equality worked.
2019-04-01 12:03:57 +01:00
Rebecca Law
814d9bee99 Merge pull request #2426 from alphagov/fix-platfom-admin-bug
Fix BST date bug for platform admin summary page.
2019-04-01 11:58:36 +01:00
Rebecca Law
a47ecf0711 Fix test where the year was static
Remove print
2019-04-01 11:43:02 +01:00
Rebecca Law
d83ee6171e Fix codestyle 2019-04-01 11:02:02 +01:00
Rebecca Law
b9b81bca8f Fix BST date bug for platform admin summary page.
Added test for that.
2019-04-01 10:56:55 +01:00
Rebecca Law
0561fe5e11 Merge pull request #2424 from alphagov/use-ft_notification_status-for-perf-platform-updates
Use ft_notification_status instead of notification_history
2019-03-29 16:08:14 +00:00
Rebecca Law
fe30d2879a Fix test 2019-03-29 15:46:36 +00:00
Rebecca Law
e9607f227d Remove query that's no longer needed. 2019-03-29 15:38:48 +00:00