and return data for one more day.
we're not really limiting to 7 days - we're returning 7 entire days,
plus whatever time has elapsed since midnight today. I felt it would be
best to rename the variable to `whole_days` to imply that it's not
"limit this data set to seven days", it's "give me at least seven days".
the endpoint is backwards compatible so we can rename the variable on the front-end later
We no longer need the `/platform-stats` route in the service blueprint,
because admin is using the new `/platform-stats` route in the platform stats
blueprint instead.
The list of top-level dependencies is moved to requirements-app.txt,
which is used by `make freeze-requirements` to generate the full
list of requirements in requirements.txt.
This is based on alphagov/digitalmarketplace-api#615, so rationale
from that PR applies here.
We had a problem with unpinned packages on new deployments leading
to failed tests (e.g. alphagov/notifications-admin#2144) which is
why we're implementing this now.
After re-evaluating pipenv again, this still seems like the least
disruptive approach:
* pyup.io has experimental support for Pipfile, but doesn't respect
version ranges or updating hashes in the lock file
* CloudFoundry buildpack recognizes and supports Pipfiles out of the
box, but the support is relatively new. For example until recently
CF would install dev packages during deployment. It's also based on
generating a requirements file from the Pipfile, which doesn't
properly support pinning VCS dependencies (eg it doesn't set the
#egg= version, meaning pip will not upgrade the package if it's
already installed).
* pipenv has a strict dependency resolution algorithm, which doesn't
appear to be well documented and can cause some unexpected failures.
For example, pipenv doesn't seem to be able to install `awscli-cwlogs`
package at all, believing it to have a version conflict for `botocore`
(which it doesn't list as a direct dependency) while neither `pip` nor
`pip-tools` highlight any issues with it.
* While trying out `pipenv install` on our list of dependencies it would
regularly fail to install utils with a "Will try again." message.
While the installation succeeds after a retry, this doesn't inspire
confidence.
* The switch to Pipfile and pipenv-managed virtualenvs requires a series
of changes to `make` targets and scripts - replacing `pip install` with
`pipenv`, removing references to requirements files and prefixing
commands with `pipenv run`. While it's likely to simplify the overall
process of managing dependencies, it would require time to properly
implement across our applications and environments (Jenkins, PaaS,
docker containers, and dev machines).
Added the letter_rate table to the list of tables which does not get
deleted after each test run and changed the tests to use the real letter
rates.
Also removed the letter rate DAO since this was only being used in
tests, so was no longer needed.
We now support letters of up to 5 sheets long, so we need to store the
rates for 4 and 5 sheet letters (both crown and non-crown) in the
`letter_rates` table.
really, it'll be somewhere btween 7 and 8 depending on what time of day
you request it at. But if today is monday, then seven days ago is last
tuesday - but we should return data for last monday as well so that
users see a full week's worth of data
also update/clarify the tests to make sure this is being honored for
all the different widgets on the dashboard
We have a few old jobs which don’t have a `processing_started` date.
This means that they always sort to the top of the jobs list in admin,
no matter how old they are. We think this is due to an old bug where
jobs would not be updated if a deploy was in progress.
This commit backfills the `processing_started` data for these jobs,
which will be roughly accurate. Complete accuracy is not the goal;
having these jobs not sort to the top of the list is.
This will affect 5 jobs across 3 services on production:
```sql
select service_id, job_status, created_at, updated_at, processing_started, processing_finished, notification_count, notifications_sent, notifications_delivered, notifications_failed from jobs where processing_started is null and job_status = 'in progress';
```
```
service_id | job_status | created_at | updated_at | processing_started | processing_finished | notification_count | notifications_sent | notifications_delivered | notifications_failed
--------------------------------------+-------------+----------------------------+----------------------------+--------------------+---------------------+--------------------+--------------------+-------------------------+----------------------
d47e5a1b-a04b-4398-8935-c8a266ce1d44 | in progress | 2017-09-29 13:49:41.512356 | 2017-10-01 02:01:05.281162 | | | 10615 | 0 | 0 | 0
128b91b6-2996-4107-bb65-51b7c24a728d | in progress | 2017-09-29 09:25:39.802623 | 2017-09-29 16:01:02.154291 | | | 10240 | 0 | 0 | 0
128b91b6-2996-4107-bb65-51b7c24a728d | in progress | 2017-09-29 09:31:52.455919 | 2017-09-29 16:01:01.990054 | | | 9930 | 0 | 0 | 0
128b91b6-2996-4107-bb65-51b7c24a728d | in progress | 2017-08-22 08:15:39.125999 | 2017-08-22 16:01:07.758805 | | | 6967 | 0 | 0 | 0
95316ff0-e555-462d-a6e7-95d26fbfd091 | in progress | 2016-05-27 14:44:18.114564 | 2016-06-13 00:18:14.542795 | | | 2742 | 2238 | 525 | 1713
(5 rows)
```
Both service api tasks work fine if the object is unexpectedly deleted
halfway through - they both check to see if the api details are still
in the DB before trying to send the request.
it now uses the ft_notification_status table - nice and quick. if you
ask for the current tax year it'll top up the data with numbers from
the notification table.
the data is in exactly the same shape as the old endpoint - no changes
to the admin app are necessary
By adding the service id, the query performance has improved greatly. It went from 6200ms to 0.04ms.
This should stop the 500s when a template is deleted.
