pytest.ini

[pytest]
xfail_strict=true
testpaths = tests
log_level = 999
env =
    NOTIFY_ENVIRONMENT=test
    ADMIN_CLIENT_SECRET=dev-notify-secret-key
    API_HOST_NAME=http://you-forgot-to-mock-an-api-call-to
    DANGEROUS_SALT=dev-notify-salt
    SECRET_KEY=dev-notify-secret-key
    ZENDESK_API_KEY=test
    REDIS_ENABLED=0

filterwarnings =
    error:Applying marks directly:pytest.RemovedInPytest4Warning
addopts = -p no:warnings
Automatically set environment vars before tests Sometimes you just wanna run some tests directly using the `pytest` command. But you’re in a new shell, and have forgotten to do `source environment_test.sh`. The screen fills with red, and your day just got a little bit worse. This commit will stop this from ever happening again, by making the setting environment variables part of running Pytest. It does this with a plugin called pytest-env[1]. pytest.ini is the standard way of configuring pytest. Creating this file where it didn’t exist before changes the behaviour of pytest, in that it will now look for tests in the same directory as the file, rather than defaulting to the `tests/` directory. So we also have to explicitly configure pytest[2] to tell it that it should only look in this directory. Otherwise it gets lost in the weeds of `node_modules`. 1. https://github.com/MobileDynasty/pytest-env 2. https://docs.pytest.org/en/latest/customize.html#confval-testpaths 2018-02-14 09:49:27 +00:00			`[pytest]`
add fieldsets to template/folder form and update docstrings also remove erroneous pytest.ini cfg and reset button 2018-11-27 14:33:50 +00:00			`xfail_strict=true`
Automatically set environment vars before tests Sometimes you just wanna run some tests directly using the `pytest` command. But you’re in a new shell, and have forgotten to do `source environment_test.sh`. The screen fills with red, and your day just got a little bit worse. This commit will stop this from ever happening again, by making the setting environment variables part of running Pytest. It does this with a plugin called pytest-env[1]. pytest.ini is the standard way of configuring pytest. Creating this file where it didn’t exist before changes the behaviour of pytest, in that it will now look for tests in the same directory as the file, rather than defaulting to the `tests/` directory. So we also have to explicitly configure pytest[2] to tell it that it should only look in this directory. Otherwise it gets lost in the weeds of `node_modules`. 1. https://github.com/MobileDynasty/pytest-env 2. https://docs.pytest.org/en/latest/customize.html#confval-testpaths 2018-02-14 09:49:27 +00:00			`testpaths = tests`
Reduce logging noise in traceback output When running tests locally Pytest returns a lot of captured logging info. This is redundant because Pytest also captures stdout. This commit effectively disables logging output when running tests by setting the log level to higher than anything a real logging call would ever emit. The logging output is still captured by stdout, so nothing is lost here, we’re just reducing duplication. 2018-10-25 10:44:01 +01:00			`log_level = 999`
Automatically set environment vars before tests Sometimes you just wanna run some tests directly using the `pytest` command. But you’re in a new shell, and have forgotten to do `source environment_test.sh`. The screen fills with red, and your day just got a little bit worse. This commit will stop this from ever happening again, by making the setting environment variables part of running Pytest. It does this with a plugin called pytest-env[1]. pytest.ini is the standard way of configuring pytest. Creating this file where it didn’t exist before changes the behaviour of pytest, in that it will now look for tests in the same directory as the file, rather than defaulting to the `tests/` directory. So we also have to explicitly configure pytest[2] to tell it that it should only look in this directory. Otherwise it gets lost in the weeds of `node_modules`. 1. https://github.com/MobileDynasty/pytest-env 2. https://docs.pytest.org/en/latest/customize.html#confval-testpaths 2018-02-14 09:49:27 +00:00			`env =`
			`NOTIFY_ENVIRONMENT=test`
			`ADMIN_CLIENT_SECRET=dev-notify-secret-key`
testing fixed 2025-04-29 15:41:06 -07:00			`API_HOST_NAME=http://you-forgot-to-mock-an-api-call-to`
Automatically set environment vars before tests Sometimes you just wanna run some tests directly using the `pytest` command. But you’re in a new shell, and have forgotten to do `source environment_test.sh`. The screen fills with red, and your day just got a little bit worse. This commit will stop this from ever happening again, by making the setting environment variables part of running Pytest. It does this with a plugin called pytest-env[1]. pytest.ini is the standard way of configuring pytest. Creating this file where it didn’t exist before changes the behaviour of pytest, in that it will now look for tests in the same directory as the file, rather than defaulting to the `tests/` directory. So we also have to explicitly configure pytest[2] to tell it that it should only look in this directory. Otherwise it gets lost in the weeds of `node_modules`. 1. https://github.com/MobileDynasty/pytest-env 2. https://docs.pytest.org/en/latest/customize.html#confval-testpaths 2018-02-14 09:49:27 +00:00			`DANGEROUS_SALT=dev-notify-salt`
			`SECRET_KEY=dev-notify-secret-key`
send zendesk rather than deskpro tickets 2018-04-24 17:37:15 +01:00			`ZENDESK_API_KEY=test`
Add Redis cache between admin and API Most of the time spent by the admin app to generate a page is spent waiting for the API. This is slow for three reasons: 1. Talking to the API means going out to the internet, then through nginx, the Flask app, SQLAlchemy, down to the database, and then serialising the result to JSON and making it into a HTTP response 2. Each call to the API is synchronous, therefore if a page needs 3 API calls to render then the second API call won’t be made until the first has finished, and the third won’t start until the second has finished 3. Every request for a service page in the admin app makes a minimum of two requests to the API (`GET /service/…` and `GET /user/…`) Hitting the database will always be the slowest part of an app like Notify. But this slowness is exacerbated by 2. and 3. Conversely every speedup made to 1. is multiplied by 2. and 3. So this pull request aims to make 1. a _lot_ faster by taking nginx, Flask, SQLAlchemy and the database out of the equation. It replaces them with Redis, which as an in-memory key/value store is a lot faster than Postgres. There is still the overhead of going across the network to talk to Redis, but the net improvement is vast. This commit only caches the `GET /service` response, but is written in such a way that we can easily expand to caching other responses down the line. The tradeoff here is that our code is more complex, and we risk introducing edge cases where a cache becomes stale. The mitigations against this are: - invalidating all caches after 24h so a stale cache doesn’t remain around indefinitely - being careful when we add new stuff to the service response --- Some indicative numbers, based on: - `GET http://localhost:6012/services/<service_id>/template/<template_id>` - with the admin app running locally - talking to Redis running locally - also talking to the API running locally, itself talking to a local Postgres instance - times measured with Chrome web inspector, average of 10 requests ╲ \| No cache \| Cache service \| Cache service and user \| Cache service, user and template -- \| -- \| -- \| -- \| -- Request time \| 136ms \| 97ms \| 73ms \| 37ms Improvement \| 0% \| 41% \| 88% \| 265% --- Estimates of how much storage this requires: - Services: 1,942 on production × 2kb = 4Mb - Users: 4,534 on production × 2kb = 9Mb - Templates: 7,079 on production × 4kb = 28Mb 2018-04-06 13:37:49 +01:00			`REDIS_ENABLED=0`
don't apply marks directly in pytest specifically - don't use `pytest.mark.xfail` directly in parametrize, instead use `pytest.param(*args, marks=pytest.mark.xfail)`. the old way is deprecated in pytest4 - for more information see https://docs.pytest.org/en/latest/deprecations.html#marks-in-pytest-mark-parametrize Also, make this an error in pytest.ini so if someone adds a new xfail, it'll crash 2018-12-31 13:10:19 +00:00
			`filterwarnings =`
			`error:Applying marks directly:pytest.RemovedInPytest4Warning`
utils bump also dont use splitlines on csv data, there might be quoted newlines we dont want to split on 2019-08-02 14:34:05 +01:00			`addopts = -p no:warnings`