darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2026-02-01 23:55:58 -05:00
Code Issues Packages Projects Releases Wiki Activity

Update limit to 1MB and update tests

Browse Source 
SES rejects email messages bigger than 10485760 bytes (just over 10 MB per message (after base64 encoding)):
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/quotas.html#limits-message

Base64 is apparently wasteful because we use just 64 different values per byte, whereas a byte can represent
256 different characters. That is, we use bytes (which are 8-bit words) as 6-bit words. There is
a waste of 2 bits for each 8 bits of transmission data. To send three bytes of information
(3 times 8 is 24 bits), you need to use four bytes (4 times 6 is again 24 bits). Thus the base64 version
of a file is 4/3 larger than it might be. So we use 33% more storage than we could.
https://lemire.me/blog/2019/01/30/what-is-the-space-overhead-of-base64-encoding/

That brings down our max safe size to 7.5 MB == 7500000 bytes before base64 encoding

But this is not the end! The message we send to SES is structured as follows:
"Message": {
    'Subject': {
        'Data': subject,
    },
    'Body': {'Text': {'Data': body}, 'Html': {'Data': html_body}}
},
Which means that we are sending the contents of email message twice in one request: once in plain text
and once with html tags. That means our plain text content needs to be much shorter to make sure we
fit within the limit, especially since HTML body can be much byte-heavier than plain text body.

Hence, we decided to put the limit at 1MB, which is equivalent of between 250 and 500 pages of text.
That's still an extremely long email, and should be sufficient for all normal use, while at the same
time giving us safe margin while sending the emails through Amazon SES.
This commit is contained in:
Pea Tyczynska
2020-10-14 10:59:13 +01:00
parent 9708b09ba3
commit 41d1cf453d
3 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
tests/app/notifications/test_validators.py
View File

@@ -338,21 +338,21 @@ def test_check_message_is_not_too_long_for_too_long_sms_messages(notify_db_sessi
check_message_is_not_too_long(template_with_content)
assert e.value.status_code == 400
assert e.value.message == f'Text messages cannot be longer than {SMS_CHAR_COUNT_LIMIT} characters. ' \
f'Your message is {char_count} characters.'
f'Your message is {char_count} characters long.'
assert e.value.fields == []
def test_check_message_is_not_too_long_for_too_big_email_messages(notify_db_session):
with pytest.raises(BadRequestError) as e:
service = create_service()
t = create_template(service=service, content='a' * 7500000, template_type='email')
t = create_template(service=service, content='a' * 1000000, template_type='email')
template = templates_dao.dao_get_template_by_id_and_service_id(template_id=t.id, service_id=service.id)
template_with_content = get_template_instance(template=template.__dict__, values={})
check_message_is_not_too_long(template_with_content)
assert e.value.status_code == 400
assert e.value.message == (
f"Email messages cannot be longer than 7500000 bytes. "
f"Your message is 7500081 bytes."
f"Emails cannot be longer than 1000000 bytes. "
f"Your message is 1000081 bytes."
)
assert e.value.fields == []