mirror of
https://github.com/GSA/notifications-admin.git
synced 2026-05-30 11:00:14 -04:00
0ce7f72b07
This is a quick additional check to protect the user: - From getting a CloudFront 502 error if the file takes too long to upload. I was surprised to find it takes about 1 minute to upload a 70Mb file to S3.* - From getting a CloudFront 502 error when we follow the redirect and run through the slow processing code in utils that builds a RecipientCSV [1]. For context, a CSV with 100K rows and a few columns is around 5Mb, so a 10Mb limit should be enough. Analysis over the past week shows that the vast majority of CSV uploads are actually < 2.5Mb. I haven't added any tests for this because: - The check isn't critical, as the worst case scenario is the user gets a worse error than this in-app one. - There's no easy way to mock the validation, and I didn't want to have a test that depends on a 10Mb+ file. *We're using "key.put" to upload the file, when we could be doing a multipart upload [2]. However, I tried this myself with a chunk size of 1000 bytes and found it only led to a marginal improvement. [1]: https://github.com/alphagov/notifications-utils/pull/930 [2]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html