Commit Graph

1805 Commits

Author SHA1 Message Date
Rebecca Law
93c4f37d08 Add lxml 3.6.0 to requirements. 2016-05-17 08:48:17 +01:00
Adam Shimali
ed74280044 Merge pull request #570 from alphagov/reply-to-email
Admin app settings to save reply to email address for service.
2016-05-16 17:51:05 +01:00
Chris Hill-Scott
37fff2da63 Revert "Install lxml first on ubuntu"
This reverts commit 844f9c4276.
2016-05-16 17:22:26 +01:00
Rebecca Law
844f9c4276 Install lxml first on ubuntu 2016-05-16 16:43:55 +01:00
Chris Hill-Scott
d39717d348 Merge pull request #567 from alphagov/uncomment-out-tests
Uncomment commented-out tests
2016-05-16 14:29:10 +01:00
Adam Shimali
18a0b94d98 Admin app settings to save reply to email address for service. 2016-05-16 13:10:51 +01:00
Chris Hill-Scott
b35f2d7e61 Merge pull request #565 from alphagov/accept-common-spreadsheet-formats
Accept common spreadsheet formats, not just CSV
2016-05-16 13:05:14 +01:00
Chris Hill-Scott
f2a5d77853 Uncomment commented-out tests
Accidentally(?) introduced in 2792bece54
2016-05-16 10:53:15 +01:00
Chris Hill-Scott
7bbc307a3e Handle files that can’t be interpreted as spreadsheets
There shouldn’t be a case where we see a `ValueError` on upload any
more. Our file handling should be robust enough to deal with whatever is
thrown at it.

This commit:
- adds test files with bad data (PNG files with their extensions changed to look
  like spreadsheets)
- catches whatever exceptions are raised by trying to parse these files
- returns a helpful flash message to the user

Anything else should raise a `500`, eg if the file can’t be uploaded to S3.
2016-05-15 22:11:01 +01:00
Chris Hill-Scott
1409ca36ca Accept common spreadsheet formats, not just CSV
We require users to export their spreadsheets as CSV files before
uploading them. But this seems like the sort of thing a computer should
be able to do.

So this commit adds a wrapper class which:
- takes a the uploaded file
- returns it in a normalised format, or reads it using pyexcel[1]
- gives the data back in CSV format

This allows us to accept `.csv`, `.xlsx`, `.xls` (97 and 95), `.ods`,
`.xlsm` and `.tsv` files. We can upload the resultant CSV just like
normal, and process it for errors as before.

Testing
---

To test this I’ve added a selection of common spreadsheet files as test
data. They all contain the same data, so the tests look to see that the
resultant CSV output is the same for each.

UI changes
---

This commit doesn’t change the UI, apart from to give a different error
message if a user uploads a file type that we still don’t understand.

I intend to do this as a separate pull request, in order to fulfil
https://www.pivotaltracker.com/story/show/119371637
2016-05-15 22:10:58 +01:00
Chris Hill-Scott
a348bce191 Remove .gitmodules
We don’t use submodules any more. Having this file stick around is
causing problems in Atom, detailed here:
https://github.com/libgit2/libgit2/issues/3414
2016-05-14 11:50:34 +01:00
minglis
cccf2c4590 Merge pull request #562 from alphagov/standardise-coding-examples
Docs: tidy up indentation in JSON examples
2016-05-13 09:00:32 +01:00
minglis
d160061408 Merge pull request #561 from alphagov/record-request-time
Record start of request so we can time all requests
2016-05-13 09:00:04 +01:00
Martyn Inglis
914a546ef5 Bumnped version of utils 2016-05-12 16:34:46 +01:00
Henry Hadlow
8e85b22157 Docs: tidy up indentation in JSON examples 2016-05-12 15:44:38 +01:00
catherineheywood
860b4fd8cc Remove more tabs 2016-05-12 15:17:49 +01:00
catherineheywood
730988173c Remove more tabs 2016-05-12 14:30:08 +01:00
catherineheywood
0e57ebae92 Remove extra lines in coding 2016-05-12 14:24:03 +01:00
catherineheywood
4f9737bd35 Replace tabs with spaces in coding 2016-05-12 14:22:49 +01:00
Martyn Inglis
af59ecb379 Record start of request so we can time all requests 2016-05-12 13:56:14 +01:00
Chris Hill-Scott
3fd7457a21 Merge pull request #560 from alphagov/code-formatting-in-docs
Give the <code> example in docs better CSS
2016-05-12 13:09:08 +01:00
Chris Hill-Scott
f1bf1d3d42 Merge pull request #559 from alphagov/test-message-not-run
Change "test run" to "test message"
2016-05-12 13:08:39 +01:00
Chris Hill-Scott
a659c75276 Give the <code> example in docs better CSS
- adds a background colour, to words wrapped in `<code>` tags, like we
  have for whole snippets of code
- reduces the font size of all code blocks
  a) to differentiate them further
  b) to fit more on the screen
2016-05-12 11:34:57 +01:00
Chris Hill-Scott
bda3aa98f8 Change "test run" to "test message"
Because reasons.
Obvs.
2016-05-12 11:22:14 +01:00
minglis
daf33b03ad Merge pull request #557 from alphagov/notification_show_updated_at
Notifications display updated_at instead of created_at.
2016-05-11 16:55:37 +01:00
minglis
2388c395e3 Merge pull request #552 from alphagov/primary-providers
Added provider management pages in.
2016-05-11 16:49:47 +01:00
Henry Hadlow
e3fa1ac253 Tidy up text on forgot your password screens
Display updated_at instead of created_at for notifications.
2016-05-11 16:30:15 +01:00
Martyn Inglis
8ef0712c82 Merge branch 'primary-providers' of github.com:alphagov/notifications-admin into primary-providers 2016-05-11 15:39:28 +01:00
Martyn Inglis
31f14fd0ec Removed print statements 2016-05-11 15:39:03 +01:00
Pete Herlihy
f18972755a Merge pull request #547 from alphagov/make-forgot-your-password-text-consistent
Tidy up text on forgot your password screens
2016-05-11 15:10:46 +01:00
Rebecca Law
9232ab51d5 Merge pull request #554 from alphagov/add-created_by-to-jobs
Update the jobs and activity page to show the user that created the job.
2016-05-11 14:55:29 +01:00
Rebecca Law
53fb1dcc4b Use Uploaded by instead of Created by for the label.
Move status to the last column on the table
2016-05-11 14:45:20 +01:00
Rebecca Law
1a07162f5d Shorten the list of required mocks to login. 2016-05-11 13:16:59 +01:00
Adam Shimali
461581d299 Only trigger dependent built for master, staging and live 2016-05-11 12:33:11 +01:00
NIcholas Staples
cd9c8f06f6 Merge pull request #553 from alphagov/update_to_utcnow
Update all dates to use utc, only in the template is it converted to …
2016-05-11 12:07:07 +01:00
Rebecca Law
46d5065297 Update the jobs and activity page to show the user that created the job. 2016-05-11 11:57:31 +01:00
Nicholas Staples
af500a96e7 Update all dates to use utc, only in the template is it converted to british time. 2016-05-11 11:20:45 +01:00
Martyn Inglis
6699442f6b Added provider management pages in.
- see priority
- change priority
2016-05-11 09:43:55 +01:00
Adam Shimali
521fdc26ab Pass branch name to dependent build 2016-05-10 14:10:08 +01:00
Henry Hadlow
36da188d04 Tidy up text on forgot your password screens 2016-05-10 11:36:49 +01:00
NIcholas Staples
90140e1d47 Merge pull request #545 from alphagov/no_permissions_banner_msg_fix
Fix for user with only send permissions.
2016-05-09 16:02:24 +01:00
catherineheywood
3fa6d87a16 reword error message 2016-05-09 14:47:06 +01:00
Adam Shimali
4d3697419a Should dependent build env var be set global? 2016-05-09 14:42:51 +01:00
catherineheywood
05553d4058 more sms changed to text 2016-05-09 14:39:58 +01:00
Adam Shimali
506ef3a30f I think json for dependent build env var was incorrect 2016-05-09 14:10:07 +01:00
Adam Shimali
2c230fb6d8 Another attempt to trigger dependent build 2016-05-09 13:50:18 +01:00
Adam Shimali
10d5f03124 Another attempt at triggering dependent build 2016-05-09 13:33:52 +01:00
Adam Shimali
5e73fa9a2e Try to pass test env to run as part of trigger depdendent build 2016-05-09 12:57:30 +01:00
catherineheywood
18f4d1e8c1 POST and GET in link 2016-05-09 12:51:02 +01:00
catherineheywood
617134781b correct link 2016-05-09 12:48:26 +01:00