Add Data - CSV Upload UI #6845
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| <file-upload ng-if="!wizard.file" on-locate="wizard.file = file" upload-selector="button.upload"> | ||
| <h2><em>Pick a CSV file to get started.</em> | ||
| Please follow the instructions below. | ||
| </h2> | ||
|
|
||
| <div class="upload-wizard-file-upload-container"> | ||
| <div class="upload-instructions">Drop your file here</div> | ||
| <div class="upload-instructions-separator">or</div> | ||
| <button class="btn btn-primary btn-lg controls upload" ng-click> | ||
| Select File | ||
| </button> | ||
| <div>Maximum upload file size: 1 GB</div> | ||
| </div> | ||
| </file-upload> | ||
|
|
||
| <div class="upload-wizard-file-preview-container" ng-if="wizard.file"> | ||
| <h2><em>Review the sample below.</em> | ||
| Click next if it looks like we parsed your file correctly. | ||
| </h2> | ||
|
|
||
| <div ng-if="!!wizard.formattedErrors.length" class="alert alert-danger parse-error"> | ||
| <ul> | ||
| <li ng-repeat="error in wizard.formattedErrors track by $index">{{ error }}</li> | ||
| </ul> | ||
| </div> | ||
|
|
||
| <div ng-if="!!wizard.formattedWarnings.length" class="alert alert-warning"> | ||
| <ul> | ||
| <li ng-repeat="warning in wizard.formattedWarnings track by $index">{{ warning }}</li> | ||
| </ul> | ||
| </div> | ||
|
|
||
| <div class="advanced-options form-inline"> | ||
| <span class="form-group"> | ||
| <label>Delimiter</label> | ||
| <select ng-model="wizard.parseOptions.delimiter" | ||
| ng-options="option.value as option.label for option in wizard.delimiterOptions" | ||
| class="form-control"> | ||
| </select> | ||
| </span> | ||
| <span class="form-group"> | ||
| <label>Filename:</label> | ||
| {{ wizard.file.name }} | ||
| </span> | ||
| </div> | ||
|
|
||
| <div class="preview"> | ||
| <table class="table table-condensed"> | ||
| <thead> | ||
| <tr> | ||
| <th ng-repeat="col in wizard.columns track by $index"> | ||
| <span title="{{ col }}">{{ col | limitTo:12 }}{{ col.length > 12 ? '...' : '' }}</span> | ||
| </th> | ||
| </tr> | ||
| </thead> | ||
| <tbody> | ||
| <tr ng-repeat="row in wizard.rows"> | ||
| <td ng-repeat="cell in row track by $index">{{ cell }}</td> | ||
|
There was a problem hiding this comment. Why are we limiting the length of the column headers to 12 characters, but not limiting the data (which in theory could be significantly longer than 12 characters)? Is that intentional? There was a problem hiding this comment. Because the data can wrap but the columns shouldn't. At least that's my thinking. The column limit idea came from @alt74, what do you think Jurgen? There was a problem hiding this comment. It can wrap, but the css as it is now will not force it to wrap if it gets big. There was a problem hiding this comment. You're right, right now it's left up to the browser to figure out the best sizing for each column. Is there a better way to handle it? |
||
| </tr> | ||
| </tbody> | ||
| </table> | ||
| </div> | ||
| </div> | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| import _ from 'lodash'; | ||
| import Papa from 'papaparse'; | ||
| import modules from 'ui/modules'; | ||
| import template from './parse_csv_step.html'; | ||
| import './styles/_add_data_parse_csv_step.less'; | ||
|
|
||
| modules.get('apps/settings') | ||
| .directive('parseCsvStep', function () { | ||
| return { | ||
| restrict: 'E', | ||
| template: template, | ||
| scope: { | ||
| file: '=', | ||
| parseOptions: '=', | ||
| samples: '=' | ||
| }, | ||
| bindToController: true, | ||
| controllerAs: 'wizard', | ||
| controller: function ($scope) { | ||
| const maxSampleRows = 10; | ||
| const maxSampleColumns = 20; | ||
|
|
||
| this.delimiterOptions = [ | ||
| { | ||
| label: 'comma', | ||
| value: ',' | ||
| }, | ||
| { | ||
| label: 'tab', | ||
| value: '\t' | ||
| }, | ||
| { | ||
| label: 'space', | ||
| value: ' ' | ||
| }, | ||
| { | ||
| label: 'semicolon', | ||
| value: ';' | ||
| }, | ||
| { | ||
| label: 'pipe', | ||
| value: '|' | ||
| } | ||
| ]; | ||
|
|
||
| this.parse = () => { | ||
| if (!this.file) return; | ||
| let row = 1; | ||
| let rows = []; | ||
| let data = []; | ||
|
|
||
| const config = _.assign( | ||
| { | ||
| header: true, | ||
| dynamicTyping: true, | ||
| step: (results, parser) => { | ||
| if (row > maxSampleRows) { | ||
| parser.abort(); | ||
|
|
||
| // The complete callback isn't automatically called if parsing is manually aborted | ||
| config.complete(); | ||
| return; | ||
| } | ||
| if (row === 1) { | ||
| // Collect general information on the first pass | ||
| if (results.meta.fields.length > _.uniq(results.meta.fields).length) { | ||
| this.formattedErrors.push('Column names must be unique'); | ||
| } | ||
| _.forEach(results.meta.fields, (field) => { | ||
| if (_.isEmpty(field)) { | ||
| this.formattedErrors.push('Column names must not be blank'); | ||
| } | ||
| }); | ||
| if (results.meta.fields.length > maxSampleColumns) { | ||
| this.formattedWarnings.push(`Preview truncated to ${maxSampleColumns} columns`); | ||
| } | ||
|
|
||
| this.columns = results.meta.fields.slice(0, maxSampleColumns); | ||
| this.parseOptions = _.defaults({}, this.parseOptions, {delimiter: results.meta.delimiter}); | ||
| } | ||
|
|
||
| this.formattedErrors = _.map(results.errors, (error) => { | ||
| return `${error.type} at line ${row + 1} - ${error.message}`; | ||
| }); | ||
|
|
||
| data = data.concat(results.data); | ||
|
|
||
| rows = rows.concat(_.map(results.data, (row) => { | ||
| return _.map(this.columns, (columnName) => { | ||
| return row[columnName]; | ||
| }); | ||
| })); | ||
|
|
||
| ++row; | ||
| }, | ||
| complete: () => { | ||
| $scope.$apply(() => { | ||
| this.rows = rows; | ||
|
|
||
| if (_.isUndefined(this.formattedErrors) || _.isEmpty(this.formattedErrors)) { | ||
| this.samples = data; | ||
| } | ||
| else { | ||
| delete this.samples; | ||
| } | ||
| }); | ||
| } | ||
| }, | ||
| this.parseOptions | ||
| ); | ||
|
|
||
| Papa.parse(this.file, config); | ||
|
There was a problem hiding this comment. Should this be wrapped in a try/catch? I don't know enough about Papaparse, but I assume this would be a good place to wrap. There was a problem hiding this comment. As far as I know papaparse doesn't throw any errors, it returns them as a part of the results. |
||
| }; | ||
|
|
||
| $scope.$watch('wizard.parseOptions', (newValue, oldValue) => { | ||
| // Delimiter is auto-detected in the first run of the parse function, so we don't want to | ||
| // re-parse just because it's being initialized. | ||
| if (!_.isUndefined(oldValue)) { | ||
| this.parse(); | ||
| } | ||
| }, true); | ||
|
|
||
| $scope.$watch('wizard.file', () => { | ||
| delete this.rows; | ||
| delete this.columns; | ||
| delete this.formattedErrors; | ||
| this.formattedWarnings = []; | ||
| this.parse(); | ||
| }); | ||
| } | ||
| }; | ||
| }); | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| @import (reference) "../../../styles/_add_data_wizard"; | ||
|
|
||
| .upload-wizard-file-upload-container { | ||
| min-height: 300px; | ||
| display: flex; | ||
| flex-direction: column; | ||
| justify-content: center; | ||
| background-color: @settings-add-data-wizard-form-control-bg; | ||
| border: @settings-add-data-wizard-parse-csv-container-border 1px dashed; | ||
| text-align: center; | ||
|
|
||
| .upload-instructions { | ||
| font-size: 2em; | ||
| } | ||
|
|
||
| .upload-instructions-separator { | ||
| margin: 15px 0; | ||
| } | ||
|
|
||
| button { | ||
| width: inherit; | ||
| } | ||
|
|
||
| button.upload { | ||
| align-self: center; | ||
| margin-bottom: 15px; | ||
| } | ||
| } | ||
|
|
||
| .upload-wizard-file-preview-container { | ||
| .preview { | ||
| overflow: auto; | ||
| max-height: 500px; | ||
| border: @settings-add-data-wizard-parse-csv-container-border 1px solid; | ||
|
|
||
| table { | ||
| margin-bottom: 0; | ||
|
|
||
| .table-striped() | ||
| } | ||
| } | ||
|
|
||
| .parse-error { | ||
| margin-top: 2em; | ||
| } | ||
|
|
||
| .advanced-options { | ||
| display: flex; | ||
| align-items: center; | ||
|
|
||
| .form-group { | ||
| display: flex; | ||
| align-items: center; | ||
| padding-right: 15px; | ||
|
|
||
| label { | ||
| padding-right: 8px; | ||
| margin-bottom: 0; | ||
| } | ||
| } | ||
|
|
||
| padding-bottom: 10px; | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this value available from config, etc so that it doesn't get out of sync with the actual limit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately it's only available server side.