Wrangler: cleanse-column-names directive requires a trailing space
Description
Release Notes
None
Activity
Show:

Bhooshan Mogal
changed the AssigneeJune 16, 2020 at 7:24 PM
Bhooshan Mogal

Mikkin Patel

Bhooshan Mogal
updated the Fix versionsJune 16, 2020 at 7:24 PMNone
7.1.0

Bhooshan Mogal
updated the ComponentsJune 16, 2020 at 7:23 PMNone
Data Prep

Yuki Jung
changed the AssigneeMay 11, 2020 at 8:04 PM
Trishka Fernandes

Bhooshan Mogal

Yuki Jung
updated the DescriptionMay 11, 2020 at 8:04 PMWrangler does not convert String to Integer when 30% (or more) of the column's data is null.
Steps to reproduce:
1) Upload the attached .csv file using Wrangler. Notice that all the fields are of type String
2) Go to Column transformations at the column NumericValue60 (which is made up of 40% null values), change the data type to Integer
3) The console won't display any error but the column will be shown as String
The backend response has a "success" message but does not change the type of the column NumericValue60 to Integer.
Compare to following steps 2-3 for NumericValue70, which successfully changes the type to Integer.
Wrangler does not convert String to Integer when 30% (or more) of the column's data is null.
Steps to reproduce:
1) Upload the attached .csv file using Wrangler. Notice that all the fields are of type String
2) Go to Column transformations at the column NumericValue60 (which is made up of 40% null values), change the data type to Integer
3) The console won't display any error but the column will be shown as String
The backend response has a "success" message but does not change the type of the column NumericValue60 to Integer. Compare to following steps 2-3 for NumericValue70, which successfully changes the type to Integer.
Another issue is also found when trying to convert the String column to Float, then back to String. The zeros as decimal places will still be shown. In addition, following these steps and trying to convert to Integer, an error message will pop up. The steps to reproduce are below: 1) Select column transformation at column NumericValues100 2) Convert to Float. So, the decimal zeros are shown 3) Convert to String. The decimal zeros are still shown, even thought the data type is changed. 4) Convert to Integer. An error will get returned from the backend.

Robin Rielley
created the IssueJune 8, 2020 at 4:24 PMPinned fields
Click on the next to a field label to start pinning.
Details
Details
Assignee
Bhooshan Mogal
Bhooshan MogalReporter
Robin Rielley
Robin RielleyLabels
Affects versions
Components
Fix versions
Priority
Created June 8, 2020 at 4:24 PM
Updated July 7, 2020 at 6:01 PM
When I use cleanse-column-names without a trailing space, the CLI adds a space after the directive, but nothing happens. The columns don't change and there's no error message. If I add a trailing space after cleanse-column-names, the directive works.
Seems like a bug to require a space after a directive that doesn't have arguments.