Record Splitter (Delimited) - With pipe | value delimiter the records splits with each letter. With [|] and \\| inserted in BQ, then same letters splits.
Description
Release Notes
None
Attachments
2
- 02 Jun 2022, 09:27 AM
- 02 Jun 2022, 09:27 AM
Activity
Show:
Pinned fields
Click on the next to a field label to start pinning.
Details
Details
Created June 2, 2022 at 9:26 AM
Updated June 2, 2022 at 9:46 AM
BQsource->Record Splitter->GCSsink, provide | pipe delimiter in delimiter field in record splitter plugin, having record values in BQ table the output generated is letter split.
Steps to reproduce:
Launch the latest CDAP (6.8.0)
Open BigQuery source, record splitter plugin and GCS sink, now establish the connection from source to transform to sink (BQ source->Record Splitter->GCS sink). Fill the properties, provide | pipe delimiter value and click on validate. Already inserted the data in BQ table with | values.
Inserted the data with double slash in BQ table as below.
insert into `cdf-athena.test_automation.delimiterslashtest`(user, item, action) values ('pipe5|pipe6|pipe7|pipe8|pipe9','donut','buy');
Now click on preview of pipeline.
And deploy the pipeline, record splits with each letter.
Also when value inserted with [|] \\| in BQ table like below,
insert into `cdf-athena.test_automation.Delimited`(user, item, action) values ('pipe5\\|pipe6\\|pipe7\\|pipe8\\|pipe9','donut','buy');
then output value generated as letter split.
Attached the screenshots reference.
Expected: | pipe delimiter splits record with each letter in output.
Actual: Only value before pipe | delimiter should be splitted.