Record Splitter (Delimited) - With pipe | value delimiter the records splits with each letter. With [|] and \\| inserted in BQ, then same letters splits.

Description

BQsource->Record Splitter->GCSsink, provide | pipe delimiter in delimiter field in record splitter plugin, having record values in BQ table the output generated is letter split.

Steps to reproduce:

  1. Launch the latest CDAP (6.8.0)

  2. Open BigQuery source, record splitter plugin and GCS sink, now establish the connection from source to transform to sink (BQ source->Record Splitter->GCS sink). Fill the properties, provide | pipe delimiter value and click on validate. Already inserted the data in BQ table with | values.

  3. Inserted the data with double slash in BQ table as below.

  4. insert into `cdf-athena.test_automation.delimiterslashtest`(user, item, action) values ('pipe5|pipe6|pipe7|pipe8|pipe9','donut','buy');
  5. Now click on preview of pipeline.

  6. And deploy the pipeline, record splits with each letter.

  7. Also when value inserted with [|] \\| in BQ table like below,

  8. insert into `cdf-athena.test_automation.Delimited`(user, item, action) values ('pipe5\\|pipe6\\|pipe7\\|pipe8\\|pipe9','donut','buy');

    then output value generated as letter split.

  9. Attached the screenshots reference.

Expected: | pipe delimiter splits record with each letter in output.
Actual: Only value before pipe | delimiter should be splitted.

Release Notes

None

Attachments

2
  • 02 Jun 2022, 09:27 AM
  • 02 Jun 2022, 09:27 AM

Activity

Show:
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Components

Priority

Created June 2, 2022 at 9:26 AM
Updated June 2, 2022 at 9:46 AM