Add option to write to files using proper JSON

Description

When the format is set to JSON, and Content Type is set to application/json, the individual records are saved as JSON objects, but are not comma-separated, and aren't in a JSON array, so the file itself is not a valid JSON file.

Actual file output:

Expected File Output:

Release Notes

None

Activity

Show:
Albert Shau
February 11, 2021, 6:00 AM

This is the intended behavior, it’s not technically a valid json element, but this is the format that the source reads and that many other distributed systems work with (for example, you can load this file directly to BigQuery). The reason has to do with being able to split the file in order to read it efficiently. If it is a proper json array, there isn’t a way to split it.

 

That being said, it would be worthwhile to add an option to write it out as proper json, assuming that is what is wanted by the user.

Assignee

Unassigned

Reporter

Shahmir Noorani

Labels

None

Product Requirement Doc

None

Reviewer

None

Dev Complete Date

None

Publish Date

None

Docs Impact

None

UX Impact

None

Components

Priority

Major