File based sources should have an optional field for source file
Description
It would be nice if the File based source plugins had an option to include the file path the record was read from as a field in the output record. Not sure if this can always be done, as it depends on the input format being used.
For example, FileSplit has a way to get the path, but something like CombineFileSplit is across multiple paths.
It would be nice if the File based source plugins had an option to include the file path the record was read from as a field in the output record. Not sure if this can always be done, as it depends on the input format being used.
For example, FileSplit has a way to get the path, but something like CombineFileSplit is across multiple paths.