...
This example will read from a SnapshotFileSet named ‘users’. It will read data in Avro format using the given schema. Every time the pipeline runs, only the most recently added snapshot will be read:
...
Property
...
Value
...
Dataset Name
...
| Code Block |
|---|
{
"name": "SnapshotAvro",
"type": "batchsource",
"properties": {
"name": "users",
"schema": "{
\"type\":\"record\",
\"name\":\"user\",
\"fields\":[
{\"name\":\"id\",\"type\":\"long\"},
{\"name\":\"name\",\"type\":\"string\"},
{\"name\":\"birthyear\",\"type\":\"int\"}
]
}"
}
} |