Wrangler UI : should throw error when unable to parse AVRO

Description

Repro steps :
- a avro file while a field of schema union
Example field schema :

{ "name" : "Body", "type" : [ "null", "bytes" ] } ]
  • When using a connection and importing this as AVRO in wrangler,
    It doesn’t show any data.

  • On checking the wrangler service we find :

    2023-09-14 07:25:19,952 - WARN [service-http-executor-56:i.c.c.e.p.c.SampleResponseCodec@68] - Error converting the json string {"SequenceNumber":14208103,"Offset":"73362348596784","EnqueuedTimeUtc":"3/12/2023 2:17:24 AM","SystemProperties":{"x-opt-enqueued-time":1678587444563},"Properties":{},"Body": [large byte array removed for this comment]} to StructuredRecord java.io.IOException: Unable to convert string to type UNION at io.cdap.cdap.format.io.JsonStructuredRecordDatumReader.convertKey(JsonStructuredRecordDatumReader.java:209) at io.cdap.cdap.format.io.JsonStructuredRecordDatumReader.decodeMap(JsonStructuredRecordDatumReader.java:133) at io.cdap.cdap.format.io.StructuredRecordDatumReader.decode(StructuredRecordDatumReader.java:76) at io.cdap.cdap.format.io.JsonStructuredRecordDatumReader.decode(JsonStructuredRecordDatumReader.java:98) at io.cdap.cdap.format.io.JsonStructuredRecordDatumReader.decodeRecord(JsonStructuredRecordDatumReader.java:156) at io.cdap.cdap.format.io.StructuredRecordDatumReader.read(StructuredRecordDatumReader.java:40) at io.cdap.cdap.format.io.JsonStructuredRecordDatumReader.read(JsonStructuredRecordDatumReader.java:87) at io.cdap.cdap.format.StructuredRecordStringConverter.fromJsonString(StructuredRecordStringConverter.java:61) at io.cdap.cdap.etl.proto.connection.SampleResponseCodec.lambda$deserialize$0(SampleResponseCodec.java:64) at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:901) at io.cdap.cdap.etl.proto.connection.SampleResponseCodec.deserialize(SampleResponseCodec.java:61) at io.cdap.cdap.etl.proto.connection.SampleResponseCodec.deserialize(SampleResponseCodec.java:43) at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58) at com.google.gson.Gson.fromJson(Gson.java:810) at com.google.gson.Gson.fromJson(Gson.java:775) at com.google.gson.Gson.fromJson(Gson.java:724) at com.google.gson.Gson.fromJson(Gson.java:696) at io.cdap.wrangler.service.directive.ConnectionDiscoverer.retrieveResult(ConnectionDiscoverer.java:202) at io.cdap.wrangler.service.directive.ConnectionDiscoverer.execute(ConnectionDiscoverer.java:134) at io.cdap.wrangler.service.directive.ConnectionDiscoverer.retrieveSample(ConnectionDiscoverer.java:106) at io.cdap.wrangler.service.directive.WorkspaceHandler.lambda$createWorkspace$1(WorkspaceHandler.java:153) at io.cdap.wrangler.service.common.AbstractWranglerHandler.respond(AbstractWranglerHandler.java:202) at io.cdap.wrangler.service.directive.WorkspaceHandler.createWorkspace(WorkspaceHandler.java:136) at io.cdap.wrangler.service.directive.WorkspaceHandlerf667fa1c55614e7a62f467e95f30ef13.lambda$createWorkspace_Lio_cdap_cdap_api_service_http_HttpServiceRequest_Lio_cdap_cdap_api_service_http_HttpServiceResponder_Ljava_lang_String__V(Unknown Source) at io.cdap.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:608) at io.cdap.cdap.internal.app.services.ServiceHttpServer$HandlerDelegatorContext$1.execute(ServiceHttpServer.java:332) at io.cdap.cdap.internal.app.runtime.service.http.AbstractHttpHandlerDelegator.execute(AbstractHttpHandlerDelegator.java:69) at io.cdap.wrangler.service.directive.WorkspaceHandlerf667fa1c55614e7a62f467e95f30ef13.createWorkspace(Unknown Source)



We should throw error message on UI that we don’t support UNION type.

Release Notes

None

Activity

Show:
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Affects versions

Triaged

No

Components

Priority

Created September 20, 2023 at 12:58 PM
Updated September 20, 2023 at 12:59 PM