Schema class is not serializable by Kryo

Description

In CDH 5.11 prerelease running the test "BatchJoinerTest" fails when running the Spark jobs when the the Spark property "spark.serializer" is set to "org.apache.spark.serializer.KryoSerializer". It works when set to "org.apache.spark.serializer.JavaSerializer".

Previously all Spark jobs would fail when the property "org.apache.spark.serializer.KryoSerializer" was used. This was fixed in https://issues.cask.co/browse/CDAP-8980 and the PR https://github.com/caskdata/cdap/pull/8440

Release Notes

Made a StructuredRecord serializable by Kryo in a Spark program when the Kryo Serializer is used.

Activity

Show:
Terence Yim
April 9, 2017, 9:07 AM
Fixed

Assignee

Terence Yim

Reporter

Matt Wuenschel

Labels

None

Docs Impact

None

UX Impact

None

Fix versions

Affects versions

Priority

Major