Handle java.sql.DECIMAL type correctly in existing database plugins

Description

Current implementation of database plugins does not support mapping of java.sql.DECIMAL type to cdap decimal types.

In addition to that, for java.sql.DECIMAL types, existing implementation tries to map it to either integer or long depending on the precision value. This can be confusing to users because based on the precision the type of the schema changes. https://github.com/cdapio/hydrator-plugins/blob/develop/database-plugins/src/main/java/io/cdap/plugin/DBUtils.java#L264

 

 

Also, this type is interpreted as BigDecimal when it is read from the connection. However, when the view schema is clicked, the associated cdap schema uses double.

https://github.com/data-integrations/wrangler/blob/develop/wrangler-service/src/main/java/io/cdap/wrangler/service/database/DatabaseHandler.java#L503 

 

The user should only see one type that is associated with java.sql.DECIMAL in wrangler as well as database plugins. The proper fix would be to use cdap decimal schema type for java.sql.DECIMAL types in wrangler as well as generic database plugins in hydrator repository.

Release Notes

None

Activity

Show:
Bajram Bojku
January 21, 2021, 10:32 PM
Edited

cherrypicks for dbsink additions:

2.3

2.4

2.5

Vinisha Shah
December 17, 2020, 12:41 AM
Edited

Please make sure fixes are added for DB sink as well as Wrangler. Also please add unit tests for the fixes added. We should not mark item as completed without that.

Bajram Bojku
December 16, 2020, 4:33 PM
Edited
Vjosa Dika
November 5, 2020, 2:46 PM
Shifu Xu
April 6, 2020, 9:34 PM
Edited

The fixes are done for cdc database source plugins. 

Assignee

Bajram Bojku

Reporter

Vinisha Shah

Labels

None

Dev Complete Date

None

Publish Date

None

Reviewer

None

Sprint

None

Fix versions

Priority

Major