Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Add release notes and update documentation about JDBC support from 6.9.1

Description

Default dataproc version from 6.9.1 will be Dataproc 2.1 which will be running in Java11. From 6.9.1 only the JDBC drivers which can run in Java11 will work in pipelines. This is applicable for JDBC drivers used in batch as well as replication pipelines.

It needs to be captured in the documentation and release notes.

Release Notes

Instances in CDAP version 6.9.1 use Dataproc image 2.1 as default compute engine, which runs in Java11. JBDC drivers that are used in database plugins in those instances must be compatible with Java11 for batch and replication pipelines to succeed. If existing behavior must be retained, see [Troubleshoot: Use Dataproc 2.0 as the compute engine for the instance with 6.9.1](link TBD).

Activity

Show:

Olivia EmeryJune 1, 2023 at 9:03 PM

I’m closing this OS documentation bug. The doc updates for this issue are occurring primarily in CDF, and external CDF doc changes happen in a different workflow. Feel free to reopen this if needed.

Olivia EmeryJune 1, 2023 at 8:45 PM

Docs are being tracked in an internal component.

Divya BMay 16, 2023 at 10:34 AM
Edited

Hi Olivia,

We have decided not to go ahead with 2.1 as default dataproc version. However we still want customer to be able to use 2.1 if needed.

We will introduce the support for dataproc-2.1 from 6.9.1

  1. We need a document similar to this, that explains how to change the default dataproc image to 2.1. There is also another option to change the version,

    1. Configure -> compute config -> customize(dataproc) -> Advanced settings -> image version -> Enter 2.1

  2. We will have some exceptions for using dataproc-2.1.

    1. It wont work with Map reduce jobs.

    2. JDBC driver versions of the database plugins will have to be updated to have support for Java11. Driver versions that work with Dataproc2.1 Java11 are listed here,

Divya BMay 9, 2023 at 1:17 PM

We need to add a document similar to this, . This is needed because customer need to know a way to run their instance using dataproc-2.0 since the default version from 6.9.1 is dataproc2-1

Olivia EmeryMay 9, 2023 at 6:54 AM

Can you confirm the accuracy/clarity of the release note?

Duplicate
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Affects versions

Triaged

No

Components

Fix versions

Priority

Created May 5, 2023 at 1:22 PM
Updated June 1, 2023 at 9:06 PM
Resolved June 1, 2023 at 9:06 PM