`send-to-error-and-continue` directive not initializing `dq_failure` when condition is false

Description

When experimenting with directive send-to-error-and-continue, we found couple of issues:

Even though send-to-error condition is false, record was send to error port.

When condition specified in send-to-error-and-continue is false, dq_failure is not initialized and fails with NPE.

code ref: https://github.com/data-integrations/wrangler/blob/ddff0132eec8a359178837388e81d0cbae415da3/wrangler-core/src/main/java/io/cdap/directives/row/SendToErrorAndContinue.java#L106

Release Notes

None

Activity

Show:

Sagar Ahire February 13, 2024 at 12:07 PM

Seems like the behavior of the plugin was changed in May 2020 by PR. This change may need to be undone.

Sagar Ahire February 13, 2024 at 11:54 AM

I’m seeing that when Send-To-Error-And-Continue encounters a true condition, the record gets sent to the error port immediately. This even happens when the Send-To-Error condition occurring later is obviously false (recipe 1), or if the Send-To-Error condition doesn’t exist at all (recipe 2).

Wrangler recipe 1:

Wrangler recipe 2:

When there is no true condition in Send-To-Error-And-Continue, the Send-To-Error condition is then checked. For example, in recipe 3 below, the record is sent to Error:

Wrangler recipe 3: