Transaction timeout should be configurable per app or program
Description
Currently, this can only be set in cdap-site.xml and is therefore site-wide. It is often desirable to control on a per-use-case basis.
In this particular case, this could also be solved by providing a specific API to a program to control its transaction timeout.
But in general, it should be possible to configure system properties on a per-app or per-program basis.
Release Notes
Allows configuration of the default transaction timeout at the level of a program, flowlet, or custom workflow action.
Activity
Design
The idea is similar to CDAP-5770: to have the transaction timeout settable through a runtime argument:
system.tx.timeout - transaction timeout in seconds.
For MapReduce and Spark the timeout applies only to the client (that is, it's initialize/destroy), as all other tasks will be inside a long transaction.
For program types that have sub-components (Flow, Workflow), a prefix can be added to narrow the scope of the argument.
Flow
When prefixed with "flowlet.[flowletName].", the timeout will only affect the given flowlet.
Workflow Actions
Prefix with "action.[actionName]." to set the timeout for the named custom action only.
For MR and Spark, the timeout should only affect the client lifecycle method, right?
yes, and that is actually mentioned in the 3rd line. Oh, it says driver, not client. Will fix.
Yes I saw that, just want to make it clear it's not the driver of the Spark program
