Currently, this can only be set in cdap-site.xml and is therefore site-wide. It is often desirable to control on a per-use-case basis.
In this particular case, this could also be solved by providing a specific API to a program to control its transaction timeout.
But in general, it should be possible to configure system properties on a per-app or per-program basis.
Allows configuration of the default transaction timeout at the level of a program, flowlet, or custom workflow action.
The idea is similar to CDAP-5770: to have the transaction timeout settable through a runtime argument:
system.tx.timeout - transaction timeout in seconds.
For MapReduce and Spark the timeout applies only to the client (that is, it's initialize/destroy), as all other tasks will be inside a long transaction.
For program types that have sub-components (Flow, Workflow), a prefix can be added to narrow the scope of the argument.
When prefixed with "flowlet.[flowletName].", the timeout will only affect the given flowlet.
Prefix with "action.[actionName]." to set the timeout for the named custom action only.
For MR and Spark, the timeout should only affect the client lifecycle method, right?
yes, and that is actually mentioned in the 3rd line. Oh, it says driver, not client. Will fix.
Yes I saw that, just want to make it clear it's not the driver of the Spark program