Using readless increments in concurrent transactions causes conflicts
Description
Release Notes
relates to
Activity
Andreas Neumann September 2, 2016 at 7:59 PM
Also see this github issue: https://github.com/cdap-guides/cdap-cube-guide/issues/21
Andreas Neumann September 2, 2016 at 7:01 PM
It would make sense to enable that for Cubes. In fact I am wondering whether that should be the default.
However, the Cube's FactTable performs both increment() and put(), and I am not sure whether it is safe to disable conflict detection. It might be ok, because even with conflict detection, if concurrent processes emit gauges, they would overwrite each other values.
This needs a little more thinking, but I definitely vote for adding a section to the documentation, both for Table and Cube (and perhaps TimeseriesTable, too?).
Alan Steckley September 2, 2016 at 4:10 PM
Would it also be appropriate to add the option to enable readless increments to the Cube documentation?
Andreas Neumann September 2, 2016 at 3:46 PM
See https://issues.cask.co/browse/CDAP-2560 for a discussion. I do think that we should improve the documentation here: http://docs.cdap.io/cdap/current/en/developers-manual/building-blocks/datasets/table.html#increment, and recommend that conflict detection should also be configured as NONE.
Andreas Neumann September 2, 2016 at 3:41 PM
The reason why readless increments still participate in conflict detection - if conflict detection is turned on for the table - is that a readless increment, even though it does not conflict with other readless increments, does conflict with other operations, such as delete or put.
Also, tables with readless increment enabled do allow other operations (possibly on columns/rows other than the ones participating in readless increments) that may conflict with each other. That is why we do not turn off conflict detection automatically.
Today a readless increments table needs conflict detection to be turned off in both SDK and in distributed mode. However when a readless increment table is created, conflict detection is not automatically turned off. Hence when readless increments are applied on a given cell in concurrent transactions, we see conflicts.
This can be fixed in two ways -
1. Automatically turn off conflict detection when a readless increment table is created.
2. Remove the restriction that readless increments table needs conflict detection to be turned off.