ConcurrentPartitionConsumer WorkingSet does not aged out partitions that no longer exists

Description

Partitions that are in the WorkingSet and are later dropped will remain in the WorkingSet forever. To reproduce, added two partitions to a PartitionedFileSet and set the ConcurrentPartitionConsumer's ConsumerConfiguration with a maxWorkingSetSize of 1 and the timeout of 60. Execute consumePartitions and mark the partition as failed with onFinish. Drop the partition that was marked as failed. Call consumePartitions again and nothing will return because the failed partition that was dropped is still in the workingSet.

Release Notes

PartitionConsumer appropriately drops partitions that have been deleted from a corresponding PartitionedFileSet

Activity

Show:

Ali Anwar September 7, 2016 at 11:23 PM
Edited

Opened a PR for this (targeting 3.5.1): https://github.com/caskdata/cdap/pull/6676

Jeff Dix August 25, 2016 at 1:49 PM

Hey Ali, thanks for checking. I will keep my copy of ConcurrentPartitionConsumer with the fix.

Ali Anwar August 24, 2016 at 7:50 PM

Hey Jeff.

This JIRA wasn't tracked for 3.5, so it wasn't given much attention. I checked ConcurrentPartitionConsumer, and it doesn't have the fix I commented about earlier (https://github.com/caskdata/cdap/compare/bugfix/CDAP-6215-remove-deleted-partitions-from-workingset?expand=1).

Jeff Dix August 24, 2016 at 7:46 PM

Hey Ali, did you fix this in 3.5? I thought I saw the fix in the code.

Ali Anwar July 8, 2016 at 5:20 AM

Right - if the onFinish of the partitions (of the partition consumer) is done within the MapReduce job (such as in the onFinish method), then the MapReduce's transaction would get aborted.
However, if the failure/exception is thrown from an action after the MapReduce, then the output of the MR won't get rollbacked, since it's in a separate transaction. I forgot about that earlier.

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Affects versions

Components

Fix versions

Priority

Created June 16, 2016 at 7:23 PM
Updated September 8, 2016 at 6:48 PM
Resolved September 8, 2016 at 4:46 PM