Delete Kafka logs on unrecoverable reset
Activity
Show:
Alex Baranau February 5, 2015 at 11:20 PM
When https://issues.cask.co/browse/CDAP-1418 is implemented this issue will be fixed indirectly.
Alex Baranau February 5, 2015 at 11:20 PM
Moving to backlog. Not sure who planned it for 2.8.
Overall, unrecoverable reset functionality needs to be changed. See https://cdap.atlassian.net/browse/CDAP-1418#icft=CDAP-1418
Sreevatsan Raman January 13, 2015 at 9:25 PM
There is a similar change discussed for metrics. Maybe we can tackle these together.
Terence Yim October 20, 2014 at 9:08 PM
It would be very difficult to delete kafka log from CDAP. Instead, it maybe better if we record a generation id and increments it on unrecoverable reset and we construct the kafka topic name with the generation id in it.
Terence
Won't Fix
Pinned fields
Click on the next to a field label to start pinning.
Created October 20, 2014 at 8:18 PM
Updated June 9, 2020 at 1:26 AM
Resolved June 9, 2020 at 1:26 AM
During an unrecoverable reset all data in CDAP is deleted, including all tables in HBase. We will need to delete the Kafka logs too to be consistent.
Also as unrecoverable reset removes the metadata stored by Log Saver (and possibly Metrics Processor), if Kafka logs are not deleted then Log Saver will start processing Kafka logs from beginning which is wasteful.