Improve JobQueueDataset/ConstraintCheckerService implementation

Description

This is a JIRA to track changes that need to happen to JobQueueDataset and ConstraintCheckerService.

1. ConstraintCheckerService#runReadyJobs should not limit the number of programs launched after checking constraints to one.

2. Constraints of the Job should be able to return a 'nextCheckTime', and the ConstraintCheckerService shouldn't need to check those constraints again until that time.
https://github.com/caskdata/cdap/pull/8720#discussion_r116291037

3. If the scheduler service crashes while a job is in PENDING_LAUNCH and in the in-memory `readyJobs`, then upon startup, they will have state PENDING_LAUNCH, but not be in `readyJobs`. Because of their state, we will not check their constraints, and they will not ever be executed.
https://github.com/caskdata/cdap/pull/8720#discussion_r116331593

Release Notes

None

Activity

Show:
Ali Anwar
June 1, 2017, 12:14 AM

Tasks 1 and 3 are implemented in https://github.com/caskdata/cdap/pull/8983 (for 4.2.0).

Ali Anwar
June 1, 2017, 5:08 AM

Moving the remainder (Task #2) out to 4.3.0, as that is not critical.

Sreevatsan Raman
August 16, 2017, 5:53 PM

Pending item is an optimization to improve the frequency of the checking next runtime. Moving it out.

Assignee

Ali Anwar

Reporter

Ali Anwar

Labels

None

Docs Impact

None

UX Impact

None

Components

Priority

Major
Configure