Realtime pipeline fails to run

Description

The Pubsub cloud integration test is failing because the pipeline fails to start. The pipeline gets stuck in the starting state. When I SSH to the dataproc master node, I see the following in the program.log for the run:

This error never makes it to the CDAP logs. In addition, the remote process is stuck forever shutting down. Taking a stack dump, I see a single thread:

which looks related.

Release Notes

None

Activity

Show:
Vinisha Shah
April 22, 2020, 2:12 AM
Vinisha Shah
April 21, 2020, 1:45 PM
Albert Shau
April 20, 2020, 8:10 PM

Integration test is at https://github.com/cdapio/cdap-integration-tests/blob/develop/integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/PubSubTest.java#L180.

Also attached the pipeline generated by the integration test (with project id redacted)

Fixed

Assignee

Vinisha Shah

Reporter

Albert Shau

Labels

None

Docs Impact

None

UX Impact

None

Components

Fix versions

Affects versions

Priority

Blocker