...
See Scaling Instances and Resource Guarantees for CDAP Programs in YARN.
Upgrading a Pipeline
If you want to upgrade 4.2.x pipelines, you can use the ETL upgrade tool packaged with the distributed version of CDAP. You would want to run this tool to upgrade applications that were created with earlier versions of the artifacts, which you want to open in the 6.2.0 version of the Pipeline Studio.
The tool will connect to an instance of CDAP, look for any applications that use 4.2.x versions of the system artifacts, and then update the application to use the 6.2.0 version of those artifacts.
CDAP must be running when you run the command:
| Code Block |
|---|
$ java -cp /opt/cdap/master/libexec/cdap-etl-tools-6.2.0.jar io.cdap.cdap.etl.tool.UpgradeTool -u http://<host>:<port> -e /tmp/failedUpgrades upgrade |
The first argument is the host and port for the CDAP router. The second argument is a directory to write the configurations of any pipelines that could not be upgraded. A pipeline may fail to upgrade if the new version of a plugin used in the pipeline is not backwards compatible. For example, this may happen if the plugin added a new required property.
You can also upgrade just the ETL applications within a specific namespace:
| Code Block |
|---|
$ java -cp /opt/cdap/master/libexec/cdap-etl-tools-6.2.0.jar io.cdap.cdap.etl.tool.UpgradeTool -u http://<host>:<port> -n <namespace-id> upgrade |
You can also upgrade just one ETL application:
| Code Block |
|---|
$ java -cp /opt/cdap/master/libexec/cdap-etl-tools-6.2.0.jar io.cdap.cdap.etl.tool.UpgradeTool -u http://<host>:<port> -n <namespace-id> -p <app-name> upgrade |
If you have authentication turned on, you also need to store an access token in a file and pass the file to the tool:
| Code Block |
|---|
$ java -cp /opt/cdap/master/libexec/cdap-etl-tools-6.2.0.jar io.cdap.cdap.etl.tool.UpgradeTool -u http://<host>:<port> -a <tokenfile> upgrade |
For instance, if you have obtained an access token (as shown in the example in the security documentation <testing-security>) such as:
| Code Block |
|---|
{"access_token":"AghjZGFwAI7e8p65Uo7OpfG5UrD87psGQE0u0sFDoqxtacdRR5GxEb6bkTypP7mXdqvqqnLmfxOS",
"token_type":"Bearer","expires_in":86400} |
The access token itself (AghjZGFwAI7e8p65Uo7OpfG5UrD87psGQE0u0sFDoqxtacdRR5GxEb6bkTypP7mXdqvqqnLmfxOS) would be placed in a file and then the file's path would be used in the above command.