Java Client API

The CDAP Java Client API provides methods for interacting with CDAP from Java applications.

Note: Datasets are deprecated and will be removed in CDAP 7.0.0.

Maven Dependency

To use the Java Client API in your project, add this Maven dependency:

1 2 3 4 5 <dependency> <groupId>io.cdap.cdap</groupId> <artifactId>cdap-client</artifactId> <version>${cdap.version}</version> </dependency>

Components

The Java Client API allows you to interact with these CDAP components:

  • ApplicationClient: interacting with applications

  • DatasetClient: interacting with datasets

  • DatasetModuleClient: interacting with dataset Modules

  • DatasetTypeClient: interacting with dataset Types

  • MetricsClient: interacting with metrics

  • MonitorClient: monitoring system services

  • PreferencesClient: interacting with preferences

  • ProgramClient: interacting with MapReduce Programs, user services, workflows, and workers

  • QueryClient: querying datasets

  • ServiceClient: interacting with user services

  • StreamClient: interacting with streams

The above list links to the examples below for each portion of the API.

Configuring your *Client

Every *Client constructor requires a ClientConfig instance which configures the hostname and port of the CDAP instance that you wish to interact with.

In a non-secure (default) CDAP instance, instantiate as follows:

1 2 3 4 // Interact with the CDAP instance located at example.com, port 11015 ClientConfig clientConfig = ClientConfig.builder() .setConnectionConfig(new ConnectionConfig("example.com", 11015, false)) .build();

In a secure CDAP instance, first pull in the cdap-authentication-client Maven dependency:

1 2 3 4 5 <dependency> <groupId>io.cdap.cdap</groupId> <artifactId>cdap-authentication-client</artifactId> <version>${cdap.client.version}</version> </dependency>

Then, instantiate as follows:

1 2 3 4 5 6 7 8 9 10 11 12 // Obtain AccessToken AuthenticationClient authenticationClient = new BasicAuthenticationClient(); authenticationClient.setConnectionInfo("example.com", 11015, sslEnabled); // Configure the AuthenticationClient as documented in // https://github.com/cdapio/cdap-clients/blob/develop/cdap-authentication-clients/java AccessToken accessToken = authenticationClient.getAccessToken(); // Interact with the secure CDAP instance located at example.com, port 11015, with the provided accessToken ClientConfig clientConfig = ClientConfig.builder() .setConnectionConfig(new ConnectionConfig("example.com", 11015, sslEnabled)) .setAccessToken(accessToken) .build();

ApplicationClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ClientConfig clientConfig; // Construct the client used to interact with CDAP ApplicationClient appClient = new ApplicationClient(clientConfig); // Fetch the list of applications List<ApplicationRecord> apps = appClient.list(NamespaceId.DEFAULT); // Deploy an application File appJarFile = new File("your-app.jar"); appClient.deploy(NamespaceId.DEFAULT, appJarFile); // Delete an application appClient.delete(NamespaceId.DEFAULT.app("Purchase")); // List programs belonging to an application appClient.listPrograms(NamespaceId.DEFAULT.app("Purchase"));

DatasetClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ClientConfig clientConfig; // Construct the client used to interact with CDAP DatasetClient datasetClient = new DatasetClient(clientConfig); // Fetch the list of datasets List<DatasetSpecificationSummary> datasets = datasetClient.list(NamespaceId.DEFAULT); // Create a dataset DatasetId datasetId = NamespaceId.DEFAULT.dataset("someDataset"); datasetClient.create(datasetId, "someDatasetType"); // Truncate a dataset datasetClient.truncate(datasetId); // Delete a dataset datasetClient.delete(datasetId);

DatasetModuleClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ClientConfig clientConfig; // Construct the client used to interact with CDAP DatasetModuleClient datasetModuleClient = new DatasetModuleClient(clientConfig); // Add a dataset module File moduleJarFile = createAppJarFile(SomeDatasetModule.class); DatasetModuleId datasetModuleId = NamespaceId.DEFAULT.datasetModule("someDatasetModule"); datasetModuleClient.add(datasetModuleId, SomeDatasetModule.class.getName(), moduleJarFile); // Fetch the dataset module information DatasetModuleMeta datasetModuleMeta = datasetModuleClient.get(datasetModuleId); // Delete all dataset modules datasetModuleClient.deleteAll(NamespaceId.DEFAULT);

DatasetTypeClient

1 2 3 4 5 6 7 8 9 10 ClientConfig clientConfig; // Construct the client used to interact with CDAP DatasetTypeClient datasetTypeClient = new DatasetTypeClient(clientConfig); // Fetch the dataset type information using the type name DatasetTypeMeta datasetTypeMeta = datasetTypeClient.get(NamespaceId.DEFAULT.datasetType("someDatasetType")); // Fetch the dataset type information using the classname datasetTypeMeta = datasetTypeClient.get(NamespaceId.DEFAULT.datasetType(SomeDataset.class.getName()));

MetricsClient

1 2 3 4 5 6 7 8 ClientConfig clientConfig; // Construct the client used to interact with CDAP MetricsClient metricsClient = new MetricsClient(clientConfig); // Fetch the total number of requests that have been processed by a service RuntimeMetrics metric = metricsClient.getServiceMetrics(NamespaceId.DEFAULT.app("SportResults").service("RetrieveCounts")); long processed = metric.getProcessed();

MonitorClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ClientConfig clientConfig; // Construct the client used to interact with CDAP MonitorClient monitorClient = new MonitorClient(clientConfig); // Fetch the list of system services List<SystemServiceMeta> services = monitorClient.listSystemServices(); // Fetch status of system transaction service String serviceStatus = monitorClient.getSystemServiceStatus("transaction"); // Fetch the number of instances of the system transaction service int systemServiceInstances = monitorClient.getSystemServiceInstances("transaction"); // Set the number of instances of the system transaction service monitorClient.setSystemServiceInstances("transaction", 1);

PreferencesClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 ClientConfig clientConfig; // Construct the client used to interact with CDAP PreferencesClient preferencesClient = new PreferencesClient(clientConfig); Map<String, String> propMap = Maps.newHashMap(); propMap.put("k1", "v1"); // Set preferences at the Instance level preferencesClient.setInstancePreferences(propMap); // Get preferences at the Instance level Map<String, String> currentPropMap = preferencesClient.getInstancePreferences(); // Delete preferences at the Instance level preferencesClient.deleteInstancePreferences(); // Set preferences of MyApp application which is deployed in the Dev namespace preferencesClient.setApplicationPreferences(new NamespaceId("Dev").app("MyApp"), propMap); // Get only the preferences of MyApp application which is deployed in the Dev namespace Map<String, String> appPrefs = preferencesClient.getApplicationPreferences(new NamespaceId("Dev").app("MyApp"), false); // Get the resolved preferences (collapsed with higher level(s) of preferences) Map<String, String> resolvedAppPrefs = preferencesClient.getApplicationPreferences(new NamespaceId("Dev").app("MyApp"), true);

ProgramClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ClientConfig clientConfig; // Construct the client used to interact with CDAP ProgramClient programClient = new ProgramClient(clientConfig); // Start a service in the WordCount example programClient.start(NamespaceId.DEFAULT.app("WordCount").service("RetrieveCounts")); // formatted in JSON programClient.getLiveInfo(NamespaceId.DEFAULT.app("SportResults").service("UploadService")); // Fetch program logs in the WordCount example programClient.getProgramLogs(NamespaceId.DEFAULT.app("WordCount").service("RetrieveCounts"), 0, Long.MAX_VALUE); // Scale a service in the SportResults example programClient.setServiceInstances(NamespaceId.DEFAULT.app("SportResults").service("UploadService"), 3); // Stop a service in the SportResults example programClient.stop(NamespaceId.DEFAULT.app("SportResults").service("UploadService"));

ServiceClient

1 2 3 4 5 6 7 ClientConfig clientConfig; // Construct the client used to interact with CDAP ServiceClient serviceClient = new ServiceClient(clientConfig); // Fetch service information using the service in the PurchaseApp example ServiceSpecification serviceSpec = serviceClient.get(NamespaceId.DEFAULT.app("PurchaseApp").service("CatalogLookup"));

StreamClient

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 ClientConfig clientConfig; // Construct the client used to interact with CDAP StreamClient streamClient = new StreamClient(clientConfig); // Fetch the stream list List streams = streamClient.list(NamespaceId.DEFAULT); // Create a stream, using the Purchase example StreamId streamId = NamespaceId.DEFAULT.stream("purchases"); streamClient.create(streamId); // Fetch a stream's properties StreamProperties config = streamClient.getConfig(streamId); // Send events to a stream streamClient.sendEvent(streamId, "Tom bought 5 apples for $10"); // Read all events from a stream (results in events) List<StreamEvent> events = Lists.newArrayList(); streamClient.getEvents(streamId, 0, Long.MAX_VALUE, Integer.MAX_VALUE, events); // Read first 5 events from a stream (results in events) events = Lists.newArrayList(); streamClient.getEvents(streamId, 0, Long.MAX_VALUE, 5, events); // Read 2nd and 3rd events from a stream, after first calling getEvents long startTime = events.get(1).getTimestamp(); long endTime = events.get(2).getTimestamp() + 1; events.clear(); streamClient.getEvents(streamId, startTime, endTime, Integer.MAX_VALUE, events); // Write asynchronously to a stream streamId = NamespaceId.DEFAULT.stream("testAsync"); events = Lists.newArrayList(); streamClient.create(streamId); // Send 10 async writes int msgCount = 10; for (int i = 0; i < msgCount; i++) { streamClient.asyncSendEvent(streamId, "Testing " + i); } // Read them back; need to read it multiple times as the writes happen asynchronously while (events.size() != msgCount) { events.clear(); streamClient.getEvents(streamId, 0, Long.MAX_VALUE, msgCount, events); } // Check that there are no more events events.clear(); while (events.isEmpty()) { events.clear(); streamClient.getEvents(streamId, 0, Long.MAX_VALUE, msgCount, events); }