Installing the CDAP Sandbox Docker Image

Docker is one of the easiest ways to start working with CDAP without having to manually configure anything. A Docker image with the CDAP Sandbox pre-installed is available on the Docker Hub for download.

To use the Docker image, you can either start the container from a command line or use Docker's Kitematic (on Mac OS X and Windows), a graphical user interface for running Docker containers.

Docker from a Command Line

Docker environments are available for a variety of platforms. Download and install Docker for your platform by following the platform-specific installation instructions from Docker.com, and verify that the Docker environment is working and has started correctly.

1. If you are not running on Linux, you will need to create and start a Docker Virtual Machine (VM) before you can use containers. For example:
   
   

This will create a new Docker virtual machine using VirtualBox named cdap; once created and running, the second command will print out the environment.

2. When you run docker-machine env cdap, it will print a message on the screen such as:

It is essential to run these export commands (or the single eval command, eval $(docker-machine env cdap)). Otherwise, subsequent Docker commands will fail because they won't be able to connect to the correct Docker VM.

3. If you are running Docker on either Mac OS X or Microsoft Windows, Docker is running a virtual Linux machine on top of your host OS. You will need to use the address shown above (such as 192.168.99.100) as the host name when either connecting to the CDAP UI or making an HTTP request.

4. If you are running Docker on Microsoft Windows, note that paths used employ Linux's forward-slashes (/) and not back-slashes (\).

5. Once Docker has started, pull down the CDAP Docker Image from the Docker Hub using:

6. Start the CDAP Sandbox Docker container with:

This will start the container (in the background), name it cdap-sandbox, and set the proxying of ports.

7. Start the CDAP Sandbox Docker container with:

This will start the container (in the foreground, the default), enable debugging, name it cdap-sandbox-debugging, and set the proxying of ports.

8. CDAP will start automatically once the container starts. CDAP’s software directory is under /opt/cdap/sandbox.

9. Once CDAP starts, it will instruct you to connect to the CDAP UI with a web browser at http://localhost:11011/.

10. If you are running Docker on either Mac OS X or Microsoft Windows, replace localhost with the Docker VM's IP address (such as 192.168.99.100) that you obtained earlier. Start a browser and enter the address to access the CDAP UI from outside Docker.

Options Starting CDAP Containers

• Starting the CDAP Sandbox, in the background (default execution)

• Use the CDAP CLI within the above cdap-sandbox container:

• Use the CDAP CLI in its own container (cdap-cli), against a remote CDAP instance at ${CDAP_HOST}:

• Starting the CDAP Sandbox, in the foreground, with ports forwarded:

• Starting the CDAP Sandbox, in the foreground, with ports forwarded, and with debugging enabled:

For information on mounting volumes and sharing data with the container, see the examples in Docker's documentation on data volumes. You can either let Docker manage the storage in the container (the easiest) or you can mount a host directory as a data volume. However, if you mount a host directory, you must make sure that it exists and that permissions are set correctly. You pass such a directory using -v /my/own/datadir:/opt/cdap/sandbox/my/own/datadir, which mounts the /my/own/datadir from the host system as /opt/cdap/sandbox/my/own/datadir in the container.

Controlling the CDAP Instance

• To control the CDAP instance, use this command, substituting one of start, restart, stop, or status for <command>:

• When you are finished, stop CDAP and then shutdown the Docker machine:

Docker Resources

For a full list of Docker Commands, see the Docker Command Line Documentation.

Docker using Kitematic

Docker Kitematic is available as part of the Docker for either Mac OS X or Microsoft Windows. It is a graphical user interface for running Docker containers. Follow these steps to install Kitematic and then download, start, and connect to a CDAP container.

1. Download and install Docker for either Mac OS X or Microsoft Windows.

2. Download and install Kitematic for either Mac OS X or Microsoft Windows. The easiest method is to select Open Kitematic from the Docker menu, and follow the instructions for downloading and installing it:
   
   

   Open

   

    

3. Start Kitematic. On Mac OS X, it will be installed in /Applications/Docker/Kitematic; on Windows, in Start Menu > Docker > Kitematic.

4. Once Kitematic has started, search for the CDAP image by using the search box at the top of the window and entering caskdata:cdap-sandbox. Once you have found the page, click on the repository menu, circled in red here:
   
   

   Open

   

5. Click the tags button:
   
   

6. Select the desired version. Note that the tag latest is the last version that was put up at Docker Hub, which is not the necessarily the desired version, which is 6.3.0 (3.5.0 shown as an illustration):
   
   

7. Close the menu by pressing the X in the circle. Click "Create" to download and start the CDAP image. When it has started up, you will see in the logs a message that the CDAP UI is listening on port 11011:
   
   

8. To connect a web browser for the CDAP UI, you'll need to find the external IP addresses and ports that the Docker host is exposing. These are listed on the right-hand side of the previous illustration:
   
   

9. This shows that the CDAP container is listening on the internal port 11011 within the Docker host, while the Docker host proxies that port on the virtual machine IP address and port (localhost:32773). Enter that address and port into your system web browser to connect to the CDAP UI, such as http://localhost:32773:
   
   

Docker and CDAP Applications

• To begin building CDAP applications, have our recommended software and tools installed in your environment.

Building and Running CDAP Applications

See Building and Running CDAP Applications for information on accessing the CDAP CLI and CDAP Local Sandbox bin utilities, building examples, starting CDAP, and deploying, starting, and stopping applications.

CDAP Sandbox Metadata

All CDAP metadata is stored in the cdap/data directory. As a best practice, consider backing up the entire data directory on a regular basis.