/
Test Scenarios

Test Scenarios

Oct 05, 2016
About

This document contains the test cases that will be used for automating using CDAP integration test suite. The first pass of test will be manual and these test cases will be automated in 3.6 timeframe.

Test Scenarios for application versioning

ID

Scenario

Steps

Expected Result

Actual Result

Status

ID

Scenario

Steps

Expected Result

Actual Result

Status

1

Snapshot versioning deployment and running services - failure

  1. Deploy Purchase app with Snapshot version

  2. Start Purchase HIstory Service

  3. Deploy updated version of the app, still with snapshot version

  4. Attempt to start the Purchase History Service

The start from step 4 should fail with 409 status code

 

PASSED (09/21)

PASSED (10/03)

2

Snapshot versioning deployment and running services - success

  1. Deploy Purchase app with Snapshot version

  2. Start Purchase HIstory Service

  3. Deploy updated version of the app, still with snapshot version

  4. Stop the PurchaseHistory Service

  5. Attempt to start the Purchase History Service

The start from Step 5 should succeed. The new code should be in effect

 

PASSED (09/21)

PASSED (10/03)

3

Deleting application with Snapshot versioning - failure

  1. Deploy Purchase app with Snapshot version

  2. Start Purchase HIstory Service

  3. Delete application

The delete from Step 3 should fail

 

PASSED (09/21)

PASSED (10/03)

4

Deleting application with Snapshot versioning - success

  1. Deploy Purchase app with Snapshot version

  2. Start Purchase HIstory Service

  3. Stop Purchase History Service

  4. Delete application

The delete from Step 4 should succeed and PurchaseHistory app should not appear in the listing end point

 

FAILED (09/21)

There is another app with artifact version is created.

PASSED (10/03)

5

Concurrent non-snapshot services - success

  1. Deploy Purchase app with non-Snapshot version

  2. Start Purchase HIstory Service

  3. Deploy updated version of the app, still with non-snapshot version

  4. Attempt to start the Purchase History Service

  5. Route traffic to both services

Both services should start, traffic should be routeable to both the instance of the running services

 

PASSED (09/21)

PASSED (10/03)

 

6

Setting routing configuration - failure

  1. Deploy Purchase app with non-snapshot version

  2. Configure 80% of traffic for Purchase History service

The configuration call in step 2 should fail, because it is not 100% traffic

 

PASSED (09/21)

Faced an issue without deploying a versioned app.

 

Other issues while testing

 

PASSED (10/03)

 

7

Setting routing configuration - failure

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Configure traffic to Purchase History Service v1-60%, v2 - 30%

The configuration call in step 3 should fail, because it is not 100% traffic

 

PASSED (09/21)

Faced in a issue without deploying versioned app

Other issues while testing

PASSED (10/03)

8

Setting routing configuration - success

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Configure traffic to Purchase History Service v1-60%, v2 - 40%

The configuration in step 3 should succeed

 

PASSED (09/21)

PASSED (10/03)

 

 

9

Setting routing traffic - success

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Configure traffic to Purchase History Service v1-60%, v2 - 40%

  4. Send 100 requests to the non versioned endpoint for service

The traffic to v1 should be around 60%

The traffic to v2 should be around 40%

 

PASSED (09/21)

Percentages were

Run 1: 63 - 37

Run 2: 62 - 38

Run 3: 66 - 34

PASSED (10/03) (Config was 70%, 30%) 

Run 1: 72 - 28 
Run 2: 76 - 24 

10

Deleting app - versioned

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Delete app with version v1

  4. Start services with version v2

Should be able to start the services and route traffic

(What should be the expected behavior of routing traffic after one version is deleted?)

Sree: Delete should fail if there is a route for the version to be deleted.

 

PASSED (09/21)

PASSED (10/03)

11

Deleting app

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Use the non-versioned end-point to delete the app

Should delete all versions of the app

 

FAILED (09/21)

Non versioned delete endpoint doesn't exist

PASSED (10/03)

12

Namespace delete - failure

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Start services with v1 app

  4. Start services with v2 app

  5. Delete namespace in which the app is deployed

The namespace delete should fail

 

PASSED (09/21)

PASSED (10/03)

13

Namespace delete - success

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Start services with v1 app

  4. Start services with v2 app

  5. Stop services with v1

  6. Stop services with v2

  7. Delete namespace in which the app is deployed

The namespace delete should succeed and on recreating the namespace the application should not exist

 

FAILED (09/21)

PASSED (10/03)

14

Deterministic routing - success

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Configure traffic to Purchase History Service v1-60%, v2 - 40%

  4. Send requests to v1 service

  5. Send requests to v2 service

v1 requests should go to v1 service, and v2 requests should go to v2 service

 

Cannot verify. The endpoints for deterministic routing is not present

As a workaround:

  1. Set the routing percetage to 100, 0 to old vs new and send requests to non-versioned endpoint Ex:

    v3/namespaces/default/apps/WordCount/services/RetrieveCounts/methods/greeting

  2. And flip the percentage to 0, 100 to redirect all traffic to new endpoint

PASSED (10/03)

15

No route configuration

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Send request without setting route config

Requests should got to v1 or v2 based on default behavior configured. (Note: this test will fail now since this is not yet implemented

)

 

Not implemented

PASSED (10/03)

16

Invalid route configuration

  1. Deploy Purchase app with non-snapshot version - v1

  2. Deploy Purchase app with non-snapshot version - v2

  3. Set route config for service with versions (v1, v2, v3) where v3 is non-existing version

The request should fail with Bad Request

 

FAILED

PASSED (10/03)

Created in 2020 by Google Inc.