Cursors should encapsulate all the information required to replay a query

Description

In current APIs, we allow any combination of query options and cursors as input the search request. However, a cursor is a manifestation of a past query and meant to resume with the next set of results for that same request. But it is allowed to use a cursor with a different query, or namespace, or target type.

Our implementations differ in how they treat such situations: 

  • In Elasticsearch, if the corresponding scroll is still live, we return the next set of results and ignore any other parameters. If the scroll has expired, we run a new search with the parameters from the request.

  • In noSQL (dataset) based storage, we combine the cursor with the other parameters, resulting in entirely unpredictable behavior. 

To fix this, the cursor itself should encapsulate all search parameters, and when a cursor is used, that alone determines the search parameters (and the rest of the request is ignored).

In addition, we could change the API to make it impossible to construct such a request, both in the SPI (SearchRequest.Builder) and the MetadataClient (search methods).  

Release Notes

None
Fixed

Assignee

Andreas Neumann

Reporter

Andreas Neumann

Labels

None

Docs Impact

None

UX Impact

None

Components

Fix versions

Affects versions

Priority

Major