Scroll large amount of existing documents

Step 1 - Start scrolling the documents

By specifying the property scroll.timeout you initialize searching with a scrolling enabled. The property is a number, that indicates for how many seconds the scroll context should be opened. This value refreshes with each consecutive search scroll request, so there is usually no need to specify more than a few seconds.

Scrolling enables you to traverse big amount of documents (possibly all of them). You can control the amount of returned documents by specifying the size property. Usage of the from property is technically possible (but discouraged) only in this starting request, all consecutive scroll requests must not specify it.

Endpoint

CODE

POST /search

Request

JSON

POST /search?token=Zkog0h42lalQzMijutD8LoVZoGz6aN7p3WBNtQdaxvpU3IUqWyEj6C1xyWIRw2u1

Content-Type: application/json

{
  "sort": {
    "createDate": {
      "order": "asc"
    }
  },
  "size": 100,
  "scroll": {
    "timeout": 10
  }
}

Response

Note following properties in response object:

scrollId - In response object at position scrollId.
- It may have value of ien3mh5vs7hO3MZRiptFqFeaKMMzPYeTBpnW4Txna7lCrMaqxbwUoJ11fPvmEKVw.

The response contains single "scroll" of the documents. To continue with the scrolling, please use the same request and specify the returned scrollId property.

Step 2 - Continue scrolling the documents

To continue scrolling, please always use the same query as in the first scroll request with scroll.id value specified. It is also crucial to not specify the from parameter, but you can modify the size to set the amount of returned documents.

Endpoint

CODE

POST /search

Request

JSON

POST /search?token=Zkog0h42lalQzMijutD8LoVZoGz6aN7p3WBNtQdaxvpU3IUqWyEj6C1xyWIRw2u1

Content-Type: application/json

{
  "sort": {
    "createDate": {
      "order": "asc"
    }
  },
  "size": 100,
  "scroll": {
    "id": "ien3mh5vs7hO3MZRiptFqFeaKMMzPYeTBpnW4Txna7lCrMaqxbwUoJ11fPvmEKVw",
    "timeout": 10
  }
}

Response

Note following properties in response object:

scrollId - In response object at position scrollId.
- It may have value of ien3mh5vs7hO3MZRiptFqFeaKMMzPYeTBpnW4Txna7lCrMaqxbwUoJ11fPvmEKVw.

You can repeat this request as long as it will yield you the results, or as will be needed. Please, always use the scrollId property from the latest response (it will be usually, but not always, the same through all the requests).

Step 3 - Close scroll context

Once all the documents were traversed, or once you no longer need to scroll any more of them, it is advisable to close the scroll context. To do so, please specify the latest scrollId value in the following request.

Endpoint

CODE

DELETE /search/scroll

Request

JSON

DELETE /search/scroll?token=Zkog0h42lalQzMijutD8LoVZoGz6aN7p3WBNtQdaxvpU3IUqWyEj6C1xyWIRw2u1

Content-Type: application/json

{
  "scrollId": "ien3mh5vs7hO3MZRiptFqFeaKMMzPYeTBpnW4Txna7lCrMaqxbwUoJ11fPvmEKVw"
}

The scroll context is now closed.