3. Transaction Report Scrolling | Circularo Developers

Transaction Report Scrolling

This scenario demonstrates how to efficiently retrieve and process large volumes of transaction detail reports using the scrolling mechanism. When dealing with extensive document collections, scrolling provides a more efficient approach than standard pagination for processing complete transaction datasets for analysis, reporting, or archiving.

Key features:

Efficiently retrieve large volumes of transaction reports without pagination limitations
Maintain a consistent view of transaction data during the retrieval process
Process complete transaction datasets for compliance reporting and business analysis
Handle thousands or millions of document records with optimal performance

Prerequisites

Before scrolling transaction reports, you need:

A valid authentication token
The use_all_documents_export permission assigned to your user account
Understanding of the scroll mechanism and its lifecycle

Understanding Transaction Detail Reports

Transaction detail reports provide comprehensive information about document activities:

Document Metadata: Title, ID, type, creation date, and owner information
Workflow States: Current status of documents (completed, in progress, rejected, etc.)
Transaction History: Record of actions performed on documents

For more information about transaction reports and their contents, see the Circularo documentation - Transaction Detail Report

Understanding the Scroll Mechanism

The scroll API is designed for efficiently retrieving large datasets:

Scroll Context: A server-side cursor that maintains position in the result set
Scroll ID: A unique identifier for the scroll context that must be used in subsequent requests
Scroll Timeout: The duration in seconds the server keeps the scroll context alive between requests
Batch Size: The number of transaction records retrieved in each scroll request

Unlike pagination, scrolling provides a consistent view of the data even if documents are being added or modified during the scrolling process.

Step 1 - Initialize Scroll Context for Transaction Reports

This endpoint demonstrates how to initialize a scroll context for retrieving large volumes of transaction detail reports. The scroll context acts as a cursor that maintains your position in the result set, allowing you to efficiently process all matching transaction records.

Creates a server-side scroll context that preserves your position in the result set
Returns the first batch of transaction records based on your criteria
Provides a scroll ID that must be used in subsequent scroll requests
Maintains a consistent view of the data even if new documents are being added

The scroll timeout specifies how long (in seconds) the server should keep the scroll context alive between requests. This value refreshes with each request, so you typically only need a few seconds unless processing is very slow.

Endpoint

POST /search/reports

Request

JSON

POST /search/reports?token=2KCyEB2RTDxQ7voUpkU6GP56eKWnbfgYOS85S29HYi6dF2Aeej7zb7y5rB7UPnwL

Content-Type: application/json

{
  "query": {
    "size": 500,  //Number of transaction records to retrieve in each batch
    "scroll": {
      "timeout": 10  //How long to keep the scroll context alive (in seconds)
    }
  }
}

Response

JSON

{
  "from": 0,  //Starting position in the result set
  "total": 18992,  //Total number of matching transaction records
  "scrollId": "7elxMT8tykPNFMbcmQrdUFjVk4ndEqvSufeJL9HvIuc6w4d5aQo61MUNjdKOg25I",  //Unique identifier for the scroll context
  "results": [  //First batch of transaction records
    {
      "documentTitle": "Invoices",
      "documentId": "f8ed7a82-0a37-4de8-a3e5-2ea8c94c79c0",
      ...
    },
    ...
  ]
}

The response contains the following important properties:

scrollId - Located at scrollId in the response.
- Example value: 7elxMT8tykPNFMbcmQrdUFjVk4ndEqvSufeJL9HvIuc6w4d5aQo61MUNjdKOg25I

The response contains the first batch of transaction records and a scroll ID for retrieving subsequent batches.

The results array contains the first batch of transaction records
The total field indicates the total number of records matching your criteria
The scrollId is a unique identifier for your scroll context that must be used in subsequent requests
Process these records and then use the scroll ID to retrieve the next batch

The scroll ID may be a long, complex string. Always use it exactly as returned without modification.

Step 2 - Retrieve Next Batch of Transaction Records

This endpoint demonstrates how to retrieve subsequent batches of transaction records using the scroll ID from previous requests. This process continues until all matching records have been retrieved.

Uses the scroll ID from the previous response to maintain position in the result set
Returns the next batch of transaction records based on your criteria
Refreshes the scroll timeout to keep the context alive
Must be called repeatedly until all records have been retrieved

Do not specify the 'from' parameter in scroll requests as it's incompatible with scrolling. The scroll context already maintains your position in the result set.

Endpoint

POST /search/reports

Request

JSON

POST /search/reports?token=2KCyEB2RTDxQ7voUpkU6GP56eKWnbfgYOS85S29HYi6dF2Aeej7zb7y5rB7UPnwL

Content-Type: application/json

{
  "query": {
    "size": 500,  //Number of transaction records to retrieve in this batch
    "scroll": {
      "id": "7elxMT8tykPNFMbcmQrdUFjVk4ndEqvSufeJL9HvIuc6w4d5aQo61MUNjdKOg25I",  //Scroll ID from the previous response
      "timeout": 10  //Refresh the scroll timeout
    }
  }
}

Response

JSON

{
  "total": 18992,  //Total number of matching transaction records
  "scrollId": "7elxMT8tykPNFMbcmQrdUFjVk4ndEqvSufeJL9HvIuc6w4d5aQo61MUNjdKOg25I",  //Scroll ID for the next request
  "results": [  //Next batch of transaction records
    {
      "documentTitle": "Memo 2025",
      "documentId": "b8e130ce-2974-478c-a8f3-81726c4403b9",
      ...
    },
    ...
  ]
}

The response contains the following important properties:

scrollId - Located at scrollId in the response.
- Example value: 7elxMT8tykPNFMbcmQrdUFjVk4ndEqvSufeJL9HvIuc6w4d5aQo61MUNjdKOg25I

The response contains the next batch of transaction records and a scroll ID for retrieving subsequent batches.

The results array contains the next batch of transaction records
The total field remains the same as in the initial request
The scrollId may be the same or different from the previous request
Continue this process until the results array is empty or smaller than the requested size

Always use the scrollId from the most recent response, as it may change between requests.

Step 3 - Close Scroll Context

This endpoint demonstrates how to properly close a scroll context when you've finished retrieving all needed transaction records. Closing the context releases server resources and is an important best practice.

Explicitly releases server resources associated with the scroll context
Should be called when you've retrieved all needed records or no longer need the context
Requires the scroll ID from your most recent scroll request

Endpoint

DELETE /search/scroll

Request

JSON

DELETE /search/scroll?token=2KCyEB2RTDxQ7voUpkU6GP56eKWnbfgYOS85S29HYi6dF2Aeej7zb7y5rB7UPnwL

Content-Type: application/json

{
  "scrollId": "7elxMT8tykPNFMbcmQrdUFjVk4ndEqvSufeJL9HvIuc6w4d5aQo61MUNjdKOg25I"  //Scroll ID from the most recent response
}

Response

JSON

{
  "status": "SUCCESS"
}

The scroll context has been successfully closed and all associated server resources have been released.

After closing a scroll context, its scroll ID becomes invalid. If you need to retrieve more transaction records, you'll need to initialize a new scroll context.

Transaction Report Scrolling Summary

You have successfully learned how to efficiently retrieve and process large volumes of transaction detail reports using the scrolling mechanism.

Key Concepts

Scroll Context: A server-side cursor that maintains position in the result set
Scroll ID: A unique identifier for the scroll context used in subsequent requests
Scroll Lifecycle: Initialize, retrieve batches, close when finished
Batch Processing: Retrieve and process transaction records in manageable chunks
Resource Management: Properly close contexts to prevent server resource leaks

Use Scrolling: For processing complete transaction sets, data exports, or deep analysis
Use Pagination: For displaying transaction records in user interfaces or when only a subset of records is needed

Next Steps

With your understanding of transaction report scrolling, you can now:

Implement efficient report processing for compliance reporting
Create data export functionality for archiving or external analysis
Build comprehensive business intelligence tools that process complete transaction datasets
Combine with filtering capabilities to focus on specific transaction subsets while maintaining efficiency

Example Implementation

See our OpenAPI documentation to learn about the full set of API endpoints and parameters.

Please use proper exception handling and function decomposition in your own code. The code is provided for illustrative purposes only and is not intended for production use.

JavaScript

// Transaction report scrolling example
const URL = "https://sandbox.circularo.com";
const API_PATH = "/api/v1";
const TOKEN = "YOUR_TOKEN"; // Must have "use_all_documents_export" permission

// Helper function to process each batch of transaction records
function processTransactionBatch(transactions) {
    for (const transaction of transactions) {
        console.log(`Processing transaction for document: ${transaction.documentTitle} (${transaction.documentId})`);

        // Add your transaction processing logic here
        // For example:
        // - Extract data for reporting
        // - Update your database with transaction information
        // - Generate statistics or analytics
    }
}

try {
    // Step 1: Initialize scroll context
    const batchSize = 500;
    const scrollTimeout = 10; // seconds
    let scrollId;

    const initScrollResponse = await fetch(`${URL}${API_PATH}/search/reports?token=${TOKEN}`, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify({
            query: {
                size: batchSize,
                scroll: {
                    timeout: scrollTimeout
                }
            }
        })
    });
    if (!initScrollResponse.ok) {
        throw new Error(`Failed to initialize scroll: ${initScrollResponse.status} ${initScrollResponse.statusText}`);
    }

    let responseData = await initScrollResponse.json();
    console.log(`Found ${responseData.total} transaction records. Processing in batches...`);
    scrollId = responseData.scrollId;

    // Step 2: Process the batch and continue scrolling until all reports are processed
    while (responseData.results.length > 0) {
        processTransactionBatch(responseData.results);

        const scrollResponse = await fetch(`${URL}${API_PATH}/search/reports?token=${TOKEN}`, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json'
            },
            body: JSON.stringify({
                query: {
                    size: batchSize,
                    scroll: {
                        id: scrollId,
                        timeout: scrollTimeout
                    }
                }
            })
        });
        if (!scrollResponse.ok) {
            throw new Error(`Scroll request failed: ${scrollResponse.status} ${scrollResponse.statusText}`);
        }

        responseData = await scrollResponse.json();
        scrollId = responseData.scrollId;
    }

    console.log('Transaction report processing complete');

} catch (error) {
    console.error('Error processing transaction reports:', error.message);

} finally {
    // Step 3: Always try to close the scroll context if we have a scrollId
    if (scrollId) {
        try {
            await fetch(`${URL}${API_PATH}/search/scroll?token=${TOKEN}`, {
                method: 'DELETE',
                headers: { 'Content-Type': 'application/json' },
                body: JSON.stringify({
                    scrollId: scrollId
                })
            });
            console.log('Scroll context closed successfully');

        } catch (closeError) {
            console.error('Failed to close scroll context:', closeError.message);
        }
    }
}

Transaction Report Scrolling

Prerequisites

Understanding Transaction Detail Reports

Understanding the Scroll Mechanism

Step 1 - Initialize Scroll Context for Transaction Reports

Step 2 - Retrieve Next Batch of Transaction Records

Step 3 - Close Scroll Context

Transaction Report Scrolling Summary

Key Concepts

When to Use Scrolling vs. Pagination

Next Steps

Example Implementation