Reindex throws error with batch_size as low as 5

Hi,

Reindexing if throwing following error with batch size as low as 5.

2019-09-24 05:43:00 MDT root         ERROR    Execution aborted
Traceback (most recent call last):
  File "reindex_indices.py", line 71, in main
    copy_indices(args.source_cluster, args.dest_cluster, args.pattern)
  File "reindex_indices.py", line 49, in copy_indices
    }, wait_for_completion=True, request_timeout=4800)
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/client/__init__.py", line 999, in reindex
    "POST", "/_reindex", params=params, body=body
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/transport.py", line 350, in perform_request
    timeout=timeout,
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
    self._raise_error(response.status, raw_data)
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/connection/base.py", line 181, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'Remote responded with a chunk that was too large. Use a smaller batch size.')

Same problem when i tried to set size = 2.

Here's the reindex definition:

result = es_dest.reindex({
            "source": {"remote": {"host": source_cluster}, "index": source_index, "size": 2},
            "dest": {"index": destination_index}
        }, wait_for_completion=True, request_timeout=4800)

index details:

number of docs: 81999
size of index: 5 GB

Reindexing contents on the same cluster. elasticsearch version 7.3. python elasticsearch package version 7.0.4

Thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.