Reindex throws error with batch_size as low as 5

Elastic Stack Elasticsearch
1

Hi,

Reindexing if throwing following error with batch size as low as 5.

2019-09-24 05:43:00 MDT root         ERROR    Execution aborted
Traceback (most recent call last):
  File "reindex_indices.py", line 71, in main
    copy_indices(args.source_cluster, args.dest_cluster, args.pattern)
  File "reindex_indices.py", line 49, in copy_indices
    }, wait_for_completion=True, request_timeout=4800)
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/client/__init__.py", line 999, in reindex
    "POST", "/_reindex", params=params, body=body
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/transport.py", line 350, in perform_request
    timeout=timeout,
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
    self._raise_error(response.status, raw_data)
  File "/root/py3/lib64/python3.6/site-packages/elasticsearch/connection/base.py", line 181, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'Remote responded with a chunk that was too large. Use a smaller batch size.')

Same problem when i tried to set size = 2.

Here's the reindex definition:

result = es_dest.reindex({
            "source": {"remote": {"host": source_cluster}, "index": source_index, "size": 2},
            "dest": {"index": destination_index}
        }, wait_for_completion=True, request_timeout=4800)

index details:

number of docs: 81999
size of index: 5 GB

Reindexing contents on the same cluster. elasticsearch version 7.3. python elasticsearch package version 7.0.4

Thanks

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.