(429) Too Many Requests during indexing pdf

Hello,

we have around 20000 products in 25 different languages. Each language has an own index (products_en, products_fr, ...)
when 1000 product data (code, name, description) per language is gathered, we send for every language a bulk request.
this works. but once we also index a pdf (200-400KB), then elasticsearch returns after some time "(429) Too Many Requests"
but why? the number of requests did not change, only the size of the bulk json and only one bulk request is send at a time

this is performed on a Windows Server 2012 (73GB memory, 8 core 2.60 GHz) which is for development

index config:

{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  }
}

ingest attachment config:

{
  "description": "Extract attachment information",
  "processors": [
    {
      "foreach": {
        "field": "attachments",
        "processor": {
          "attachment": {
            "field": "_ingest._value.data",
            "target_field": "_ingest._value.attachment",
            "properties": [ "content" ]
          }
        }
      }
    }, {
        "foreach": {
        "field": "attachments",
        "processor" : {
            "remove" : { "field" : "_ingest._value.data" }
          }
        }
     }
  ]
}

Regards

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.