Setting id field in client request and also using date name processor

Elastic Stack Elasticsearch
1

Hi all,

I had a question about sending id field in the client request and also using the date name processor to create indices based on a field value.

So here is my use case , I want to produce daily indices based on a particular field value. I am sending my data using python script from Lambda function

my PUT call looks something like this

PUT /my_index/doc/_id

(here again I am passing id field in the request to eliminate duplicates)

in the dev console, I created an ingest pipeline

PUT _ingest/pipeline/daily
{
  "description": "daily date index naming",
  "processors" : [
    {
      "date_index_name" : {
        "field" : "datetimefield",
        "index_name_prefix" : "my_index-",
        "date_rounding" : "d",
        "date_formats": ["yyyy-MM-dd"]
      }
    }
  ]
}

and

set the default pipeline for

PUT my_index
{
  "settings": {
    "default_pipeline": "daily"
  }
}

when I test it out, its not sending any data to the cluster nor created any new index. Not sure what I am doing wrong here. Any insight would be helpful.

Thank you!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.