Add weekend field via ingest pipeline

Elastic Stack Elasticsearch
1

Hello,

i need to add a field to check whether the date field is weekend or not.

I found below script for weekend identification.

doc[alrm_occr_ddt'].date.dayOfWeek == 6 || doc[alrm_occr_ddt'].date.dayOfWeek == 7"

so i tries to put ingest pipelines as below.

PUT _ingest/pipeline/weekend_true

{
  
  "processors" : [

    {
      
      "set": {
       
          "field": "weekend",
        
          "value": "true"
      
       }
    
    }   
  
  ]

}


PUT _ingest/pipeline/addWeek

{
  
  "description": "check weekend by using alrm_occr_ddt.date.dayOfWeek",
  
  "version": 1,
  
  "processors": [
    
  {
      
     "pipeline": {
        
         "if": "doc[alrm_occr_ddt'].date.dayOfWeek == 6 || doc[alrm_occr_ddt'].date.dayOfWeek == 7",
        
         "name": "weekend_false"
      
      }
    
  }
  
  ]

}

but i am experiencing below error when i tried to put a test index.

PUT test1/doc/2?pipeline=addWeek
{
  "alrm_occr_ddt" : "2019-04-15T02:03:14.000Z"
}


#ERROR
{
  "error": {
    "root_cause": [
      {
        "type": "exception",
        "reason": "java.lang.IllegalArgumentException: ScriptException[compile error]; nested: IllegalArgumentException[unexpected character ['].date.dayOfWeek == 2].]; nested: LexerNoViableAltException;",
        "header": {
          "processor_type": "conditional"
        }
      }
    ],
    "type": "exception",
    "reason": "java.lang.IllegalArgumentException: ScriptException[compile error]; nested: IllegalArgumentException[unexpected character ['].date.dayOfWeek == 2].]; nested: LexerNoViableAltException;",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "ScriptException[compile error]; nested: IllegalArgumentException[unexpected character ['].date.dayOfWeek == 2].]; nested: LexerNoViableAltException;",
      "caused_by": {
        "type": "script_exception",
        "reason": "compile error",
        "script_stack": [
          "doc[alrm_occr_ddt'].date.dayOfWeek == 2",
          "                 ^---- HERE"
        ],
        "script": "doc[alrm_occr_ddt'].date.dayOfWeek == 2",
        "lang": "painless",
        "caused_by": {
          "type": "illegal_argument_exception",
          "reason": "unexpected character ['].date.dayOfWeek == 2].",
          "caused_by": {
            "type": "lexer_no_viable_alt_exception",
            "reason": null
          }
        }
      }
    },
    "header": {
      "processor_type": "conditional"
    }
  },
  "status": 500
}

can anyone please advise how i can avoid the error and put correct weekend info?

thank you!

2

Hey Jason,

When asking a question on this forum, please format any code with the </> button, as explained here. It makes it much easier to read. :slight_smile:

Your script has several issues, the most important of which is that you're using the doc['field'] notation to access field values. That notation only works at query and aggregation time. At index time, you need to access fields with the ctx.field notation.

The following pipeline should work:

PUT _ingest/pipeline/addWeek
{
  "description": "check weekend by using alrm_occr_ddt.date.dayOfWeek",
  "version": 1,
  "processors": [
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.SATURDAY || dayOfWeek == DayOfWeek.SUNDAY)",
        "name": "weekend_true"
      }
    }
  ]
}
1 Like
3 
PUT _ingest/pipeline/adddayofweek
{
  "description": "check weekend by using alrm_occr_ddt",
  "version": 1,
  "processors": [
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.MONDAY)",
        "name": "dayofweek_monday"
      }
    },
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.TUESDAY)",
        "name": "dayofweek_tuesday"
      }
    },
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.WEDNESDAY)",
        "name": "dayofweek_wednesday"
      }
    },
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.THURSDAY)",
        "name": "dayofweek_thursday"
      }
    },
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.FRIDAY)",
        "name": "dayofweek_friday"
      }
    },
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.SATURDAY)",
        "name": "dayofweek_saturday"
      }
    },
    {
      "pipeline": {
        "if": "def dayOfWeek = OffsetDateTime.parse(ctx.alrm_occr_ddt).getDayOfWeek(); return (dayOfWeek == DayOfWeek.SUNDAY)",
        "name": "dayofweek_sunday"
      }
    }
  ]
}

i successfully add dayofweek field with above ingest pipeline.

thank you for your reply!!! :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.