# Script params via Logstash, is it possible?

**URL:** <https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586>\
**Category:** Logstash\
**Created:** [April 5, 2020, 6:29pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586 "2020-04-05T18:29:38Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![crickes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickes/32/18009_2.png) [@crickes](https://discuss.elastic.co/u/crickes)\
**Post date:** [April 5, 2020, 6:29pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/1 "2020-04-05T18:29:39Z")

</div>

Is it possible to send 'params' to a stored script in Elasticsearch from Logstash, or do I have to use a script defined inline or as a file in Logstash in order to define a 'params' block for the script?

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [April 5, 2020, 11:32pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/2 "2020-04-05T23:32:55Z")

</div>

Hello @crickes,

The [Logstash Elasticsearch output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-script_type) offers an option named `script_type`:

- `inline` (default), meaning the `script` option must contain the script to be executed
- `indexed`, meaning the script has already been stored in Elasticsearch and you must specify its ID / name on the `script` option
- `file`, the name of the script stored in Elasticsearch configuration directory

Logstash will send the content of the event as `event` attribute inside the `params`.  
This means the stored script can access the parameters using `params.event.get('<name of the field>')`.

Those parameters work only when the `action` option is set to `update`!  
Example:

```auto
elasticsearch {
  hosts => [...]
  index => ...
  action => "update"
  script_type => "indexed"
  script => "script-id"
...
}

```

---

<div class="post-metadata">

**Author:** ![crickes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickes/32/18009_2.png) [@crickes](https://discuss.elastic.co/u/crickes)\
**Post date:** [April 6, 2020, 8:28am UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/3 "2020-04-06T08:28:04Z")

</div>

The golden nugget of information here is:

> Logstash will send the content of the event as `event` attribute inside the `params` .

I spent quite some time trying to find this in the documentation, and if it is in there, I couldn't find it.

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [April 6, 2020, 9:12am UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/4 "2020-04-06T09:12:44Z")

</div>

I've submitted [https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/931](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/931) to clarify this point in the documentation.

---

<div class="post-metadata">

**Author:** ![parosio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parosio/32/27367_2.png) [@parosio](https://discuss.elastic.co/u/parosio)\
**Post date:** [April 15, 2020, 4:07pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/5 "2020-04-15T16:07:12Z")

</div>

Hello, a clarification could really help...

I've a similar situation, and couldn't understand how to pass the event/field from within logstash, to achive an effect like this:

```auto
  ## Upsert with update of array field
  POST test-logs-ces/doc/1242272-10.127.0.1/_update
  {
  "script" : {
          "source": """
          if (ctx._source.files != null) {
             if (! ctx._source.files.contains( params.filename )) {
               ctx._source.files.add( params.filename )
            }
          } else 
            ctx._source.files = [params.filename]""",
          "lang": "painless",
          "params" : {
              "filename" : "extract1.xlsx"
          }
      },
    "upsert" : {
      "MID" : "1242272",
      "field_1" : "Some value"
    }
  }

```

so, i tried to unroll the param within the script, but hit the `[script] Too many dynamic script compilations within ... please use indexed, or scripts with parameters instead`

So, if my logstash event has a field `filename` I suppose I could use something like:

```auto
output {
 if [filename] { 
  elasticsearch {   
    document_id => "%{doc_id}"
    action => "update"
    doc_as_upsert => true
    script_lang => "painless"
    script => " fn=params.event.get('filename'); if (ctx._source.filenames != null) {ctx._source.filenames.add( fn )} else {ctx._source.filenames = [fn]} "
    script_type => "inline"
     . . . 
  }
 }
...

```

to have the new filename appended to `filenames` array.

Do I have advantages with `indexed` or `file`scripts over an `inline` one?

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [April 16, 2020, 11:28pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/6 "2020-04-16T23:28:52Z")

</div>

> [@parosio](#):
>
> Do I have advantages with `indexed` or `file` scripts over an `inline` one?

I would avoid the `file`.

If you use `inline`, the script is being sent together with the request each time.  
It will be cached after the first compilation, so there are no major performance improvements.  
The script will be sent each time.

If you use `indexed`, the script must be setup before starting Logstash and there will be less data to be transferred over network.

@Alex_Marquardt has a good tutorial in his blog: [Using Logstash and Elasticsearch scripted upserts to transform eCommerce purchasing data](https://alexmarquardt.com/2019/12/17/logstash-and-elasticsearch-painless-scripted-upserts-transform-data/)

If you enable `doc_as_upsert` you might miss the first `filename`.

I think the correct `elasticsearch` output should be:

```auto
 elasticsearch {
    index => "ecommerce_ls_transformed"
    document_id => "%{doc_id}"
    action => "update"
    scripted_upsert => true
    script_lang => "painless"
    script => "def fn = params.event.get('filename'); if (ctx._source.filenames != null) {ctx._source.filenames.add(fn)} else {ctx._source.filenames = [fn]} "
  }

```

Regarding the following error:

> [@parosio](#):
>
> [script] Too many dynamic script compilations within ... please use indexed, or scripts with parameters instead

If the `painless` script is valid, I wouldn't expect it to be recompiled several times (as it is cached at the first execution, if there is no dynamic content in it - meaning you access `params.event` and there is no templated text in the script).  
Are you sure this didn't occur just because you did some attempts adjusting the script and the compilation failed too many times in a short time frame?

---

<div class="post-metadata">

**Author:** ![parosio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parosio/32/27367_2.png) [@parosio](https://discuss.elastic.co/u/parosio)\
**Post date:** [April 17, 2020, 7:32pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/7 "2020-04-17T19:32:51Z")

</div>

About the errors...

> Are you sure this didn't occur just because you did some attempts adjusting the script and the compilation failed too many times in a short time frame?

They occurred before I finally found your example of how to use event values as parameters (and had the values "cabled" inside the script ☹ )

Hence my initial comment about a clarification in the documentation.  
Without a working example it isn't really simple to come up with the solution; there are a few other conversation about this topic, here and on stackoverflow, but none with the (simple) answer.

About the `scripted_upsert`... I decided it would have been too difficult to put into the script all the other fields of my document.

---

<div class="post-metadata">

**Author:** ![parosio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parosio/32/27367_2.png) [@parosio](https://discuss.elastic.co/u/parosio)\
**Post date:** [April 17, 2020, 7:48pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/8 "2020-04-17T19:48:52Z")

</div>

And, about documentation again:

> [@Luca\_Belluccini](#):
>
> If you use `indexed` , the script must be setup before starting Logstash and there will be less data to be transferred over network.

an example of an indexed script would be really useful an example within logstash documentation, (currently only:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/e/0e796cdbecbe1634389129c7e872b245bacc8d90.png)  
with a reference to elastic one

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 15, 2020, 7:48pm UTC](https://discuss.elastic.co/t/script-params-via-logstash-is-it-possible/226586/9 "2020-05-15T19:48:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.