# Logstash http\_poller json parsing error

**URL:** <https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990>\
**Category:** Logstash\
**Created:** [August 22, 2020, 3:48pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990 "2020-08-22T15:48:27Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 22, 2020, 3:48pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/1 "2020-08-22T15:48:27Z")

</div>

```auto
Hello , 
I am using http_poller logstash input plugin to poll rest API ( JIRA ) .

When i poll it , these are the sample fields 
maxresults
startAt
tags 
issues 

Fields under issues 
    issues 
      {
       fields 
         { 
         emailadress
         key
         name }}
my filter 

filter {
json 
{
source => "issues"
}
mutate
{
add_field => { "username" => "%"{[fields][name]} }}
} }

But i am not able to parse the value . Could someone help

```

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 22, 2020, 4:31pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/2 "2020-08-22T16:31:50Z")

</div>

```auto
add_field => { "username" => [issues][fields][name]} 

```

Try that.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 22, 2020, 4:40pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/3 "2020-08-22T16:40:54Z")

</div>

If [issues] contains a string of JSON then that json filter should result in there being a field called [fields][name] which you would reference using

```
mutate { add_field => { "username" => "%{[fields][name]}" } }
```

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 22, 2020, 5:16pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/5 "2020-08-22T17:16:41Z")

</div>

```auto
Thanks , 
its not working . its not parsing instead its adding the field name = %{[fields][assignee][name]}
below is my json content 
"_index": "index_name",

  "_type": "_doc",

  "_id": "fCIYF3QBw3RYG9-rSeAc",

  "_version": 1,

  "_score": null,

  "_source": {

    "expand": "names,schema",

    "total": 1,

    "startAt": 0,

    "issues": [

      {

        "expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",

        "key": "test-1234",

        "id": "132468",

        "self": "http://dns/rest/api/2/issue/132468",

        "fields": {

          "issuetype": {

            "subtask": false,

            "name": "Task",

            "self": "http://dns/rest/api/2/issuetype/3",

            "id": "3",

          },

          "assignee": {

            "name": "fred",

         }

  ]

}}

 
Below are the filter tried 

“username" => "%{[fields][assignee][name]}" or "%{[issues][fields][assignee][name]}"

“key” => "%{[fields][key]}" or "%{[issues][fields][key]}"

Index fields available already

issues.fields.assignee.name

issues.fields.key

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 22, 2020, 5:19pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/6 "2020-08-22T17:19:09Z")

</div>

[issues] is an array, so it would be %{[issues][0][fields][assignee][name]}

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 22, 2020, 5:19pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/7 "2020-08-22T17:19:46Z")

</div>

```auto
Thanks , but its not working . Logstash is going down
```

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 22, 2020, 5:20pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/8 "2020-08-22T17:20:34Z")

</div>

```auto
thanks , let me try that 

```

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 22, 2020, 5:49pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/9 "2020-08-22T17:49:38Z")

</div>

```auto
it works , thanks a lot but have another question 
There is another array inside of the issues
"issues": [
{
"fields": {
"app": [
{
value: test
}
]
}

]

How do i access the "app" field which is a nested array ? Thanks in advance

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 22, 2020, 5:54pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/10 "2020-08-22T17:54:54Z")

</div>

That would be

```
[issues][0][fields][app][0][value]
```

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 22, 2020, 5:58pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/11 "2020-08-22T17:58:12Z")

</div>

```auto
Thanks , was trying the same :) 

```

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 24, 2020, 11:06am UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/12 "2020-08-24T11:06:43Z")

</div>

> [@Badger](#):
>
> `name`

```auto
it works , thanks . I have an another question though 

i am trying to remove the fields 

This is my filter 

f
 

filter {

 

        mutate {

        add_field => {

                               

                               

                                "status" => "%{[issues][0][fields][status][name]}"

                                }

 

        }

                                ruby {

           code => '

        event.to_hash.each { |k, v|

             if v == "" or v.to_s.start_with?("%{[issues]")

                event.remove(k)

            end

        }

but its not working , is this right ? i am trying to remove the field if the value is not updated 

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2020, 3:34pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/13 "2020-08-24T15:34:29Z")

</div>

I would have expected that to work.

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 24, 2020, 3:42pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/14 "2020-08-24T15:42:22Z")

</div>

```auto
Thanks for the response . Below is the error 
Error:

[2020-08-24T15:38:43,990][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 51, column 1 (byte 1556) after ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:44:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:356:in `block in converge_state'"]}

Filter:

filter {

 

        mutate {

        add_field => {

                               

                                "status" => "%{[issues][0][fields][status][name]}"

                                }

 

        }

        ruby {

           code => '

                                      event.to_hash.each { |k, v|

                                                  if v == "" or v.to_s.start_with?("%{[issues]")

                                                  event.remove(k)

                                                  end

                                                                                      }

                                                  '

                                                                      }

                                                  }

}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2020, 3:46pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/15 "2020-08-24T15:46:15Z")

</div>

You seem to have an extra } at the end of that configuration.

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 24, 2020, 3:49pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/16 "2020-08-24T15:49:09Z")

</div>

> [@jerin](#):
>
> ```auto
> filter {
> 
>  
> 
> mutate {
> 
> add_field => {
> 
>                                
> 
> "status" => "%{[issues][0][fields][status][name]}"
> 
> }
> 
>  
> 
> }
> 
> ruby {
> 
> code => '
> 
> event.to_hash.each { |k, v|
> 
> if v == "" or v.to_s.start_with?("%{[issues]")
> 
> event.remove(k)
> 
> end
> 
> }
> 
> '
> 
> }
> 
> }
> 
> }
> 
> ```

```auto
hi , that's to close the filter 

filter {

}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2020, 3:52pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/17 "2020-08-24T15:52:29Z")

</div>

> [@jerin](#):
>
> `hi , that's to close the filter `

No, it is not. If you format your filter section like this:

```
filter {
    mutate { add_field => { "status" => "%{[issues][0][fields][status][name]}" } }
    ruby {
        code => '
            event.to_hash.each { |k, v|
                if v == "" or v.to_s.start_with?("%{[issues]")
                     event.remove(k)
                end
            }
        '
    }
}
}

```

you will see there is an extra }

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 24, 2020, 4:06pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/18 "2020-08-24T16:06:46Z")

</div>

```auto
is this required to have if v == "" or v.to_s.start_with?("% **{** [issues]") **braces** before issues . because i checked again , there is no extra braces

There are 3 open braces in ruby code and 1 to close filter ... total 7 open braces and 7 closed ones.. sorry if i am making wrong statement here . 

```

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 24, 2020, 4:22pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/19 "2020-08-24T16:22:10Z")

</div>

```auto
filter {
    mutate { add_field => { "status" => "%{[issues][0][fields][status][name]}" } }
    ruby {
        code => '
            event.to_hash.each { |k, v|
                if v == "" or v.to_s.start_with?("%{[issues]")
                     event.remove(k)
                end
            }
        '
    }
}
}
Is this right ? still its not working

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2020, 4:24pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/20 "2020-08-24T16:24:28Z")

</div>

No, it is not right, that's the point. Delete the final }

---

<div class="post-metadata">

**Author:** ![jerin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerin/32/74357_2.png) [@jerin](https://discuss.elastic.co/u/jerin)\
**Post date:** [August 24, 2020, 4:27pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990/21 "2020-08-24T16:27:26Z")

</div>

> [@jerin](#):
>
> `}`

```auto
ah .. silly of me ..thanks ..

```

[Next page](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990.md?page=2)