# Logstash not creating index - json parse failure

**URL:** <https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860>\
**Category:** Logstash\
**Created:** [April 9, 2016, 3:04am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860 "2016-04-09T03:04:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bluethundr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bluethundr/32/409_2.png) [@bluethundr](https://discuss.elastic.co/u/bluethundr)\
**Post date:** [April 9, 2016, 3:04am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860/1 "2016-04-09T03:04:35Z")

</div>

Hello,

Suddenly logstash is not creating an index in ES. And I'm seeing errors like this in the debug output of logstash:

```
JSON parse failure. Falling back to plain-text {:error=>#<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values
 at [Source: [B@267c9565; line: 1, column: 6]>, :data=>"2016-04-08 18:40:27 [I] processing report for logs.jokefire.com", :level=>:error, :file=>"logstash/codecs/json.rb", :l
ine=>"67", :method=>"legacy_parse"}

```

And I'm also seeing these errors as well:

```
Beats input: the pipeline is blocked, temporary refusing new connection. {:reconnect_backoff_sleep=>0.5, :level=>:warn, :file=>"logstash/inputs/beats.rb", :line=>"164", :method=>"run"}
Beats input: the pipeline is blocked, temporary refusing new connection. {:reconnect_backoff_sleep=>0.5, :level=>:warn, :file=>"logstash/inputs/beats.rb", :line=>"164", :method=>"run"}

```

The configs had been in place both for logstash and beats, and everything was working fine. But now I just went to restart logstash after fixing a disk space issue. And those errors started popping up and now it won't create it's index.

Here's my configs:

```
cat 10-logstash-input.conf
input {

     beats {
     port => 5000
     tags => "beats"
     codec => "json"
     ssl => true
     ssl_certificate => "/etc/pki/tls/certs/logstash.crt"
     ssl_key => "/etc/pki/tls/private/logstash.key"
     congestion_threshold => 10
     }

     syslog {
        type => "syslog"
        port => "5514"

    }

     #redis {
     #host => "216.120.248.98"
     #type => "redis-input"
     #data_type => "list"
     #key => "logstash"
    #}
}

```

root@logs:/etc/logstash/conf.d] #cat 10-logstash-input.conf  
input {

```
 beats {
 port => 5000
 tags => "beats"
 codec => "json"
 ssl => true
 ssl_certificate => "/etc/pki/tls/certs/logstash.crt"
 ssl_key => "/etc/pki/tls/private/logstash.key"
 congestion_threshold => 10
 }

 syslog {
    type => "syslog"
    port => "5514"

}

 #redis {
 #host => "216.120.248.98"
 #type => "redis-input"
 #data_type => "list"
 #key => "logstash"
#}

```

}

```
#cat 30-logstash-output.conf
output {
  elasticsearch {
    hosts => ["xx.xx.xx.xx", "xx.xx.xx.xx", "xx.xx.xx.xx"]
    index => "logstash-%{+YYYY.MM.dd}"
    user => "ls_user"
    password => "secret"
  }

   # Alerting
   if [log_message] == "POSSIBLE BREAK-IN ATTEMPT!" {
       email {
          body => "Triggered in: %{message}"
          subject => "This is a Logstash alert for POSSIBLE BREAK-IN ATTEMPTS!"
          from => "logstash.alert@jokefire.com"
          to => "admin@example.com"
          via => "sendmail"
       }
    }

}

```

Filter file appears in the reply since it's too long for this post,

Any ideas on why this isn't working?

Thanks

---

<div class="post-metadata">

**Author:** ![bluethundr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bluethundr/32/409_2.png) [@bluethundr](https://discuss.elastic.co/u/bluethundr)\
**Post date:** [April 9, 2016, 3:05am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860/2 "2016-04-09T03:05:12Z")

</div>

> #cat 20-filters.conf  
> filter {  
> if "\_jsonparsefailure" in [tags] {  
> mutate{  
> remove\_tag =\> ["\_jsonparsefailure"]  
> }

> ```
> grok {
> match => { 'message' => '%{SYSLOGTIMESTAMP:event_timestamp} %{HOSTNAME:jf_host} %{SYSLOGPROG}: %{GREEDYDATA:log_message}'}
> }
> }
> 
> ```

> ```
> if "_jsonparsefailure" in [tags] {
> mutate{
> remove_tag => ["_jsonparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '%{SYSLOGTIMESTAMP:event_timestamp} %{HOSTNAME:jf_host} %{SYSLOGPROG}: %{GREEDYDATA:log_message}'}
> }
> 
> ```

> ```
> }
> 
> ```

> ```
> if "_grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '%{IP:ipaddress} - - \[%{HTTPDATE:http_date}] %{QS:http_message}%{GREEDYDATA:log_message}'}
> add_tag => ["http_query"]
> }
> 
> ```

> ```
> }
> 
> ```

> ```
> if "_grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '\[%{CISCO_REASON:date}%{NOTSPACE} %{YEAR:year}] %{NOTSPACE:loglevel} %{SYSLOG5424SD} %{NOTSPACE:apache_module} %{SYSLOG5424SD:client_ip_and_port} %{WORD:apache_code}: %{GREEDYDATA:message}'}
> add_tag => ["apache_log"]
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![bluethundr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bluethundr/32/409_2.png) [@bluethundr](https://discuss.elastic.co/u/bluethundr)\
**Post date:** [April 9, 2016, 3:05am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860/3 "2016-04-09T03:05:22Z")

</div>

> ```
> }
> 
> ```

> ```
> if "_grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => 'File %{QS:data_dog_file}, line %{BASE10NUM:line_number}, in %{WORD:option}'}
> add_tag => ["dada_dog_file", "datadog"]
> }
> }
> 
> ```

> ```
> if "_grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '%{NOTSPACE} %{SYSLOG5424SD} %{CISCO_REASON:log_message}'}
> add_tag => ["io_error"]
> }
> }
> 
> ```

> ```
> if "_grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '%{WORD} \(%{JAVAFILE}\):'}
> add_tag => "Traceback"
> }
> }
> 
> ```

> ```
> if "_grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '%{SYSLOGTIMESTAMP:event_timestamp} %{HOSTNAME:jf_host} %{SYSLOGPROG}: %{WORD:request}: invalid user %{USERNAME:user} %{NOTSPACE:auth_method}'}
> add_tag => ["invalid_login"]
> }
> }
> 
> ```

> ```
> if "_grokparsefailure_sysloginput" in [tags] or " _grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure", "_grokparsefailure_sysloginput"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '# semanage fcontext -a -t FILE_TYPE %{QS:log_file}'}
> }
> }
> 
> ```

> ```
> if "_grokparsefailure_sysloginput, _grokparsefailure" in [tags] {
> mutate{
> remove_tag => ["_grokparsefailure_sysloginput", "_grokparsefailure"]
> }
> 
> ```

> ```
> grok {
> match => { 'message' => '# semodule -i %{PROG:module}'}
> }
> }
> 
> ```

> ```
> ## Add GeoIP to apache logs
> 
> ```

> ```
> if [type] == "apache_ref_access" or [type] == "apache_dev_access" or [type] == "apache_beta_access" {
> 
> ```

> ```
> geoip {
> source => "clientip"
> target => "geoip"
> database => "/etc/logstash/data.d/GeoLiteCity.dat"
> add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ]
> add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ]
> }
> mutate {
> convert => ["[geoip][coordinates]", "float"]
> }
> 
> ```

> ```
> }
> 
> ```

> ```
> ## Tagging section begins here
> 
> ```

> ```
> if [program] == "dd.forwarder" or [program] == "dd.dogstatsd" {
> mutate{
> add_tag => "datadog"
> }
> } else if [program] == "sshd" {
> mutate{
> add_tag => "ssh"
> }
> } else if [program] == "puppet-master" or [program] == "puppet" or [program] == "puppet-agent" {
> mutate{
> add_tag => "puppet"
> }
> } else if [_type] == "security" {
> mutate{
> add_tag => "security"
> }
> } else if [program] == "NetworkManager" {
> mutate{
> add_tag => "NetworkManager"
> }
> } else if [log_message] == "password check failed for user (root)" or [log_message] == "Failed password for root" {
> mutate{
> add_tag => "password failed for root"
> }
> } else if [program] == "systemd" {
> mutate{
> add_tag => "systemd"
> }
> } else if [program] == "run-parts(/etc/cron.daily)" or [program] == "run-parts(/etc/cron.hourly)" or [program] == "cron" or [program] == "CROND" or [program] == "crontab" or [program] == "anacron" {
> mutate{
> add_tag => "cron"
> }
> } else if [program] == "dbus" {
> mutate{
> add_tag => "dbus"
> }
> } else if [program] == "dhclient" or [program] == "avahi-daemon" {
> mutate{
> add_tag => "DHCP"
> }
> } else if [type] == "security" {
> mutate{
> add_tag => "security"
> }
> } else if [program] == "mcollectived" {
> mutate{
> add_tag => "mcollectived"
> }
> } else if [type] == "postfix" {
> mutate{
> add_tag => "postfix"
> }
> }
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![Jigar\_Shah](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@Jigar\_Shah](https://discuss.elastic.co/u/Jigar_Shah)\
**Post date:** [October 11, 2016, 4:53am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860/4 "2016-10-11T04:53:35Z")

</div>

Hi Guys,

I am getting below error  
LogStash::Json::ParserError: Unexpected character ('.' (code 46)): Expected space separating root-level values

Below is my logstash.conf

input {  
beats {  
port =\> 5044  
type =\> beats  
codec =\> json\_lines  
}  
gelf {  
port =\>5043  
type =\> xxx  
}  
tcp {  
port =\> 5045  
codec =\> json\_lines  
type =\> xxx  
}  
}  
filter {  
if [type] == "beats" {  
json {  
source =\> "message"  
}}

if [type] == "beats" {  
geoip {  
source =\> "remote\_ip"  
target =\> "geoip"  
database =\> "/etc/logstash/GeoLiteCity.dat"  
add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ]  
add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ]  
}  
mutate {  
convert =\> ["[geoip][coordinates]", "float"]  
}}  
}  
output {  
if [type] == "beats" {  
amazon\_es {  
hosts =\> [“xxx”]  
region =\> “xx”  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
codec =\> "json"  
}  
}  
if [type] == "ecs" {  
amazon\_es {  
hosts =\> [“xx”]  
region =\> “x”  
index =\> "%{[tag]}-%{+YYYY.MM.dd}"  
#index =\> "testing-%{+YYYY.MM.dd}"  
document\_type =\> "%{[type]}"  
codec =\> "json"  
}  
}  
if [type] == "ecstcp" {  
amazon\_es {  
hosts =\> [“xx”]  
region =\> “xx”  
index =\> "%{[logstash\_index]}-%{+YYYY.MM.dd}"  
#index =\> "filetesting-%{+YYYY.MM.dd}"  
document\_type =\> "%{[type]}"  
codec =\> "json"  
}  
}  
}

Please suggest ASAP. unable to figure it out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:34am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-json-parse-failure/46860/5 "2017-07-06T04:34:49Z")

</div>