# Logstash not working after changing password

**URL:** https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788
**Category:** Logstash
**Created:** [June 14, 2019, 7:57am UTC](https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788 "2019-06-14T07:57:09Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![turnip](https://avatars.discourse-cdn.com/v4/letter/t/b77776/32.png) [@turnip](https://discuss.elastic.co/u/turnip)
#### Post date: [June 14, 2019, 7:57am UTC](https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788/1 "2019-06-14T07:57:09Z")

</div>

Hi all i have recently started on elk stack, and i have encounter some problem after changing the default password.  
Current stack version 7.0.1 based off this link  
[https://github.com/deviantony/docker-elk](https://github.com/deviantony/docker-elk)  
after setting the stack up on docker, i have proceed to change the password using:  
`docker-compose exec -T elasticsearch 'bin/elasticsearch-setup-passwords' auto --batch`  
and have replaced the username and password in`logstash.yml` & `kibana.yml` then proceeded to change the password in `logstash.conf` and restarted the whole stack

At this point all is working then i proceed to set my own password user: elastic, and updated the password in `logstash.conf` once this is done i restarted logstash.

From this point on the logstash fail to start.

Below is the logs shown  
logstash\_1 | [2019-06-14T07:52:39,639][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, { at line 101, column 16 (byte 2020) after output {\r\n\telasticsearch {\r\n\t\thosts =\> "elasticsearch:9200"\r\n\t\tuser =\> elastic\r\n\t\tpassword =\> P", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in`compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in`map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:151:in`initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:23:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:36:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:325:in`block in converge\_state'"]}  
logstash\_1 | [2019-06-14T07:52:56,899][WARN][logstash.outputs.elasticsearch] You are using a deprecated config setting "document\_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=\>"document\_type", :plugin=\>\<LogStash::Outputs::ElasticSearch bulk\_path=\>"/\_monitoring/bulk?system\_id=logstash&system\_api\_version=7&interval=1s", password=\>, hosts=\>[[http://elasticsearch:9200](http://elasticsearch:9200)], sniffing=\>false, manage\_template=\>false, id=\>"eadcdc69b7355983ca3a69ecac563286a667376c099396f2ec7dac2089060a4d", user=\>"logstash\_system", document\_type=\>"%{[@metadata][document\_type]}", enable\_metric=\>true, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_a30a99f4-045a-4849-8837-cec8ccb8969d", enable\_metric=\>true, charset=\>"UTF-8"\>, workers=\>1, template\_name=\>"logstash", template\_overwrite=\>false, doc\_as\_upsert=\>false, script\_type=\>"inline", script\_lang=\>"painless", script\_var\_name=\>"event", scripted\_upsert=\>false, retry\_initial\_interval=\>2, retry\_max\_interval=\>64, retry\_on\_conflict=\>1, ilm\_enabled=\>"auto", ilm\_rollover\_alias=\>"logstash", ilm\_pattern=\>"{now/d}-000001", ilm\_policy=\>"logstash-policy", action=\>"index", ssl\_certificate\_verification=\>true, sniffing\_delay=\>5, timeout=\>60, pool\_max=\>1000, pool\_max\_per\_route=\>100, resurrect\_delay=\>5, validate\_after\_inactivity=\>10000, http\_compression=\>false\>}  
logstash\_1 | [2019-06-14T07:52:57,649][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://logstash\_system:xxxxxx@elasticsearch:9200/](http://logstash_system:xxxxxx@elasticsearch:9200/)]}}  
logstash\_1 | [2019-06-14T07:52:57,868][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://logstash\_system:xxxxxx@elasticsearch:9200/](http://logstash_system:xxxxxx@elasticsearch:9200/)"}  
logstash\_1 | [2019-06-14T07:52:57,941][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>7}  
logstash\_1 | [2019-06-14T07:52:57,947][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7}  
logstash\_1 | [2019-06-14T07:52:58,071][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[http://elasticsearch:9200](http://elasticsearch:9200)"]}  
logstash\_1 | [2019-06-14T07:52:58,182][INFO][logstash.javapipeline] Starting pipeline {:pipeline\_id=\>".monitoring-logstash", "pipeline.workers"=\>1, "pipeline.batch.size"=\>2, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>2, :thread=\>"#\<Thread:0x5d43d92 run\>"}  
logstash\_1 | [2019-06-14T07:52:58,856][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=\>".monitoring-logstash"}  
logstash\_1 | [2019-06-14T07:53:01,439][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
logstash\_1 | [2019-06-14T07:53:07,471][INFO][logstash.javapipeline] Pipeline terminated {"pipeline.id"=\>".monitoring-logstash"}  
logstash\_1 | [2019-06-14T07:53:08,217][INFO][logstash.runner] Logstash shut down.  
docker-elk\_logstash\_1 exited with code 0

---

<div class="post-metadata">

### Author: ![turnip](https://avatars.discourse-cdn.com/v4/letter/t/b77776/32.png) [@turnip](https://discuss.elastic.co/u/turnip)
#### Post date: [June 14, 2019, 8:02am UTC](https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788/2 "2019-06-14T08:02:34Z")

</div>

Here is a look at the `logstash.conf` file that may be causing the problem.  
I cant identify where the problem is caused

```auto
input {
	tcp {
		port => 5000
	}
	gelf {
    port => 12200
    codec => json
    type => nginx
	}

	gelf {
    port => 12201
    codec => json
    type => app
	}

}
filter {

  if [type] == "nginx" {
     json {
      source => message
      add_tag => ["%{tag}"]
      tag_on_failure => ["error"]
    }

    date {
      match => ["timestamp", "ISO8601"]
      target => "@timestamp"
    }

    if "error" in [tags] {
      mutate {
        rename => {"message" => "error"}
      }
    }

    mutate {
      rename => {"tag" => "server_name"}
      remove_field => ["@version", "timestamp", "command", "message", "level"]
    }
  }

  if [type] == "app" {
    json {
      source => message
      add_tag => ["%{tag}"]
    }

    date {
      match => ["timestamp", "ISO8601"]
      target => "@timestamp"
    }

    mutate {
      remove_field => ["@version", "timestamp", "command", "level"]
    }
    # drop debug log
    # if [server_name] == "oauth-server" and [log_level] == "DEBUG" {
    # drop { }
    # }

    # if [server_name] == "oauth-server" and [log_level] == "INFO" {
    # drop { }
    # }

    # if [server_name] == "bff-server" and "/user/authorise" in [message] {
    # drop { }
    # }

    # if [server_name] == "bff-server" and "/user/signup" in [message] {
    # drop { }
    # }

    # if [server_name] == "bff-server" and "/user/reset_password" in [message] {
    # drop { }
    # }

    # if [server_name] == "bff-server" and "/user/change_password" in [message] {
    # drop { }
    # }

    # if [server_name] == "bff-server" and "/user/password/reset" in [message] {
    # drop { }
	# }

    # if [server_name] == "bff-server" and [log_level] == "DEBUG" and "/oauth/token" in [message] {
    # drop { }
    # }
  }
}

## Add your filters / logstash plugins configuration here

output {
	elasticsearch {
		hosts => "elasticsearch:9200"
		user => elastic
		password => newPassword
	}
}

```

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [June 14, 2019, 11:29pm UTC](https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788/3 "2019-06-14T23:29:49Z")

</div>

> [@turnip](#):
>
> Expected one of #, { at line 101, column 16 (byte 2020) after output {\r\n\telasticsearch {\r\n\t\thosts =\> "elasticsearch:9200"\r\n\t\tuser =\> elastic\r\n\t\tpassword =\> P

I believe you need to quote your password, adding `"` before and after it. While the logstash configuration language allows some "barewords" to be included without quoting, more complex strings typically break the rules with special characters and need to be provided in a quoted manner.

I would advise using a environment variables or storing the credentials in the Java Keystore, which would enable you to have:

```auto
output {
  elasticsearch {
    hosts => "elasticsearch:9200"
    user => "${ELASTICSEARCH_USERNAME}"
    password => "${ELASTICSEARCH_PASSWORD}"
  }
}

```

Docs on the Keystore can be found [here](https://www.elastic.co/guide/en/logstash/current/keystore.html), and docs on environment variable substitution can be found [here](https://www.elastic.co/guide/en/logstash/current/environment-variables.html).

* * *

Additionally, I believe the error message you have pasted includes the first character of your password, so it would be wise to change the password again.

---

<div class="post-metadata">

### Author: ![turnip](https://avatars.discourse-cdn.com/v4/letter/t/b77776/32.png) [@turnip](https://discuss.elastic.co/u/turnip)
#### Post date: [June 17, 2019, 2:27am UTC](https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788/4 "2019-06-17T02:27:08Z")

</div>

thanks, adding quotation to user and password as you said worked well!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 15, 2019, 2:36am UTC](https://discuss.elastic.co/t/logstash-not-working-after-changing-password/185788/5 "2019-07-15T02:36:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
