Settings from keystore doesnot load

[Thanura_Kannangara]

I'm running OSS version of winlogbeat
winlogbeat-oss-6.7.1-windows-x86_64

I created winlogbeat keystore and stored ES_PWD using instructions available here

and I'm using refering to the key as mentioned in the above mentioned article as below

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["https://<HostDNS>:9200"]
  ssl.verification_mode: "none"
  username: "admin"
  password: ${ES_PWD}
 # password: "${ES_PWD}" <-- tried this too

However, i can't get WInlogbeat started with this config as it gives me below error

2019-05-22T20:11:50.339Z ERROR instance/beat.go:906 Exiting: error initializing publisher: missing field accessing 'output.elasticsearch.password' (source:'C:\Program Files\winlogbeat\winlogbeat-6.7.1-windows-x86_64\winlogbeat.yml')

However, when i provide plain-text password it works

What am I doing wrong?

[Thanura_Kannangara]

Note: same issue happens on Metricbeat as well,

Im suspecting that service doesnot know where is the keystore, is Keystore hardcoded to a path?
right now i can see the keystore file is available hre
C:\Program Files\winlogbeat\winlogbeat-6.7.1-windows-x86_64\data\winlogbeat.keystore

similar location on Metricbeat

[Thanura_Kannangara]

Found the root cause
For some reason, windows service thinks path.data is in below location
-path.data "C:\ProgramData\winlogbeat"

Thats how its specified in
"install-service-winlogbeat.ps1"

but, when we generate the keystore it get stored in below location
C:\Program Files\winlogbeat\winlogbeat-6.7.1-windows-x86_64\data

if I copy the newly generated keystore to C:\ProgramData\winlogbeat and start the service, everything works fine..

Is this a bug, or im not doing something right

Note:

when I generate the keystore/ add keys
I CDed to this folder and ran the commands like this
> cd C:\Program Files\winlogbeat\winlogbeat-6.7.1-windows-x86_64
> echo " ThePass " | .\winlogbeat.exe keystore add ES_PWD --stdin

[pierhugues (Pier-Hugues Pellerin)]

@Thanura_Kannangara Did you use the install-service-winlogbeat.ps1 script to install winlogbeat? if this is the case you have to add the option -path.data "C:\ProgramData\winlogbeat" when you are calling the keystore subcommand.

winlogbeat keystore create -path.data `"C:\ProgramData\winlogbeat`"

[Thanura_Kannangara]

Thanks @pierhugues
for anyone else who run into this issue,
below command will work.
Even you don't create the the keystore first, this will create the keystore in correct location and add the new key

echo "Password123" | .\metricbeat.exe keystore add ES_PWD --stdin --force --path.data "C:\ProgramData\metricbeat"