`.
> #cloud.auth:
>
> #================================ Outputs =====================================
>
> # Configure what output to use when sending the data collected by the beat.
>
> #-------------------------- Elasticsearch output ------------------------------
> #output.elasticsearch:
> # Array of hosts to connect to.
> # hosts: ["localhost:9200"]
>
> # Optional protocol and basic auth credentials.
> #protocol: "https"
> #username: "elastic"
> #password: "changeme"
>
> #----------------------------- Logstash output --------------------------------
> output.logstash:
> # The Logstash hosts
> hosts: ["myipAddress:5044"]
>
> # Optional SSL. By default is off.
> # List of root certificates for HTTPS server verifications
> #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
>
> # Certificate for SSL client authentication
> #ssl.certificate: "/etc/pki/client/cert.pem"
>
> # Client Certificate Key
> #ssl.key: "/etc/pki/client/cert.key"
>
> #================================ Processors =====================================
>
> # Configure processors to enhance or manipulate events generated by the beat.
>
> processors:
> - add_host_metadata: ~
> - add_cloud_metadata: ~
>
> #================================ Logging =====================================
>
> # Sets log level. The default log level is info.
> # Available log levels are: error, warning, info, debug
> logging.level: debug
>
> # At debug level, you can selectively enable logging only for some components.
> # To enable all selectors use ["*"]. Examples of other selectors are "beat",
> # "publish", "service".
> #logging.selectors: ["*"]
>
> #============================== Xpack Monitoring ===============================
> # filebeat can export internal metrics to a central Elasticsearch monitoring
> # cluster. This requires xpack monitoring to be enabled in Elasticsearch. The
> # reporting is disabled by default.
>
> # Set to true to enable the monitoring reporter.
> #xpack.monitoring.enabled: false
>
> # Uncomment to send the metrics to Elasticsearch. Most settings from the
> # Elasticsearch output are accepted here as well. Any setting that is not set is
> # automatically inherited from the Elasticsearch output configuration, so if you
> # have the Elasticsearch output configured, you can simply uncomment the
> # following line.
> #xpack.monitoring.elasticsearch:
>
> #================================= Migration ==================================
>
> # This allows to enable 6.7 migration aliases
> #migration.6_to_7.enabled: true
```
---
### Author:  [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [June 26, 2019, 11:31am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/7 "2019-06-26T11:31:02Z")
@markov, `filebeat.yml` file configuration seems fine. Are you able to telnet your logstash server from client machine (where you installed filebeat) on port 5044. Please run below command from your filebeat machine:
```auto
$ telnet logstash_server_ip 5044
```
If not able to telnet open your firewall rule else please provide some more error logs.
Also please provide the output of below command from logstash server:
```auto
$ netstat -an|grep 5044
```
Thanks.
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 26, 2019, 11:40am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/8 "2019-06-26T11:40:20Z")
> [@Tek\_Chand](#):
>
> netstat -an|grep 5044
this is the `logstash-plain.log`
`[2019-06-26T13:37:42,604][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>403, :url=>"http://51.15.188.74:9200/_bulk"}`
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 26, 2019, 11:41am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/9 "2019-06-26T11:41:41Z")
> [@markov](#):
>
> netstat -an|grep 5044
```
root@sd-131865:~# netstat -an|grep 5044
tcp6 0 0 :::5044 :::* LISTEN
tcp6 0 0 xx.xxx.xx.xx:5044 105.188.133.167:50718 ESTABLISHED
```
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 26, 2019, 11:45am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/10 "2019-06-26T11:45:12Z")
```
:~$ telnet logstash_server_ip 5044
Trying logstash_server_ip...
Connected to logstash_server_ip.
Escape character is '^]'.
Connection closed by foreign host.
```
---
### Author:  [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [June 26, 2019, 11:47am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/11 "2019-06-26T11:47:47Z")
@markov,
> [@markov](#):
>
> tcp6 0 0 xx.xxx.xx.xx:5044 105.188.133.167:50718 ESTABLISHED
From above output seems your are getting traffic at logstash server from your one machine i.e `105.188.133.167`
If its true then you need to setup your filebeat according to that server.
Thanks.
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 26, 2019, 11:52am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/12 "2019-06-26T11:52:08Z")
yes that's it, and i've configured filebeat to send to logstash server with the ip xx.xxx.xx.xx, i think everything is ok but !!!!!!!!
---
### Author:  [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [June 26, 2019, 12:00pm UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/13 "2019-06-26T12:00:22Z")
@markov,
Please verify your logstash configuration also. Its should be look like below:
```auto
input {
beats {
port => 5044
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => ["received_at", "%{@timestamp}"]
add_field => ["received_from", "%{host}"]
}
syslog_pri { }
date {
match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}
}
}
output {
elasticsearch {
hosts => ["192.168.56.4:9200"]
user => "logstash_internal"
password => "admin@123"
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
```
You should have `user => "logstash_internal"` and `password` in output filter of logstash.
Hope so it will fix your issue.
Thanks.
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 26, 2019, 1:16pm UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/14 "2019-06-26T13:16:51Z")
02-beats-input.conf
```
input {
beats {
port => 5044
}
}
```
10-syslog-filter.conf
```
filter {
if [source] =~ "gid" {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:timestamp}\|(\[%{BASE10NUM:nbr}\])\|%{IPORHOST:ClientIP}\|%{USERNAME:User};%{DATA:Node}\|%{URIPATH:Url}\|%{NOTS$
}
}else if [source] =~ "server" {
grok {
match =>{ "message" => [
"(?m)%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?[^\]]+)\] %{GREEDYDATA:Message}",
"%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?[^\]]+)\] %{WORD:n} \| %{NUMBER:number} \| %{WORD:b} \| %{DATA:Url} \| %{GREEDYDAT$
"%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?[^\]]+)\] %{GREEDYDATA:Message}"] }
}
}
date {match => ["timestamp" , "MMM dd yyyy HH:mm:ss","MMM d yyyy HH:mm:ss", "ISO8601"]
target => "@timestamp"}
mutate {
remove_field => ["[beat][name]", "[beat][version]", "[beat][hostname]", "[host][architecture]", "[host][containerized]", "[host][id]", "[host][name]", "[host]$
}
}
```
30-elasticsearch-output.conf
```
output {
elasticsearch {
hosts => ["myIpAdress:9200"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
user => "logstash_internal"
password => "admin10@"
document_type => "%{[@metadata][type]}"
}
}
```
it's all working before integrating the xpack for security reasons
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 27, 2019, 1:16pm UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/15 "2019-06-27T13:16:53Z")
retrying failed action with response code: 403 ({"type"=\>"security\_exception", "reason"=\>"action [indices:admin/create] is unauthorized for user [logstash\_internal]"})
---
### Author:  [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [June 28, 2019, 4:45am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/16 "2019-06-28T04:45:16Z")
@markov,
> [@markov](#):
>
> action [indices:admin/create] is unauthorized for user [logstash\_internal]"
Please follow the below documentation. You need to assign logstash\_writer role to logstash\_internal user. Please read document and follow it.
[document](https://www.elastic.co/guide/en/logstash/7.1/ls-security.html)
Thanks.
---
### Author:  [@markov](https://discuss.elastic.co/u/markov)
#### Post date: [June 28, 2019, 10:21am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/17 "2019-06-28T10:21:58Z")
the documentation said that i have to configure credentials separately for each of the Elasticsearch plugins in my Logstash `.conf` file; is that mean :
input.conf
```
> input {
> beats {
> port => 5044
> }
> elasticsearch {
> hosts => ["ipadress:9200"]
> user => "logstash_internal"
> password => "admin@"
> }
> }
```
filter.conf
```
> filter {
> elasticsearch {
> hosts => ["ipadress:9200"]
> user => "logstash_internal"
> password => "admin@"
> }
>
> if [source] =~ "gid" {
> grok {
> match => {"message" => "%{TIMESTAMP_ISO8601:timestamp}\|(\[%{BASE10NUM:nbr}\])\|%{IPORHOST:ClientIP}\|%{USERNAME:User};%{DATA:Node}\|%{URIPATH:Url}\|%{NOTS$
> }
> }else if [source] =~ "server" {
>
> grok {
> match =>{ "message" => [
>
> "(?m)%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?[^\]]+)\] %{GREEDYDATA:Message}",
> "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?[^\]]+)\] %{WORD:n} \| %{NUMBER:number} \| %{WORD:b} \| %{DATA:Url} \| %{GREEDYDAT$
> "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?[^\]]+)\] %{GREEDYDATA:Message}"] }
> }
> }
> date {match => ["timestamp" , "MMM dd yyyy HH:mm:ss","MMM d yyyy HH:mm:ss", "ISO8601"]
> target => "@timestamp"}
>
> mutate {
> remove_field => ["[beat][name]", "[beat][version]", "[beat][hostname]", "[host][architecture]", "[host][containerized]", "[host][id]", "[host][name]", "[host]$
> }
> }
```
output.conf
```
> output {
> elasticsearch {
> hosts => ["ipadress:9200"]
> index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
> user => "logstash_internal"
> password => "admin@"
> document_type => "%{[@metadata][type]}"
> }
> stdout { codec => rubydebug }
> }
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 26, 2019, 10:21am UTC](https://discuss.elastic.co/t/logstash-with-xpack-and-filebeat/187533/18 "2019-07-26T10:21:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.