My filebeat in windows runs fine. but i am unable to send logs from filebeat in windows to logstash in ubuntu server.
Im running both windows and ubuntu in VM's.
Here is my config file and the error.
The error is:
2017-05-27T23:25:33+05:30 ERR Failed to publish events caused by: read tcp 172.16.1.240:55392->172.16.1.84:5443: wsarecv: An existing connection was forcibly closed by the remote host.
2017-05-27T23:25:33+05:30 INFO Error publishing events (retrying): read tcp 172.16.1.240:55392->172.16.1.84:5443: wsarecv: An existing connection was forcibly closed by the remote host.
Hello,
My logstash logs do not show any errors relating to the windows server. It just shows old logs. I will upload the old logs if you need to see, but they dont really relate to the errors in windows server.
UPDATE: I changed the output of filebeat to elasticsearch instead of logstash. So now the logs go directly into elasticsearch. It works. But the problem is I cannot apply filters in elasticsearch. I need logstash to do the filtering. But through logstash connection is being refused. Need assistance.
LS- output ES config:
output {
elasticsearch { hosts => ["172.16.1.84:9200"]
hosts => "172.16.1.84:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
These are the config files in logstash.
Filters which process the data according to the need of our company.
I think the problem is with ssl certificate. I havent included ssl in filebeat output config in windows. If i include ssl in the configuration the filebeat wont start.
If you define ssl in beats logstash input, then filebeat logstash output will also require ssl configuration. When you say that filebeat won't start, what error do you get? Can you show us the log?
It is the ssl certificate. I just removed ssl certificate everywhere and its working. The filebeat in windows server send logs to logstash in ubuntu. But, I know it is not advisable to use it without ssl certificate.
Here is the error that i get when i include ssl:
Restart-Service : Service 'filebeat (filebeat)' cannot be started due to the following error: Cannot start service
filebeat on computer '.'.
At line:1 char:1
Your configuration has a set of template.* configuration that the logstash output does not supports, but that should not be blocking it from starting up.
But yet, it is not clear to me why it is not starting. Are you sure that the file C:\Program Files\filebeat\logstash exists and it is a valid certificate authority file? Can you get the full filebeat log file?
I dont understand what u mean by valid certificate authority file. I've downloaded the same certificate that ive generated while installing ELK-Stack.
In the image you can see that it has logstash in it.
I think you are missing a proper file extension in that logstash CA file. Can you go into command prompt, execute dir inside the C:\Program Files\filebeat directory and post the result here?
I have changed it. But the error is the same when i try to restart it.
PS C:\Program Files\Filebeat> Restart-Service filebeat
Restart-Service : Service 'filebeat (filebeat)' cannot be started due to the following error: Cannot start service
filebeat on computer '.'.
At line:1 char:1
2017-05-29T16:50:44+05:30 INFO Stopping filebeat
2017-05-29T16:50:44+05:30 INFO Stopping Crawler
2017-05-29T16:50:44+05:30 INFO Stopping 1 prospectors
2017-05-29T16:50:44+05:30 INFO Prospector channel stopped because beat is stopping.
2017-05-29T16:50:44+05:30 INFO Scan aborted because prospector stopped.
2017-05-29T16:50:45+05:30 INFO Prospector ticker stopped
2017-05-29T16:50:45+05:30 INFO Stopping Prospector: 7528349933066091638
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125046-notification[5680].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125028-authorization[1716].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-165016-insightv2[4140].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164524-insightv2[7652].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164724-insightv2[9092].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164424-insightv2[6580].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164624-insightv2[7608].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164324-insightv2[5828].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164224-insightv2[8152].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125043-gateway[5152].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-165024-insightv2[8304].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125029-providerv2[4272].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125040-billingv2[272].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125041-background[4728].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125039-apiv2[4944].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125031-workloadsv2[4472].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125044-resourcesv2[5348].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125025-authentication[3160].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125021-CnRestApiConsole-INTERNAL[3440].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125024-archive[3744].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-125019-monitorservice[4004].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164824-insightv2[8212].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164924-insightv2[6512].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Reader was closed: C:\computenext\tracelogs\ABTIS-QA-APP-20170529-164516-insightv2[7416].txt. Closing.
2017-05-29T16:50:45+05:30 INFO Crawler stopped
2017-05-29T16:50:45+05:30 INFO Stopping spooler
2017-05-29T16:50:45+05:30 INFO Stopping Registrar
2017-05-29T16:50:45+05:30 INFO Ending Registrar
2017-05-29T16:50:45+05:30 INFO Total non-zero values: filebeat.harvester.closed=24 filebeat.harvester.started=24 libbeat.logstash.call_count.PublishEvents=30 libbeat.logstash.publish.read_bytes=750 libbeat.logstash.publish.write_bytes=114833 libbeat.logstash.publish.write_errors=1 libbeat.logstash.published_and_acked_events=2639 libbeat.logstash.published_but_not_acked_events=8 libbeat.publisher.published_events=2639 publish.events=11880 registrar.states.current=8777 registrar.states.update=11880 registrar.writes=34
2017-05-29T16:50:45+05:30 INFO Uptime: 4m2.7424832s
2017-05-29T16:50:45+05:30 INFO filebeat stopped.
indent preformatted text by 4 spaces
This is the error when i uncomment "ssl.certificate_authorities" line and try to restart filebeat.
This is not an error, it is simply telling g that filebeat is stopping. Is it all that is inside log file? Can you paste the complete log file in pastebin.com and attach link here?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.