# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=729 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 730 --- ## [Ingesting CSV reports through Filebeat to Logstash](https://discuss.elastic.co/t/ingesting-csv-reports-through-filebeat-to-logstash/137680)
**Author:** [@joshua625](https://discuss.elastic.co/u/joshua625)\ **Replies:** 20\ **Last updated:** [July 3, 2018, 8:41pm UTC](https://discuss.elastic.co/t/ingesting-csv-reports-through-filebeat-to-logstash/137680 "2018-07-03T20:41:59Z")
Hello, I am facing a critical problem on writing a filter config for logstash to parse the incoming csv data (having 50+ headers and 300 MB) . I tried many ways, but, I was unable to index the data on Elasticsearch. I… --- ## [Logstash with beats issue](https://discuss.elastic.co/t/logstash-with-beats-issue/137686)
**Author:** [@HRG](https://discuss.elastic.co/u/HRG)\ **Replies:** 16\ **Last updated:** [July 3, 2018, 7:22pm UTC](https://discuss.elastic.co/t/logstash-with-beats-issue/137686 "2018-07-03T19:22:08Z")
Installed Logstash beats plug in ./logstash-plugin install logstash-input-beats Validating logstash-input-beats Installing logstash-input-beats Installation successful When i run the logstash with configuration file… --- ## [Need Help parsing timestamps](https://discuss.elastic.co/t/need-help-parsing-timestamps/134988)
**Author:** [@Mugen](https://discuss.elastic.co/u/Mugen)\ **Replies:** 7\ **Last updated:** [July 3, 2018, 6:50pm UTC](https://discuss.elastic.co/t/need-help-parsing-timestamps/134988 "2018-07-03T18:50:53Z")
I am curently parsing azure messages. Sadly azure sends some dates in the following format: 2018-06-03 01:00:12Z at first I thought this would be a normal ISO8601 but this did not match. I tried a Formats like yyyy-M… --- ## [Forward before processing](https://discuss.elastic.co/t/forward-before-processing/138422)
**Author:** [@criddock](https://discuss.elastic.co/u/criddock)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 6:32pm UTC](https://discuss.elastic.co/t/forward-before-processing/138422 "2018-07-03T18:32:35Z")
Greetings and Happy 4th to Everyone... My question is this... I have a customer that wants to forward a copy of all received event traffic to a cold storage server before Logstash can process the event...We are doing so… --- ## [Logstash - Grok parse failure](https://discuss.elastic.co/t/logstash-grok-parse-failure/138403)
**Author:** [@Aby](https://discuss.elastic.co/u/Aby)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 4:33pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure/138403 "2018-07-03T16:33:05Z")
Hi - i am trying to parse the below log file: \[2018-05-29 08:04:23,687\] DEBUG - http-outgoing-2845 \<\< HTTP/1.1 500 Internal Server Error {org.apache.synapse.transport.http.headers} \[2018-05-29 08:04:18,037\] DEBUG - h… --- ## [I am unable to access Logstash(windows service) through http://localhost:5044 (within the logstash server) and also unable to access http://192.168.27.189:5044 (outside of the logstash server)](https://discuss.elastic.co/t/i-am-unable-to-access-logstash-windows-service-through-http-localhost-5044-within-the-logstash-server-and-also-unable-to-access-http-192-168-27-189-5044-outside-of-the-logstash-server/137877)
**Author:** [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Replies:** 2\ **Last updated:** [July 3, 2018, 3:48pm UTC](https://discuss.elastic.co/t/i-am-unable-to-access-logstash-windows-service-through-http-localhost-5044-within-the-logstash-server-and-also-unable-to-access-http-192-168-27-189-5044-outside-of-the-logstash-server/137877 "2018-07-03T15:48:14Z")
I am unable to access Logstash(windows service) through http://localhost:5044 (within the logstash server) and also unable to access http://192.168.27.189:5044 (outside of the logstash server) I can see Logstash window… --- ## [Unexpected character ('\<' (code 60))](https://discuss.elastic.co/t/unexpected-character-code-60/138279)
**Author:** [@Pixarjunkie](https://discuss.elastic.co/u/Pixarjunkie)\ **Replies:** 3\ **Last updated:** [July 3, 2018, 3:27pm UTC](https://discuss.elastic.co/t/unexpected-character-code-60/138279 "2018-07-03T15:27:39Z")
Hi, I'm new to using elasticsearch/logstash and I am trying to trouble shoot an error (this is also my first post here so apologies if I miss something. I have created a logstash\_test.config file: \> input { \> … --- ## [Multipipeline with Filebeat, am I doing it right?](https://discuss.elastic.co/t/multipipeline-with-filebeat-am-i-doing-it-right/135402)
**Author:** [@FOXHOUNDER](https://discuss.elastic.co/u/FOXHOUNDER)\ **Replies:** 6\ **Last updated:** [July 3, 2018, 3:00pm UTC](https://discuss.elastic.co/t/multipipeline-with-filebeat-am-i-doing-it-right/135402 "2018-07-03T15:00:11Z")
Hi, I'm new to the ELK Stack and I've been assigned the simple task to use this technology to store and analyze our logs. Quick intro: we have a folder containing logs, one for each microservice. A new log is produced e… --- ## [Use subset of json entry as document](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 3\ **Last updated:** [July 3, 2018, 2:25pm UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240 "2018-07-03T14:25:40Z")
I have several json entries in a huge file, in the form of: {"preview":false,"offset":0,"result":{"@timestamp":"2016-11-30T21:59:43.000Z","@version":"1","\_raw":"..}} What I want to use this file as input but keep every… --- ## [Port drop statistics](https://discuss.elastic.co/t/port-drop-statistics/138239)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 3\ **Last updated:** [July 3, 2018, 1:54pm UTC](https://discuss.elastic.co/t/port-drop-statistics/138239 "2018-07-03T13:54:29Z")
Hello, I'm currently using netflow plugin in my ELK stack. I was wondering if there is any way to figure out how many packets are being dropped when feeding netflow data to the 2055 port of logstash. Thanks --- ## [Centralized logstash configuration to run on 'n' servers](https://discuss.elastic.co/t/centralized-logstash-configuration-to-run-on-n-servers/138401)
**Author:** [@pkshara](https://discuss.elastic.co/u/pkshara)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 1:46pm UTC](https://discuss.elastic.co/t/centralized-logstash-configuration-to-run-on-n-servers/138401 "2018-07-03T13:46:26Z")
Hi, I have 3 different servers which has log files in each server. 10.x.x.01 , 10.x.x.02 and 10.x.x.03 is my server list. I have logstash installed in 10.x.x.01 server and will run config file from the same, so is the… --- ## [Logstash multiple pipelines issue](https://discuss.elastic.co/t/logstash-multiple-pipelines-issue/137615)
**Author:** [@Jack37](https://discuss.elastic.co/u/Jack37)\ **Replies:** 3\ **Last updated:** [July 3, 2018, 1:45pm UTC](https://discuss.elastic.co/t/logstash-multiple-pipelines-issue/137615 "2018-07-03T13:45:34Z")
Hi guys, I have a strange issue with Logstash 5.6.9 and multiple pipelines I have two pipelines that send messages to two different elastic servers, with different IP and subnet Yesterday one of this server gone offli… --- ## [Logstash - Trim value](https://discuss.elastic.co/t/logstash-trim-value/138379)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 1:43pm UTC](https://discuss.elastic.co/t/logstash-trim-value/138379 "2018-07-03T13:43:27Z")
I have huge json files with logs, which are all timestamped in the form result":{"@timestamp":"2016-11-30T21:59:59.699265Z" The problem is that due to some reason, the timestamp format is in various forms, e.g. the fol… --- ## [How to find out which logshatsh filter is best for your data](https://discuss.elastic.co/t/how-to-find-out-which-logshatsh-filter-is-best-for-your-data/138153)
**Author:** [@Ambuj\_Saxena](https://discuss.elastic.co/u/Ambuj_Saxena)\ **Replies:** 15\ **Last updated:** [July 3, 2018, 1:33pm UTC](https://discuss.elastic.co/t/how-to-find-out-which-logshatsh-filter-is-best-for-your-data/138153 "2018-07-03T13:33:16Z")
Hi, I am new with ELK, I start learning with ELK and saw some issue with my own data. That time I realized that I am using the wrong filter that is why I am not getting desirable output. This is the sample of my data, … --- ## [\_grokparsefailure but it work on grok debugger](https://discuss.elastic.co/t/grokparsefailure-but-it-work-on-grok-debugger/138127)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 18\ **Last updated:** [July 2, 2018, 2:20pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-it-work-on-grok-debugger/138127 "2018-07-02T14:20:51Z")
Hello everybody, I use logstash to grok logs line. For exemple one of these lines is : INFO;0000;000003;\* /data/EDT/batchs/files/logs/WKF998MT-20180621-20180621-1301.log \* I use a first grok to cut in 3 field this eve… --- ## [Lost logs with logrotate](https://discuss.elastic.co/t/lost-logs-with-logrotate/138400)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 0\ **Last updated:** [July 3, 2018, 1:28pm UTC](https://discuss.elastic.co/t/lost-logs-with-logrotate/138400 "2018-07-03T13:28:34Z")
I have a logstash process that writes a log file ( syslog.log) and a logstash process that reads the log file. When logrotate starts, it changes the filename into syslog-\[date\].log and some logs that aren't read yet fro… --- ## [Logstash - Unable to match date pattern](https://discuss.elastic.co/t/logstash-unable-to-match-date-pattern/138351)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 12:17pm UTC](https://discuss.elastic.co/t/logstash-unable-to-match-date-pattern/138351 "2018-07-03T12:17:51Z")
I am using a json input to filter some data and sent it to elasticsearch; sample document from input json file: {"preview":false,"offset":0,"result":{"@timestamp":"2016-11-30T21:59:43.000Z","@version":"1","\_raw":"{ ..}… --- ## [ELK 6.3 logstash not monitoring](https://discuss.elastic.co/t/elk-6-3-logstash-not-monitoring/137512)
**Author:** [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Replies:** 10\ **Last updated:** [July 3, 2018, 12:09pm UTC](https://discuss.elastic.co/t/elk-6-3-logstash-not-monitoring/137512 "2018-07-03T12:09:19Z")
I am unable to get monitoring to work on Logstash. My logstash yml file xpack.security.enabled: false xpack.graph.enabled: true xpack.ml.enabled: true xpack.monitoring.enabled: true xpack.watcher.enabled: true xpa… --- ## [An unexpected error occurred! {:error=\>#\\>](https://discuss.elastic.co/t/an-unexpected-error-occurred-error-errno-broken-pipe-stdout/138373)
**Author:** [@sunilkumar](https://discuss.elastic.co/u/sunilkumar)\ **Replies:** 0\ **Last updated:** [July 3, 2018, 12:00pm UTC](https://discuss.elastic.co/t/an-unexpected-error-occurred-error-errno-broken-pipe-stdout/138373 "2018-07-03T12:00:59Z")
Im getting a fatal error while running logstash . this is my logstash log file \[2018-07-03T17:14:51,463\]\[INFO \]\[logstash.modules.scaffold\] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/home/automaton/log… --- ## [Should Logstash be installed on same server as FileBeat?](https://discuss.elastic.co/t/should-logstash-be-installed-on-same-server-as-filebeat/138354)
**Author:** [@solo](https://discuss.elastic.co/u/solo)\ **Replies:** 0\ **Last updated:** [July 3, 2018, 10:32am UTC](https://discuss.elastic.co/t/should-logstash-be-installed-on-same-server-as-filebeat/138354 "2018-07-03T10:32:32Z")
Hi, I have 4 development servers: Webserver Apache - Filebeat Integration server 1 Java-code - Filebeat Integration server 2 Java-code - Filebeat Payment server - .Net And I have one server that runs: Logstash Elas… --- ## [Creat two index patterns](https://discuss.elastic.co/t/creat-two-index-patterns/138352)
**Author:** [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Replies:** 0\ **Last updated:** [July 3, 2018, 10:26am UTC](https://discuss.elastic.co/t/creat-two-index-patterns/138352 "2018-07-03T10:26:13Z")
hello everyone , i am looking for an simple example or tutorial of configuration that creat 2 index into ES , because i have 2 type of file logs , i specified 2 path for the 2 types in filebeat.yml , but when i follow… --- ## [Logstash instance shutdown after some time atomatically](https://discuss.elastic.co/t/logstash-instance-shutdown-after-some-time-atomatically/138119)
**Author:** [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)\ **Replies:** 7\ **Last updated:** [July 3, 2018, 9:48am UTC](https://discuss.elastic.co/t/logstash-instance-shutdown-after-some-time-atomatically/138119 "2018-07-03T09:48:45Z")
I'm running logstash instance using data directory. But after sometime it stops shipping logs though instance keep running . Then after removing the data directory and creating same directory again it start shipping. Ple… --- ## [Generate output index from input filename](https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339)
**Author:** [@n1ur0](https://discuss.elastic.co/u/n1ur0)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 8:50am UTC](https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339 "2018-07-03T08:50:49Z")
Hi, I'm trying to parse and integrate multiple .csv files into elasticsearch through logstash csv plugin. However, as I have around 4000 csv files, is there a way to make the same input csv filename the same as the elas… --- ## [Config problem with logstash](https://discuss.elastic.co/t/config-problem-with-logstash/138194)
**Author:** [@ankit211](https://discuss.elastic.co/u/ankit211)\ **Replies:** 5\ **Last updated:** [July 3, 2018, 8:34am UTC](https://discuss.elastic.co/t/config-problem-with-logstash/138194 "2018-07-03T08:34:30Z")
in loagstash choosing correct path alougth it's not config properly. file given below. input { file { path =\> "C:\\Ankit\\kibana\_files\\Client\_4\_May\_2018\_16\_31.csv" start\_position =\> "beginning" sincedb\_path=\>"c:\\Anki… --- ## [Logstash: Can not output to elasticsearch when using s3 input plugin](https://discuss.elastic.co/t/logstash-can-not-output-to-elasticsearch-when-using-s3-input-plugin/138326)
**Author:** [@mumehta](https://discuss.elastic.co/u/mumehta)\ **Replies:** 0\ **Last updated:** [July 3, 2018, 8:06am UTC](https://discuss.elastic.co/t/logstash-can-not-output-to-elasticsearch-when-using-s3-input-plugin/138326 "2018-07-03T08:06:21Z")
Can anyone help please, I am trying to analyse files in s3 bucket, however, it seems output is not getting processed or kibana is not showing any output on the dashboard. Only output i can see is from filebeat, which i … --- ## [Increase logstash logging](https://discuss.elastic.co/t/increase-logstash-logging/138214)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 4\ **Last updated:** [July 3, 2018, 7:26am UTC](https://discuss.elastic.co/t/increase-logstash-logging/138214 "2018-07-03T07:26:56Z")
Hi all, I've been using logstash since February in order to index data about CDR (Call Detail Record) from our international central (basically a cvs with the data) So far so good: logstash behaved in a stable way, wi… --- ## [Logstash use the "--config.reload.automatic" fail](https://discuss.elastic.co/t/logstash-use-the-config-reload-automatic-fail/138316)
**Author:** [@sun\_changlong](https://discuss.elastic.co/u/sun_changlong)\ **Replies:** 2\ **Last updated:** [July 3, 2018, 7:22am UTC](https://discuss.elastic.co/t/logstash-use-the-config-reload-automatic-fail/138316 "2018-07-03T07:22:45Z")
I use the bin/logstash -f sys.conf --config.reload.automatic run the logstash,The first time is normal operation，However, after I change the configuration file, logstash will report an error when reloading. \[ERROR\]\[logs… --- ## [Logstash unable to parse a simple json](https://discuss.elastic.co/t/logstash-unable-to-parse-a-simple-json/138315)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 1\ **Last updated:** [July 3, 2018, 7:22am UTC](https://discuss.elastic.co/t/logstash-unable-to-parse-a-simple-json/138315 "2018-07-03T07:22:24Z")
I am just trying to read the simplest json possible via logstash and send it to elasticsearch. input json file: { "preview": false } input config input { file { path =\> "/usr/share/logstash/files/sample\_data… --- ## [Error Starting Logstash(Any version)](https://discuss.elastic.co/t/error-starting-logstash-any-version/138313)
**Author:** [@JamJamJames98](https://discuss.elastic.co/u/JamJamJames98)\ **Replies:** 0\ **Last updated:** [July 3, 2018, 6:35am UTC](https://discuss.elastic.co/t/error-starting-logstash-any-version/138313 "2018-07-03T06:35:59Z")
I am trying to run Logstash on my machine (mac osx high sierra), but am encountering an error when i run ./bin/logstash I get the error when trying the "hello world", and without any config file still gives this error w… --- ## [No index created](https://discuss.elastic.co/t/no-index-created/137410)
**Author:** [@retxedue](https://discuss.elastic.co/u/retxedue)\ **Replies:** 6\ **Last updated:** [July 3, 2018, 5:57am UTC](https://discuss.elastic.co/t/no-index-created/137410 "2018-07-03T05:57:31Z")
I have this below logstash config. However, it is not creating any index. file { type =\> "monitor1" path =\> "/root/Documents/scripts/data/monitor.out" start\_position =\> "beginning" sincedb\_path =\> "/opt/sincedb/.sin… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=728) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=730)