# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=850 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 851 --- ## [Logstash - setting ElasticSearch index based on different input type](https://discuss.elastic.co/t/logstash-setting-elasticsearch-index-based-on-different-input-type/113078)
**Author:** [@Bikash\_Behuria](https://discuss.elastic.co/u/Bikash_Behuria)\ **Replies:** 1\ **Last updated:** [December 27, 2017, 2:08pm UTC](https://discuss.elastic.co/t/logstash-setting-elasticsearch-index-based-on-different-input-type/113078 "2017-12-27T14:08:30Z")
Hi , I have two input plugins in Logstash i.e. filebeat , and kafka topics for two different type and sources of logs. Now i want to filter , process the logs and send to different Elasticsearch indexes . Now since I a… --- ## [Logstash - json log line has field that can be either string or another json object: seems to anger elasticsearch](https://discuss.elastic.co/t/logstash-json-log-line-has-field-that-can-be-either-string-or-another-json-object-seems-to-anger-elasticsearch/113087)
**Author:** [@Mike\_Nielsen](https://discuss.elastic.co/u/Mike_Nielsen)\ **Replies:** 1\ **Last updated:** [December 27, 2017, 2:02pm UTC](https://discuss.elastic.co/t/logstash-json-log-line-has-field-that-can-be-either-string-or-another-json-object-seems-to-anger-elasticsearch/113087 "2017-12-27T14:02:37Z")
I have a log file in which each line is a syntactically-correct JSON object. The problem is that there is a particular field that can either be a simple string or another JSON object (see the field "v" in the example be… --- ## [Correlating Asynchronous Requests](https://discuss.elastic.co/t/correlating-asynchronous-requests/113312)
**Author:** [@Samuel\_Lima1](https://discuss.elastic.co/u/Samuel_Lima1)\ **Replies:** 0\ **Last updated:** [December 27, 2017, 12:14pm UTC](https://discuss.elastic.co/t/correlating-asynchronous-requests/113312 "2017-12-27T12:14:36Z")
Hi, I have a situation where many requests and responses are sent over Logstash and I have to correlate each pair request/response. I don't know when neither wheather the response is coming, but when and if it comes I'… --- ## [When I'm trying to install logstash 6. 0 .0](https://discuss.elastic.co/t/when-im-trying-to-install-logstash-6-0-0/113142)
**Author:** [@kartheek91](https://discuss.elastic.co/u/kartheek91)\ **Replies:** 2\ **Last updated:** [December 27, 2017, 10:13am UTC](https://discuss.elastic.co/t/when-im-trying-to-install-logstash-6-0-0/113142 "2017-12-27T10:13:27Z")
When I'm trying to install logstash i.e I have unzipped the directory in c:\\Program Files and I have created sample.conf and I tried to run that sample.conf file using bin/logstash -f sample.conf but it is throwing me fo… --- ## [Unable to install logstash-filter-geoip](https://discuss.elastic.co/t/unable-to-install-logstash-filter-geoip/111875)
**Author:** [@jbeck](https://discuss.elastic.co/u/jbeck)\ **Replies:** 3\ **Last updated:** [December 27, 2017, 7:39am UTC](https://discuss.elastic.co/t/unable-to-install-logstash-filter-geoip/111875 "2017-12-27T07:39:39Z")
Installing logstash-filter-geoip is failing as of this morning. bin/logstash-plugin install logstash-filter-geoip Validating logstash-filter-geoip Installing logstash-filter-geoip Error Bundler::InstallError, retrying 1… --- ## [Logstash error while executing](https://discuss.elastic.co/t/logstash-error-while-executing/113266)
**Author:** [@syed](https://discuss.elastic.co/u/syed)\ **Replies:** 0\ **Last updated:** [December 27, 2017, 5:28am UTC](https://discuss.elastic.co/t/logstash-error-while-executing/113266 "2017-12-27T05:28:34Z")
Dear Team, we received below error when we are executing logstash. "we have a trail license till 9th of jan 2018" \[ERROR\]\[logstash.licensechecker.licensemanager\] Unable to retrieve license information from license serv… --- ## [Need clarity - logstash 'filter' patterns](https://discuss.elastic.co/t/need-clarity-logstash-filter-patterns/113207)
**Author:** [@subin](https://discuss.elastic.co/u/subin)\ **Replies:** 2\ **Last updated:** [December 27, 2017, 2:23am UTC](https://discuss.elastic.co/t/need-clarity-logstash-filter-patterns/113207 "2017-12-27T02:23:33Z")
In the docs and various examples, I'm seeing if statements in Input section, like: if \[message\] =~ /dovecot: (imap|pop3|managesieve)-login Apart from \[message\], what other fields can if take? I don't have much unders… --- ## [Lostash problem](https://discuss.elastic.co/t/lostash-problem/112555)
**Author:** [@narasiman1986](https://discuss.elastic.co/u/narasiman1986)\ **Replies:** 5\ **Last updated:** [December 27, 2017, 1:07am UTC](https://discuss.elastic.co/t/lostash-problem/112555 "2017-12-27T01:07:46Z")
My logstash file code as follows input { jdbc { jdbc\_driver\_library =\> "D:\\mysql-connector-java-5.1.44\\mysql-connector-java-5.1.44\\mysql-connector-java-5.1.44-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" … --- ## [Old logs?](https://discuss.elastic.co/t/old-logs/110223)
**Author:** [@Jerry\_Hoffmeister](https://discuss.elastic.co/u/Jerry_Hoffmeister)\ **Replies:** 6\ **Last updated:** [December 26, 2017, 10:29pm UTC](https://discuss.elastic.co/t/old-logs/110223 "2017-12-26T22:29:03Z")
I recently upgrade our elasticstack from 2.4 to 5.6.4 which went fairly smoothly. I'm seeing lots of the following in the logstash logs tho: \[2017-12-04T22:29:34,620\]\[INFO \]\[logstash.outputs.elasticsearch\] retrying fai… --- ## [Grok - How do I make it so a field is indexed in ES?](https://discuss.elastic.co/t/grok-how-do-i-make-it-so-a-field-is-indexed-in-es/113238)
**Author:** [@E\_Marshall](https://discuss.elastic.co/u/E_Marshall)\ **Replies:** 1\ **Last updated:** [December 26, 2017, 7:07pm UTC](https://discuss.elastic.co/t/grok-how-do-i-make-it-so-a-field-is-indexed-in-es/113238 "2017-12-26T19:07:14Z")
First thing, I do have a Grok configuration file which is working just fine in parsing my log files. I even have it writing to a separate file for any non matches. My issue is that when looking at the created index for … --- ## [Logstash is shutting down when avro format message is directly sent to kafka topic accidently, throws below error](https://discuss.elastic.co/t/logstash-is-shutting-down-when-avro-format-message-is-directly-sent-to-kafka-topic-accidently-throws-below-error/113213)
**Author:** [@NandhaKrishna](https://discuss.elastic.co/u/NandhaKrishna)\ **Replies:** 0\ **Last updated:** [December 26, 2017, 1:29pm UTC](https://discuss.elastic.co/t/logstash-is-shutting-down-when-avro-format-message-is-directly-sent-to-kafka-topic-accidently-throws-below-error/113213 "2017-12-26T13:29:27Z")
\[2017-12-22T08:41:10,876\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\, :backtrace=\>\["org/jruby/ext/stringio/StringIO.java:829:in read'", "/Data/tools/logsta… --- ## [No indexes, no error logs](https://discuss.elastic.co/t/no-indexes-no-error-logs/113015)
**Author:** [@subin](https://discuss.elastic.co/u/subin)\ **Replies:** 10\ **Last updated:** [December 26, 2017, 9:25am UTC](https://discuss.elastic.co/t/no-indexes-no-error-logs/113015 "2017-12-26T09:25:04Z")
I've been trying to push logs with Logstash into 3-node elasticsearch cluster. I start logstash, logstash logs all fine, but no logs appear in ES. I've 3 nodes, both data and master-eligible nodes, and one among them is … --- ## [How can i start the netflow module through the systemctl](https://discuss.elastic.co/t/how-can-i-start-the-netflow-module-through-the-systemctl/113162)
**Author:** [@antony.y](https://discuss.elastic.co/u/antony.y)\ **Replies:** 11\ **Last updated:** [December 26, 2017, 7:35am UTC](https://discuss.elastic.co/t/how-can-i-start-the-netflow-module-through-the-systemctl/113162 "2017-12-26T07:35:28Z")
Hi all, now i can start my netflow module in my logstash through the command line.But i'd like to start it by systemctl.But i failed. /usr/share/logstash/bin/logstash --modules netflow --setup -M netflow.var.input.udp.p… --- ## [Need help Mutating a Date column from CSV](https://discuss.elastic.co/t/need-help-mutating-a-date-column-from-csv/113093)
**Author:** [@Celtics2018](https://discuss.elastic.co/u/Celtics2018)\ **Replies:** 1\ **Last updated:** [December 26, 2017, 7:18am UTC](https://discuss.elastic.co/t/need-help-mutating-a-date-column-from-csv/113093 "2017-12-26T07:18:34Z")
Hi everyone, I am quite new to using the Stack so I apologize for what may be a quite easy question, I am having trouble figuring this one out! I am loading in a large CSV file into ES and I am mutating some of the fie… --- ## [Can't start the logstash process with different user](https://discuss.elastic.co/t/cant-start-the-logstash-process-with-different-user/113134)
**Author:** [@3a6299333cf24ec3fbbb](https://discuss.elastic.co/u/3a6299333cf24ec3fbbb)\ **Replies:** 1\ **Last updated:** [December 26, 2017, 7:14am UTC](https://discuss.elastic.co/t/cant-start-the-logstash-process-with-different-user/113134 "2017-12-26T07:14:01Z")
When I use root to start the logstash-indexer , it's fine everything. If I use different user, I only can start for about 12 logstash-indexer. I have changed the /etc/security/limits.conf, still have the same issue. T… --- ## [Is plugin id shared across pipelines?](https://discuss.elastic.co/t/is-plugin-id-shared-across-pipelines/113161)
**Author:** [@vb4t](https://discuss.elastic.co/u/vb4t)\ **Replies:** 2\ **Last updated:** [December 26, 2017, 2:28am UTC](https://discuss.elastic.co/t/is-plugin-id-shared-across-pipelines/113161 "2017-12-26T02:28:11Z")
Can I use the same plugin id across different pipelines? So plugin is is logstash unique or per pipeline unique? --- ## [Logstash twiter input plugin error](https://discuss.elastic.co/t/logstash-twiter-input-plugin-error/113150)
**Author:** [@Amouzigh\_Driss](https://discuss.elastic.co/u/Amouzigh_Driss)\ **Replies:** 0\ **Last updated:** [December 25, 2017, 12:09pm UTC](https://discuss.elastic.co/t/logstash-twiter-input-plugin-error/113150 "2017-12-25T12:09:43Z")
Hello guys, I'm using Logstash 6.1.0 and ElasticSearch 6.1.0. I'm using testing logstash twitter input plugin with this configuration. (passwords from twitter dev API end ES configuration). input { twitter { consume… --- ## [How setting about logstash with proxy connect to es cloud?](https://discuss.elastic.co/t/how-setting-about-logstash-with-proxy-connect-to-es-cloud/112346)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 6\ **Last updated:** [December 25, 2017, 6:15am UTC](https://discuss.elastic.co/t/how-setting-about-logstash-with-proxy-connect-to-es-cloud/112346 "2017-12-25T06:15:57Z")
how setting about logstash with proxy connect to es cloud? packetbeat can seding logs to es cloud ? but logstash can't seding system\_log in to es cloud. also use proxy can to es cloud. logstash logs. \[2017-12-19T12… --- ## [Netflow codec application\_id field incorrect translation](https://discuss.elastic.co/t/netflow-codec-application-id-field-incorrect-translation/112967)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [December 25, 2017, 3:05am UTC](https://discuss.elastic.co/t/netflow-codec-application-id-field-incorrect-translation/112967 "2017-12-25T03:05:40Z")
Hi, I've noticed that in case of Fortigate devices the application\_id field is translated incorrectly. By default logstash outputs 20:53068615909376 as the application\_id. However it should look something like 15921. T… --- ## [Document\_type deprecated elasticsearch output logstash](https://discuss.elastic.co/t/document-type-deprecated-elasticsearch-output-logstash/112822)
**Author:** [@Amouzigh\_Driss](https://discuss.elastic.co/u/Amouzigh_Driss)\ **Replies:** 6\ **Last updated:** [December 23, 2017, 8:24pm UTC](https://discuss.elastic.co/t/document-type-deprecated-elasticsearch-output-logstash/112822 "2017-12-23T20:24:31Z")
Hello guys, I'm using logstash 6.1.0. The output section is configured for ElasticSearch as following: output { elasticsearch { hosts =\> "http://localhost:9200" index =\> "Data" document\_type =\> "MyType" } stdou… --- ## [Logstash date parsing error ( date field doesn't have any time)](https://discuss.elastic.co/t/logstash-date-parsing-error-date-field-doesnt-have-any-time/112737)
**Author:** [@Arjun\_subramanyam\_va](https://discuss.elastic.co/u/Arjun_subramanyam_va)\ **Replies:** 4\ **Last updated:** [December 23, 2017, 1:21am UTC](https://discuss.elastic.co/t/logstash-date-parsing-error-date-field-doesnt-have-any-time/112737 "2017-12-23T01:21:18Z")
down vote favorite My data has date in the format yyyy-MM-dd ex : "2015-10-12" My logstash date filter is as below date { match =\> \["start\_date", "yyyy-MM-dd"\] target =\> "start\_date" } However, my es instance is … --- ## [Help with conf file to process log with binary data](https://discuss.elastic.co/t/help-with-conf-file-to-process-log-with-binary-data/110734)
**Author:** [@sconrod](https://discuss.elastic.co/u/sconrod)\ **Replies:** 15\ **Last updated:** [December 22, 2017, 11:06pm UTC](https://discuss.elastic.co/t/help-with-conf-file-to-process-log-with-binary-data/110734 "2017-12-22T23:06:05Z")
Hi I have log files that contains some binary data. For now I just wanted to try to ingest them using stdin and then later build a grok filter. However, it is not ingesting the first log file, no error, it just hangs. M… --- ## [Issue with Logstash Grok and special character in field](https://discuss.elastic.co/t/issue-with-logstash-grok-and-special-character-in-field/112897)
**Author:** [@stef35340](https://discuss.elastic.co/u/stef35340)\ **Replies:** 3\ **Last updated:** [December 22, 2017, 6:39pm UTC](https://discuss.elastic.co/t/issue-with-logstash-grok-and-special-character-in-field/112897 "2017-12-22T18:39:17Z")
Hello, I've an issue that I don't understand why I have it... Here is the example: Log line to parse: 2017-12-21 20:26:29.253;TEST;5236;10792;General;Information;-1; METHODE=EnvoyerDoc.Mlbx\_SendMailSmtpEx RETOUR=0 ME… --- ## [Logstash csv input parsing](https://discuss.elastic.co/t/logstash-csv-input-parsing/113041)
**Author:** [@Amouzigh\_Driss](https://discuss.elastic.co/u/Amouzigh_Driss)\ **Replies:** 0\ **Last updated:** [December 22, 2017, 5:40pm UTC](https://discuss.elastic.co/t/logstash-csv-input-parsing/113041 "2017-12-22T17:40:12Z")
Hello I'm using logstash 6.1.0 to parse csv files and send them to elasticsearch. With elastic 2.1.0 it's working quite fine. But with elastic 6.1.0 I have a logstash error: \[2017-12-22T17:34:04,553\]\[INFO \]\[logstash.… --- ## [Logstash only accepts first event](https://discuss.elastic.co/t/logstash-only-accepts-first-event/113031)
**Author:** [@Timshs](https://discuss.elastic.co/u/Timshs)\ **Replies:** 0\ **Last updated:** [December 22, 2017, 3:46pm UTC](https://discuss.elastic.co/t/logstash-only-accepts-first-event/113031 "2017-12-22T15:46:59Z")
Hi all For some reason my logstash channel locks or stops after first event which comes in fine. After that I have to restart the service and then it accepts the first event again ... and so on ... I have this from pla… --- ## [Logstash input displayed wrong in Kibana](https://discuss.elastic.co/t/logstash-input-displayed-wrong-in-kibana/109650)
**Author:** [@Timshs](https://discuss.elastic.co/u/Timshs)\ **Replies:** 5\ **Last updated:** [December 22, 2017, 3:33pm UTC](https://discuss.elastic.co/t/logstash-input-displayed-wrong-in-kibana/109650 "2017-12-22T15:33:57Z")
I get \\xC0\\t\\xC0\\b\\xC00\\xC0/\\xC0(\\xC0'\\xC0\\u0014\\xC0\\u0013\\xC0\\u0012\\u0000\\x9D\\u0000\\x9C\\u0000=\\u0000\<\\u00005\\u0000/\\u0000 as message in Kibana when sending tcp input to logstash from IOS. My logstash.conf is: input { … --- ## [How can I find the documents causing error messages?](https://discuss.elastic.co/t/how-can-i-find-the-documents-causing-error-messages/113022)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 1\ **Last updated:** [December 22, 2017, 3:25pm UTC](https://discuss.elastic.co/t/how-can-i-find-the-documents-causing-error-messages/113022 "2017-12-22T15:25:21Z")
I'm seeing this error in my Logstash logs: \[2017-12-22T15:16:21,203\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"my-index-2017-0… --- ## [Grok failed to parse simple string and logstash stuck at 100% CPU](https://discuss.elastic.co/t/grok-failed-to-parse-simple-string-and-logstash-stuck-at-100-cpu/112895)
**Author:** [@Vivien](https://discuss.elastic.co/u/Vivien)\ **Replies:** 4\ **Last updated:** [December 22, 2017, 2:34pm UTC](https://discuss.elastic.co/t/grok-failed-to-parse-simple-string-and-logstash-stuck-at-100-cpu/112895 "2017-12-22T14:34:13Z")
Hi! I've got a strange error: \[2017-12-21T20:54:31,450\]\[WARN \]\[logstash.filters.grok \] Timeout executing grok '%{UNIXPATH:path}/(?\[^\]\*)(?\\[-.0-9a-z\]+)\_(?\\[A-Z\]+).logs' against field 'source… --- ## [Elastic search filter plugin is not working](https://discuss.elastic.co/t/elastic-search-filter-plugin-is-not-working/113013)
**Author:** [@Sai\_Jana\_Selvakumar](https://discuss.elastic.co/u/Sai_Jana_Selvakumar)\ **Replies:** 0\ **Last updated:** [December 22, 2017, 2:14pm UTC](https://discuss.elastic.co/t/elastic-search-filter-plugin-is-not-working/113013 "2017-12-22T14:14:09Z")
input { http { host =\> "####" # default: 0.0.0.0 port =\> # # ##default: 8080 } } filter { json { source =\> "message" #remove\_field =\> \[ "message","path","type","host" \] } ruby { code … --- ## [Understanding pipelines v6.x](https://discuss.elastic.co/t/understanding-pipelines-v6-x/112888)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 13\ **Last updated:** [December 22, 2017, 1:54pm UTC](https://discuss.elastic.co/t/understanding-pipelines-v6-x/112888 "2017-12-22T13:54:17Z")
Elasticsearch/Logstash version 6.1.1 When I start logstash I receive the following error with the below settings: I have my logstash.yml with: path.data: /var/lib/logstash path.logs: /var/log/logstash # ------------… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=849) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=851)