# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=727 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 728 --- ## [Tried to pass multiple conf file not working](https://discuss.elastic.co/t/tried-to-pass-multiple-conf-file-not-working/138287)
**Author:** [@Krishna2](https://discuss.elastic.co/u/Krishna2)\ **Replies:** 6\ **Last updated:** [July 6, 2018, 7:00am UTC](https://discuss.elastic.co/t/tried-to-pass-multiple-conf-file-not-working/138287 "2018-07-06T07:00:58Z")
I. had passed as below to Logstash. It is not taking. I. have to pass multiple input files. to Logstash , so I had created multiple conf files as below -------I. passed like below bin/logstash -f logstash-filter\*.con… --- ## [Newbie question -- ELK 6.2, Logstash error status 400](https://discuss.elastic.co/t/newbie-question-elk-6-2-logstash-error-status-400/138286)
**Author:** [@corpengineer](https://discuss.elastic.co/u/corpengineer)\ **Replies:** 11\ **Last updated:** [July 6, 2018, 6:58am UTC](https://discuss.elastic.co/t/newbie-question-elk-6-2-logstash-error-status-400/138286 "2018-07-06T06:58:41Z")
Below is my config for ELK v6.2. Within Kibana, I see a few default fields, in addition to the 'message' field. However, I do not see (what I think that my Grok filter does) the 'message' field parsed out into several … --- ## [Logstash does not work with windows 10](https://discuss.elastic.co/t/logstash-does-not-work-with-windows-10/138812)
**Author:** [@GGUERRA](https://discuss.elastic.co/u/GGUERRA)\ **Replies:** 0\ **Last updated:** [July 6, 2018, 2:16am UTC](https://discuss.elastic.co/t/logstash-does-not-work-with-windows-10/138812 "2018-07-06T02:16:20Z")
Hi everyone, It is very likely that logstash will not work with windows 10. I'm learning to use logstash so I downloaded its latest version and I want to test it on my computer that has windows 10. When I get to the test… --- ## [Abcde](https://discuss.elastic.co/t/abcde/138289)
**Author:** [@GGUERRA](https://discuss.elastic.co/u/GGUERRA)\ **Replies:** 0\ **Last updated:** [July 2, 2018, 11:28pm UTC](https://discuss.elastic.co/t/abcde/138289 "2018-07-02T23:28:17Z")
ABCDE ABCDE --- ## [How can a public field be filtered by a grok regular expression for multiple unordered logs containing public fields?](https://discuss.elastic.co/t/how-can-a-public-field-be-filtered-by-a-grok-regular-expression-for-multiple-unordered-logs-containing-public-fields/138673)
**Author:** [@sun\_changlong](https://discuss.elastic.co/u/sun_changlong)\ **Replies:** 3\ **Last updated:** [July 6, 2018, 1:51am UTC](https://discuss.elastic.co/t/how-can-a-public-field-be-filtered-by-a-grok-regular-expression-for-multiple-unordered-logs-containing-public-fields/138673 "2018-07-06T01:51:20Z")
For example, the following are the contents of the log: label1=aaa ecorder=taw vsid=3 sub\_type=bbbb dst\_addr=1.1.1.1 label2=ttt dst\_addr=1.1.1.1 vsid=3 sub\_type=bbbbtest der=taaa label3=www dst\_addr=1.1.1.1 vsid=3 s… --- ## [How to speed up plugin development cycle](https://discuss.elastic.co/t/how-to-speed-up-plugin-development-cycle/137116)
**Author:** [@ThickFreeze](https://discuss.elastic.co/u/ThickFreeze)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 10:48pm UTC](https://discuss.elastic.co/t/how-to-speed-up-plugin-development-cycle/137116 "2018-07-05T22:48:48Z")
I am developing a custom plugin and the "build, test, fix" cycle is very slow. I have to: Write the code. Build the gem Install the gem Run logstash (which takes quite a while to startup) Is there a way to speed up t… --- ## [Logstash Sourceloader - No configuration found in the configured sources](https://discuss.elastic.co/t/logstash-sourceloader-no-configuration-found-in-the-configured-sources/138441)
**Author:** [@Pixarjunkie](https://discuss.elastic.co/u/Pixarjunkie)\ **Replies:** 9\ **Last updated:** [July 5, 2018, 9:41pm UTC](https://discuss.elastic.co/t/logstash-sourceloader-no-configuration-found-in-the-configured-sources/138441 "2018-07-05T21:41:49Z")
I have a config file that was running alright until this morning. The contents are: input { file { path =\> "C:/Users/data/test.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { … --- ## [Parsing date without a time stamp](https://discuss.elastic.co/t/parsing-date-without-a-time-stamp/138777)
**Author:** [@jspeer](https://discuss.elastic.co/u/jspeer)\ **Replies:** 4\ **Last updated:** [July 5, 2018, 8:20pm UTC](https://discuss.elastic.co/t/parsing-date-without-a-time-stamp/138777 "2018-07-05T20:20:38Z")
We have a field that includes the date with no timestamp. When it's indexed into elastic it's set to UTC time, so the date displayed is always the previous date. Is there a way to set either a filter to convert the date … --- ## [HTTP Poller not working. cgroups error](https://discuss.elastic.co/t/http-poller-not-working-cgroups-error/138787)
**Author:** [@juandanielpujols](https://discuss.elastic.co/u/juandanielpujols)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 7:35pm UTC](https://discuss.elastic.co/t/http-poller-not-working-cgroups-error/138787 "2018-07-05T19:35:45Z")
Running logstash docker. HTTP Poller scheduler not working. Error: \[2018-07-05T19:31:26,816\]\[DEBUG\]\[logstash.instrument.periodicpoller.cgroup\] Error, cannot retrieve cgroups information {:exception=\>"Errno::ENOENT", :m… --- ## [Hyperlink field combining multiple fields in query string](https://discuss.elastic.co/t/hyperlink-field-combining-multiple-fields-in-query-string/138681)
**Author:** [@thegrockq](https://discuss.elastic.co/u/thegrockq)\ **Replies:** 5\ **Last updated:** [July 5, 2018, 5:40pm UTC](https://discuss.elastic.co/t/hyperlink-field-combining-multiple-fields-in-query-string/138681 "2018-07-05T17:40:46Z")
I'm trying to add a hyperlink field to Kibana where the values of multiple fields are used to construct the query string. For example, I need to do something like this: Sample Fields id: 135805885 session: 088015099519… --- ## [Dedicated Logstash Cluster](https://discuss.elastic.co/t/dedicated-logstash-cluster/138565)
**Author:** [@Precision](https://discuss.elastic.co/u/Precision)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 5:19pm UTC](https://discuss.elastic.co/t/dedicated-logstash-cluster/138565 "2018-07-05T17:19:55Z")
Hi there, currently I am using Logstash and the GELF plugin to send logfiles to Graylog. So: Logstash -\> GELF -\> Graylog Basically it works, but sometimes Logstash goes crazy and consumes almost all CPU resources - wh… --- ## [Parsing different date formats (difficult)](https://discuss.elastic.co/t/parsing-different-date-formats-difficult/138638)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 8\ **Last updated:** [July 5, 2018, 5:12pm UTC](https://discuss.elastic.co/t/parsing-different-date-formats-difficult/138638 "2018-07-05T17:12:46Z")
Hi all, I have two types of timestamps coming into my logstash input from different logfiles: \[6/13/18 8:11:25:022 CEST\] 2018-04-17T15:19:20.313 My grok below works for both: if \[fields\]\[log\_type\] == "p8\_server\_err… --- ## [How to replace backslash from a string](https://discuss.elastic.co/t/how-to-replace-backslash-from-a-string/138607)
**Author:** [@praveen\_buya](https://discuss.elastic.co/u/praveen_buya)\ **Replies:** 5\ **Last updated:** [July 5, 2018, 3:04pm UTC](https://discuss.elastic.co/t/how-to-replace-backslash-from-a-string/138607 "2018-07-05T15:04:22Z")
Payload:"Raw Customer Details Stored Successfully." tried to use below filters mutate { gsub =\> \[ "Payload", "\[\\/\]+", "" \] } --- ## [If-then json or grok](https://discuss.elastic.co/t/if-then-json-or-grok/138742)
**Author:** [@randomuserid](https://discuss.elastic.co/u/randomuserid)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 2:28pm UTC](https://discuss.elastic.co/t/if-then-json-or-grok/138742 "2018-07-05T14:28:01Z")
So I am ingesting two json formatted logs plus most of the contents of /var/log including /var/log/nginx/. After some work I can parse the latter two categories of messages well with grok. I can send all of these to logs… --- ## [Value too large to output (365 bytes)](https://discuss.elastic.co/t/value-too-large-to-output-365-bytes/138413)
**Author:** [@Toni\_Riera](https://discuss.elastic.co/u/Toni_Riera)\ **Replies:** 4\ **Last updated:** [July 5, 2018, 2:09pm UTC](https://discuss.elastic.co/t/value-too-large-to-output-365-bytes/138413 "2018-07-05T14:09:15Z")
Good Afternoon, I finally presented my final grade work with ELK, and i'm quite happy of the results. By the way, we're adding new features when it talks to data analysis and painting graphics. One of the problems that… --- ## [Logstash : parse multiple lines, each in one field in a single event](https://discuss.elastic.co/t/logstash-parse-multiple-lines-each-in-one-field-in-a-single-event/138498)
**Author:** [@Jeremie\_Myara](https://discuss.elastic.co/u/Jeremie_Myara)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 8:36am UTC](https://discuss.elastic.co/t/logstash-parse-multiple-lines-each-in-one-field-in-a-single-event/138498 "2018-07-05T08:36:53Z")
Hi everyone, According to this thread :Multiline conf file to parse log file to elasticsearch - #5 by michaellizhou The file input isn't meant to read entire files so you're sort of fighting the system here. I suggest… --- ## [Trouble Parsing Unstructured Logs With Custom Grok Patterns](https://discuss.elastic.co/t/trouble-parsing-unstructured-logs-with-custom-grok-patterns/138380)
**Author:** [@Alper\_Mehmet\_Ozdemir](https://discuss.elastic.co/u/Alper_Mehmet_Ozdemir)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 1:50pm UTC](https://discuss.elastic.co/t/trouble-parsing-unstructured-logs-with-custom-grok-patterns/138380 "2018-07-05T13:50:40Z")
I was given a task to visualize some unstructured logs in a file that contains many logs in different formats. I tried to use the grok filter for parsing but I couldn't quite understand the documentation for custom patte… --- ## [Logstash: creating new fields from windows event log](https://discuss.elastic.co/t/logstash-creating-new-fields-from-windows-event-log/138541)
**Author:** [@solaris](https://discuss.elastic.co/u/solaris)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 1:13pm UTC](https://discuss.elastic.co/t/logstash-creating-new-fields-from-windows-event-log/138541 "2018-07-05T13:13:58Z")
Hi, please don't judge me too harsh. I'm new to ELK and have question regarding creation of new fields from windows log in logstash. Windows logs are forwarded to logstash by winlogbeat plugin. I need extract eventlog … --- ## [Logstash.service: Unit not found (CentOS 7)](https://discuss.elastic.co/t/logstash-service-unit-not-found-centos-7/138446)
**Author:** [@weyderfs](https://discuss.elastic.co/u/weyderfs)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 12:34pm UTC](https://discuss.elastic.co/t/logstash-service-unit-not-found-centos-7/138446 "2018-07-05T12:34:36Z")
Hi everyone, I'm having trouble starting Logstash in my Centos 7. I found several articles on Web, but not found a solution. My configs: Oracle Java 8 $ echo $JAVA\_HOME $ /usr/bin/java $java -version java version … --- ## [Grok failure with Cisco ASA using built-in plugin](https://discuss.elastic.co/t/grok-failure-with-cisco-asa-using-built-in-plugin/138679)
**Author:** [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 9:46am UTC](https://discuss.elastic.co/t/grok-failure-with-cisco-asa-using-built-in-plugin/138679 "2018-07-05T09:46:25Z")
So we have huge number of documents with the tag "\_grokparsefailure" and i think i figured out why. We have Cisco ASA and we are using the following filter: filter { if "syslog" in \[tags\] and "pre-processed" not in \[… --- ## [Logstash hangs on using document\_id while creating index for first time](https://discuss.elastic.co/t/logstash-hangs-on-using-document-id-while-creating-index-for-first-time/138701)
**Author:** [@aashishgahlawat](https://discuss.elastic.co/u/aashishgahlawat)\ **Replies:** 5\ **Last updated:** [July 5, 2018, 11:37am UTC](https://discuss.elastic.co/t/logstash-hangs-on-using-document-id-while-creating-index-for-first-time/138701 "2018-07-05T11:37:06Z")
input { jdbc { jdbc\_driver\_library =\> "sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_connection\_string =\> "jdbc:sqlserver://localhost:1433;databaseName=transact… --- ## [K8s/Docker autodiscover for Kafka plugin?](https://discuss.elastic.co/t/k8s-docker-autodiscover-for-kafka-plugin/138704)
**Author:** [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 11:20am UTC](https://discuss.elastic.co/t/k8s-docker-autodiscover-for-kafka-plugin/138704 "2018-07-05T11:20:00Z")
Possibly a slightly unusual use case here, but we have Kafka instances coming and going, in different K8s namespaces, and there's no obvious way to cope with this (other than writing an input configuration for each possi… --- ## [Mapping based on template not working](https://discuss.elastic.co/t/mapping-based-on-template-not-working/138702)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 11:18am UTC](https://discuss.elastic.co/t/mapping-based-on-template-not-working/138702 "2018-07-05T11:18:00Z")
I am importing some data to elasticsearch after passing it through some lostash pipelinening. I am declaring in the template (snippet below) some fields e.g. as ip and some others as integer "src": {"type": "ip… --- ## [Logstash Elasticsearch Input Plugin no source data](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin-no-source-data/138697)
**Author:** [@stefah](https://discuss.elastic.co/u/stefah)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 10:50am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin-no-source-data/138697 "2018-07-05T10:50:04Z")
Hi, I've encountered the following problem with ES 5.4 and LS 5.4./6.3: I want to read from an elasticsearch cluster with logstash but no document data is returned, just: {"@timestamp":"2018-07-05T10:34:57.049Z","@vers… --- ## [LumberJack - timestamp got added to JSON message](https://discuss.elastic.co/t/lumberjack-timestamp-got-added-to-json-message/138430)
**Author:** [@thomasdang](https://discuss.elastic.co/u/thomasdang)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 9:37am UTC](https://discuss.elastic.co/t/lumberjack-timestamp-got-added-to-json-message/138430 "2018-07-05T09:37:12Z")
Dear Experts: I have filebeat send logs (JSON messages) to the first Logstash server. From the first Logstash server I forward the logs to the second Logstash server using LumberJack. The connectivity is working fine i… --- ## [Logstash not pushing data to AWS Elasticsearch endpoint](https://discuss.elastic.co/t/logstash-not-pushing-data-to-aws-elasticsearch-endpoint/138669)
**Author:** [@Shweta\_Priyadarshani](https://discuss.elastic.co/u/Shweta_Priyadarshani)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 9:34am UTC](https://discuss.elastic.co/t/logstash-not-pushing-data-to-aws-elasticsearch-endpoint/138669 "2018-07-05T09:34:52Z")
Hi all, I am trying to push my logs from logstash to elasticsearch but its failing. here is my logstash.conf file : input { file { path =\> "D:/shweta/ELK\_poc/test3.txt" start\_p… --- ## [Filter \*.tmp](https://discuss.elastic.co/t/filter-tmp/138452)
**Author:** [@Arlen\_Gomes](https://discuss.elastic.co/u/Arlen_Gomes)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 9:30am UTC](https://discuss.elastic.co/t/filter-tmp/138452 "2018-07-05T09:30:12Z")
Dear, I am sending the logs from my file server with winlogbeat. I just send the log with event 4663, to track file deletion, however, the Kibana view shows several temporary file events (\* .tmp). Is it possible to do … --- ## [Extract data from one field and add that in another filed](https://discuss.elastic.co/t/extract-data-from-one-field-and-add-that-in-another-filed/138649)
**Author:** [@Avinash\_Kumar](https://discuss.elastic.co/u/Avinash_Kumar)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 7:16am UTC](https://discuss.elastic.co/t/extract-data-from-one-field-and-add-that-in-another-filed/138649 "2018-07-05T07:16:08Z")
all i have is events with content of log in message field , now i want to add few new fields and add relevant data to that field , how to achieve that? --- ## [Logstash multiple configuration files](https://discuss.elastic.co/t/logstash-multiple-configuration-files/138566)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 4\ **Last updated:** [July 5, 2018, 7:15am UTC](https://discuss.elastic.co/t/logstash-multiple-configuration-files/138566 "2018-07-05T07:15:31Z")
Hi everybody, Do you think it is possible to have several configuration for logstash filters files and load only one according to the filebeat agent version ? Thanks a lot for your help. BR, Khaled --- ## [Filebeat--\>Logstash receiving index issues](https://discuss.elastic.co/t/filebeat-logstash-receiving-index-issues/138472)
**Author:** [@bobbytan](https://discuss.elastic.co/u/bobbytan)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 4:05am UTC](https://discuss.elastic.co/t/filebeat-logstash-receiving-index-issues/138472 "2018-07-04T04:05:35Z")
Hi, After running logstash, the index"filebeatsample" could not be found in elasticsearch and data are not displayed in the stdout within the shell running logstash.Please advice. Filebeat config file content - typ… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=726) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=728)