# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=477 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 478 --- ## [Logstash 6.6.1 Crashing](https://discuss.elastic.co/t/logstash-6-6-1-crashing/197103)
**Author:** [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Replies:** 2\ **Last updated:** [September 16, 2019, 5:23am UTC](https://discuss.elastic.co/t/logstash-6-6-1-crashing/197103 "2019-09-16T05:23:05Z")
I have a regular problem of logstash crashing with FATAL on Linux RHEL 6.9. The logstash version is 6.6.1. The pipeline.yml contains following config : pipeline.id: sys\_username\_mapping pipeline.workers: 1 path.conf… --- ## [Is there a way to remove duplicate code in logstash?](https://discuss.elastic.co/t/is-there-a-way-to-remove-duplicate-code-in-logstash/199569)
**Author:** [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\ **Replies:** 1\ **Last updated:** [September 16, 2019, 5:21am UTC](https://discuss.elastic.co/t/is-there-a-way-to-remove-duplicate-code-in-logstash/199569 "2019-09-16T05:21:41Z")
Can I include duplicate code in a specific file in the log? For example, when importing data from multiple db tables, you need to keep writing jdbc { jdbc\_driver\_library =\> "mysql-connector-java-5.1.36-bin.jar" jdbc\_… --- ## [Https://discuss.elastic.co/t/feed-lost-data-to-elk/198997](https://discuss.elastic.co/t/https-discuss-elastic-co-t-feed-lost-data-to-elk-198997/199561)
**Author:** [@Rajith\_konara](https://discuss.elastic.co/u/Rajith_konara)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 3:43am UTC](https://discuss.elastic.co/t/https-discuss-elastic-co-t-feed-lost-data-to-elk-198997/199561 "2019-09-16T03:43:20Z")
Continuing the discussion from Feed lost data to ELK: --- ## [One elasticsearch to input from other elasticsearches](https://discuss.elastic.co/t/one-elasticsearch-to-input-from-other-elasticsearches/199540)
**Author:** [@Marta\_Zagrajek](https://discuss.elastic.co/u/Marta_Zagrajek)\ **Replies:** 1\ **Last updated:** [September 15, 2019, 8:04pm UTC](https://discuss.elastic.co/t/one-elasticsearch-to-input-from-other-elasticsearches/199540 "2019-09-15T20:04:53Z")
Hi there :slight\_smile: I want to create one big elastichsearch cluster which will combine logs from other elasticsearch clusters (even up to 10). Is there a possibility to do that using logstash? My further explanati… --- ## [Basic grok query - logstash](https://discuss.elastic.co/t/basic-grok-query-logstash/199520)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 5\ **Last updated:** [September 15, 2019, 3:24pm UTC](https://discuss.elastic.co/t/basic-grok-query-logstash/199520 "2019-09-15T15:24:34Z")
Hi Team, I am still learning grok and have certain basic query which I tried searching but could not find the perfect answer hence asking to this community. My apologies if that was already asked. What is \[type\] in fil… --- ## [How to run conf file in two different machines/nodes?](https://discuss.elastic.co/t/how-to-run-conf-file-in-two-different-machines-nodes/199179)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 7\ **Last updated:** [September 15, 2019, 3:22pm UTC](https://discuss.elastic.co/t/how-to-run-conf-file-in-two-different-machines-nodes/199179 "2019-09-15T15:22:17Z")
Hi All I have two conf file confile1.conf and confile2.conf The confile1.conf is sitting on Machine 1 The confile2.conf is sitting on Machine 2 Now i want to operate everything from Machine 1 From Machine 1 , i am … --- ## [Can't upload CSV file to elastic search via logstash](https://discuss.elastic.co/t/cant-upload-csv-file-to-elastic-search-via-logstash/199361)
**Author:** [@vedansh](https://discuss.elastic.co/u/vedansh)\ **Replies:** 2\ **Last updated:** [September 15, 2019, 1:14pm UTC](https://discuss.elastic.co/t/cant-upload-csv-file-to-elastic-search-via-logstash/199361 "2019-09-15T13:14:04Z")
logstash-car.conf I will go through all the question related to this topic but didn't find the solution yet input { file { path =\> "C:\\Users\\vedansh.p\\Desktop\\ELK\\CSVLogsDemo\\cars.csv" start\_position =\> "beginning" … --- ## [Logstash-output-mongodb plugin not support login authentication](https://discuss.elastic.co/t/logstash-output-mongodb-plugin-not-support-login-authentication/199535)
**Author:** [@wys1010](https://discuss.elastic.co/u/wys1010)\ **Replies:** 0\ **Last updated:** [September 15, 2019, 12:38pm UTC](https://discuss.elastic.co/t/logstash-output-mongodb-plugin-not-support-login-authentication/199535 "2019-09-15T12:38:45Z")
Why does the \[logstash-output-mongodb\] plugin not support login authentication? I hope to add this feature, thanks! --- ## [Enrich message](https://discuss.elastic.co/t/enrich-message/199514)
**Author:** [@sparmar](https://discuss.elastic.co/u/sparmar)\ **Replies:** 2\ **Last updated:** [September 15, 2019, 12:25pm UTC](https://discuss.elastic.co/t/enrich-message/199514 "2019-09-15T12:25:42Z")
I have a java application that is writing out message as follows for example: 00:00:13,950 INFO \[com.companyurl.application\] (bq-thread-pool-threads - 881) ABC response by ApplicationManager - received \ **Author:** [@kolli\_dilip](https://discuss.elastic.co/u/kolli_dilip)\ **Replies:** 6\ **Last updated:** [August 22, 2019, 3:12pm UTC](https://discuss.elastic.co/t/add-a-new-filed-in-csv-based-on-a-value-in-colum-and-cal-the-time-diff/196117 "2019-08-22T15:12:17Z") Hi all I have a set of csv data as below i am interested in adding a new field 'status' based on the 'endtime' column value, i.e if its not 'null' then the status should be completed else open. also need to calculat… --- ## [Calculate time difference considering only office hours and ignoring the weekend in logstash](https://discuss.elastic.co/t/calculate-time-difference-considering-only-office-hours-and-ignoring-the-weekend-in-logstash/199527)
**Author:** [@kolli\_dilip](https://discuss.elastic.co/u/kolli_dilip)\ **Replies:** 0\ **Last updated:** [September 15, 2019, 9:56am UTC](https://discuss.elastic.co/t/calculate-time-difference-considering-only-office-hours-and-ignoring-the-weekend-in-logstash/199527 "2019-09-15T09:56:11Z")
can we calculate difference of time ignoring the weekdays... like i should not take the time during the weekend and the time beyond the office hours example consider the time between 08.00 to 18.00 ... And only weekdays… --- ## [Logstash not parser correctly with dissect](https://discuss.elastic.co/t/logstash-not-parser-correctly-with-dissect/199511)
**Author:** [@McFly](https://discuss.elastic.co/u/McFly)\ **Replies:** 1\ **Last updated:** [September 15, 2019, 9:43am UTC](https://discuss.elastic.co/t/logstash-not-parser-correctly-with-dissect/199511 "2019-09-15T09:43:05Z")
Hello everyone! how are they? I need to consult them. I am parsing the logs of a service, the following come this way: |@ TIMESTAMP || DATE1 || DATE2 || DATE3 || DATE4 || DATE5 || DATE6 @| Howeve… --- ## [How do I combine 2 index join data in elasticsearch?](https://discuss.elastic.co/t/how-do-i-combine-2-index-join-data-in-elasticsearch/199202)
**Author:** [@mr\_searchng](https://discuss.elastic.co/u/mr_searchng)\ **Replies:** 1\ **Last updated:** [September 14, 2019, 10:31am UTC](https://discuss.elastic.co/t/how-do-i-combine-2-index-join-data-in-elasticsearch/199202 "2019-09-14T10:31:21Z")
that's exactly what I want. My purpose was this. In elasticsearch, solve this with a code structure that does this --- ## [Logstash timestamp appearing as text in Elasticsearch](https://discuss.elastic.co/t/logstash-timestamp-appearing-as-text-in-elasticsearch/199239)
**Author:** [@khat33b](https://discuss.elastic.co/u/khat33b)\ **Replies:** 1\ **Last updated:** [September 14, 2019, 10:29am UTC](https://discuss.elastic.co/t/logstash-timestamp-appearing-as-text-in-elasticsearch/199239 "2019-09-14T10:29:14Z")
I am using Elasticsearch 7.3.1 and Logstash 7.3.1. I am trying to make a field of mine as the Elasticsearch timestamp using the date filter. The data is being inserted properly but the type of @timestamp is coming text. … --- ## [Hi All, Please help to resolve this issue. Due to line break after "Command Line : test" in the log data is not coming to single object](https://discuss.elastic.co/t/hi-all-please-help-to-resolve-this-issue-due-to-line-break-after-command-line-test-in-the-log-data-is-not-coming-to-single-object/199293)
**Author:** [@praveena77](https://discuss.elastic.co/u/praveena77)\ **Replies:** 1\ **Last updated:** [September 14, 2019, 10:22am UTC](https://discuss.elastic.co/t/hi-all-please-help-to-resolve-this-issue-due-to-line-break-after-command-line-test-in-the-log-data-is-not-coming-to-single-object/199293 "2019-09-14T10:22:37Z")
log data Job Id : 181202632 Job Name : yutrw Batch Id : 181202632 Command Line : test Execution User : WINNTDOM Execution Queue : General output { Job Nam… --- ## [The index pattern syslog-pfsense-\* does not contain any of the following compatible field types: geo\_point](https://discuss.elastic.co/t/the-index-pattern-syslog-pfsense-does-not-contain-any-of-the-following-compatible-field-types-geo-point/199491)
**Author:** [@nghia\_huynh](https://discuss.elastic.co/u/nghia_huynh)\ **Replies:** 1\ **Last updated:** [September 14, 2019, 5:25am UTC](https://discuss.elastic.co/t/the-index-pattern-syslog-pfsense-does-not-contain-any-of-the-following-compatible-field-types-geo-point/199491 "2019-09-14T05:25:49Z")
Dear ELK Team, Today I have an issue when I try to create Coordinate Map. This error is described in below. And then, I checked parsing rule config like that: Please help me to fix this error in other to create ma… --- ## [How can merge logs based on the multi parameters](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 4\ **Last updated:** [September 14, 2019, 4:19am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494 "2019-09-14T04:19:43Z")
hi all I have a question, how can merge multi lines of logs based on the multi parameters into one event?. for example, log is as following: IT=1 AC=12345 p1=12345 IW=1 AC=12345 A1=check1 IW=1 E1=0 IW=2 AC=12345 A1=c… --- ## [Logstash 6.8 or 7x comaptibility with Kafka 0.9](https://discuss.elastic.co/t/logstash-6-8-or-7x-comaptibility-with-kafka-0-9/199482)
**Author:** [@stecino](https://discuss.elastic.co/u/stecino)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 9:31pm UTC](https://discuss.elastic.co/t/logstash-6-8-or-7x-comaptibility-with-kafka-0-9/199482 "2019-09-13T21:31:06Z")
Hello, Has anyone tried to read topics in Kafka 0.9 using these logstash versions? I am currently using Logstash 7.3.2 which supposedly connects to it, but I am not seeing any consumers in kafka manager being connected… --- ## [A Failed Logstash pipeline is causing other pipelines to fail!](https://discuss.elastic.co/t/a-failed-logstash-pipeline-is-causing-other-pipelines-to-fail/199275)
**Author:** [@baselai](https://discuss.elastic.co/u/baselai)\ **Replies:** 3\ **Last updated:** [September 13, 2019, 3:32pm UTC](https://discuss.elastic.co/t/a-failed-logstash-pipeline-is-causing-other-pipelines-to-fail/199275 "2019-09-13T15:32:57Z")
I have Logstash V 7.3.0 running with 4 pipelines: pipeline\_a.conf pipeline\_b.conf pipeline\_c.conf pipeline\_d.conf and they're all in the same folder. When pipeline\_a.conf is failing for some reason, which it's no… --- ## [Difference between dissect filter and dissect processor](https://discuss.elastic.co/t/difference-between-dissect-filter-and-dissect-processor/199437)
**Author:** [@sahishnu.patil](https://discuss.elastic.co/u/sahishnu.patil)\ **Replies:** 1\ **Last updated:** [September 13, 2019, 2:50pm UTC](https://discuss.elastic.co/t/difference-between-dissect-filter-and-dissect-processor/199437 "2019-09-13T14:50:37Z")
What is the actual difference between dissect filter and dissect processor and they both seem to have same functionality and different syntax. Syntax for dissect filter plugin . filter { dissect { mapping =\> { "mess… --- ## [Pipeline-to-pipeline communication - will passing variables work?](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-will-passing-variables-work/199399)
**Author:** [@nickbabkin](https://discuss.elastic.co/u/nickbabkin)\ **Replies:** 1\ **Last updated:** [September 13, 2019, 1:02pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-will-passing-variables-work/199399 "2019-09-13T13:02:35Z")
Hi! We're designing our logstash configuration and trying to setup a flow when a pipeline receives events, then, based on the content of the event, assigns codec and destination file values (by adding custom fields), th… --- ## [Logstash Holding on to Deleted File Handles](https://discuss.elastic.co/t/logstash-holding-on-to-deleted-file-handles/199426)
**Author:** [@dlahn](https://discuss.elastic.co/u/dlahn)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 12:50pm UTC](https://discuss.elastic.co/t/logstash-holding-on-to-deleted-file-handles/199426 "2019-09-13T12:50:30Z")
We are currently using Logstash and the input file plugin to ship logs from Docker to Elasticsearch. We are using Logstash 7.1 and the latest version of the plugin. The problem we are running into is that after about 7-… --- ## [Get EventLogFile - Salesforce input plugin](https://discuss.elastic.co/t/get-eventlogfile-salesforce-input-plugin/199401)
**Author:** [@Nelson\_Nunes](https://discuss.elastic.co/u/Nelson_Nunes)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 9:31am UTC](https://discuss.elastic.co/t/get-eventlogfile-salesforce-input-plugin/199401 "2019-09-13T09:31:59Z")
Hi, input { salesforce { client\_id =\> 'XXXXXXXXXXXXXrEIKI89zOgB8e8Uo9PrZk..9jFs' client\_secret =\> '7C4D2BXXXXXXXXXXXXX' username =\> 'XXXXXXXXXXXXX@test-pkxswlXXXXXXXXXXXXX' password =\> 'XXXXXXXXXXXXX' security\_tok… --- ## [Logstash input kafka not reading messages from topic](https://discuss.elastic.co/t/logstash-input-kafka-not-reading-messages-from-topic/199395)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [September 13, 2019, 9:30am UTC](https://discuss.elastic.co/t/logstash-input-kafka-not-reading-messages-from-topic/199395 "2019-09-13T09:30:43Z")
I am using kafka as message broker and writing and reading messages using logstash-6.3.1. Attached document shows all the configuration. I have producer pipeline, which creates messages in kafka and consumr pipeline w… --- ## [4624 log stoms](https://discuss.elastic.co/t/4624-log-stoms/195684)
**Author:** [@james\_007](https://discuss.elastic.co/u/james_007)\ **Replies:** 10\ **Last updated:** [September 13, 2019, 8:00am UTC](https://discuss.elastic.co/t/4624-log-stoms/195684 "2019-09-13T08:00:16Z")
Hi All - was wondering if anyone can help on the following: event ID 4624 : this event logs everything that speaks to the domain, I just want to log user who below to the DD1 domain and forget and drop the rest of the e… --- ## [Gem file Error](https://discuss.elastic.co/t/gem-file-error/199379)
**Author:** [@Vineeth\_Ananthula1](https://discuss.elastic.co/u/Vineeth_Ananthula1)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 7:25am UTC](https://discuss.elastic.co/t/gem-file-error/199379 "2019-09-13T07:25:32Z")
We are unable to start logstash due to gemfile error \[ERROR\] 2019-09-13 12:52:22.884 \[main\] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (GemNotFound) Could not find publi… --- ## [How do we know logstash is reading from dead letter queue or not](https://discuss.elastic.co/t/how-do-we-know-logstash-is-reading-from-dead-letter-queue-or-not/199378)
**Author:** [@yestoprince](https://discuss.elastic.co/u/yestoprince)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 7:24am UTC](https://discuss.elastic.co/t/how-do-we-know-logstash-is-reading-from-dead-letter-queue-or-not/199378 "2019-09-13T07:24:40Z")
Hi, I have added a input pipeline to read from dead queue then i restart the logstash and there is no error in logs but don't know if it is reading from dead queue or not. by the way there is no change in the size of dea… --- ## [Specifying template using Order](https://discuss.elastic.co/t/specifying-template-using-order/199376)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 7:05am UTC](https://discuss.elastic.co/t/specifying-template-using-order/199376 "2019-09-13T07:05:14Z")
Hi, When specifying the template file in logstash is it possible to utilise multiple template files using the Order functionality within the templates, i was looking to build a base template then branch that off with ex… --- ## [Desired output not achieved while logs parsing](https://discuss.elastic.co/t/desired-output-not-achieved-while-logs-parsing/199357)
**Author:** [@gaurav.itsoft](https://discuss.elastic.co/u/gaurav.itsoft)\ **Replies:** 0\ **Last updated:** [September 13, 2019, 5:02am UTC](https://discuss.elastic.co/t/desired-output-not-achieved-while-logs-parsing/199357 "2019-09-13T05:02:30Z")
I have system log file for one of my application having pattern like below , i want to capture only the lines having sql exception like: +java.sql.SQLException: ORA-\*\*\* to be shown in kibana while rest of log lines ar… --- ## [Logstash not parsing default nginx logs](https://discuss.elastic.co/t/logstash-not-parsing-default-nginx-logs/199313)
**Author:** [@Kushal\_Das](https://discuss.elastic.co/u/Kushal_Das)\ **Replies:** 1\ **Last updated:** [September 13, 2019, 1:59am UTC](https://discuss.elastic.co/t/logstash-not-parsing-default-nginx-logs/199313 "2019-09-13T01:59:27Z")
Hi, I am using the latest 7.x series of the stack. The filter pattern is the same for the default nginx. But, I can not find any of the nginx.\* fields, and instead can see the message inside of the \_source field. Any … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=476) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=478)