# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=569 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 570 --- ## [Mapping issues with BRO/Zeek and Logstash](https://discuss.elastic.co/t/mapping-issues-with-bro-zeek-and-logstash/175645)
**Author:** [@KyleC](https://discuss.elastic.co/u/KyleC)\ **Replies:** 1\ **Last updated:** [April 9, 2019, 11:57am UTC](https://discuss.elastic.co/t/mapping-issues-with-bro-zeek-and-logstash/175645 "2019-04-09T11:57:31Z")
Hey folks, I am trying to integrate Bro ids with the elastic stack and keep running into this error message: \[2019-04-05T14:12:05,234\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:s… --- ## [Get and visualize job execution average duration](https://discuss.elastic.co/t/get-and-visualize-job-execution-average-duration/173792)
**Author:** [@kovas6](https://discuss.elastic.co/u/kovas6)\ **Replies:** 4\ **Last updated:** [April 9, 2019, 11:51am UTC](https://discuss.elastic.co/t/get-and-visualize-job-execution-average-duration/173792 "2019-04-09T11:51:37Z")
Hi, I am trying to visualize job execution average duration in hours in Kibana. I query source Duration field from SQL server in Unix format with LogStash filter: filter{ date { match =\> \[ "duration", "UNIX" \] targ… --- ## [Create index with the same name as request path value, using ElasticSearch output](https://discuss.elastic.co/t/create-index-with-the-same-name-as-request-path-value-using-elasticsearch-output/175986)
**Author:** [@Miao](https://discuss.elastic.co/u/Miao)\ **Replies:** 6\ **Last updated:** [April 9, 2019, 11:25am UTC](https://discuss.elastic.co/t/create-index-with-the-same-name-as-request-path-value-using-elasticsearch-output/175986 "2019-04-09T11:25:01Z")
This is my logstash.conf: input { http { host =\> "127.0.0.1" port =\> 31311 } } output { elasticsearch { hosts =\> \["http://localhost:9200"\] } stdout { codec =\> "rubydebug" } } As a test, I r… --- ## [How to indexing to elastic specific tag of xml?](https://discuss.elastic.co/t/how-to-indexing-to-elastic-specific-tag-of-xml/175976)
**Author:** [@mandar.raj](https://discuss.elastic.co/u/mandar.raj)\ **Replies:** 1\ **Last updated:** [April 9, 2019, 11:17am UTC](https://discuss.elastic.co/t/how-to-indexing-to-elastic-specific-tag-of-xml/175976 "2019-04-09T11:17:58Z")
How to indexing to elastic specific tag of xml? I want to index specific tag only from below xml format could you please help me? \ **Author:** [@ipv6ok](https://discuss.elastic.co/u/ipv6ok)\ **Replies:** 0\ **Last updated:** [April 9, 2019, 10:05am UTC](https://discuss.elastic.co/t/multiple-csv-files-input-and-output-abnormal-only-read-first-line/176004 "2019-04-09T10:05:18Z") if path parameter fill a single file : like this: /logdata/data1/warn/StutterAlarm-STB-10min/20190409.csv then Everything is all right but if path parameter fill a Multiple file（about sum 144 files）: like this: … --- ## [Empty message with exec input plugin](https://discuss.elastic.co/t/empty-message-with-exec-input-plugin/172992)
**Author:** [@sab](https://discuss.elastic.co/u/sab)\ **Replies:** 4\ **Last updated:** [April 9, 2019, 9:53am UTC](https://discuss.elastic.co/t/empty-message-with-exec-input-plugin/172992 "2019-04-09T09:53:58Z")
The ls command returns empty message. input { exec { command =\> "ls -l path/to/a/specific/file" interval =\> 30 } } filter : grok { match =\> { "message" =\> "^%{DATA:permission}\\s+%{NUMBER:links\_nb}\\s… --- ## [\[LOGSTASH\] logstash-filter-translate plugin](https://discuss.elastic.co/t/logstash-logstash-filter-translate-plugin/175996)
**Author:** [@Alessio\_Creo](https://discuss.elastic.co/u/Alessio_Creo)\ **Replies:** 0\ **Last updated:** [April 9, 2019, 9:28am UTC](https://discuss.elastic.co/t/logstash-logstash-filter-translate-plugin/175996 "2019-04-09T09:28:39Z")
Hi there, I've inserted in one of my Logstash pipeline the translate filter tha uses a dictionary in a Windows produced CSV file. When I was using the 6.2.4 version of logstash all was working fine. Upgrading to 6.7.0 … --- ## [Need help with creating index pattern for .txt log files](https://discuss.elastic.co/t/need-help-with-creating-index-pattern-for-txt-log-files/174321)
**Author:** [@Kalyan\_MB](https://discuss.elastic.co/u/Kalyan_MB)\ **Replies:** 2\ **Last updated:** [April 9, 2019, 7:56am UTC](https://discuss.elastic.co/t/need-help-with-creating-index-pattern-for-txt-log-files/174321 "2019-04-09T07:56:49Z")
Hello, i have installed elastic search and configured it in RHEL, installed kibana as well. Elasticsearch: elasticsearch-6.7.0 Kibana: kibana-6.7.0-linux-x86\_64 both are in running state, ports 5061(kibana) and 9200(… --- ## [Intellij Plugin to for logstash pipeline files?](https://discuss.elastic.co/t/intellij-plugin-to-for-logstash-pipeline-files/175972)
**Author:** [@holgerbrandl](https://discuss.elastic.co/u/holgerbrandl)\ **Replies:** 0\ **Last updated:** [April 9, 2019, 7:47am UTC](https://discuss.elastic.co/t/intellij-plugin-to-for-logstash-pipeline-files/175972 "2019-04-09T07:47:26Z")
Hi there, is there a plugin for Intellij for logstash pipeline files? The files don't seem to be ruby-lang but something more custom, so having IDE support would be great here. Is there a ticket I could vote for? If n… --- ## [LogStash.conf with multiple outputs: Data displayed on console, but not sent to ElasticSearch server](https://discuss.elastic.co/t/logstash-conf-with-multiple-outputs-data-displayed-on-console-but-not-sent-to-elasticsearch-server/175856)
**Author:** [@Miao](https://discuss.elastic.co/u/Miao)\ **Replies:** 3\ **Last updated:** [April 9, 2019, 7:17am UTC](https://discuss.elastic.co/t/logstash-conf-with-multiple-outputs-data-displayed-on-console-but-not-sent-to-elasticsearch-server/175856 "2019-04-09T07:17:10Z")
Here is my logstash.conf: input { http { host =\> "127.0.0.1" port =\> 31311 } } output { elasticsearch { hosts =\> \["http://localhost:9200"\] } stdout { codec =\> "rubydebug" } } As a test, I ran th… --- ## [How to set output rotating every 15 minutes in logstash?](https://discuss.elastic.co/t/how-to-set-output-rotating-every-15-minutes-in-logstash/175966)
**Author:** [@keep](https://discuss.elastic.co/u/keep)\ **Replies:** 0\ **Last updated:** [April 9, 2019, 7:13am UTC](https://discuss.elastic.co/t/how-to-set-output-rotating-every-15-minutes-in-logstash/175966 "2019-04-09T07:13:39Z")
where are config? how to use rotating in logstast? input { ............. } filter { ............. } output { if (\[IAM\]) { file { path =\> "/logstash-6.5.1/data/%{+yyyyMMddHH}.log" codec =\> line { format =\> "L… --- ## [Repo for Grok Patterns](https://discuss.elastic.co/t/repo-for-grok-patterns/175606)
**Author:** [@tyler\_hilsabeck](https://discuss.elastic.co/u/tyler_hilsabeck)\ **Replies:** 0\ **Last updated:** [April 5, 2019, 1:37pm UTC](https://discuss.elastic.co/t/repo-for-grok-patterns/175606 "2019-04-05T13:37:53Z")
Hi Everyone! I was wondering if there is a central repository for community created grok patterns? I often find really useful patterns around and was wondering if there is place that they are often hosted in one place. … --- ## [How to create a new field using value from another field?](https://discuss.elastic.co/t/how-to-create-a-new-field-using-value-from-another-field/175885)
**Author:** [@Kagerou](https://discuss.elastic.co/u/Kagerou)\ **Replies:** 4\ **Last updated:** [April 8, 2019, 8:10pm UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-using-value-from-another-field/175885 "2019-04-08T20:10:08Z")
I have json data that I'm using the Json filter on to turn into fields. I want to use the elapsed filter so I need the value of one of the fields to act as the start and end tag. Is there any filter that allows you to tu… --- ## [Logstash -- inconsistent result with Date](https://discuss.elastic.co/t/logstash-inconsistent-result-with-date/175904)
**Author:** [@Nsearch](https://discuss.elastic.co/u/Nsearch)\ **Replies:** 6\ **Last updated:** [April 8, 2019, 7:59pm UTC](https://discuss.elastic.co/t/logstash-inconsistent-result-with-date/175904 "2019-04-08T19:59:29Z")
Version: 5.6.3 Operating System: RedHat 7.4 Sample Data: "2019-03-10 02:00:00" will not work. Steps to Reproduce: create a file with the following value: "2019-03-10 02:00:00" Here is a Logstash config. file:: input… --- ## [Unable to update the elasticsearch document properly](https://discuss.elastic.co/t/unable-to-update-the-elasticsearch-document-properly/175897)
**Author:** [@Vijaykumar\_Gundavara](https://discuss.elastic.co/u/Vijaykumar_Gundavara)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 5:41pm UTC](https://discuss.elastic.co/t/unable-to-update-the-elasticsearch-document-properly/175897 "2019-04-08T17:41:48Z")
Hello, I have this config file where I am trying to update an existing document with matching UUID only if the new document has greater timestamp than previous document. However, the field previoustime never seems to be… --- ## [Apache web server logs - best practice?](https://discuss.elastic.co/t/apache-web-server-logs-best-practice/175887)
**Author:** [@Krypto\_1919](https://discuss.elastic.co/u/Krypto_1919)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 4:28pm UTC](https://discuss.elastic.co/t/apache-web-server-logs-best-practice/175887 "2019-04-08T16:28:05Z")
Hi, I'm trying to send my Apache access & error logs to Logstash, and found the various resources a bit confusing as it's all a bit new to me; I'd appreciate some clarifications: Is there a way to use Syslog input pl… --- ## [Extracting data from 3'rd party API](https://discuss.elastic.co/t/extracting-data-from-3rd-party-api/175880)
**Author:** [@drenriza](https://discuss.elastic.co/u/drenriza)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 3:59pm UTC](https://discuss.elastic.co/t/extracting-data-from-3rd-party-api/175880 "2019-04-08T15:59:49Z")
I am tasked with extracting data from a 3'rd party API that return JSON objects. While reading through the documentation for logstash, \*beat and elasticsearch, i read that logstash can't connect to 3'rd party API's (wit… --- ## [Using conditional statement in grok](https://discuss.elastic.co/t/using-conditional-statement-in-grok/175816)
**Author:** [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Replies:** 1\ **Last updated:** [April 8, 2019, 3:07pm UTC](https://discuss.elastic.co/t/using-conditional-statement-in-grok/175816 "2019-04-08T15:07:35Z")
I am having a log which contains some lines which are entirely different from remaining lines. could you please help me with, how can i use conditional statement like if under filter conditions --- ## [Logstash 6: Log message on Could not index event to Elasticsearch exception](https://discuss.elastic.co/t/logstash-6-log-message-on-could-not-index-event-to-elasticsearch-exception/175863)
**Author:** [@filipe.ferreira](https://discuss.elastic.co/u/filipe.ferreira)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 2:54pm UTC](https://discuss.elastic.co/t/logstash-6-log-message-on-could-not-index-event-to-elasticsearch-exception/175863 "2019-04-08T14:54:51Z")
Hi, In logstash 5, when we had a "Could not index event to Elasticsearch", logstash logged the event with the message that caused the exception. Ex: \[2019-04-08T14:47:58,939\]\[WARN \]\[logstash.outputs.elasticsearch\] Cou… --- ## [How :sql\_last\_value works with jdbc\_page\_size?](https://discuss.elastic.co/t/how-sql-last-value-works-with-jdbc-page-size/175861)
**Author:** [@vitaliikobrin](https://discuss.elastic.co/u/vitaliikobrin)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 2:52pm UTC](https://discuss.elastic.co/t/how-sql-last-value-works-with-jdbc-page-size/175861 "2019-04-08T14:52:02Z")
I created logstash config to index data from MySQL DB in Elasticsearch by schedule. Here is my config: \> input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://mysql:3306/smartlikes\_staging" jdbc\_user =\> … --- ## [Logstash - IP range to CIDR](https://discuss.elastic.co/t/logstash-ip-range-to-cidr/175706)
**Author:** [@HaranKumar](https://discuss.elastic.co/u/HaranKumar)\ **Replies:** 8\ **Last updated:** [April 8, 2019, 2:19pm UTC](https://discuss.elastic.co/t/logstash-ip-range-to-cidr/175706 "2019-04-08T14:19:36Z")
Hello Logstash Community, I have parsed a set of IOC which is continuously pulled from TCP port. I have a field called @indicator which has IP values in range format (though it is a single IP value). for example my f… --- ## [Multiple ELK cluster output](https://discuss.elastic.co/t/multiple-elk-cluster-output/175810)
**Author:** [@Gidi\_Kalef](https://discuss.elastic.co/u/Gidi_Kalef)\ **Replies:** 4\ **Last updated:** [April 8, 2019, 2:13pm UTC](https://discuss.elastic.co/t/multiple-elk-cluster-output/175810 "2019-04-08T14:13:51Z")
Hi All, Is it possible to send to two ELK cluster by the following format: output { elasticsearch { hosts=\> \["node1","node2","node3"\] } elasticsearch { hosts=\> \["server1","server2","server3"\] } } Thank you alo… --- ## [Header issue in logstash csv output](https://discuss.elastic.co/t/header-issue-in-logstash-csv-output/175781)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 2\ **Last updated:** [April 8, 2019, 1:51pm UTC](https://discuss.elastic.co/t/header-issue-in-logstash-csv-output/175781 "2019-04-08T13:51:35Z")
hi all, i am using following logstash configuration: input { elasticsearch { hosts =\> \["localhost:9200"\] index =\> "disp\_2019-04-08" } } filter { mutate { add\_field =\> { "P\_date" =\> "%{PDate}" } } mutate { … --- ## [How to integrate multiple config files in Logstash?](https://discuss.elastic.co/t/how-to-integrate-multiple-config-files-in-logstash/174763)
**Author:** [@vignesh.s2](https://discuss.elastic.co/u/vignesh.s2)\ **Replies:** 2\ **Last updated:** [April 8, 2019, 12:53pm UTC](https://discuss.elastic.co/t/how-to-integrate-multiple-config-files-in-logstash/174763 "2019-04-08T12:53:01Z")
As there are 2 config files in Logstash, logs are not received and forwarded to Kibana. I need to know whether i can have more than one conf file for logstash and if so, How to integrate the conf files. I have mentioned… --- ## [Windows logstash setting issue!](https://discuss.elastic.co/t/windows-logstash-setting-issue/175784)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 1\ **Last updated:** [April 8, 2019, 10:44am UTC](https://discuss.elastic.co/t/windows-logstash-setting-issue/175784 "2019-04-08T10:44:37Z")
I have tried this code. my template is creating properly but index are not generating from the file. this is logstash.conf file. i am guessing it is 'sincedb' issue but I am confused. OS : windows 10, ELK version : 6.5.… --- ## [Multiple timestamped log file format](https://discuss.elastic.co/t/multiple-timestamped-log-file-format/174754)
**Author:** [@markov](https://discuss.elastic.co/u/markov)\ **Replies:** 6\ **Last updated:** [April 8, 2019, 10:25am UTC](https://discuss.elastic.co/t/multiple-timestamped-log-file-format/174754 "2019-04-08T10:25:35Z")
Hi, i have two timestamped log file names "server\_log11032019", "app\_log11032019" and in one of those log file there is multiple format for example(app\_log) : 2019-03-11 10:53:55,741 ERROR \[com.website.classname\] Error\_… --- ## [Http\_poller certificate error](https://discuss.elastic.co/t/http-poller-certificate-error/175232)
**Author:** [@HaranKumar](https://discuss.elastic.co/u/HaranKumar)\ **Replies:** 9\ **Last updated:** [April 8, 2019, 9:15am UTC](https://discuss.elastic.co/t/http-poller-certificate-error/175232 "2019-04-08T09:15:12Z")
Hello Team, I have a URL to be used as input of logstash and output to be saved as a file. I am getting no data in my output file if i use CSV and getting htto\_request failure message as data if i use json. kindly hel… --- ## [How logstash takes new file and sends it to elasticsearch](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770)
**Author:** [@Hari\_Krishnan](https://discuss.elastic.co/u/Hari_Krishnan)\ **Replies:** 3\ **Last updated:** [April 8, 2019, 8:56am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770 "2019-04-08T08:56:09Z")
Hi , I have a script to create a file that logstash takes as input, and i configured logstash with path for the input file like this ./logs/\*/allfiles\_decode.json My concern is , for input file creation my script take… --- ## [Parsing jenkins config.xml using logstash](https://discuss.elastic.co/t/parsing-jenkins-config-xml-using-logstash/175764)
**Author:** [@sowndharya.s](https://discuss.elastic.co/u/sowndharya.s)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 6:34am UTC](https://discuss.elastic.co/t/parsing-jenkins-config-xml-using-logstash/175764 "2019-04-08T06:34:00Z")
Hi All, Can u give me some idea how to parse jenkins config.xml file using logstash. --- ## [How can i migrate multiple tables from mysql to elasticsearch with realtime schedule with Logstash?](https://discuss.elastic.co/t/how-can-i-migrate-multiple-tables-from-mysql-to-elasticsearch-with-realtime-schedule-with-logstash/175751)
**Author:** [@kietruong](https://discuss.elastic.co/u/kietruong)\ **Replies:** 0\ **Last updated:** [April 8, 2019, 4:43am UTC](https://discuss.elastic.co/t/how-can-i-migrate-multiple-tables-from-mysql-to-elasticsearch-with-realtime-schedule-with-logstash/175751 "2019-04-08T04:43:30Z")
error here is my logstash.conf file i duplicates it for 2 tables [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=568) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=570)