# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=805 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 806 --- ## [Extract Data from message to display each field as a column in kibana](https://discuss.elastic.co/t/extract-data-from-message-to-display-each-field-as-a-column-in-kibana/122924)
**Author:** [@Prabh\_Natt](https://discuss.elastic.co/u/Prabh_Natt)\ **Replies:** 9\ **Last updated:** [March 9, 2018, 9:56pm UTC](https://discuss.elastic.co/t/extract-data-from-message-to-display-each-field-as-a-column-in-kibana/122924 "2018-03-09T21:56:06Z")
I want to be able to extract the fields i need from message and be able to select them as their own fields, as well as index everything dynamically using the "clientCode". I have been working on this for the past couple … --- ## [Multiple events in the same input XML](https://discuss.elastic.co/t/multiple-events-in-the-same-input-xml/123325)
**Author:** [@S\_T](https://discuss.elastic.co/u/S_T)\ **Replies:** 8\ **Last updated:** [March 9, 2018, 9:49pm UTC](https://discuss.elastic.co/t/multiple-events-in-the-same-input-xml/123325 "2018-03-09T21:49:26Z")
After countless hours of trying, I'm about to give up and I'm posting here as a final shot. My goal is to index a XML file that contains multiple events on the same line, and I'm using Logstash to pre-process the XML fi… --- ## [Setting up logstash config file question](https://discuss.elastic.co/t/setting-up-logstash-config-file-question/123170)
**Author:** [@jason\_smith](https://discuss.elastic.co/u/jason_smith)\ **Replies:** 15\ **Last updated:** [March 9, 2018, 7:12pm UTC](https://discuss.elastic.co/t/setting-up-logstash-config-file-question/123170 "2018-03-09T19:12:59Z")
Hello Good folks, New to ELK stack. I have setup Elasticsearch and Kibanan on my windows 8 box and practice index data into elasticsearch from Kibana. Now I want to index data thru Logstash. I downloaded logstash (ZIP … --- ## [Logstash Date filter ISO8601 Issue](https://discuss.elastic.co/t/logstash-date-filter-iso8601-issue/122938)
**Author:** [@RichardLaing](https://discuss.elastic.co/u/RichardLaing)\ **Replies:** 2\ **Last updated:** [March 9, 2018, 7:08pm UTC](https://discuss.elastic.co/t/logstash-date-filter-iso8601-issue/122938 "2018-03-09T19:08:29Z")
Hi there I am having some trouble with the date creation for my logs when they enter logstash, the end goal is to have the @timestamp field match the timestamp for the Auth or Syslog as thy are processed. These are in hi… --- ## [Logstash grok OSSEC](https://discuss.elastic.co/t/logstash-grok-ossec/120188)
**Author:** [@Jose\_Campos](https://discuss.elastic.co/u/Jose_Campos)\ **Replies:** 2\ **Last updated:** [March 9, 2018, 5:24pm UTC](https://discuss.elastic.co/t/logstash-grok-ossec/120188 "2018-03-09T17:24:17Z")
Hello A few days ago I was trying to make a grok to get the values ​​of a specific field. For example, within my text lines I want to extract the data from 'Source Network Address' Process Information: Process ID: 0x0 … --- ## [Overwrite doesn't happen when message empty](https://discuss.elastic.co/t/overwrite-doesnt-happen-when-message-empty/123095)
**Author:** [@huyouiqq](https://discuss.elastic.co/u/huyouiqq)\ **Replies:** 1\ **Last updated:** [March 9, 2018, 4:15pm UTC](https://discuss.elastic.co/t/overwrite-doesnt-happen-when-message-empty/123095 "2018-03-09T16:15:56Z")
logstash 5.3.0 filter { grok { patterns\_dir =\> \["/etc/logstash/patterns"\] match =\> \[ "message", "%{NGINXACCESS} %{GREEDYDATA:message}", "message", "%{NGINXACCESSAUTH}%{GREEDYDATA:message}", "message", "%{NGINXERRO… --- ## [Csv columnnames utf-8](https://discuss.elastic.co/t/csv-columnnames-utf-8/122065)
**Author:** [@TclasenITVT](https://discuss.elastic.co/u/TclasenITVT)\ **Replies:** 1\ **Last updated:** [March 9, 2018, 2:21pm UTC](https://discuss.elastic.co/t/csv-columnnames-utf-8/122065 "2018-03-09T14:21:49Z")
I am trying to use the CSV Filter in my Logstashpipeline. When I try to specify Columnnames that contain ü, ä, § or € the JSon that is display by Kibana shows something like "Anzahl Monate (für NN GP abs. €)". Is ther… --- ## [Crash of logstash through KV plugin](https://discuss.elastic.co/t/crash-of-logstash-through-kv-plugin/122186)
**Author:** [@Jeff\_Bolle](https://discuss.elastic.co/u/Jeff_Bolle)\ **Replies:** 2\ **Last updated:** [March 9, 2018, 1:59pm UTC](https://discuss.elastic.co/t/crash-of-logstash-through-kv-plugin/122186 "2018-03-09T13:59:02Z")
I'm running logstash 6.1.3 and have been intermittently getting an Array index out of bounds. Nothing turns up in the logs, and as such I'm not able to provide exact data that can replicate it. Also, its intermittent. … --- ## [Obtain document ID just after insert](https://discuss.elastic.co/t/obtain-document-id-just-after-insert/123262)
**Author:** [@panchicore](https://discuss.elastic.co/u/panchicore)\ **Replies:** 1\ **Last updated:** [March 9, 2018, 1:18pm UTC](https://discuss.elastic.co/t/obtain-document-id-just-after-insert/123262 "2018-03-09T13:18:58Z")
The idea is to send the record to ES using the output filter, this works great, but there is posibility to obtain the document id after to continue with other enrichment pipeline triggered by, for example a kafka output?… --- ## [Grok filter for parsing SSL ciphers?](https://discuss.elastic.co/t/grok-filter-for-parsing-ssl-ciphers/123266)
**Author:** [@burnersk](https://discuss.elastic.co/u/burnersk)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 1:14pm UTC](https://discuss.elastic.co/t/grok-filter-for-parsing-ssl-ciphers/123266 "2018-03-09T13:14:01Z")
Has anyone already created a Grok filter for extracting key exchange, encryption, signature and integrity information out of the OpenSSL cipher? For example, I have the following raw information "ECDHE-RSA-CHACHA20-POLY… --- ## [Changing Date Format in logstash](https://discuss.elastic.co/t/changing-date-format-in-logstash/123206)
**Author:** [@Janardhan\_Randhi](https://discuss.elastic.co/u/Janardhan_Randhi)\ **Replies:** 3\ **Last updated:** [March 9, 2018, 1:04pm UTC](https://discuss.elastic.co/t/changing-date-format-in-logstash/123206 "2018-03-09T13:04:35Z")
How can i delete the t and z date format in elastic search using through logstash. my input is :2017-05-08 07:27:03.121 after push into elastic search its shows like 2017-05-08T07:27:03.121Z format ,how can i remove … --- ## [Aggregate Filter exception for Oracle Query](https://discuss.elastic.co/t/aggregate-filter-exception-for-oracle-query/123261)
**Author:** [@vigneshaj](https://discuss.elastic.co/u/vigneshaj)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 12:51pm UTC](https://discuss.elastic.co/t/aggregate-filter-exception-for-oracle-query/123261 "2018-03-09T12:51:38Z")
Hi Team I am applying an aggreagate filter to the simple oracle query and getting below error. We have multiple names for a single ID and I am trying to create something like below. Kindly find error got and yml used … --- ## [DNS request timed out - logstash filter](https://discuss.elastic.co/t/dns-request-timed-out-logstash-filter/123221)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 2\ **Last updated:** [March 9, 2018, 11:46am UTC](https://discuss.elastic.co/t/dns-request-timed-out-logstash-filter/123221 "2018-03-09T11:46:52Z")
Hi All, Hi there, This with regards to DNS request time out in logstash filter for cisco firewall logs, after restarting the logstash service it will work for one day and next day i get the logstash stops parsing data … --- ## [Replacing Special Characters](https://discuss.elastic.co/t/replacing-special-characters/123254)
**Author:** [@phillipgibb](https://discuss.elastic.co/u/phillipgibb)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 11:26am UTC](https://discuss.elastic.co/t/replacing-special-characters/123254 "2018-03-09T11:26:30Z")
Hi, I have a filter with: mutate { gsub =\> \[ "message", "\\u003e", "\>", "message", "\\u003c", "\<", "message", "\[\\n\]", "", "message", "\\"", "'" \] } that I apply to : {"log":"screen=None callback\_url='/api' reques… --- ## [Combine fields inside aggregation filter](https://discuss.elastic.co/t/combine-fields-inside-aggregation-filter/123048)
**Author:** [@daniender](https://discuss.elastic.co/u/daniender)\ **Replies:** 4\ **Last updated:** [March 9, 2018, 10:24am UTC](https://discuss.elastic.co/t/combine-fields-inside-aggregation-filter/123048 "2018-03-09T10:24:44Z")
Hi all. I've been googling this for a couple of days and have not found any help, although I think this problem should be very easy to solve. I'm using Logstash to import data from MySQL database and export it to Elast… --- ## [Using expression in Elastic search](https://discuss.elastic.co/t/using-expression-in-elastic-search/121428)
**Author:** [@bismayam](https://discuss.elastic.co/u/bismayam)\ **Replies:** 1\ **Last updated:** [February 26, 2018, 8:23am UTC](https://discuss.elastic.co/t/using-expression-in-elastic-search/121428 "2018-02-26T08:23:10Z")
We have logs like below. 2018-02-24 18:24:21 GET /test/customer/api/abc/getCustomer 2018-02-24 18:24:21 GET /test/customer/10000 2018-02-24 18:24:21 GET /test/customer/10000/updateCustomer We are trying to p… --- ## [Netflow and type long (fwd\_flow\_delta\_bytes)](https://discuss.elastic.co/t/netflow-and-type-long-fwd-flow-delta-bytes/122716)
**Author:** [@adankert](https://discuss.elastic.co/u/adankert)\ **Replies:** 3\ **Last updated:** [March 9, 2018, 9:56am UTC](https://discuss.elastic.co/t/netflow-and-type-long-fwd-flow-delta-bytes/122716 "2018-03-09T09:56:43Z")
I have an issue where data passed from logstash to es is to big for type long: Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"elastiflow-2018.03.06", :\_type=\>"doc", :\_ro… --- ## [Split string into array](https://discuss.elastic.co/t/split-string-into-array/123079)
**Author:** [@Lycan](https://discuss.elastic.co/u/Lycan)\ **Replies:** 10\ **Last updated:** [March 9, 2018, 9:47am UTC](https://discuss.elastic.co/t/split-string-into-array/123079 "2018-03-09T09:47:07Z")
Hi. What I'm trying to do is accessing a nested string in "doc" called "message" (doc.message?), and this string contains a couple of things, including a case specification. That specification is what I am trying to ext… --- ## [Several tables in from](https://discuss.elastic.co/t/several-tables-in-from/123105)
**Author:** [@Amandine](https://discuss.elastic.co/u/Amandine)\ **Replies:** 4\ **Last updated:** [March 9, 2018, 8:47am UTC](https://discuss.elastic.co/t/several-tables-in-from/123105 "2018-03-09T08:47:47Z")
Hello, I have a JDBC driver with SQL Server ELK ... I have an error message with a syntax problem with the FROM when doing a ./logstash -f /etc/logstash/conf.d/file.conf. I want to add several tables in the same FROM. Th… --- ## [Null objets after mapping input from MySQL](https://discuss.elastic.co/t/null-objets-after-mapping-input-from-mysql/123222)
**Author:** [@daniender](https://discuss.elastic.co/u/daniender)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 8:37am UTC](https://discuss.elastic.co/t/null-objets-after-mapping-input-from-mysql/123222 "2018-03-09T08:37:49Z")
Hi all. I have a working config file for getting a list of objects from MySQL database and inserting them on elastic. But I have a weir behaviour and I can not figure out the reason. I receive a list of 'ticket' objec… --- ## [Field doesn't create when field value is empty](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 6\ **Last updated:** [March 9, 2018, 7:18am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839 "2018-03-09T07:18:56Z")
hi elastic team, i would like to ask one thing. when i check my log in https://grokdebug.herokuapp.com/, i got expect result my test log is 2018-03-07 10:13:56,256 (null) 24 WARN : TEST.Controllers.PaymentCont… --- ## [Filebeat to Logstash over SSL](https://discuss.elastic.co/t/filebeat-to-logstash-over-ssl/122735)
**Author:** [@Lazarbeam](https://discuss.elastic.co/u/Lazarbeam)\ **Replies:** 5\ **Last updated:** [March 9, 2018, 7:17am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-over-ssl/122735 "2018-03-09T07:17:35Z")
Team -- We're encountering some issues getting Filebeat to send data to Logstash over SSL. All products are versioned 6.2.2. Filebeat is on a linux machine, and Logstash on Winodws Server. For testing/development purpos… --- ## [Unable to parse Fortinet Netflow with logstash module](https://discuss.elastic.co/t/unable-to-parse-fortinet-netflow-with-logstash-module/122941)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [March 9, 2018, 7:13am UTC](https://discuss.elastic.co/t/unable-to-parse-fortinet-netflow-with-logstash-module/122941 "2018-03-09T07:13:36Z")
Hi, I am trying to monitor Fortinet Netflow logs using elasticsearch stack 6.2.2. I am using logstash Netflow module but logs are not completely being parsed and seeing below errors. Any clue what could be the issue? \[… --- ## [How to match all pattern found in message](https://discuss.elastic.co/t/how-to-match-all-pattern-found-in-message/123200)
**Author:** [@Eaky\_Ake](https://discuss.elastic.co/u/Eaky_Ake)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 6:09am UTC](https://discuss.elastic.co/t/how-to-match-all-pattern-found-in-message/123200 "2018-03-09T06:09:59Z")
I have a log file that record multi-event log (multiline log). How to config logstash to match same pattern into one. I try to grok with option break\_on\_match =\> false but not working 18:16:16.075 Customer selected choi… --- ## [Logstash gives only first part of the “msg” field](https://discuss.elastic.co/t/logstash-gives-only-first-part-of-the-msg-field/123199)
**Author:** [@SashaAlex](https://discuss.elastic.co/u/SashaAlex)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 6:09am UTC](https://discuss.elastic.co/t/logstash-gives-only-first-part-of-the-msg-field/123199 "2018-03-09T06:09:15Z")
This is one of my log message. time="2018-03-08T17:24:23+05:30" level=error msg="{\\"Status\\":\\"404 Not Found\\",\\"StatusCode\\":404,\\"Body\\":\\"{\\\\\\"error\\\\\\":{\\\\\\"code\\\\\\":\\\\\\"005056A509B11EE1B9A8FEC11C21D78E\\\\\\",\\\\\\"mess… --- ## [JDBC Connection to SAP HANA](https://discuss.elastic.co/t/jdbc-connection-to-sap-hana/123193)
**Author:** [@Chandra\_Bhandaru](https://discuss.elastic.co/u/Chandra_Bhandaru)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 5:41am UTC](https://discuss.elastic.co/t/jdbc-connection-to-sap-hana/123193 "2018-03-09T05:41:01Z")
Below is my config and I am not able to connect to SAP HANA. I have right driver and am able to connect via standalone java program. input { jdbc { jdbc\_connection\_string =\> "jdbc:sap://\" jdbc\_user … --- ## [TCP connection failure to ELK stack](https://discuss.elastic.co/t/tcp-connection-failure-to-elk-stack/123189)
**Author:** [@csudaya](https://discuss.elastic.co/u/csudaya)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 5:14am UTC](https://discuss.elastic.co/t/tcp-connection-failure-to-elk-stack/123189 "2018-03-09T05:14:11Z")
I have setup the elk stack based on https://github.com/cisco/bigmuddy-network-telemetry-stacks. I have build and run the stack successfully. cisco@ubuntu:~/bigmuddy-network-telemetry-stacks/stack\_elk$ sudo ./stack\_run %… --- ## [Logstash 6.2.2 Fatal error crashing background running logstash when stdout output plugin is used](https://discuss.elastic.co/t/logstash-6-2-2-fatal-error-crashing-background-running-logstash-when-stdout-output-plugin-is-used/122973)
**Author:** [@LLK](https://discuss.elastic.co/u/LLK)\ **Replies:** 1\ **Last updated:** [March 9, 2018, 4:57am UTC](https://discuss.elastic.co/t/logstash-6-2-2-fatal-error-crashing-background-running-logstash-when-stdout-output-plugin-is-used/122973 "2018-03-09T04:57:41Z")
Hello, Same issue as with the other user's thread: Running logstash in the background with stdout plugin enabled crashes it: \[2018-03-07T17:18:19,623\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {… --- ## [Parsing docker container logs](https://discuss.elastic.co/t/parsing-docker-container-logs/123184)
**Author:** [@aysala](https://discuss.elastic.co/u/aysala)\ **Replies:** 0\ **Last updated:** [March 9, 2018, 4:22am UTC](https://discuss.elastic.co/t/parsing-docker-container-logs/123184 "2018-03-09T04:22:17Z")
Hi @magnusbaeck , I'm using filebeat to ship my logs to Logstash and I need help with parsing docker container logs along with few other logs. My Filebeat configuration: filebeat.prospectors: - input\_ty… --- ## [JVM segmentation fault in logstash 6.2.0 while building rake artifacts:rpm in Docker Container](https://discuss.elastic.co/t/jvm-segmentation-fault-in-logstash-6-2-0-while-building-rake-artifacts-rpm-in-docker-container/121401)
**Author:** [@Sandeep\_Sarkar](https://discuss.elastic.co/u/Sandeep_Sarkar)\ **Replies:** 11\ **Last updated:** [March 9, 2018, 4:14am UTC](https://discuss.elastic.co/t/jvm-segmentation-fault-in-logstash-6-2-0-while-building-rake-artifacts-rpm-in-docker-container/121401 "2018-03-09T04:14:58Z")
Hi @all, I was trying to build logstash 6.2.0 from source and I am getting segmentation fault. My system information is as follows: Linux el7uek.x86\_64. openjdk 1.8.0\_151. jruby version 9.1.15.0 When I am trying to b… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=804) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=806)