# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=926 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 927 --- ## [Splitting 1 column of integers into 2 columns](https://discuss.elastic.co/t/splitting-1-column-of-integers-into-2-columns/94994)
**Author:** [@kahelek](https://discuss.elastic.co/u/kahelek)\ **Replies:** 4\ **Last updated:** [August 14, 2017, 8:30pm UTC](https://discuss.elastic.co/t/splitting-1-column-of-integers-into-2-columns/94994 "2017-08-14T20:30:13Z")
Hi all. I have been playing with ELK for a few months, and been making decent progress so far. I have an interesting problem I came across, which I haven’t been able to figure out. Let’s say you have a column that has … --- ## [Csv filter doesn't activate](https://discuss.elastic.co/t/csv-filter-doesnt-activate/93827)
**Author:** [@reswob](https://discuss.elastic.co/u/reswob)\ **Replies:** 2\ **Last updated:** [August 14, 2017, 8:01pm UTC](https://discuss.elastic.co/t/csv-filter-doesnt-activate/93827 "2017-08-14T20:01:24Z")
OK, I've been working this all morning and I've tried everything I can think of and googled. We are doing some processing of netflow data and it produces a csv file. We want to ingest each csv file into Elastic, but f… --- ## [TypeError: can't convert Array into String, HTTP Input Plugin](https://discuss.elastic.co/t/typeerror-cant-convert-array-into-string-http-input-plugin/97006)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 2\ **Last updated:** [August 14, 2017, 5:48pm UTC](https://discuss.elastic.co/t/typeerror-cant-convert-array-into-string-http-input-plugin/97006 "2017-08-14T17:48:46Z")
Logstash: v5.5.1 I’ve got the HTTP Input plugin which was working great for about a month and then without anything seeming to change, I consistently get this error with any data I attempt to push to Logstash: \[2017-08… --- ## [Run Logstash-input-jdbc SQL query on-demand](https://discuss.elastic.co/t/run-logstash-input-jdbc-sql-query-on-demand/96991)
**Author:** [@Apolitosz](https://discuss.elastic.co/u/Apolitosz)\ **Replies:** 0\ **Last updated:** [August 14, 2017, 3:03pm UTC](https://discuss.elastic.co/t/run-logstash-input-jdbc-sql-query-on-demand/96991 "2017-08-14T15:03:32Z")
Hi, I know that with a cron-like syntax we can schedule a jdbc input to run regularly, however our underlying database tables (postgres) are refreshing irregularly, therefore we are looking for a way to only run the SQL… --- ## [Wildcard for 2 character in logstash input path](https://discuss.elastic.co/t/wildcard-for-2-character-in-logstash-input-path/96960)
**Author:** [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\ **Replies:** 4\ **Last updated:** [August 14, 2017, 1:11pm UTC](https://discuss.elastic.co/t/wildcard-for-2-character-in-logstash-input-path/96960 "2017-08-14T13:11:51Z")
Hello, My problem is very simple… In my input file i have this : input { file { path =\> "/data/logstash/celisible\_elkf/DCfw\*7\_\*.txt" exclude =\> "DCfw\*6\_\*.txt" type =\> "cra" max\_open\_files =\> 30000 … --- ## [Global variable?](https://discuss.elastic.co/t/global-variable/96794)
**Author:** [@Chemse](https://discuss.elastic.co/u/Chemse)\ **Replies:** 3\ **Last updated:** [August 14, 2017, 2:37pm UTC](https://discuss.elastic.co/t/global-variable/96794 "2017-08-14T14:37:41Z")
Hello, Since logstash deals with a log file line by line, I have a field just in the begining of the file, I want to use it value as a condition for the rest of the file. Is there a method to do it ? Make this variable … --- ## [Multiline filter in logstash 5.5](https://discuss.elastic.co/t/multiline-filter-in-logstash-5-5/96832)
**Author:** [@aliyesami](https://discuss.elastic.co/u/aliyesami)\ **Replies:** 4\ **Last updated:** [August 14, 2017, 2:24pm UTC](https://discuss.elastic.co/t/multiline-filter-in-logstash-5-5/96832 "2017-08-14T14:24:47Z")
I am unable to find any example or document giving example of it , this code is from 2.4 version , what would be the equivalent in 5.5 ? I have tried "codec =\> multiline" but it gives syntax error in the "filte… --- ## [Configure Logstash to create an Elasticsearch document with nested arrays](https://discuss.elastic.co/t/configure-logstash-to-create-an-elasticsearch-document-with-nested-arrays/96979)
**Author:** [@EricPSU](https://discuss.elastic.co/u/EricPSU)\ **Replies:** 0\ **Last updated:** [August 14, 2017, 2:24pm UTC](https://discuss.elastic.co/t/configure-logstash-to-create-an-elasticsearch-document-with-nested-arrays/96979 "2017-08-14T14:24:17Z")
I’m indexing my PostgreSQL data for Elasticsearch using the Logstash JDBC Input Plugin. I have two tables called REQUEST and ASSIGNMENT listed below. How can I use Logstash to index the two tables into one Elasticsearch… --- ## [Logstash processing the same file again and again from a remote shared path](https://discuss.elastic.co/t/logstash-processing-the-same-file-again-and-again-from-a-remote-shared-path/96955)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 5\ **Last updated:** [August 14, 2017, 1:55pm UTC](https://discuss.elastic.co/t/logstash-processing-the-same-file-again-and-again-from-a-remote-shared-path/96955 "2017-08-14T13:55:30Z")
Hi, I have a logfile in a shared path (//XXXX/storage/files/test.log) with 100 lines. I am processing the it with logstash with the below input cong: input { file { path =\> "//XXXX/storage/files/test.log" start\_pos… --- ## [Date filter not working properly](https://discuss.elastic.co/t/date-filter-not-working-properly/96941)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 5\ **Last updated:** [August 14, 2017, 11:34am UTC](https://discuss.elastic.co/t/date-filter-not-working-properly/96941 "2017-08-14T11:34:13Z")
i have a string timestamp field in my log,i need to change the string to date format, This is my sample log file: {“LogMsg”:"{“type”:“GAUGE”, “name”:“io.dropwizard.jetty.MutableServletContextHandler.percent-4xx-15m”, “… --- ## [How to connect to Sybase SQL Anywhere with JDBC plugin](https://discuss.elastic.co/t/how-to-connect-to-sybase-sql-anywhere-with-jdbc-plugin/96463)
**Author:** [@nikaalv](https://discuss.elastic.co/u/nikaalv)\ **Replies:** 2\ **Last updated:** [August 14, 2017, 10:38am UTC](https://discuss.elastic.co/t/how-to-connect-to-sybase-sql-anywhere-with-jdbc-plugin/96463 "2017-08-14T10:38:08Z")
Hello! I need to import data from Sybase SQL Anywhere 17 to Logstash. How can I do it? --- ## [Logstash logs parse error](https://discuss.elastic.co/t/logstash-logs-parse-error/96948)
**Author:** [@w33d](https://discuss.elastic.co/u/w33d)\ **Replies:** 0\ **Last updated:** [August 14, 2017, 10:37am UTC](https://discuss.elastic.co/t/logstash-logs-parse-error/96948 "2017-08-14T10:37:24Z")
Hello All. I have Logstash 5.5.1 on debian 9 x64. Get apache2 logs from beat and put in elasticsearch and all fine. But i try get nginx custom logs from FreeBSD 10.3 filebeats 5.5.0 and logstash is print error and free… --- ## [Element size differs](https://discuss.elastic.co/t/element-size-differs/96932)
**Author:** [@Lukas\_Tilch](https://discuss.elastic.co/u/Lukas_Tilch)\ **Replies:** 0\ **Last updated:** [August 14, 2017, 9:09am UTC](https://discuss.elastic.co/t/element-size-differs/96932 "2017-08-14T09:09:54Z")
Helllo, when I try to parse a XML document with logstash i get an error The data seems to be consistent, the original document has about 2XXX data fields. but i got the error Ruby exception occurred: element size diff… --- ## [Replace @timestamp in kibana / logstash with time attribute of my log](https://discuss.elastic.co/t/replace-timestamp-in-kibana-logstash-with-time-attribute-of-my-log/94064)
**Author:** [@diooda](https://discuss.elastic.co/u/diooda)\ **Replies:** 21\ **Last updated:** [August 14, 2017, 8:51am UTC](https://discuss.elastic.co/t/replace-timestamp-in-kibana-logstash-with-time-attribute-of-my-log/94064 "2017-08-14T08:51:31Z")
I want to do replace @timestamp in kibana / logstash, want to change to the time attribute in my logfile ,, what should i do? thank you :slight\_smile: --- ## ["\_grokparsefailure" even though the grok pattern matches](https://discuss.elastic.co/t/-grokparsefailure-even-though-the-grok-pattern-matches/96783)
**Author:** [@Hisushi](https://discuss.elastic.co/u/Hisushi)\ **Replies:** 8\ **Last updated:** [August 14, 2017, 7:41am UTC](https://discuss.elastic.co/t/-grokparsefailure-even-though-the-grok-pattern-matches/96783 "2017-08-14T07:41:00Z")
Hi everyone, I’m encountering the following problem: I’ve been testing my grok filter including my patterns on both Grok Debugger and Grok Constructor which works fine. Also, when running logstash with option -t no erro… --- ## [Logstash filter customize](https://discuss.elastic.co/t/logstash-filter-customize/96903)
**Author:** [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Replies:** 1\ **Last updated:** [August 14, 2017, 5:28am UTC](https://discuss.elastic.co/t/logstash-filter-customize/96903 "2017-08-14T05:28:44Z")
my data type: \[INFO \] status =\> FAIL | client : \[MTB\] | cell : \[1746710009\] | message\_delivery\_time : \[2017-08-07 09:46:27,807\] | operator: \[GP\] \[INFO \] status =\> SENT | client : \[IPDC\] | cell : \[1746710009\] | messag… --- ## [Can't parse a CSV file - message is coming through though](https://discuss.elastic.co/t/cant-parse-a-csv-file-message-is-coming-through-though/96855)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 2\ **Last updated:** [August 14, 2017, 4:41am UTC](https://discuss.elastic.co/t/cant-parse-a-csv-file-message-is-coming-through-though/96855 "2017-08-14T04:41:03Z")
update - problem solved, explanations in the replies section Hi, I am using the conf file below to parse a csv. I cannot get the message to populate into the fields that are mapped. Instead I see the message in kibana,… --- ## [Logstash Monitor API get 500: Unknown internal error after upgrade to 5.5](https://discuss.elastic.co/t/logstash-monitor-api-get-500-unknown-internal-error-after-upgrade-to-5-5/96897)
**Author:** [@byakuinss](https://discuss.elastic.co/u/byakuinss)\ **Replies:** 0\ **Last updated:** [August 14, 2017, 3:12am UTC](https://discuss.elastic.co/t/logstash-monitor-api-get-500-unknown-internal-error-after-upgrade-to-5-5/96897 "2017-08-14T03:12:22Z")
Hi all, I used to run python script to monitor logstash performance through monitor API. However, it return 500 status code when query hot\_threads after I upgraded logstash from 5.4.1 to 5.5.1. I checked 5.5 release n… --- ## [Logstash config file error](https://discuss.elastic.co/t/logstash-config-file-error/96878)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 1\ **Last updated:** [August 13, 2017, 6:33pm UTC](https://discuss.elastic.co/t/logstash-config-file-error/96878 "2017-08-13T18:33:16Z")
I got an error like below when indexing in logstash Log file model is date=2017-01-24 time=07:24:24 logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=192.168.1.110 srcport=60846 srcintf="intern… --- ## [Logstash fails to start with central configuration(bug in 6.0.0-beta1)](https://discuss.elastic.co/t/logstash-fails-to-start-with-central-configuration-bug-in-6-0-0-beta1/96648)
**Author:** [@arthur](https://discuss.elastic.co/u/arthur)\ **Replies:** 5\ **Last updated:** [August 13, 2017, 7:16am UTC](https://discuss.elastic.co/t/logstash-fails-to-start-with-central-configuration-bug-in-6-0-0-beta1/96648 "2017-08-13T07:16:13Z")
I have Elasticsearch6.0.0beta1 and logstash6.0.0beta1 installed on my Mac. Both have x-pack plugin installed. My Elasticsearch.yml contains “xpack.security.enabled: false” and everything works. When trying to start Logs… --- ## [Failed to configure JDBC Input Plugin with MongoDB](https://discuss.elastic.co/t/failed-to-configure-jdbc-input-plugin-with-mongodb/95254)
**Author:** [@alejandrobarrios07](https://discuss.elastic.co/u/alejandrobarrios07)\ **Replies:** 1\ **Last updated:** [August 13, 2017, 4:51am UTC](https://discuss.elastic.co/t/failed-to-configure-jdbc-input-plugin-with-mongodb/95254 "2017-08-13T04:51:20Z")
Hello, I’m trying to set up logstash with JDBC to consume data from MongoDB without success. I am testing with the java drivers for MongoDb (https://mongodb.github.io/mongo-java-driver/) in its different versions: mon… --- ## [Logstash Pipeline Monitoring Shows incorrect versions (6.0.0-beta1)](https://discuss.elastic.co/t/logstash-pipeline-monitoring-shows-incorrect-versions-6-0-0-beta1/96777)
**Author:** [@arthur](https://discuss.elastic.co/u/arthur)\ **Replies:** 2\ **Last updated:** [August 12, 2017, 10:15am UTC](https://discuss.elastic.co/t/logstash-pipeline-monitoring-shows-incorrect-versions-6-0-0-beta1/96777 "2017-08-12T10:15:47Z")
Hi, I have elastic stack 6.0.0 beta 1 installed and trying to push some logs from file input to Elasticsearch through Logstash using the new centralized config management functionality. Everything seem to work, but, the… --- ## [How do I create a grok match pattern for a log?](https://discuss.elastic.co/t/how-do-i-create-a-grok-match-pattern-for-a-log/96830)
**Author:** [@e\_dd](https://discuss.elastic.co/u/e_dd)\ **Replies:** 0\ **Last updated:** [August 12, 2017, 1:34am UTC](https://discuss.elastic.co/t/how-do-i-create-a-grok-match-pattern-for-a-log/96830 "2017-08-12T01:34:06Z")
Hi It would be a great help if you write it in a pattern that can be distinguished from kibana. Thank you. {“EventTime”:“2017-08-12 10:33:15”,“Hostname”:“ISCHOOL”,“Keywords”:-9214364837600034816,“EventType”:“AUDIT\_SU… --- ## [Logstash 5.5.1 index creation issue](https://discuss.elastic.co/t/logstash-5-5-1-index-creation-issue/96828)
**Author:** [@aliyesami](https://discuss.elastic.co/u/aliyesami)\ **Replies:** 0\ **Last updated:** [August 12, 2017, 1:10am UTC](https://discuss.elastic.co/t/logstash-5-5-1-index-creation-issue/96828 "2017-08-12T01:10:57Z")
I am new to logstash , I am trying to convert an logstash 2.4 conf file to latest 5.x release but I am getting strange behavior . It does create the elastic index but that index is not making sense to me . why is he c… --- ## [Custom logstash plugin dependency hell](https://discuss.elastic.co/t/custom-logstash-plugin-dependency-hell/94167)
**Author:** [@chancerose91](https://discuss.elastic.co/u/chancerose91)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 9:35pm UTC](https://discuss.elastic.co/t/custom-logstash-plugin-dependency-hell/94167 "2017-08-11T21:35:09Z")
I am trying to build a custom LogStash plugin for use in an environment which is not connected to the internet. Our enterprise also has very strict control over which artifacts are brought into our isolated environment. … --- ## [Kafka Timestamp](https://discuss.elastic.co/t/kafka-timestamp/96825)
**Author:** [@ashvita](https://discuss.elastic.co/u/ashvita)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 9:23pm UTC](https://discuss.elastic.co/t/kafka-timestamp/96825 "2017-08-11T21:23:28Z")
Hi, Is there a way to get timestamp that the kafka producer has in its message into my logs? Thanks --- ## [Not working JDBC Mysql in UBUNTU 16 --- Homestead VAGRANT MACHINE](https://discuss.elastic.co/t/not-working-jdbc-mysql-in-ubuntu-16-homestead-vagrant-machine/96818)
**Author:** [@David\_Barrera\_Bernal](https://discuss.elastic.co/u/David_Barrera_Bernal)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 8:05pm UTC](https://discuss.elastic.co/t/not-working-jdbc-mysql-in-ubuntu-16-homestead-vagrant-machine/96818 "2017-08-11T20:05:11Z")
Hello ! Ive been trying to migrate the data from a mysql database into elasticsearch using LOGSTASH But not working. Both Elasticsearch and kibana are running and i created the logstash config file in /etc/logstash/con… --- ## [Forward avro event natively to another kafka topic](https://discuss.elastic.co/t/forward-avro-event-natively-to-another-kafka-topic/96813)
**Author:** [@stevenh](https://discuss.elastic.co/u/stevenh)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 6:43pm UTC](https://discuss.elastic.co/t/forward-avro-event-natively-to-another-kafka-topic/96813 "2017-08-11T18:43:38Z")
How do I read an avro event from one topic and put it on a different topic without doing any processing on the event. As I understand it, there is at minimum a codec that is required which converts input records to logs… --- ## [XML Filter Not Working](https://discuss.elastic.co/t/xml-filter-not-working/93840)
**Author:** [@httpex](https://discuss.elastic.co/u/httpex)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 6:24pm UTC](https://discuss.elastic.co/t/xml-filter-not-working/93840 "2017-08-11T18:24:31Z")
After adding the xml filter below, it's not doing anything. It just says that it successfully started. Help is much appreciated. Not sure what to do next... \[2017-07-19T16:50:54,105\]\[INFO \]\[logstash.outputs.elastic… --- ## [Ship filbeat data to public ip server(logstash)](https://discuss.elastic.co/t/ship-filbeat-data-to-public-ip-server-logstash/93856)
**Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 6:23pm UTC](https://discuss.elastic.co/t/ship-filbeat-data-to-public-ip-server-logstash/93856 "2017-08-11T18:23:09Z")
I have 2 machine (1 remote and 1 server) My server is on public IP which holds the logstash and elasticsearch. It is possible to send data(filebeat) to the public ip server(logstash/elasticsearch) Can you share a config … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=925) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=927)