# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=466 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 467 --- ## [Longitude and latitude in different rows](https://discuss.elastic.co/t/longitude-and-latitude-in-different-rows/202544)
**Author:** [@Alisa\_Faingold](https://discuss.elastic.co/u/Alisa_Faingold)\ **Replies:** 1\ **Last updated:** [October 7, 2019, 3:48pm UTC](https://discuss.elastic.co/t/longitude-and-latitude-in-different-rows/202544 "2019-10-07T15:48:34Z")
Hi, I have log with location info and I want to turn it to geo point. However, this info is in different rows: 10-17 18:59:10.997 2361 2361 D CAP:Location: latitude: 45.08371837 10-17 18:59:10.997 2361 2361 D CAP:… --- ## [IF less than value is not working](https://discuss.elastic.co/t/if-less-than-value-is-not-working/202381)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 7\ **Last updated:** [October 7, 2019, 1:52pm UTC](https://discuss.elastic.co/t/if-less-than-value-is-not-working/202381 "2019-10-07T13:52:10Z")
Hello, I am trying to check if the YEAR value is less than 100 other the value in YEAR is for example 2012 otherwise it'll be 12 or so on for different year. However, it is throwing an error. I can't seem to compare and … --- ## [Ship logs from remote machine](https://discuss.elastic.co/t/ship-logs-from-remote-machine/201921)
**Author:** [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\ **Replies:** 0\ **Last updated:** [October 2, 2019, 9:23am UTC](https://discuss.elastic.co/t/ship-logs-from-remote-machine/201921 "2019-10-02T09:23:01Z")
Hello, How to get logs from remote machine ? --- ## [XML Plugin and xpath usage](https://discuss.elastic.co/t/xml-plugin-and-xpath-usage/202541)
**Author:** [@ajojohnk](https://discuss.elastic.co/u/ajojohnk)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 1:23pm UTC](https://discuss.elastic.co/t/xml-plugin-and-xpath-usage/202541 "2019-10-07T13:23:20Z")
On XML filter How is the event been decided by the logstash for its processing ? In my following input xml, how should I flatten this xml using xpath to ingest json to ES My case data: … --- ## [Integrate Okta with Logstash](https://discuss.elastic.co/t/integrate-okta-with-logstash/202537)
**Author:** [@NADEERA\_DASUN](https://discuss.elastic.co/u/NADEERA_DASUN)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 1:16pm UTC](https://discuss.elastic.co/t/integrate-okta-with-logstash/202537 "2019-10-07T13:16:30Z")
I'm trying to obtain data from Okta through logstash and display them in Kibana. Please suggest a correct way to fulfill this manner. Thanks in advance Nadeera --- ## [Indexing Aggregate Results](https://discuss.elastic.co/t/indexing-aggregate-results/202528)
**Author:** [@Artisanal](https://discuss.elastic.co/u/Artisanal)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 12:35pm UTC](https://discuss.elastic.co/t/indexing-aggregate-results/202528 "2019-10-07T12:35:03Z")
Hi all, I'm attempting to use the Logstash elasticsearch input to perform an aggregating query against the auditbeat index, add some metadata to the results, then send them back to Elasticsearch using the elasticsearch … --- ## [How to transfer multiple json documents using translate pluging](https://discuss.elastic.co/t/how-to-transfer-multiple-json-documents-using-translate-pluging/202518)
**Author:** [@namali](https://discuss.elastic.co/u/namali)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 11:15am UTC](https://discuss.elastic.co/t/how-to-transfer-multiple-json-documents-using-translate-pluging/202518 "2019-10-07T11:15:06Z")
wanted to transfer two json documents, one has hotel data and other one has destination data. destination object in hotel.json and code in destination.json are the common objects. hotel.json results :\[ {code:1001,name:… --- ## [Invalid JSON PARSE FAILURE DUE TO LEADING ZEROS](https://discuss.elastic.co/t/invalid-json-parse-failure-due-to-leading-zeros/201736)
**Author:** [@ashish\_kumar1490](https://discuss.elastic.co/u/ashish_kumar1490)\ **Replies:** 10\ **Last updated:** [October 7, 2019, 10:29am UTC](https://discuss.elastic.co/t/invalid-json-parse-failure-due-to-leading-zeros/201736 "2019-10-07T10:29:39Z")
I am getting the above exception when I parsing log - {"ProcessName":"c:/windows/system32/svchost.exe","cpuPerc":0,"cpuTime":"00:00","memResident":1237319680,"memSize":1090519040,"pid":660,"startTime":1569254748000,"sta… --- ## [How can trace two column in single sql query(logstash)](https://discuss.elastic.co/t/how-can-trace-two-column-in-single-sql-query-logstash/202497)
**Author:** [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 8:12am UTC](https://discuss.elastic.co/t/how-can-trace-two-column-in-single-sql-query-logstash/202497 "2019-10-07T08:12:19Z")
Dear All. i have sql query that contain union operation with different join and table. SELECT DISTINCT S.HEALTHRECORDKEY,NULL PRODUCTCODE,S.FIRSTNAME,S.LASTNAME,NULL DRUGDESCRIPTION, PUL.LABTEST\_TYPE\_DESC,PUL.LABTEST… --- ## [Logstash always fail to connect jdbc connection with sql server](https://discuss.elastic.co/t/logstash-always-fail-to-connect-jdbc-connection-with-sql-server/202483)
**Author:** [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 7:13am UTC](https://discuss.elastic.co/t/logstash-always-fail-to-connect-jdbc-connection-with-sql-server/202483 "2019-10-07T07:13:47Z")
Dear All. I am trying to fetch auto data from sql server to Elasticsearch but i am getting below failure error. \[2019-10-07T12:12:25,194\]\[INFO \]\[logstash.outputs.elasticsearch\] retrying failed action with response cod… --- ## [Trasform one object from array of objects in json to elasticseach](https://discuss.elastic.co/t/trasform-one-object-from-array-of-objects-in-json-to-elasticseach/202334)
**Author:** [@namali](https://discuss.elastic.co/u/namali)\ **Replies:** 3\ **Last updated:** [October 7, 2019, 5:36am UTC](https://discuss.elastic.co/t/trasform-one-object-from-array-of-objects-in-json-to-elasticseach/202334 "2019-10-07T05:36:00Z")
wanted to transfer only selected filed from a json file. and one field has array of objects and from that wanted to take only one object hotel.json count:25000 results: hotelcode:h001 hotelname: Pearl City Hotel c… --- ## [Error when starting up Logstash](https://discuss.elastic.co/t/error-when-starting-up-logstash/202252)
**Author:** [@sud0](https://discuss.elastic.co/u/sud0)\ **Replies:** 2\ **Last updated:** [October 7, 2019, 1:30am UTC](https://discuss.elastic.co/t/error-when-starting-up-logstash/202252 "2019-10-07T01:30:29Z")
When starting Logstash, I get the following error: \[2019-10-03T23:04:46,420\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or command line options are specified \[2019-10-03T… --- ## [Can't not bind listen port 5140 to logstash ubuntu 18.04](https://discuss.elastic.co/t/cant-not-bind-listen-port-5140-to-logstash-ubuntu-18-04/202428)
**Author:** [@Bubba\_Shakes](https://discuss.elastic.co/u/Bubba_Shakes)\ **Replies:** 24\ **Last updated:** [October 6, 2019, 5:41pm UTC](https://discuss.elastic.co/t/cant-not-bind-listen-port-5140-to-logstash-ubuntu-18-04/202428 "2019-10-06T17:41:20Z")
Nothing is using that port, guyp@ubuntu:/etc/logstash/conf.d$ netstat -an | grep 5140 tcp 0 0 127.0.0.1:9200 127.0.0.1:51406 ESTABLISHED tcp 0 0 127.0.0.1:9200 127.0.0… --- ## [Having issues with my PF sense configs](https://discuss.elastic.co/t/having-issues-with-my-pf-sense-configs/202438)
**Author:** [@JeMunos](https://discuss.elastic.co/u/JeMunos)\ **Replies:** 4\ **Last updated:** [October 6, 2019, 5:06pm UTC](https://discuss.elastic.co/t/having-issues-with-my-pf-sense-configs/202438 "2019-10-06T17:06:59Z")
I'm having issues with getting my PF sense logs to parse properly. The current result is that everything is dropped and I am uncertain why. I am using the config files from this github: https://github.com/a3ilson/pfelk … --- ## [Aggregate filter doesn't work in merging multi events from elasticsearch](https://discuss.elastic.co/t/aggregate-filter-doesnt-work-in-merging-multi-events-from-elasticsearch/201484)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 4\ **Last updated:** [October 6, 2019, 2:39pm UTC](https://discuss.elastic.co/t/aggregate-filter-doesnt-work-in-merging-multi-events-from-elasticsearch/201484 "2019-10-06T14:39:08Z")
hi all I have following lines in my log which is parsing to logstash to do some filters and store them in the elasticsearch IT=1 AC=12345 p1=12345 IW=1 AC=12345 A1=check1 IW=1 E1=0 IW=2 AC=12345 A1=check2 IW=2 E1=4 I… --- ## [Use variables (possibly set by Ruby) in Input](https://discuss.elastic.co/t/use-variables-possibly-set-by-ruby-in-input/202420)
**Author:** [@Mohamad\_Rahim](https://discuss.elastic.co/u/Mohamad_Rahim)\ **Replies:** 4\ **Last updated:** [October 5, 2019, 4:59pm UTC](https://discuss.elastic.co/t/use-variables-possibly-set-by-ruby-in-input/202420 "2019-10-05T16:59:02Z")
Hello all, So I'm using the http\_poller plugin to poll events off one of our SaaS subscriptions. The problem is that I need to specify the "From Date" and "To Date" statically. My question is how can I generate a vari… --- ## [Custom name index kibana](https://discuss.elastic.co/t/custom-name-index-kibana/202262)
**Author:** [@TheSun](https://discuss.elastic.co/u/TheSun)\ **Replies:** 2\ **Last updated:** [October 5, 2019, 2:42am UTC](https://discuss.elastic.co/t/custom-name-index-kibana/202262 "2019-10-05T02:42:41Z")
Hi, I am handling pfsense data with ELK stack. But I have 2 pfsense. How to put data of each pfsense into a different index? I tried with logstash output but it didn't work. This is my configuration. Please!! help m… --- ## [GC for persistent queue and LS clustering](https://discuss.elastic.co/t/gc-for-persistent-queue-and-ls-clustering/202400)
**Author:** [@stefsamm](https://discuss.elastic.co/u/stefsamm)\ **Replies:** 1\ **Last updated:** [October 5, 2019, 12:23am UTC](https://discuss.elastic.co/t/gc-for-persistent-queue-and-ls-clustering/202400 "2019-10-05T00:23:06Z")
I'm workin on switching our log aggregation from Graylog to full ELK setup. While reading on LS clustering found recommendation of adding queue.checkpoint.writes: 1 to the config to increase durability: https://www.elast… --- ## [\[Question\] Can you contribute a java logstash plugin to RubyGems.org](https://discuss.elastic.co/t/question-can-you-contribute-a-java-logstash-plugin-to-rubygems-org/202390)
**Author:** [@Xucito](https://discuss.elastic.co/u/Xucito)\ **Replies:** 0\ **Last updated:** [October 4, 2019, 9:14pm UTC](https://discuss.elastic.co/t/question-can-you-contribute-a-java-logstash-plugin-to-rubygems-org/202390 "2019-10-04T21:14:14Z")
I have a plugin to contribute the community, Can you push java plugins to RubyGems.org for easy installation or does that only work for JRuby plugins? Regards, Tony Nguyen --- ## [Logstash overrides input tcp codec from JSON to JSON\_Lines](https://discuss.elastic.co/t/logstash-overrides-input-tcp-codec-from-json-to-json-lines/202375)
**Author:** [@dmalhan](https://discuss.elastic.co/u/dmalhan)\ **Replies:** 0\ **Last updated:** [October 4, 2019, 6:54pm UTC](https://discuss.elastic.co/t/logstash-overrides-input-tcp-codec-from-json-to-json-lines/202375 "2019-10-04T18:54:57Z")
My config file: input { tcp { port =\> 5044 codec =\> json } } Everytime I run logstash, I get the following in my startup logs: \[INFO \]\[logstash.inputs.tcp \] Automatically switching from j… --- ## [Error consuming weblogic server topic using logstash-input-jms](https://discuss.elastic.co/t/error-consuming-weblogic-server-topic-using-logstash-input-jms/202366)
**Author:** [@Luis\_Carvalho](https://discuss.elastic.co/u/Luis_Carvalho)\ **Replies:** 0\ **Last updated:** [October 4, 2019, 4:24pm UTC](https://discuss.elastic.co/t/error-consuming-weblogic-server-topic-using-logstash-input-jms/202366 "2019-10-04T16:24:32Z")
Hi, I'm using logstash-7.3.2 and config the input with jms plugin to consume a Weblogic JMS topic, but i'm getting a "Unable to resolve 'weblogic.jms.backend.jms'. Resolved 'weblogic.jms.backend'". error. I already tes… --- ## [Load multiple csv files and update existing index](https://discuss.elastic.co/t/load-multiple-csv-files-and-update-existing-index/202324)
**Author:** [@myrtod](https://discuss.elastic.co/u/myrtod)\ **Replies:** 6\ **Last updated:** [October 4, 2019, 2:27pm UTC](https://discuss.elastic.co/t/load-multiple-csv-files-and-update-existing-index/202324 "2019-10-04T14:27:24Z")
Hi, I am trying to create a "master table" in elastic, using logstash, where many csv files will be merged according to a common column named 'A'. Given the mapping of the index, can I insert each csv and update the uni… --- ## [He given configuration is invalid. Reason: Expected one of #, input, filter, output](https://discuss.elastic.co/t/he-given-configuration-is-invalid-reason-expected-one-of-input-filter-output/202355)
**Author:** [@kenorb](https://discuss.elastic.co/u/kenorb)\ **Replies:** 2\ **Last updated:** [October 4, 2019, 2:24pm UTC](https://discuss.elastic.co/t/he-given-configuration-is-invalid-reason-expected-one-of-input-filter-output/202355 "2019-10-04T14:24:35Z")
I've got the following syntax error when running Logstash: $ sudo -u logstash /usr/share/logstash/bin/logstash -V logstash 7.1.0 $ sudo -u logstash /usr/share/logstash/bin/logstash -t -f /etc/logstash/logstash.yml WARN… --- ## [Warning in Logstash Log](https://discuss.elastic.co/t/warning-in-logstash-log/202345)
**Author:** [@as50790](https://discuss.elastic.co/u/as50790)\ **Replies:** 4\ **Last updated:** [October 4, 2019, 1:22pm UTC](https://discuss.elastic.co/t/warning-in-logstash-log/202345 "2019-10-04T13:22:35Z")
Hi, Could you please help me figuring out how to get rid of these deprecation warnings? I have used ELK version(7.3.1). Elasticsearch Version: 7.3.1 Logstash Version :7.3.1 Kibana Version :7.3.1 \[71\]\[2019-10-04T05:… --- ## [HTTP input connection refused](https://discuss.elastic.co/t/http-input-connection-refused/202311)
**Author:** [@sparkblaze](https://discuss.elastic.co/u/sparkblaze)\ **Replies:** 0\ **Last updated:** [October 4, 2019, 9:10am UTC](https://discuss.elastic.co/t/http-input-connection-refused/202311 "2019-10-04T09:10:32Z")
I am using CentOS 7.7 / Logstash 7.4 (also didn't work in 7.3.1) and trying to use the http input to accept messages sent via HTTP (some of our existing scripts log this way). The server accepts traffic to the beats and… --- ## [Logstash Pipeline Translate Filter](https://discuss.elastic.co/t/logstash-pipeline-translate-filter/202184)
**Author:** [@fim](https://discuss.elastic.co/u/fim)\ **Replies:** 4\ **Last updated:** [October 4, 2019, 8:58am UTC](https://discuss.elastic.co/t/logstash-pipeline-translate-filter/202184 "2019-10-04T08:58:45Z")
May somebody help me to solve the following problem? My index gets written into elasticsearch but the field host.name is not changed according the predefined dictionary. Data gets ingested from a metricbeat v7.3. inpu… --- ## [LogStash - Data-enhancement at Insert](https://discuss.elastic.co/t/logstash-data-enhancement-at-insert/202287)
**Author:** [@LeFrancois](https://discuss.elastic.co/u/LeFrancois)\ **Replies:** 1\ **Last updated:** [October 4, 2019, 6:51am UTC](https://discuss.elastic.co/t/logstash-data-enhancement-at-insert/202287 "2019-10-04T06:51:17Z")
Hello ! I am currently looking to considerably improve our data-enhancement and ingesting pipeline by trying to give more responsabilities to ELK. Currently before inserting time of a document, I call few APIs (gRPC bu… --- ## [Logstash 7.4 File input exclude issue](https://discuss.elastic.co/t/logstash-7-4-file-input-exclude-issue/202043)
**Author:** [@Frrrrrunkis](https://discuss.elastic.co/u/Frrrrrunkis)\ **Replies:** 1\ **Last updated:** [October 4, 2019, 5:02am UTC](https://discuss.elastic.co/t/logstash-7-4-file-input-exclude-issue/202043 "2019-10-04T05:02:01Z")
Upgraded from 7.1 to 7.4 and my logstash file input seemingly started ignoring the exclude. In this pipeline I want all mule API logs but want to consume the server oriented logs separately. In my case im seeing mule\_e… --- ## [Split timestamp to date and time](https://discuss.elastic.co/t/split-timestamp-to-date-and-time/201739)
**Author:** [@mohamad.faisel](https://discuss.elastic.co/u/mohamad.faisel)\ **Replies:** 5\ **Last updated:** [October 4, 2019, 2:59am UTC](https://discuss.elastic.co/t/split-timestamp-to-date-and-time/201739 "2019-10-04T02:59:33Z")
Hi, How do I split timestamp field and store the result into timestamp\_date and timestamp\_time in logstash. Thank you. --- ## [Converting GUIDS to readable names](https://discuss.elastic.co/t/converting-guids-to-readable-names/202255)
**Author:** [@grazer](https://discuss.elastic.co/u/grazer)\ **Replies:** 1\ **Last updated:** [October 4, 2019, 1:55am UTC](https://discuss.elastic.co/t/converting-guids-to-readable-names/202255 "2019-10-04T01:55:30Z")
Hello, I'm parsing my exchange message trackinglogs and I want to convert GUIDS to Mailbox database names in logstash, for example; "MDB:f7b324b2-7cb3-4225-8346-6526d7c84674 "=\> DB01 thanks [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=465) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=467)