# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=354 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 355 --- ## [Convert Float to Geo\_Point](https://discuss.elastic.co/t/convert-float-to-geo-point/233743)
**Author:** [@Jasmin](https://discuss.elastic.co/u/Jasmin)\ **Replies:** 1\ **Last updated:** [May 21, 2020, 2:48pm UTC](https://discuss.elastic.co/t/convert-float-to-geo-point/233743 "2020-05-21T14:48:01Z")
Hi, i am trying to convert city name to location and see them in map , my the type of location.lat and location.lon are still float , i can't convert them to geo\_point , my logstash.conf is : input { jdbc { jdbc\_c… --- ## [How to split 2 curl command output in message](https://discuss.elastic.co/t/how-to-split-2-curl-command-output-in-message/233753)
**Author:** [@aannee](https://discuss.elastic.co/u/aannee)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 2:25pm UTC](https://discuss.elastic.co/t/how-to-split-2-curl-command-output-in-message/233753 "2020-05-21T14:25:33Z")
Hi, I'm still a newbie in ELK. So, it's a big help if you can give any advise to me. :smiley: I really want to explore ELK. I just need to ask if its possible to split the output of this 2 script. And how can I split it… --- ## [In LogStash, how remove any json/xml node/field larger than specific size](https://discuss.elastic.co/t/in-logstash-how-remove-any-json-xml-node-field-larger-than-specific-size/233752)
**Author:** [@jimisdrpc](https://discuss.elastic.co/u/jimisdrpc)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 2:11pm UTC](https://discuss.elastic.co/t/in-logstash-how-remove-any-json-xml-node-field-larger-than-specific-size/233752 "2020-05-21T14:11:13Z")
In few words, I have this stack in our company for our corporate logs: All Request/Response Log Files -\> Filebeat -\> Kafka -\> Logstash - ElasiicSearch Pretty common approach. Nevertheless, there might exist in unexpec… --- ## [How to force LogStash to connect using Ip Address instead of DNS names](https://discuss.elastic.co/t/how-to-force-logstash-to-connect-using-ip-address-instead-of-dns-names/233748)
**Author:** [@jimisdrpc](https://discuss.elastic.co/u/jimisdrpc)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 1:58pm UTC](https://discuss.elastic.co/t/how-to-force-logstash-to-connect-using-ip-address-instead-of-dns-names/233748 "2020-05-21T13:58:59Z")
In my local environment, I must use IP Address instead of Localhost or other nickname. My settup is Docker over Virtual Box instead of Docker For Windows. It is not part of my question discuss why I am using Virtual Box … --- ## [Query template not found by Logstash](https://discuss.elastic.co/t/query-template-not-found-by-logstash/233736)
**Author:** [@mielsi30](https://discuss.elastic.co/u/mielsi30)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 12:30pm UTC](https://discuss.elastic.co/t/query-template-not-found-by-logstash/233736 "2020-05-21T12:30:41Z")
Hello, I am using a .conf file which uses the elasticsearch filter to query an index and enrich the documents coming in. I am using a query template called search\_by\_title.json which is stored in the same directory as t… --- ## [Logstach nested XML indexing using ruby filter](https://discuss.elastic.co/t/logstach-nested-xml-indexing-using-ruby-filter/233560)
**Author:** [@Junior\_Zilles](https://discuss.elastic.co/u/Junior_Zilles)\ **Replies:** 0\ **Last updated:** [May 20, 2020, 2:45pm UTC](https://discuss.elastic.co/t/logstach-nested-xml-indexing-using-ruby-filter/233560 "2020-05-20T14:45:15Z")
So I have this XML with 30mb and I'm trying to put into Elasticsearch with logstash, each XML can have a lot of pages and each page need to be save in Elasticsearch what I have until now is: \ … --- ## [Stalled Logstash](https://discuss.elastic.co/t/stalled-logstash/233733)
**Author:** [@ethical20](https://discuss.elastic.co/u/ethical20)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 12:05pm UTC](https://discuss.elastic.co/t/stalled-logstash/233733 "2020-05-21T12:05:47Z")
Hi, I had logstash stalled, I've used the --pipeline.unsafe\_shutdown but no luck. Here are the logs. ubuntu@elk:~$ sudo service logstash start --pipeline.unsafe\_shutdown ^C ubuntu@elk:~$ cd /usr/share/logstash/ ubuntu@… --- ## [Logstash dont send to output](https://discuss.elastic.co/t/logstash-dont-send-to-output/233580)
**Author:** [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Replies:** 15\ **Last updated:** [May 21, 2020, 11:48am UTC](https://discuss.elastic.co/t/logstash-dont-send-to-output/233580 "2020-05-21T11:48:57Z")
its a new installation im collecting sflow data using sflow plugin debugging the sflow plugin i can see flows data getting to logstash but nothing is being sent to the output. ive tried to logz.io and to local file. … --- ## [JDBC input with SSL enabled](https://discuss.elastic.co/t/jdbc-input-with-ssl-enabled/233516)
**Author:** [@chandu5565](https://discuss.elastic.co/u/chandu5565)\ **Replies:** 1\ **Last updated:** [May 21, 2020, 9:59am UTC](https://discuss.elastic.co/t/jdbc-input-with-ssl-enabled/233516 "2020-05-21T09:59:52Z")
Hi, Earlier I was using logstash jdbc plugin to input mongodb data which was working fine. Now we have made changes to our mongodb and enabled SSL , replicaset with self signed certificate. I am not sure how can I give… --- ## [Geoip filter not creating type geo\_point but only string type](https://discuss.elastic.co/t/geoip-filter-not-creating-type-geo-point-but-only-string-type/233551)
**Author:** [@vivitare](https://discuss.elastic.co/u/vivitare)\ **Replies:** 2\ **Last updated:** [May 21, 2020, 7:44am UTC](https://discuss.elastic.co/t/geoip-filter-not-creating-type-geo-point-but-only-string-type/233551 "2020-05-21T07:44:54Z")
Hello, I'm trying to create a Kibana map with coordinates points from apache log using the plugin geoip. The problem is no geo\_point type are created in Elasticsearch. However, apache log are correctly parsed by filter… --- ## [Logstash 6.8.3 is crashing 'to\_hash\_with\_metadata'](https://discuss.elastic.co/t/logstash-6-8-3-is-crashing-to-hash-with-metadata/233689)
**Author:** [@denzhel](https://discuss.elastic.co/u/denzhel)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 7:26am UTC](https://discuss.elastic.co/t/logstash-6-8-3-is-crashing-to-hash-with-metadata/233689 "2020-05-21T07:26:54Z")
Hello everyone, I have 4 identical servers that runs the same deployment of Logstash 6.8.3, one of them, only one crashes with the following error: \[2020-05-21T06:11:31,662\]\[ERROR\]\[logstash.pipeline \] Exception … --- ## [ECS for Metricbeat to use "Kibana -\> Metrics"](https://discuss.elastic.co/t/ecs-for-metricbeat-to-use-kibana-metrics/233685)
**Author:** [@Zerobot](https://discuss.elastic.co/u/Zerobot)\ **Replies:** 0\ **Last updated:** [May 21, 2020, 7:14am UTC](https://discuss.elastic.co/t/ecs-for-metricbeat-to-use-kibana-metrics/233685 "2020-05-21T07:14:36Z")
Hi! I'm trying to filter my data into ECS properly, right now I'm trying to write CPU stats like perecentage. So my question is: Which fields are required for Kibana -\> Metrics \> CPU Usage? I'm using metricbeat's ind… --- ## [Mutate message in Logstash](https://discuss.elastic.co/t/mutate-message-in-logstash/233272)
**Author:** [@Joseph-Gan-wk](https://discuss.elastic.co/u/Joseph-Gan-wk)\ **Replies:** 2\ **Last updated:** [May 21, 2020, 7:11am UTC](https://discuss.elastic.co/t/mutate-message-in-logstash/233272 "2020-05-21T07:11:32Z")
Hi all, I want to ask is that Logstash filtering support the case like this:- value field is ABCDE, abcde, 12345, @#$% if the value field is ABCDE and abcde, then overwrite ABCE and abcde to 98765. mutate { match =\>… --- ## [Logstash throws runtime exception while parsing huge XML](https://discuss.elastic.co/t/logstash-throws-runtime-exception-while-parsing-huge-xml/231621)
**Author:** [@Saravana\_Maadavan](https://discuss.elastic.co/u/Saravana_Maadavan)\ **Replies:** 5\ **Last updated:** [May 21, 2020, 7:04am UTC](https://discuss.elastic.co/t/logstash-throws-runtime-exception-while-parsing-huge-xml/231621 "2020-05-21T07:04:22Z")
Hello, In Logstash I am trying to replace a prefix in the XML which is not having any definition and then indexing it to elastic. Receiving error while parsing some huge XMLs. I would be requiring the full XML to be sto… --- ## [Mutate gsub \\ not working](https://discuss.elastic.co/t/mutate-gsub-not-working/233455)
**Author:** [@Rafael\_Pereira\_Silva](https://discuss.elastic.co/u/Rafael_Pereira_Silva)\ **Replies:** 3\ **Last updated:** [May 21, 2020, 3:41am UTC](https://discuss.elastic.co/t/mutate-gsub-not-working/233455 "2020-05-21T03:41:45Z")
Hello, I have a log .... \\"pattern\\":\\"^\[0-9\]{9}$\\"},\\"typeId\\":{\\"enum\\":\[32\], \\"mysensitivefield\\":\\"ASDASDLAASDASDASLDASDAZ\\" , I need to replace mysensitivefield information. getting like this \*\*\*\*\*\*\*\*\*\* : \*\*\*\*\*… --- ## [Logstash Translate Dictionary not working](https://discuss.elastic.co/t/logstash-translate-dictionary-not-working/233420)
**Author:** [@AliMalik](https://discuss.elastic.co/u/AliMalik)\ **Replies:** 16\ **Last updated:** [May 21, 2020, 1:41am UTC](https://discuss.elastic.co/t/logstash-translate-dictionary-not-working/233420 "2020-05-21T01:41:29Z")
Hi All, I need some help. I am trying to use a translate filter in logstash to translate winlog.event\_data.AccessList value to some meaningful field but it's not working. below is my code in the filter file. Looks like … --- ## [Configuration with Filebeat and S3](https://discuss.elastic.co/t/configuration-with-filebeat-and-s3/233622)
**Author:** [@Annette1](https://discuss.elastic.co/u/Annette1)\ **Replies:** 0\ **Last updated:** [May 20, 2020, 9:50pm UTC](https://discuss.elastic.co/t/configuration-with-filebeat-and-s3/233622 "2020-05-20T21:50:58Z")
I currently have a logstash configuration running with filebeat using pipelines that is somewhat simplistic. Logstash is load balanced across two instances in the file beats configuration. The logstash pipelines are co… --- ## [Reversing The Order Of Characters to a date valid format](https://discuss.elastic.co/t/reversing-the-order-of-characters-to-a-date-valid-format/233443)
**Author:** [@Claudio\_Ract\_Costa](https://discuss.elastic.co/u/Claudio_Ract_Costa)\ **Replies:** 4\ **Last updated:** [May 20, 2020, 9:48pm UTC](https://discuss.elastic.co/t/reversing-the-order-of-characters-to-a-date-valid-format/233443 "2020-05-20T21:48:53Z")
Hi guys, Does anyone know how can I reverse the order of the number bellow to transform it in a valid date format ? I am trying to figure out how do it using mutate gsub or ruby but it is very complex. from: 02404231… --- ## [BottleNeck in local beats input](https://discuss.elastic.co/t/bottleneck-in-local-beats-input/232871)
**Author:** [@aerodynamic](https://discuss.elastic.co/u/aerodynamic)\ **Replies:** 4\ **Last updated:** [May 20, 2020, 7:39pm UTC](https://discuss.elastic.co/t/bottleneck-in-local-beats-input/232871 "2020-05-20T19:39:48Z")
Hello, I'm currently facing a bottleneck with Logstash. Currently my project is a single linux box running the FELK stack. I have determined that Logstash is a bottleneck by having filebeats write to a file (where I get… --- ## [Logstash to send events dynamically to kafka topics](https://discuss.elastic.co/t/logstash-to-send-events-dynamically-to-kafka-topics/233602)
**Author:** [@Sreekanth3](https://discuss.elastic.co/u/Sreekanth3)\ **Replies:** 0\ **Last updated:** [May 20, 2020, 7:33pm UTC](https://discuss.elastic.co/t/logstash-to-send-events-dynamically-to-kafka-topics/233602 "2020-05-20T19:33:21Z")
Hi , Is there a way to send the events received from a s3 bucket to kafka dynamically. Eg: First 10k events to one topic and next 10k events to other topic. It should send based on the count of events . --- ## [Logstash file input very slow with lots of files](https://discuss.elastic.co/t/logstash-file-input-very-slow-with-lots-of-files/233598)
**Author:** [@ByronTN](https://discuss.elastic.co/u/ByronTN)\ **Replies:** 0\ **Last updated:** [May 20, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-file-input-very-slow-with-lots-of-files/233598 "2020-05-20T19:20:56Z")
Hello, I am working to import a ton of old logs into our elastic cache cluster. These are cloudtrail files so there are millions of very small files I am trying to import. When starting Logstash with an input to glob o… --- ## [Help needed for logstash pipeline.yml settings](https://discuss.elastic.co/t/help-needed-for-logstash-pipeline-yml-settings/233583)
**Author:** [@annk](https://discuss.elastic.co/u/annk)\ **Replies:** 1\ **Last updated:** [May 20, 2020, 5:27pm UTC](https://discuss.elastic.co/t/help-needed-for-logstash-pipeline-yml-settings/233583 "2020-05-20T17:27:44Z")
Hi, I have around 5 pipelines running on a logstash node sending output to elasticsearch (4 node cluster). two of the pipelines require high throughput in some scenarios . usually we received around 1000 events/ sec e… --- ## [Using the multiline codec with input pipeline](https://discuss.elastic.co/t/using-the-multiline-codec-with-input-pipeline/233350)
**Author:** [@Offer\_Baruch](https://discuss.elastic.co/u/Offer_Baruch)\ **Replies:** 3\ **Last updated:** [May 20, 2020, 4:01pm UTC](https://discuss.elastic.co/t/using-the-multiline-codec-with-input-pipeline/233350 "2020-05-20T16:01:44Z")
Hello everyone, I am using pipeline communication to have a different pipeline to parse different file types as input. My main input pipeline recognizes the file type and passes control to the correct pipeline. I am t… --- ## [Could not index event to Elasticsearch. {:status=\>400, :action=\> #\\], :response=\>{"index"=\>ception", "reason"=\>"failed to parse date field](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-status-400-action-logstash-0x458fc470-response-index-ception-reason-failed-to-parse-date-field/233313)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 3\ **Last updated:** [May 20, 2020, 4:01pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-status-400-action-logstash-0x458fc470-response-index-ception-reason-failed-to-parse-date-field/233313 "2020-05-20T16:01:09Z")
\[2020-05-18T09:02:52,095\]\[WARN \]\[logstash.outputs.amazonelasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"gppepo-epo-govpp-scheduler-2020.05.18", :\_type=\>"\_doc… --- ## [Unable to add new field with values of nested fields](https://discuss.elastic.co/t/unable-to-add-new-field-with-values-of-nested-fields/233403)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 7\ **Last updated:** [May 20, 2020, 3:59pm UTC](https://discuss.elastic.co/t/unable-to-add-new-field-with-values-of-nested-fields/233403 "2020-05-20T15:59:00Z")
I am attempting to create a geo-point data type so that I can utilize the map features in kibana. My index will not show as an option when I attempt to add a layer and I believe its because I do not have a geo-point data… --- ## [Why does split creates multiple output records in json filter](https://discuss.elastic.co/t/why-does-split-creates-multiple-output-records-in-json-filter/233490)
**Author:** [@Ullas7](https://discuss.elastic.co/u/Ullas7)\ **Replies:** 1\ **Last updated:** [May 20, 2020, 3:56pm UTC](https://discuss.elastic.co/t/why-does-split-creates-multiple-output-records-in-json-filter/233490 "2020-05-20T15:56:41Z")
I havea JSON with array element in it { "K1": "V1", "k2":"v2", "k3": \[{"a1":"b1","a2":"b2"},{"a1":"c1","a2":d2}\] } After logstash filter, I saving this in Elastic search, but for single json record its ins… --- ## [Problem updating counter on elastic document](https://discuss.elastic.co/t/problem-updating-counter-on-elastic-document/231722)
**Author:** [@gpolini](https://discuss.elastic.co/u/gpolini)\ **Replies:** 4\ **Last updated:** [May 20, 2020, 3:48pm UTC](https://discuss.elastic.co/t/problem-updating-counter-on-elastic-document/231722 "2020-05-20T15:48:43Z")
Hi I populate ELK database reading log files by logstash. Each document is populated using many log rows, each one inserts specific fileds. When logstash read a specific action, it update a boolean filed "action\_done… --- ## [Logstash error when attempting to ingest](https://discuss.elastic.co/t/logstash-error-when-attempting-to-ingest/233565)
**Author:** [@wild\_turkey](https://discuss.elastic.co/u/wild_turkey)\ **Replies:** 1\ **Last updated:** [May 20, 2020, 3:45pm UTC](https://discuss.elastic.co/t/logstash-error-when-attempting-to-ingest/233565 "2020-05-20T15:45:56Z")
Hi all, I'm pretty new to this, and this will be my first attempt to load a file into Elasticsearch. I have a csv file. I have Kibana and Elasticsearch running on a Windows 10 laptop I configured a config file the fo… --- ## [Sflow plugin dont work](https://discuss.elastic.co/t/sflow-plugin-dont-work/233469)
**Author:** [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Replies:** 9\ **Last updated:** [May 20, 2020, 3:14pm UTC](https://discuss.elastic.co/t/sflow-plugin-dont-work/233469 "2020-05-20T15:14:50Z")
hi im new here just install logstash as shipper toward Logz.io ive install the sflow input plugin but i dont see logstash listening in any port. i have filebeat installed on this agent as well for syslog collecting. … --- ## [Passing in complex array as environment variables?](https://discuss.elastic.co/t/passing-in-complex-array-as-environment-variables/233508)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 3\ **Last updated:** [May 20, 2020, 1:43pm UTC](https://discuss.elastic.co/t/passing-in-complex-array-as-environment-variables/233508 "2020-05-20T13:43:04Z")
Hi there, I'm using the Logstash SNMP input to poll a list of servers with a list of OIDs. We are deploying Logstash in Docker using Ansible, and we try to maintain the code using environment variables passed through t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=353) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=355)