# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=854 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 855 --- ## [Logstash/eleasticsearch - geoip failed to parse from lat long](https://discuss.elastic.co/t/logstash-eleasticsearch-geoip-failed-to-parse-from-lat-long/111797)
**Author:** [@ajwillo](https://discuss.elastic.co/u/ajwillo)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 7:09pm UTC](https://discuss.elastic.co/t/logstash-eleasticsearch-geoip-failed-to-parse-from-lat-long/111797 "2017-12-18T19:09:01Z")
I'm using elastic search 6 and logstash 6. I have added the filter below: input { udp { port =\> 9996 codec =\> netflow { versions =\> \[5, 9\] } type =\> netflow tags =\> "checkpoint-flow" } ud… --- ## [Drop old messages](https://discuss.elastic.co/t/drop-old-messages/111775)
**Author:** [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 7:06pm UTC](https://discuss.elastic.co/t/drop-old-messages/111775 "2017-12-18T19:06:56Z")
Hi all, i'm reading with logstash several files and after send to ES. I this files i have also old log lines with timestamps. How can avoid to send to ES the rows with old timestamp? I can use drop filter, but what is… --- ## [Logstash is not picking up the heap memory settings from jvm.options file](https://discuss.elastic.co/t/logstash-is-not-picking-up-the-heap-memory-settings-from-jvm-options-file/111682)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 7:04pm UTC](https://discuss.elastic.co/t/logstash-is-not-picking-up-the-heap-memory-settings-from-jvm-options-file/111682 "2017-12-18T19:04:35Z")
I have added -Xms2g -Xmx2g in jvm.options file and restarted logstash. But this change is not reflecting. Version: 5.4.3 --- ## [Convert timestamp field from yyyy-MM-dd HH:mm:ss to yyyy-MM-dd HH:mm:ss.SSS](https://discuss.elastic.co/t/convert-timestamp-field-from-yyyy-mm-dd-hh-mm-ss-to-yyyy-mm-dd-hh-mm-ss-sss/111671)
**Author:** [@cmadera\_rp](https://discuss.elastic.co/u/cmadera_rp)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 7:04pm UTC](https://discuss.elastic.co/t/convert-timestamp-field-from-yyyy-mm-dd-hh-mm-ss-to-yyyy-mm-dd-hh-mm-ss-sss/111671 "2017-12-18T19:04:02Z")
Hello I have a logstash input from amazon kinesis than one field "news\_timestamp" have an timestamp with this format "yyyy-MM-dd HH:mm:ss" I need to update this field adding the milliseconds .000 like this "yyyy-MM-dd H… --- ## [GELF Logs from Graylog to Logstash](https://discuss.elastic.co/t/gelf-logs-from-graylog-to-logstash/111730)
**Author:** [@MultiplierMultiplier](https://discuss.elastic.co/u/MultiplierMultiplier)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 9:35am UTC](https://discuss.elastic.co/t/gelf-logs-from-graylog-to-logstash/111730 "2017-12-18T09:35:03Z")
INFO: I am running version 6.1 on all of my Elastic Stack components, these are Ubuntu VM's hosted in Azure. I have listed the contents of the Gralyog logs below and the Logstash logs show nothing (Even when set to debug… --- ## [I can't have the index logstash using mydomaine:9200/\_cat/indices?v](https://discuss.elastic.co/t/i-cant-have-the-index-logstash-using-mydomaine-9200-cat-indices-v/111967)
**Author:** [@marouane](https://discuss.elastic.co/u/marouane)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 4:33pm UTC](https://discuss.elastic.co/t/i-cant-have-the-index-logstash-using-mydomaine-9200-cat-indices-v/111967 "2017-12-18T16:33:37Z")
Hello every one i'm working on adding the index logstash i followed this link : Index and Query a Document | Elasticsearch Guide \[1.4\] | Elastic and i exected this : curl -XPUT 'localhost:9200/customer/external/1?pre… --- ## [Codec gzip\_lines not working?](https://discuss.elastic.co/t/codec-gzip-lines-not-working/112271)
**Author:** [@Michel99\_7](https://discuss.elastic.co/u/Michel99_7)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 4:00pm UTC](https://discuss.elastic.co/t/codec-gzip-lines-not-working/112271 "2017-12-18T16:00:02Z")
I saw many topics talking about this subject but nobody really answer: I use this gzip\_lines codec to extract log information from .csv files contained within .gz files the codec used in the input section is uptodate =… --- ## [Heap size filling up but CPU usage remains very low](https://discuss.elastic.co/t/heap-size-filling-up-but-cpu-usage-remains-very-low/112253)
**Author:** [@bamowens](https://discuss.elastic.co/u/bamowens)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 2:51pm UTC](https://discuss.elastic.co/t/heap-size-filling-up-but-cpu-usage-remains-very-low/112253 "2017-12-18T14:51:42Z")
I have a very basic passthrough logstash instance with a single http input, one date formatting filter, and two outputs, one to s3 and one to another logstash instance. I upgraded to 6.1 recently, but am continuing to s… --- ## [How to format date in logstash](https://discuss.elastic.co/t/how-to-format-date-in-logstash/112250)
**Author:** [@saidesh\_kilaru](https://discuss.elastic.co/u/saidesh_kilaru)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 2:38pm UTC](https://discuss.elastic.co/t/how-to-format-date-in-logstash/112250 "2017-12-18T14:38:11Z")
I have date string and i need to convert that to date with specific format. for example: dattime="2017-12-12 11:12:00"; date { match =\> \[ "dattime", "yyyy-MM-dd HH:mm:ss" \] target =\> "dattime" } when iam doing this… --- ## [Split field logstash](https://discuss.elastic.co/t/split-field-logstash/112217)
**Author:** [@sureshboiina](https://discuss.elastic.co/u/sureshboiina)\ **Replies:** 3\ **Last updated:** [December 18, 2017, 2:07pm UTC](https://discuss.elastic.co/t/split-field-logstash/112217 "2017-12-18T14:07:19Z")
hi input 2017-12-18 16:50:54.462 \[http-bio-8080-exec-9\] DEBUG o.s.w.f.CommonsRequestLoggingFilter.afterRequest(55) - After request \[uri=/tng-rfid-api-gateway/gateway/api/v1/pre-checks/01;client=172.31.100.151;user=admi… --- ## [Logstash convert string field to number](https://discuss.elastic.co/t/logstash-convert-string-field-to-number/112023)
**Author:** [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)\ **Replies:** 5\ **Last updated:** [December 18, 2017, 1:56pm UTC](https://discuss.elastic.co/t/logstash-convert-string-field-to-number/112023 "2017-12-18T13:56:38Z")
Hi I'm trying to ingest durationUs=786 into numeric value in elasticsearch but no luck tried using mutate but the events still shows as string so any help would be great if \[type\] == "pump" { grok { match =\> { "mess… --- ## [Tcp input is broken on Z/os socket](https://discuss.elastic.co/t/tcp-input-is-broken-on-z-os-socket/109916)
**Author:** [@roytev](https://discuss.elastic.co/u/roytev)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 12:35pm UTC](https://discuss.elastic.co/t/tcp-input-is-broken-on-z-os-socket/109916 "2017-12-18T12:35:32Z")
Hello! i'm sending json messages over TCP sockets from Mainframe using a REEXX/socket and it worked for me flawlessly from version 5.0 through 5.50, when i upgraded the stack to version 6.0(skipped 5.6) the messages ar… --- ## [Add logstash version to record](https://discuss.elastic.co/t/add-logstash-version-to-record/112228)
**Author:** [@ericrichter](https://discuss.elastic.co/u/ericrichter)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 12:21pm UTC](https://discuss.elastic.co/t/add-logstash-version-to-record/112228 "2017-12-18T12:21:30Z")
How can I add the version of logstash to a record? The beats agent has a 'beat.version' field, but I can't find the same for logstash. --- ## [First occurrence of multiple same log messages](https://discuss.elastic.co/t/first-occurrence-of-multiple-same-log-messages/112214)
**Author:** [@Shubhank\_Gupta](https://discuss.elastic.co/u/Shubhank_Gupta)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 11:09am UTC](https://discuss.elastic.co/t/first-occurrence-of-multiple-same-log-messages/112214 "2017-12-18T11:09:31Z")
I want to note down the first occurrence of a particular log type and adding key event according to that. Also after that if same type of log comes again than don't do anything till the type of log doesn't changes. How… --- ## [TCP vs HTTP input - reliability](https://discuss.elastic.co/t/tcp-vs-http-input-reliability/112215)
**Author:** [@vb4t](https://discuss.elastic.co/u/vb4t)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 11:10am UTC](https://discuss.elastic.co/t/tcp-vs-http-input-reliability/112215 "2017-12-18T11:10:43Z")
Hello. I need to collect logs using nxlog log shipper (can not use filebeat - unsupported OS). I am able to send events using TCP or HTTP. Probably TCP input is faster but HTTP seems to be more reliable (ACK on applicati… --- ## [Multiline codec in if else condition in beat input of logstash](https://discuss.elastic.co/t/multiline-codec-in-if-else-condition-in-beat-input-of-logstash/110323)
**Author:** [@Shubhank\_Gupta](https://discuss.elastic.co/u/Shubhank_Gupta)\ **Replies:** 7\ **Last updated:** [December 18, 2017, 11:00am UTC](https://discuss.elastic.co/t/multiline-codec-in-if-else-condition-in-beat-input-of-logstash/110323 "2017-12-18T11:00:09Z")
I want to put condition on input logs through filebeat using type of log but in input section we can't put if else condition if i am right ? Suggest a method to apply multiline codec on a particular type of log only. --- ## [If statement in grok filter](https://discuss.elastic.co/t/if-statement-in-grok-filter/112186)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 10:27am UTC](https://discuss.elastic.co/t/if-statement-in-grok-filter/112186 "2017-12-18T10:27:35Z")
hi everyone! is there anyway to check aws logstream name in logstash grok filter? eg: filter { if "cloudwatch\_logs.log\_stream" == "TEST1" { grok { # type1 match =\> { "message" =\> \[ "^(?\\\d… --- ## [Snmptrap plugin is not decoding a specific field and rest of them are decoded](https://discuss.elastic.co/t/snmptrap-plugin-is-not-decoding-a-specific-field-and-rest-of-them-are-decoded/111096)
**Author:** [@George\_Cherian](https://discuss.elastic.co/u/George_Cherian)\ **Replies:** 0\ **Last updated:** [December 11, 2017, 1:19pm UTC](https://discuss.elastic.co/t/snmptrap-plugin-is-not-decoding-a-specific-field-and-rest-of-them-are-decoded/111096 "2017-12-11T13:19:36Z")
I am using snmp plugin in logstash for getting snmp trap alarms. The plugin is not decoding the trap name but the field is getting decoded as shown below: { "SNMPv2-MIB::snmpTrapOID.0" =\> "BroadworksFault… --- ## [Pushing logs to S3 vs local backups!](https://discuss.elastic.co/t/pushing-logs-to-s3-vs-local-backups/112183)
**Author:** [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 9:13am UTC](https://discuss.elastic.co/t/pushing-logs-to-s3-vs-local-backups/112183 "2017-12-18T09:13:59Z")
Hi, I wanted to know which is better to have parsed logs backup into S3 bucket or write on local disk. Local disk is cheaper. With S3 output creates temporary files into the OS temporary directory, then will transfer th… --- ## [Problem with logstash-filter-elasticsearch when connecting to SSL enabled ES cluster](https://discuss.elastic.co/t/problem-with-logstash-filter-elasticsearch-when-connecting-to-ssl-enabled-es-cluster/111717)
**Author:** [@admlko](https://discuss.elastic.co/u/admlko)\ **Replies:** 0\ **Last updated:** [December 14, 2017, 8:59am UTC](https://discuss.elastic.co/t/problem-with-logstash-filter-elasticsearch-when-connecting-to-ssl-enabled-es-cluster/111717 "2017-12-14T08:59:19Z")
Hi, I'm having serious issues when trying to query data from SSL enabled ES cluster. By serious issues, I mean that I have tried different parameters, adding and removing protocol and port from hosts-array etc etc for … --- ## [How do I create an index name based on a time field that is not the timestamp?](https://discuss.elastic.co/t/how-do-i-create-an-index-name-based-on-a-time-field-that-is-not-the-timestamp/112035)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 6:25am UTC](https://discuss.elastic.co/t/how-do-i-create-an-index-name-based-on-a-time-field-that-is-not-the-timestamp/112035 "2017-12-18T06:25:19Z")
I am using the @timestamp field for when the document was indexed, but have a separate field (\[page\]\[post\_date\]) for a date of when the event occurred. I would like to create index names based on the \[page\]\[post\_date\] fi… --- ## [Create pattern in logstash](https://discuss.elastic.co/t/create-pattern-in-logstash/111221)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 15\ **Last updated:** [December 18, 2017, 6:19am UTC](https://discuss.elastic.co/t/create-pattern-in-logstash/111221 "2017-12-18T06:19:56Z")
Greetings, My logs are of the format 2017-12-08 01:42:23,658 app="capture", level=ERROR, thread=play-thread-5, logger=play, userId="client:release", reqAction="GET", reqUrl="/api/labels", message=" console.App.html ac… --- ## [Can i run logstash without conf file, all conf data in command line?](https://discuss.elastic.co/t/can-i-run-logstash-without-conf-file-all-conf-data-in-command-line/111205)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 6\ **Last updated:** [December 17, 2017, 11:46am UTC](https://discuss.elastic.co/t/can-i-run-logstash-without-conf-file-all-conf-data-in-command-line/111205 "2017-12-17T11:46:27Z")
Hi All Can i run logstash without conf file, all conf data in command line? I am using multiple conf files, for which it is hard for me to maintain these small conf files in server. I am looking a way wherein i can us… --- ## [Access data event and field in logstash condition error](https://discuss.elastic.co/t/access-data-event-and-field-in-logstash-condition-error/112109)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 10\ **Last updated:** [December 17, 2017, 9:46am UTC](https://discuss.elastic.co/t/access-data-event-and-field-in-logstash-condition-error/112109 "2017-12-17T09:46:24Z")
Im ussing LS 6.1 When i use condition in logstash, it not work. i dont know that why? I should be run into condition (True) instead of False. It not run in " if \[nginx\]\[access\]\[remote\_ip\] { ... } " Sample log { "\_i… --- ## [How-to-setup-logstash-agent-secure-ssl-communication-with-rabbitmq (https://github.com/logstash-plugins/logstash-output-rabbitmq)](https://discuss.elastic.co/t/how-to-setup-logstash-agent-secure-ssl-communication-with-rabbitmq-https-github-com-logstash-plugins-logstash-output-rabbitmq/112077)
**Author:** [@prathap\_b](https://discuss.elastic.co/u/prathap_b)\ **Replies:** 0\ **Last updated:** [December 16, 2017, 9:55am UTC](https://discuss.elastic.co/t/how-to-setup-logstash-agent-secure-ssl-communication-with-rabbitmq-https-github-com-logstash-plugins-logstash-output-rabbitmq/112077 "2017-12-16T09:55:01Z")
Continuing the discussion from How to setup Logstash agent secure SSL communication with RabbitMQ and eventually to ELK server Hello, I've been working on similar setup as in this topic (How to setup Logstash agent sec… --- ## [How to configure Serializers / Deserializers in the logstash input plugin?](https://discuss.elastic.co/t/how-to-configure-serializers-deserializers-in-the-logstash-input-plugin/112057)
**Author:** [@huawang](https://discuss.elastic.co/u/huawang)\ **Replies:** 0\ **Last updated:** [December 16, 2017, 12:18am UTC](https://discuss.elastic.co/t/how-to-configure-serializers-deserializers-in-the-logstash-input-plugin/112057 "2017-12-16T00:18:46Z")
We have custom Deserializer to be used in the logstash input plugin. There are several parameters that need to be configured. We are hoping to be able to pass in the configure via the input plugin section in the logstash… --- ## [What happens when you use \`schedule\` with JDBC and the first import query hasn't finished yet?](https://discuss.elastic.co/t/what-happens-when-you-use-schedule-with-jdbc-and-the-first-import-query-hasnt-finished-yet/112054)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 0\ **Last updated:** [December 15, 2017, 11:32pm UTC](https://discuss.elastic.co/t/what-happens-when-you-use-schedule-with-jdbc-and-the-first-import-query-hasnt-finished-yet/112054 "2017-12-15T23:32:24Z")
Will Logstash have two or more queries running at once? Or will it be smart enough to see it hasn't finished the existing one and not pile them up? --- ## [Logstash not sending logs](https://discuss.elastic.co/t/logstash-not-sending-logs/111564)
**Author:** [@anudeepmk](https://discuss.elastic.co/u/anudeepmk)\ **Replies:** 3\ **Last updated:** [December 15, 2017, 6:52pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs/111564 "2017-12-15T18:52:53Z")
I am newbie to ELK, I am trying to send the SAP hybris logs to elasticsearch from logstash on the server input { file { path =\> "/hybris/log/hybris\*.log" type =\> "hybris" start\_po… --- ## [Filter the palo alto Global protect logs](https://discuss.elastic.co/t/filter-the-palo-alto-global-protect-logs/111405)
**Author:** [@rpendela](https://discuss.elastic.co/u/rpendela)\ **Replies:** 3\ **Last updated:** [December 15, 2017, 5:34pm UTC](https://discuss.elastic.co/t/filter-the-palo-alto-global-protect-logs/111405 "2017-12-15T17:34:25Z")
Hello Community, We recently implemented ELK stack using docker compose, I am trying to visualize the palo alto global protect logs for login status and ip address. Below snip to give idea how I am filtering for system… --- ## [Logstash http output plugin ignore ssl verification](https://discuss.elastic.co/t/logstash-http-output-plugin-ignore-ssl-verification/110888)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 0\ **Last updated:** [December 8, 2017, 7:38pm UTC](https://discuss.elastic.co/t/logstash-http-output-plugin-ignore-ssl-verification/110888 "2017-12-08T19:38:52Z")
logstash http plugin (https://github.com/logstash-plugins/logstash-output-http/blob/master/lib/logstash/outputs/http.rb) have been pulled and modifed for our usecase but call need to be ssl verification ignore . at requ… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=853) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=855)