# Logstash

**URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=961

[Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md)

**Page:** 962

---

## [Logstash have delay](https://discuss.elastic.co/t/logstash-have-delay/90187)

<div class="topic-metadata">

**Author:** [@xiao123](https://discuss.elastic.co/u/xiao123)\
**Replies:** 0\
**Last updated:** [June 21, 2017, 2:02am UTC](https://discuss.elastic.co/t/logstash-have-delay/90187 "2017-06-21T02:02:39Z")

</div>

logstash should collect the latest data , but it collect the data hours ago for example, the newest log is 19:00:00 , but it collect the 08:00:00 data this is my configure input { file { path =\> \[ "/appl…

---

## [Logstash Netflow translate and convert a field](https://discuss.elastic.co/t/logstash-netflow-translate-and-convert-a-field/90173)

<div class="topic-metadata">

**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\
**Replies:** 4\
**Last updated:** [June 21, 2017, 1:22am UTC](https://discuss.elastic.co/t/logstash-netflow-translate-and-convert-a-field/90173 "2017-06-21T01:22:12Z")

</div>

Hi there, I'm trying to add extra information to my netflow field, more specifically netflow.direction Currently it uses number format but I would like it to be a string as I'm adding a string I've ended up doing thi…

---

## [Convert Hex to IEEE-754 float](https://discuss.elastic.co/t/convert-hex-to-ieee-754-float/89863)

<div class="topic-metadata">

**Author:** [@Xavier\_Clavien](https://discuss.elastic.co/u/Xavier_Clavien)\
**Replies:** 1\
**Last updated:** [June 20, 2017, 10:09pm UTC](https://discuss.elastic.co/t/convert-hex-to-ieee-754-float/89863 "2017-06-20T22:09:46Z")

</div>

Hello I need to convert a Hex value in floatting point 32bits can I use ruby for that? if yes, how? Any other suggestions? Thanks for your help

---

## [Logstash not connecting to Oracle DB - resolved](https://discuss.elastic.co/t/logstash-not-connecting-to-oracle-db-resolved/89977)

<div class="topic-metadata">

**Author:** [@sfunk](https://discuss.elastic.co/u/sfunk)\
**Replies:** 1\
**Last updated:** [June 20, 2017, 9:49pm UTC](https://discuss.elastic.co/t/logstash-not-connecting-to-oracle-db-resolved/89977 "2017-06-20T21:49:27Z")

</div>

I am attempting to connect to an Oracle database with Logstash 5.4.1 on CentOS 7.x. I have the Oracle driver on the machine as well, ojdbc7.jar. When I execute logstash manually to test things, the config appears to be…

---

## [Grok: How to loop and number the field consecutively](https://discuss.elastic.co/t/grok-how-to-loop-and-number-the-field-consecutively/90161)

<div class="topic-metadata">

**Author:** [@Ralf2k](https://discuss.elastic.co/u/Ralf2k)\
**Replies:** 0\
**Last updated:** [June 20, 2017, 8:56pm UTC](https://discuss.elastic.co/t/grok-how-to-loop-and-number-the-field-consecutively/90161 "2017-06-20T20:56:39Z")

</div>

Hi all, I am looking at a part of my log-file. I already took this out timestamp etc and it would be too long to post it here. Now part of the interessting message are separated with pipes. I try to get the different f…

---

## [Issues with Logstash filter ignoring logical elements](https://discuss.elastic.co/t/issues-with-logstash-filter-ignoring-logical-elements/90159)

<div class="topic-metadata">

**Author:** [@Problematiq](https://discuss.elastic.co/u/Problematiq)\
**Replies:** 0\
**Last updated:** [June 20, 2017, 8:41pm UTC](https://discuss.elastic.co/t/issues-with-logstash-filter-ignoring-logical-elements/90159 "2017-06-20T20:41:57Z")

</div>

Here is a snip-it of my filter. filter { if "ASA" in \[tags\] { grok { patterns\_dir =\> \["/etc/logstash/patterns"\] match =\> \[ "message", "%{CISCOFW106023}", "message", "%{CISCOFW313005}", "message", "%{CISCOFW7331…

---

## [Mapping conflicts](https://discuss.elastic.co/t/mapping-conflicts/89990)

<div class="topic-metadata">

**Author:** [@DanMa](https://discuss.elastic.co/u/DanMa)\
**Replies:** 1\
**Last updated:** [June 20, 2017, 6:55pm UTC](https://discuss.elastic.co/t/mapping-conflicts/89990 "2017-06-20T18:55:21Z")

</div>

Hey guys, ive installed a new ELK Stack. I want to filter some Cisco ASA Firewall Logs and tried to find the best filters. Because of some strange behavior would i go to the main configuration without filters (only…

---

## [Using Logstash to modify syslog messages prior to send them to a syslog server](https://discuss.elastic.co/t/using-logstash-to-modify-syslog-messages-prior-to-send-them-to-a-syslog-server/90144)

<div class="topic-metadata">

**Author:** [@leoreginin](https://discuss.elastic.co/u/leoreginin)\
**Replies:** 0\
**Last updated:** [June 20, 2017, 6:34pm UTC](https://discuss.elastic.co/t/using-logstash-to-modify-syslog-messages-prior-to-send-them-to-a-syslog-server/90144 "2017-06-20T18:34:42Z")

</div>

Hi, Maybe this's a silly question, but I need to remove some part of a syslog message prior to send it to the proper syslog server. Is it possible use Logstash to do that ? How would be the config to, for example, remo…

---

## [Logstash nested array parsing](https://discuss.elastic.co/t/logstash-nested-array-parsing/90009)

<div class="topic-metadata">

**Author:** [@Shuxin\_Lin](https://discuss.elastic.co/u/Shuxin_Lin)\
**Replies:** 2\
**Last updated:** [June 20, 2017, 5:37pm UTC](https://discuss.elastic.co/t/logstash-nested-array-parsing/90009 "2017-06-20T17:37:01Z")

</div>

Hi, Right now I have an entry like this after applying kv filter: "b/r/n": "1/1/0", I would like this entry to be expanded as: "b/r/n": { # the field name is not strictly to be "b/r/n" "b": 1, "r"…

---

## [Problem in parsing XML files using logstash?](https://discuss.elastic.co/t/problem-in-parsing-xml-files-using-logstash/90096)

<div class="topic-metadata">

**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\
**Replies:** 5\
**Last updated:** [June 20, 2017, 2:58pm UTC](https://discuss.elastic.co/t/problem-in-parsing-xml-files-using-logstash/90096 "2017-06-20T14:58:14Z")

</div>

Hi, My xml file is like this: \<note\> \<to\>Tove\</to\> \<from\>Jani\</from\> \<heading\>Reminder\</heading\> \<body\>Don't forget me this weekend!\</body\> \</note\> \<note\> \<to\>Tove\</to\> \<from\>Jani\</from\> \<heading\>Reminder\</heading\> \<…

---

## [Get current time using ruby filter in logstash](https://discuss.elastic.co/t/get-current-time-using-ruby-filter-in-logstash/90084)

<div class="topic-metadata">

**Author:** [@sachintanpure85](https://discuss.elastic.co/u/sachintanpure85)\
**Replies:** 8\
**Last updated:** [June 20, 2017, 2:10pm UTC](https://discuss.elastic.co/t/get-current-time-using-ruby-filter-in-logstash/90084 "2017-06-20T14:10:46Z")

</div>

I need to add the filed value for my index as current time. like Updated\_date column value as current time. As reference of post :https://discuss.elastic.co/t/add-field-timestamp-with-current-time/59862 we need ruby f…

---

## [Remove header fields generated by http input plugin](https://discuss.elastic.co/t/remove-header-fields-generated-by-http-input-plugin/90108)

<div class="topic-metadata">

**Author:** [@Marc\_Aragones](https://discuss.elastic.co/u/Marc_Aragones)\
**Replies:** 1\
**Last updated:** [June 20, 2017, 2:08pm UTC](https://discuss.elastic.co/t/remove-header-fields-generated-by-http-input-plugin/90108 "2017-06-20T14:08:05Z")

</div>

Hi, When I use http input, Logstash adds the following fields to Elasticsearch: headers.http\_accept headers.content\_type headers.request\_path headers.http\_version headers.request\_method headers.http\_host:localhost h…

---

## [Improve performance of Logstash data loading into ES](https://discuss.elastic.co/t/improve-performance-of-logstash-data-loading-into-es/88828)

<div class="topic-metadata">

**Author:** [@udaykirankona](https://discuss.elastic.co/u/udaykirankona)\
**Replies:** 15\
**Last updated:** [June 20, 2017, 1:51pm UTC](https://discuss.elastic.co/t/improve-performance-of-logstash-data-loading-into-es/88828 "2017-06-20T13:51:03Z")

</div>

HI, I am trying to load data into elasticsearch using logstash, but its loading very slow. 80000/minute into elasticsearch. So, its taking 10 mins for 11GB data. I have requirement to reduce this to 10 secs. Please h…

---

## [How to split and filtring xml in LogStash](https://discuss.elastic.co/t/how-to-split-and-filtring-xml-in-logstash/90065)

<div class="topic-metadata">

**Author:** [@mohammed\_cherif](https://discuss.elastic.co/u/mohammed_cherif)\
**Replies:** 8\
**Last updated:** [June 20, 2017, 1:51pm UTC](https://discuss.elastic.co/t/how-to-split-and-filtring-xml-in-logstash/90065 "2017-06-20T13:51:03Z")

</div>

I have the following logstash conf file: input { file { path =\> "C:/logstash/data/test.xml" start\_position =\> beginning codec =\> multiline { pattern =\> "^\<\\?provi…

---

## [Logstash return nothing (stdout)](https://discuss.elastic.co/t/logstash-return-nothing-stdout/88427)

<div class="topic-metadata">

**Author:** [@Sam67000](https://discuss.elastic.co/u/Sam67000)\
**Replies:** 8\
**Last updated:** [June 20, 2017, 1:14pm UTC](https://discuss.elastic.co/t/logstash-return-nothing-stdout/88427 "2017-06-20T13:14:39Z")

</div>

Hello EveryOne, I just installed logstash on a linux server (RPM installation). I have a little problem when I run it. Here is my configuration file: input { file { path =\> "/home/ldap/user/\*.log" …

---

## [Index.translog.durability: async logstash](https://discuss.elastic.co/t/index-translog-durability-async-logstash/90095)

<div class="topic-metadata">

**Author:** [@michielp](https://discuss.elastic.co/u/michielp)\
**Replies:** 0\
**Last updated:** [June 20, 2017, 12:40pm UTC](https://discuss.elastic.co/t/index-translog-durability-async-logstash/90095 "2017-06-20T12:40:51Z")

</div>

Hi, Each morning I need to do the following two things: First I set the translog to async state. # curl -XPUT 'http://localhost:9200/\_all/\_settings?preserve\_existing=true' -d '{"index.translog.durability" : "async"…

---

## [Field conversion to integer](https://discuss.elastic.co/t/field-conversion-to-integer/89739)

<div class="topic-metadata">

**Author:** [@pmohan](https://discuss.elastic.co/u/pmohan)\
**Replies:** 3\
**Last updated:** [June 20, 2017, 12:10pm UTC](https://discuss.elastic.co/t/field-conversion-to-integer/89739 "2017-06-20T12:10:37Z")

</div>

Hi, I'm new to this elastic serach\\logstash world ! This is one filed(extra\_fields) that i'm working on at the mint | /PublishingServices/\*\*/events/\*\*: 6| /PublishingServices/\*\*/sport: 1| /PublishingServices/\*\*/event…

---

## [How to filter results based upon values using logstash ruby](https://discuss.elastic.co/t/how-to-filter-results-based-upon-values-using-logstash-ruby/89913)

<div class="topic-metadata">

**Author:** [@sachintanpure85](https://discuss.elastic.co/u/sachintanpure85)\
**Replies:** 3\
**Last updated:** [June 20, 2017, 12:02pm UTC](https://discuss.elastic.co/t/how-to-filter-results-based-upon-values-using-logstash-ruby/89913 "2017-06-20T12:02:01Z")

</div>

I want filter the results based upon some column values of the index and perform math operation. 'tier0\_cnt','tier1\_cnt','tier2\_cnt','tier0\_amt','tier1\_amt','tier2\_amt' are the columns of my index. I am using below logst…

---

## [Ignore specific fields?](https://discuss.elastic.co/t/ignore-specific-fields/89182)

<div class="topic-metadata">

**Author:** [@Kissaki](https://discuss.elastic.co/u/Kissaki)\
**Replies:** 4\
**Last updated:** [June 20, 2017, 11:02am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182 "2017-06-20T11:02:45Z")

</div>

Hey, I'm using logstash to move already parsed messages in a json format from redis to elasticsearch. To determine the index a document is indexed to, I have a @index field in that message and a @type field for the typ…

---

## [Using an ingest pipeline from logstash to update a document](https://discuss.elastic.co/t/using-an-ingest-pipeline-from-logstash-to-update-a-document/90075)

<div class="topic-metadata">

**Author:** [@parosio](https://discuss.elastic.co/u/parosio)\
**Replies:** 0\
**Last updated:** [June 20, 2017, 10:18am UTC](https://discuss.elastic.co/t/using-an-ingest-pipeline-from-logstash-to-update-a-document/90075 "2017-06-20T10:18:56Z")

</div>

Hello, I'm trying using an ingest pipeline from logstash to grok some fields. It works correctly with "action =\> "index"", but not when action =\> "update" doc\_as\_upsert =\> "true" Is this the correct wa…

---

## [X-pack installation with logstash](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175)

<div class="topic-metadata">

**Author:** [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Replies:** 12\
**Last updated:** [June 20, 2017, 9:48am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175 "2017-06-20T09:48:20Z")

</div>

hi... I have been trying to install x pack in my ELK cluster of 3 nodes. I have elasticsearch on all 3 nodes in logstash and kibana in 1 node which I operate physically and rest using ssh server . I have installed x pack…

---

## [Kinesis Stream Output Not working](https://discuss.elastic.co/t/kinesis-stream-output-not-working/90049)

<div class="topic-metadata">

**Author:** [@Amir\_Gilad](https://discuss.elastic.co/u/Amir_Gilad)\
**Replies:** 1\
**Last updated:** [June 20, 2017, 9:38am UTC](https://discuss.elastic.co/t/kinesis-stream-output-not-working/90049 "2017-06-20T09:38:49Z")

</div>

Hi, I'm trying to use the Kinesis Stream Output Filter. I've found it here: Kinesis Stream Output FIlter Git This is the output section from my .conf file: ##########################################################…

---

## [Logstash plugin mail error](https://discuss.elastic.co/t/logstash-plugin-mail-error/89964)

<div class="topic-metadata">

**Author:** [@baha1](https://discuss.elastic.co/u/baha1)\
**Replies:** 2\
**Last updated:** [June 20, 2017, 9:34am UTC](https://discuss.elastic.co/t/logstash-plugin-mail-error/89964 "2017-06-20T09:34:28Z")

</div>

Hi every body, i use mail plugin in logstash but doesn't work.i get this error: \[ERROR\]\[logstash.outputs.email \] Something happen while delivering an email {:exception=\>#\<EOFError: End of file reached this my config: …

---

## [Splitting logs to custom format](https://discuss.elastic.co/t/splitting-logs-to-custom-format/89824)

<div class="topic-metadata">

**Author:** [@tomer](https://discuss.elastic.co/u/tomer)\
**Replies:** 3\
**Last updated:** [June 20, 2017, 8:33am UTC](https://discuss.elastic.co/t/splitting-logs-to-custom-format/89824 "2017-06-20T08:33:15Z")

</div>

Hi I have logs that in general looks like: UID A\_TS B\_TS C\_TS D\_TS X Y Z index: (UID = unique Id , TS = time Stamp , X etc = other values ) I want to create from those logs tables ("logs") that will be in the follo…

---

## [Error while loading logstash](https://discuss.elastic.co/t/error-while-loading-logstash/90035)

<div class="topic-metadata">

**Author:** [@ent\_1](https://discuss.elastic.co/u/ent_1)\
**Replies:** 4\
**Last updated:** [June 20, 2017, 7:46am UTC](https://discuss.elastic.co/t/error-while-loading-logstash/90035 "2017-06-20T07:46:06Z")

</div>

Hi , i am unable to upload the data to elastic search using logstash ? config file : input { jdbc { jdbc\_driver\_library =\> "/home/ec2-user/ojdbc14.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.Orac…

---

## [Nonei](https://discuss.elastic.co/t/nonei/90020)

<div class="topic-metadata">

**Author:** [@Bharath\_Raju](https://discuss.elastic.co/u/Bharath_Raju)\
**Replies:** 0\
**Last updated:** [June 20, 2017, 2:46am UTC](https://discuss.elastic.co/t/nonei/90020 "2017-06-20T02:46:54Z")

</div>

none235768fv

---

## [FreeIPA Log Configuration for Logstash (Grok Filter)](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876)

<div class="topic-metadata">

**Author:** [@mvelez83](https://discuss.elastic.co/u/mvelez83)\
**Replies:** 5\
**Last updated:** [June 20, 2017, 1:11am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876 "2017-06-20T01:11:03Z")

</div>

Hello, I was trying to follow https://www.freeipa.org/page/Centralized\_Logging to configure the following logs from FreeIPA in logstash: /var/log/httpd/error\_log: FreeIPA API call logs (and Apache errors) /var/log/kr…

---

## [Unable to convert to geopoint](https://discuss.elastic.co/t/unable-to-convert-to-geopoint/90002)

<div class="topic-metadata">

**Author:** [@potatoshane](https://discuss.elastic.co/u/potatoshane)\
**Replies:** 3\
**Last updated:** [June 20, 2017, 1:06am UTC](https://discuss.elastic.co/t/unable-to-convert-to-geopoint/90002 "2017-06-20T01:06:17Z")

</div>

Hello, I am currently using the geoip filter to generate geopoints. However, when it is fed into Elasticsearch, the type for geoip.location is number, which I am assuming is float. I need it to be geo\_point in order to …

---

## [HEXEWKB to Lat, Long in Logstash?](https://discuss.elastic.co/t/hexewkb-to-lat-long-in-logstash/89988)

<div class="topic-metadata">

**Author:** [@infamous](https://discuss.elastic.co/u/infamous)\
**Replies:** 1\
**Last updated:** [June 19, 2017, 10:28pm UTC](https://discuss.elastic.co/t/hexewkb-to-lat-long-in-logstash/89988 "2017-06-19T22:28:02Z")

</div>

Hi all, I was wondering if Logstash has a way to change a HEXEWKB string into a Lat/Long pair for geo coordinates. Right now I can pre-process my data to do this but I want to limit my pre-processing as much as possibl…

---

## [Logstash Filter Issue and Kibana](https://discuss.elastic.co/t/logstash-filter-issue-and-kibana/89847)

<div class="topic-metadata">

**Author:** [@wtran168](https://discuss.elastic.co/u/wtran168)\
**Replies:** 1\
**Last updated:** [June 19, 2017, 9:26pm UTC](https://discuss.elastic.co/t/logstash-filter-issue-and-kibana/89847 "2017-06-19T21:26:19Z")

</div>

I'm having an issue with my logstash filter. The fields are not showing up in Kibana. I can't see what I'm doing wrong. I have used the rubydebug and the output looks fine. grok { match =\> { "message" =\> "%{DATE} %…

[Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=960)

[Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=962)