# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=666 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 667 --- ## [Crontab, ikuturso-indexer logstash 6.2 and open source threat intelligence](https://discuss.elastic.co/t/crontab-ikuturso-indexer-logstash-6-2-and-open-source-threat-intelligence/152859)
**Author:** [@mcinesi](https://discuss.elastic.co/u/mcinesi)\ **Replies:** 2\ **Last updated:** [October 17, 2018, 3:28pm UTC](https://discuss.elastic.co/t/crontab-ikuturso-indexer-logstash-6-2-and-open-source-threat-intelligence/152859 "2018-10-17T15:28:27Z")
I have an ikuturso/logstash-indexer:6.2 container which receives the data from rabbit-mq. Within the container I have a scheduled crontab script which regularly downloads the open source feed Minemeld (Paloalto Threat In… --- ## [JSON filter plugin explain example please](https://discuss.elastic.co/t/json-filter-plugin-explain-example-please/152608)
**Author:** [@humalog](https://discuss.elastic.co/u/humalog)\ **Replies:** 8\ **Last updated:** [October 16, 2018, 10:03am UTC](https://discuss.elastic.co/t/json-filter-plugin-explain-example-please/152608 "2018-10-16T10:03:21Z")
One moment, i need to correct my post. I converted the file into json. Is it easier to work with xml or json files or is it the same difficulty? --- ## [JVM Error](https://discuss.elastic.co/t/jvm-error/152849)
**Author:** [@DavisDxb](https://discuss.elastic.co/u/DavisDxb)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 1:34pm UTC](https://discuss.elastic.co/t/jvm-error/152849 "2018-10-17T13:34:04Z")
When I try to Install Logstash in Ubuntu 18.04, all time shows error, /etc/logstash not able to create. More than it JVM not recognized.These two errors. I inspect with stackflow, they replied JDK 11 not supported Logs… --- ## [Problem importing data to kibana through logstash](https://discuss.elastic.co/t/problem-importing-data-to-kibana-through-logstash/152834)
**Author:** [@mansi\_sharma](https://discuss.elastic.co/u/mansi_sharma)\ **Replies:** 1\ **Last updated:** [October 17, 2018, 1:20pm UTC](https://discuss.elastic.co/t/problem-importing-data-to-kibana-through-logstash/152834 "2018-10-17T13:20:28Z")
above error occurs whenever i try connecting. My config file contains following code and i am using windows 10 OS. input { file{ path =\> "C:\\Program Files\\data\\cars.csv" start\_position =\> "beginning" sincdb\_path =\>… --- ## [Apache log imports importing only small amount of logs?](https://discuss.elastic.co/t/apache-log-imports-importing-only-small-amount-of-logs/152789)
**Author:** [@Menso](https://discuss.elastic.co/u/Menso)\ **Replies:** 8\ **Last updated:** [October 17, 2018, 1:08pm UTC](https://discuss.elastic.co/t/apache-log-imports-importing-only-small-amount-of-logs/152789 "2018-10-17T13:08:37Z")
Hi everyone, I'm using Logstash to import around 464 MB of logs. You can have a look at my logstash.conf here: https://gist.github.com/mensoh/932968e45395420a0e28aa1b7a13c0e0 As you can see the output is also sent to s… --- ## [Kv filter for misbehaving](https://discuss.elastic.co/t/kv-filter-for-misbehaving/152840)
**Author:** [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 12:52pm UTC](https://discuss.elastic.co/t/kv-filter-for-misbehaving/152840 "2018-10-17T12:52:35Z")
I am currently using kv filter Logstash for parsing the log lines such as following type=LOGIN msg=audit(1539751621.172:6100294): pid=72964 uid=0 subj=system\_u:system\_r:crond\_t:s0-s0:c0.c1023 old auid=4294967295 new aui… --- ## [Is it possible to get Date from filename as field](https://discuss.elastic.co/t/is-it-possible-to-get-date-from-filename-as-field/152805)
**Author:** [@ajb](https://discuss.elastic.co/u/ajb)\ **Replies:** 5\ **Last updated:** [October 17, 2018, 12:38pm UTC](https://discuss.elastic.co/t/is-it-possible-to-get-date-from-filename-as-field/152805 "2018-10-17T12:38:07Z")
HI Elastic team, Im trying to process logs from liferay (a CMS application), im trying to achieve @timestamp to be same as log event timestamp but the logevent timestamp has only time and miliseconds no date log file n… --- ## [Help - No configuration found in the configured sources](https://discuss.elastic.co/t/help-no-configuration-found-in-the-configured-sources/152436)
**Author:** [@adamdharris](https://discuss.elastic.co/u/adamdharris)\ **Replies:** 2\ **Last updated:** [October 17, 2018, 12:04pm UTC](https://discuss.elastic.co/t/help-no-configuration-found-in-the-configured-sources/152436 "2018-10-17T12:04:58Z")
Having trouble running logstash on local windows 10 instances. I have looked at the other posts but cannot figure it out. Config File input{ file { path =\> "C:/Users/adamd/Data/Invoice\_01062016\_14032018.cs… --- ## [Loading not all data](https://discuss.elastic.co/t/loading-not-all-data/152818)
**Author:** [@Olha1](https://discuss.elastic.co/u/Olha1)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 11:34am UTC](https://discuss.elastic.co/t/loading-not-all-data/152818 "2018-10-17T11:34:11Z")
When data is loaded into the elastic from the database, less data is loaded than the count corresponding to the query. I have a lot of deleted docs. but how can I see which documents do not fall into the elastic? In fil… --- ## [JDBC input Value tracking, "Can't dup Fixnum"](https://discuss.elastic.co/t/jdbc-input-value-tracking-cant-dup-fixnum/150473)
**Author:** [@muyuqj](https://discuss.elastic.co/u/muyuqj)\ **Replies:** 2\ **Last updated:** [October 17, 2018, 11:42am UTC](https://discuss.elastic.co/t/jdbc-input-value-tracking-cant-dup-fixnum/150473 "2018-10-17T11:42:53Z")
rufus-scheduler intercepted an error: 2030 job: 2030 Rufus::Scheduler::CronJob "\*/1 \* \* \* \*" {} 2030 error: 2030 2030 2030 TypeError 2030 can't dup Fixnum 2030 org/jruby/RubyKernel.java:1… --- ## [Windows logs are coming in, but Linux logs are not](https://discuss.elastic.co/t/windows-logs-are-coming-in-but-linux-logs-are-not/151609)
**Author:** [@sc1](https://discuss.elastic.co/u/sc1)\ **Replies:** 1\ **Last updated:** [October 17, 2018, 11:30am UTC](https://discuss.elastic.co/t/windows-logs-are-coming-in-but-linux-logs-are-not/151609 "2018-10-17T11:30:05Z")
Hello, I have set up a single node ELK machine running the latest 6~ instances for Elasticsearch, Kibana, Logstash and Filebeat. I have been able to configure Winlogbeat on 2x Windows servers and I am receiving the logs… --- ## [Grok parse failure syslog](https://discuss.elastic.co/t/grok-parse-failure-syslog/149968)
**Author:** [@heskez](https://discuss.elastic.co/u/heskez)\ **Replies:** 11\ **Last updated:** [October 17, 2018, 10:26am UTC](https://discuss.elastic.co/t/grok-parse-failure-syslog/149968 "2018-10-17T10:26:42Z")
Hi there, I've setup syslog forwarding on my firewall. And I'm getting this error: fortigate\_subtype:%{subtype} host:192.168.100.100 logTimestamp:%{date} %{time} @version:1 syslog5424\_pri:188 fortigate\_type:syslog messa… --- ## [Comma destroys perfect events | file { codec =\> multiline { pattern](https://discuss.elastic.co/t/comma-destroys-perfect-events-file-codec-multiline-pattern/152780)
**Author:** [@humalog](https://discuss.elastic.co/u/humalog)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 8:24am UTC](https://discuss.elastic.co/t/comma-destroys-perfect-events-file-codec-multiline-pattern/152780 "2018-10-17T08:24:34Z")
I want to index a json file. I use input file, with codec multiline. My filter is json filter, output is elasticsearch. The structure of my json file looks like this: \[ { "Plugin": 92438, "Plugin Name": "WordPa… --- ## [Iconv I can not install it](https://discuss.elastic.co/t/iconv-i-can-not-install-it/152783)
**Author:** [@jmkim](https://discuss.elastic.co/u/jmkim)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 9:10am UTC](https://discuss.elastic.co/t/iconv-i-can-not-install-it/152783 "2018-10-17T09:10:48Z")
$ gem install iconv Building native extensions. This could take a while... ERROR: Error installing iconv: ERROR: Failed to build gem native extension. current directory: /usr/share/logstash/vendor/bundle/jruby/2.3.0… --- ## [Setting "xpack.monitoring.elasticsearch.url" hasn't been registered](https://discuss.elastic.co/t/setting-xpack-monitoring-elasticsearch-url-hasnt-been-registered/152779)
**Author:** [@roflmao](https://discuss.elastic.co/u/roflmao)\ **Replies:** 1\ **Last updated:** [October 17, 2018, 8:50am UTC](https://discuss.elastic.co/t/setting-xpack-monitoring-elasticsearch-url-hasnt-been-registered/152779 "2018-10-17T08:50:49Z")
Running docker.elastic.co/logstash/logstash-oss:6.4.0 The settings are set, though logstash crashes with the following: 2018/10/17 08:21:42 Setting 'path.config' from environment. 2018/10/17 08:21:42 Setting 'queue.max… --- ## [Netflow input PLugin for Capturing IPFIX from Ixia Aggregator](https://discuss.elastic.co/t/netflow-input-plugin-for-capturing-ipfix-from-ixia-aggregator/152777)
**Author:** [@Sharizan\_Sumuui](https://discuss.elastic.co/u/Sharizan_Sumuui)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 8:20am UTC](https://discuss.elastic.co/t/netflow-input-plugin-for-capturing-ipfix-from-ixia-aggregator/152777 "2018-10-17T08:20:22Z")
Hi, I'm collecting IPFIX from Ixia aggregator. I can see the IPFIX flowing into logstash on wireshark but could not get see expected fields on elasticserach/kibana. Anyone has experience in collecting IPFIX from Ixia, … --- ## [How to dispose the field of date type importing data to es by logstash?](https://discuss.elastic.co/t/how-to-dispose-the-field-of-date-type-importing-data-to-es-by-logstash/152771)
**Author:** [@shao](https://discuss.elastic.co/u/shao)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 7:53am UTC](https://discuss.elastic.co/t/how-to-dispose-the-field-of-date-type-importing-data-to-es-by-logstash/152771 "2018-10-17T07:53:29Z")
I have a question look like the title, my data has a field of date type, and I need to let it be date type in es after imported it by logstash, so I use date method: input { file { path =\> \["C:\\Users\\Desktop\\test.… --- ## [Field is present but missing](https://discuss.elastic.co/t/field-is-present-but-missing/152453)
**Author:** [@Fiothiel](https://discuss.elastic.co/u/Fiothiel)\ **Replies:** 3\ **Last updated:** [October 17, 2018, 7:32am UTC](https://discuss.elastic.co/t/field-is-present-but-missing/152453 "2018-10-17T07:32:34Z")
Hello, I'll start off by saying I am very new to Kibana. With a lot of googling I have managed to come as far as I have, but now I am stuck. I am reading log files and in the Discover tag I can finally see the fields I … --- ## [Logstash with multiple config files failed to execute action](https://discuss.elastic.co/t/logstash-with-multiple-config-files-failed-to-execute-action/151620)
**Author:** [@BolaBLG](https://discuss.elastic.co/u/BolaBLG)\ **Replies:** 5\ **Last updated:** [October 17, 2018, 7:18am UTC](https://discuss.elastic.co/t/logstash-with-multiple-config-files-failed-to-execute-action/151620 "2018-10-17T07:18:39Z")
Hi, I'm running ELK in production environment on Ubuntu. Everything worked well when I try to index only one csv data with only one logstash config file in the directory /etc/logstash/conf.d Problems started when I add… --- ## [Logstash - complicated parsing, can logstash remember states?](https://discuss.elastic.co/t/logstash-complicated-parsing-can-logstash-remember-states/152682)
**Author:** [@yoav1](https://discuss.elastic.co/u/yoav1)\ **Replies:** 2\ **Last updated:** [October 17, 2018, 5:48am UTC](https://discuss.elastic.co/t/logstash-complicated-parsing-can-logstash-remember-states/152682 "2018-10-17T05:48:26Z")
Hi, I am new in elasticsearch and logstash. my logstash receives its input from filebeat and parses the file line by line. I am trying to do compilcated parsing and wonders if logstash can remember things that seen in… --- ## [Load logs from gzip files using codec](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 2\ **Last updated:** [October 8, 2018, 4:43am UTC](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188 "2018-10-08T04:43:37Z")
There is a closed topic regarding the same from 2 years ago ... But the .rb code has been updated on GitHub. What is the latest solution to using the gzip-lines codec? I have installed the codec and am running the fo… --- ## [Could not find or load main class Files\\logstash\\logstash-core\\lib\\jars\\animal-sniffer-annotations-1.14.jar;C:Program](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files-logstash-logstash-core-lib-jars-animal-sniffer-annotations-1-14-jar-c-program/150360)
**Author:** [@kunalwagh](https://discuss.elastic.co/u/kunalwagh)\ **Replies:** 3\ **Last updated:** [October 16, 2018, 11:17pm UTC](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files-logstash-logstash-core-lib-jars-animal-sniffer-annotations-1-14-jar-c-program/150360 "2018-10-16T23:17:31Z")
Downloaded the latest package and Installed Logstash in C:\\Program Files. Installed JAVA\_HOME to " C:\\Program Files\\Java\\jdk1.8.0\_181 " Checked echo %java\_home% ==\> showing up the same Checked java -version sho… --- ## [Unable to read logs from filebeat and post it to logstash via pipeline](https://discuss.elastic.co/t/unable-to-read-logs-from-filebeat-and-post-it-to-logstash-via-pipeline/152714)
**Author:** [@majeti](https://discuss.elastic.co/u/majeti)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 8:02pm UTC](https://discuss.elastic.co/t/unable-to-read-logs-from-filebeat-and-post-it-to-logstash-via-pipeline/152714 "2018-10-16T20:02:38Z")
Though filebeat is up and running we are unable to see the logs getting parsed under logstash. Deploying logstash config through centralized pipelines. logstash and filebeat are residing on different nodes. Getting belo… --- ## [Logstash starts but does not pass csv data to index](https://discuss.elastic.co/t/logstash-starts-but-does-not-pass-csv-data-to-index/152268)
**Author:** [@Vkryvor](https://discuss.elastic.co/u/Vkryvor)\ **Replies:** 1\ **Last updated:** [October 16, 2018, 7:45pm UTC](https://discuss.elastic.co/t/logstash-starts-but-does-not-pass-csv-data-to-index/152268 "2018-10-16T19:45:00Z")
Hey all, I am trying new to Logstash and the ELK stack altogether and am trying to accomplish a simple task- reading data from a CSV file into elastic search via Logstash. I followed the tutorial and it seems as if Logs… --- ## [Accessing first element after executing split](https://discuss.elastic.co/t/accessing-first-element-after-executing-split/152282)
**Author:** [@lucas.alvarez](https://discuss.elastic.co/u/lucas.alvarez)\ **Replies:** 3\ **Last updated:** [October 16, 2018, 4:52pm UTC](https://discuss.elastic.co/t/accessing-first-element-after-executing-split/152282 "2018-10-16T16:52:55Z")
Hello everybody!. I cannot find a solution for the following thing. I'm splitting a hostname that is composed by a "name", a "." and a "domain". I would like to get only the name. I.e: lucas.google =\> lucas I've read a… --- ## [Expected one of #, input, filter, output at line 1, column 1 (byte 1) after](https://discuss.elastic.co/t/expected-one-of-input-filter-output-at-line-1-column-1-byte-1-after/152435)
**Author:** [@jambhale12](https://discuss.elastic.co/u/jambhale12)\ **Replies:** 1\ **Last updated:** [October 16, 2018, 4:31pm UTC](https://discuss.elastic.co/t/expected-one-of-input-filter-output-at-line-1-column-1-byte-1-after/152435 "2018-10-16T16:31:18Z")
Hello, Please help me to solve this issue I am getting below error message while running logstash-test.conf Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::C… --- ## [Logstash Netflow module remote setup ERROR](https://discuss.elastic.co/t/logstash-netflow-module-remote-setup-error/152265)
**Author:** [@Oleksiy\_Himranov](https://discuss.elastic.co/u/Oleksiy_Himranov)\ **Replies:** 1\ **Last updated:** [October 16, 2018, 3:13pm UTC](https://discuss.elastic.co/t/logstash-netflow-module-remote-setup-error/152265 "2018-10-16T15:13:14Z")
\# /usr/share/logstash/bin/logstash --modules netflow --setup -M "netflow.var.input.udp.port=9996" -M "netflow.var.elasticsearch.hosts=http://192.168.0.133:9200" -M "netflow.var.kibana.host=192.168.0.135:5601" -M "netflow… --- ## [Order of output fields](https://discuss.elastic.co/t/order-of-output-fields/152683)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 2:33pm UTC](https://discuss.elastic.co/t/order-of-output-fields/152683 "2018-10-16T14:33:13Z")
Can I use the Logstash to change the order of the fields in a message and generate as a new message to output file? If yes, How can I did it? In the Filter or in the output section? Thanks Sharon. --- ## [Date input filter failing intermittently after upgrade to 6.4](https://discuss.elastic.co/t/date-input-filter-failing-intermittently-after-upgrade-to-6-4/152677)
**Author:** [@atgaurav](https://discuss.elastic.co/u/atgaurav)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 1:58pm UTC](https://discuss.elastic.co/t/date-input-filter-failing-intermittently-after-upgrade-to-6-4/152677 "2018-10-16T13:58:20Z")
My scripts creates files daily which are sent from filebeat to logstash daily. I have a date input filter in logstash which changes a field (tstamp) to date data type. My logstash sends data to a newly created index in… --- ## [Logstash does not create Index in ELasticsearch](https://discuss.elastic.co/t/logstash-does-not-create-index-in-elasticsearch/152555)
**Author:** [@kunalwagh](https://discuss.elastic.co/u/kunalwagh)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 8:53pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-index-in-elasticsearch/152555 "2018-10-15T20:53:24Z")
Logstash Pipeline is running successfully. Elasticsearch and Kibana are running successfully. Have apache\_access\_xxxx.log files in logstash/event-data Checked http://localhost:9200/\_cat/indices and could not find logst… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=665) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=667)