# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=802 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 803 --- ## [Same filter which is working for 2.x in Elk not working for 6.1.3](https://discuss.elastic.co/t/same-filter-which-is-working-for-2-x-in-elk-not-working-for-6-1-3/123176)
**Author:** [@ashish\_khulbe](https://discuss.elastic.co/u/ashish_khulbe)\ **Replies:** 9\ **Last updated:** [March 14, 2018, 1:07pm UTC](https://discuss.elastic.co/t/same-filter-which-is-working-for-2-x-in-elk-not-working-for-6-1-3/123176 "2018-03-14T13:07:43Z")
Hi, I have 2 queries :- I configured Elk 2.x with Kibana 4.x a week back and everything working fine. Now I am working with 6.1.3 of Elk and file beat but I can see whole message but not filters. Is there any dif… --- ## [Please help with grok pattern](https://discuss.elastic.co/t/please-help-with-grok-pattern/123497)
**Author:** [@vitich](https://discuss.elastic.co/u/vitich)\ **Replies:** 7\ **Last updated:** [March 14, 2018, 10:47am UTC](https://discuss.elastic.co/t/please-help-with-grok-pattern/123497 "2018-03-14T10:47:59Z")
Hi! Log string: "12/Mar/2018:07:45:23 +0000"; "91.198.36.14"; "UA:-:-"; "10.10.10.1:80"; "200"; "0.000 sec"; "0.003 sec"; "www.some-site.ua"; "GET /services/ HTTP/1.1"; "327"; "https://www.some-site.ua/services/"; "Moz… --- ## [Logstash is not executing config files in conf.d in RHEL environment?](https://discuss.elastic.co/t/logstash-is-not-executing-config-files-in-conf-d-in-rhel-environment/123881)
**Author:** [@naveen\_K.N](https://discuss.elastic.co/u/naveen_K.N)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 10:34am UTC](https://discuss.elastic.co/t/logstash-is-not-executing-config-files-in-conf-d-in-rhel-environment/123881 "2018-03-14T10:34:07Z")
Hi Elastic Experts! we are trying send serverlog data from file system and error log data from database.so we placed config files in "/etc/logstash/conf.d/ " , for filesystem we are using file input plugin and for da… --- ## [Multiline in logstash version 5](https://discuss.elastic.co/t/multiline-in-logstash-version-5/123850)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 4\ **Last updated:** [March 14, 2018, 10:07am UTC](https://discuss.elastic.co/t/multiline-in-logstash-version-5/123850 "2018-03-14T10:07:01Z")
the multiline filter plugin is replaced by what in logstash version 5 --- ## [Sincedb Creation with %{type} syntaxe](https://discuss.elastic.co/t/sincedb-creation-with-type-syntaxe/123525)
**Author:** [@hzarrabi](https://discuss.elastic.co/u/hzarrabi)\ **Replies:** 7\ **Last updated:** [March 14, 2018, 9:40am UTC](https://discuss.elastic.co/t/sincedb-creation-with-type-syntaxe/123525 "2018-03-14T09:40:53Z")
Hi, I would like to be able to create different sincedb for each logstash conf file as below by inserting %{mytype} in the sincedb\_path string. input { file { type =\> "appname" path =\> "/path/to/cata… --- ## [Double Quotes in Logstash Ruby Code](https://discuss.elastic.co/t/double-quotes-in-logstash-ruby-code/123861)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 8:54am UTC](https://discuss.elastic.co/t/double-quotes-in-logstash-ruby-code/123861 "2018-03-14T08:54:41Z")
Hi, I know, the parsing would end in the ruby code of the filter plugin, if we use double quotes inside the block, but I have a regex to be used in the code, which has double quotes, something like this (/backingupsize… --- ## [Logstash CPUAffinity](https://discuss.elastic.co/t/logstash-cpuaffinity/123853)
**Author:** [@Megha\_Gautam](https://discuss.elastic.co/u/Megha_Gautam)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 7:23am UTC](https://discuss.elastic.co/t/logstash-cpuaffinity/123853 "2018-03-14T07:23:28Z")
I am running Logstash as a service on a 16 core machine, with following conf curl -XGET 'localhost:9600/\_node/pipelines?pretty' { "host" : "0.0.0.0", "version" : "6.1.2", "http\_address" : "127.0.0.1:9600", "id"… --- ## [Date filter \_dateparsefailure](https://discuss.elastic.co/t/date-filter-dateparsefailure/123843)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [March 14, 2018, 6:50am UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/123843 "2018-03-14T06:50:52Z")
Hi, I am using ELK GA 5.0.0. I am consuming from Kafka and I have a message in which has date like 05-Feb-2018:10:39:33 +0000. I used grok filter to parse date like; date { match =\> \[ "datetime", "dd-MM-YYYY:HH:mm:ss … --- ## [Logstash Mapping Template](https://discuss.elastic.co/t/logstash-mapping-template/123841)
**Author:** [@sri535](https://discuss.elastic.co/u/sri535)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 6:21am UTC](https://discuss.elastic.co/t/logstash-mapping-template/123841 "2018-03-14T06:21:07Z")
HI can anyone please help with mapping template of the Elastic stack ( logstash mapping template ) with the example. i am facing issues with the mapping template after upgrading to 6.2 --- ## [Optimizing Grok patterns](https://discuss.elastic.co/t/optimizing-grok-patterns/123256)
**Author:** [@burnersk](https://discuss.elastic.co/u/burnersk)\ **Replies:** 1\ **Last updated:** [March 14, 2018, 3:01am UTC](https://discuss.elastic.co/t/optimizing-grok-patterns/123256 "2018-03-14T03:01:37Z")
I have the following Grok pattern for parsing nginx's error log file: input { file { id =\> "nginx\_error\_file\_input" codec =\> "plain" tags =\> \[ "nginx", "nginx\_errors" \] type =\> "nginx\_error\_log" pa… --- ## [Logstash sending date instead of string](https://discuss.elastic.co/t/logstash-sending-date-instead-of-string/122866)
**Author:** [@jainbhavya53](https://discuss.elastic.co/u/jainbhavya53)\ **Replies:** 10\ **Last updated:** [March 14, 2018, 3:31am UTC](https://discuss.elastic.co/t/logstash-sending-date-instead-of-string/122866 "2018-03-14T03:31:22Z")
I am using grok filter to parse logs.In that there is a "time" field for which I am using "TIMESTAMP\_ISO8601" as the grok pattern. By default logstash should send this "time" field as "string" but it is sending it as da… --- ## [Geoip in logstash returns coordinates correctly for simple grok filter but not from more complex filter](https://discuss.elastic.co/t/geoip-in-logstash-returns-coordinates-correctly-for-simple-grok-filter-but-not-from-more-complex-filter/123674)
**Author:** [@studiorecordings](https://discuss.elastic.co/u/studiorecordings)\ **Replies:** 8\ **Last updated:** [March 13, 2018, 11:09pm UTC](https://discuss.elastic.co/t/geoip-in-logstash-returns-coordinates-correctly-for-simple-grok-filter-but-not-from-more-complex-filter/123674 "2018-03-13T23:09:30Z")
Hi, I am trying to extract an IP address from a syslog message in log stash. When I run log stash in debugging mode to stdout using only this simple filter it maps to geoip correctly and the geoip plugin processes and o… --- ## [Logstash DNS Filter Issue](https://discuss.elastic.co/t/logstash-dns-filter-issue/123818)
**Author:** [@sysadmin007](https://discuss.elastic.co/u/sysadmin007)\ **Replies:** 0\ **Last updated:** [March 13, 2018, 10:43pm UTC](https://discuss.elastic.co/t/logstash-dns-filter-issue/123818 "2018-03-13T22:43:55Z")
Hello All, I setup an ELK Stack with the latest version of logstash 6.2.2 on Ubuntu 17.10. The setup went ok i have netflow records in my Kibana dashboard and most everything seems to be working properly. The only issue… --- ## [Issue in Logstash after Upgrade](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 5\ **Last updated:** [March 13, 2018, 10:33pm UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421 "2018-03-13T22:33:38Z")
I tested upgrade of ELK stack to V5.6.8 from V5.2.2 running on Centos 7 VM . Elasticsearch upgrade & Kibana upgrade Went fine ..But logstash ran into issue details below testing Logatsh config : input { udp { type =\> … --- ## [Grok Parse Error, and input to output formatting using grok and json\_encode filter](https://discuss.elastic.co/t/grok-parse-error-and-input-to-output-formatting-using-grok-and-json-encode-filter/123285)
**Author:** [@Prabh\_Natt](https://discuss.elastic.co/u/Prabh_Natt)\ **Replies:** 3\ **Last updated:** [March 13, 2018, 9:53pm UTC](https://discuss.elastic.co/t/grok-parse-error-and-input-to-output-formatting-using-grok-and-json-encode-filter/123285 "2018-03-13T21:53:38Z")
Error in command prompt: { "@timestamp" =\> 2018-03-09T14:18:26.524Z, "tags" =\> \[ \[0\] "\_grokparsefailure" \], "json\_message" =\> "input : {" } { "@timestamp" =\> 2018-03-09T14:18:26.524Z, "tags" =\> \[ \[0\] "\_grokpar… --- ## [Logstash output to elasticsearch - how to include external synonims file?](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-how-to-include-external-synonims-file/123381)
**Author:** [@gennolo](https://discuss.elastic.co/u/gennolo)\ **Replies:** 1\ **Last updated:** [March 13, 2018, 9:22pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-how-to-include-external-synonims-file/123381 "2018-03-13T21:22:32Z")
I am using Logstash to output a big CSV file inside an ElasticSearch index and I need to include a SOLR synonims external file (https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter… --- ## [Logstash jdbc input keep sql field ordering in place](https://discuss.elastic.co/t/logstash-jdbc-input-keep-sql-field-ordering-in-place/123780)
**Author:** [@ramzey1981](https://discuss.elastic.co/u/ramzey1981)\ **Replies:** 4\ **Last updated:** [March 13, 2018, 9:20pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-keep-sql-field-ordering-in-place/123780 "2018-03-13T21:20:55Z")
hey team, I have the following logstash config file, jdbc { jdbc\_driver\_library =\> "C:\\Program Files\\sqljdbc\_6.2\\enu\\mssql-jdbc-6.2.2.jre8.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdb… --- ## [Invalid document\_id not handled well with elasticsearch output](https://discuss.elastic.co/t/invalid-document-id-not-handled-well-with-elasticsearch-output/123811)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 0\ **Last updated:** [March 13, 2018, 9:17pm UTC](https://discuss.elastic.co/t/invalid-document-id-not-handled-well-with-elasticsearch-output/123811 "2018-03-13T21:17:02Z")
Due to an issue in my multiline codec, I had an event that did not have the expected format. When I parsed the document\_id out of it, it was part of a JSON object. This results in logstash looping retrying the request an… --- ## [Collectd codec not shipping properly](https://discuss.elastic.co/t/collectd-codec-not-shipping-properly/123810)
**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\ **Replies:** 0\ **Last updated:** [March 13, 2018, 9:11pm UTC](https://discuss.elastic.co/t/collectd-codec-not-shipping-properly/123810 "2018-03-13T21:11:01Z")
I am trying to configure the collectd codec. We have it shipping every 1 minute. Interval 60 #Include "/etc/collectd.d/\*.conf" TypesDB "/usr/share/collectd/types.db" # Globals true # # ModulePath LogTraces true In… --- ## [Custom filter plugin logging configuration through log4j](https://discuss.elastic.co/t/custom-filter-plugin-logging-configuration-through-log4j/123389)
**Author:** [@srikanth\_muddu](https://discuss.elastic.co/u/srikanth_muddu)\ **Replies:** 6\ **Last updated:** [March 13, 2018, 8:57pm UTC](https://discuss.elastic.co/t/custom-filter-plugin-logging-configuration-through-log4j/123389 "2018-03-13T20:57:50Z")
I have installed new filter plugin with jar dependencies.I kept log4j2.properties for a java project and was able to create logging with a java project.But after installing plugin logs are not appending in the logstash-… --- ## [Clarification - ingesting documents using logstash into an existing index](https://discuss.elastic.co/t/clarification-ingesting-documents-using-logstash-into-an-existing-index/123662)
**Author:** [@jason\_smith](https://discuss.elastic.co/u/jason_smith)\ **Replies:** 4\ **Last updated:** [March 13, 2018, 7:05pm UTC](https://discuss.elastic.co/t/clarification-ingesting-documents-using-logstash-into-an-existing-index/123662 "2018-03-13T19:05:39Z")
Hello folks, I want to create the index structure and ingest documents into that structure using logstash. In the pipeline config file in the attachement, where should I specify the name of structure into which this da… --- ## [Parsing datapower syslog using GROK for latency calculation for given URI](https://discuss.elastic.co/t/parsing-datapower-syslog-using-grok-for-latency-calculation-for-given-uri/123118)
**Author:** [@mguttula](https://discuss.elastic.co/u/mguttula)\ **Replies:** 16\ **Last updated:** [March 13, 2018, 6:44pm UTC](https://discuss.elastic.co/t/parsing-datapower-syslog-using-grok-for-latency-calculation-for-given-uri/123118 "2018-03-13T18:44:19Z")
Hi, I'm new to GROK and Logstash. I'm trying to calculate the response time from latency field of a syslog entry of Datapower. Entry looks like following: 20180308T211418.088Z \[0x80e00073\]\[mpgw\]\[debug\] mpgw(MyServices… --- ## [Elasticsearch filter giving "TypeError: no implicit conversion of nil into String"](https://discuss.elastic.co/t/elasticsearch-filter-giving-typeerror-no-implicit-conversion-of-nil-into-string/123791)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 0\ **Last updated:** [March 13, 2018, 6:25pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-giving-typeerror-no-implicit-conversion-of-nil-into-string/123791 "2018-03-13T18:25:06Z")
I have log files from an app that logs requests and responses. I want to knit those together. I write the request to elasticsearch, then, when I process the response, I want to lookup the request that generated it. So fo… --- ## [Kafka Input Plugin: Shutdown semantics](https://discuss.elastic.co/t/kafka-input-plugin-shutdown-semantics/123785)
**Author:** [@kaihil](https://discuss.elastic.co/u/kaihil)\ **Replies:** 0\ **Last updated:** [March 13, 2018, 5:19pm UTC](https://discuss.elastic.co/t/kafka-input-plugin-shutdown-semantics/123785 "2018-03-13T17:19:37Z")
Hello, I have a question regarding the shutdown semantics of Logstash's Kafka Input Plugin: When I run it with default settings, send a batch of 10.000 message into kafka and restart\* logstash while it is processing t… --- ## [CPU usage for filebeat or logstash](https://discuss.elastic.co/t/cpu-usage-for-filebeat-or-logstash/123770)
**Author:** [@swethamahesh](https://discuss.elastic.co/u/swethamahesh)\ **Replies:** 0\ **Last updated:** [March 13, 2018, 3:54pm UTC](https://discuss.elastic.co/t/cpu-usage-for-filebeat-or-logstash/123770 "2018-03-13T15:54:46Z")
Hi, we are observing CPU usage high from logstash process. Can you please let us know the recommended RAM for running logstash process. Do we have any CPU Usage metric used by logstash service, Thanks in Advance, Sw… --- ## [No config files found after install](https://discuss.elastic.co/t/no-config-files-found-after-install/123134)
**Author:** [@dpangallo](https://discuss.elastic.co/u/dpangallo)\ **Replies:** 3\ **Last updated:** [March 13, 2018, 3:52pm UTC](https://discuss.elastic.co/t/no-config-files-found-after-install/123134 "2018-03-13T15:52:43Z")
I installed logstash 6.2.2 via apt-get on Ubuntu 16.04 and everything seem to work ok except I'm missing all the config files; logstash.yml, pipelines.yml, etc... anyone know where/how I can download just the a copy of d… --- ## [What are list of data types that we have to mutate in the pipeline config file..so far I know "Integer" and "float"](https://discuss.elastic.co/t/what-are-list-of-data-types-that-we-have-to-mutate-in-the-pipeline-config-file-so-far-i-know-integer-and-float/123655)
**Author:** [@jason\_smith](https://discuss.elastic.co/u/jason_smith)\ **Replies:** 2\ **Last updated:** [March 13, 2018, 3:15pm UTC](https://discuss.elastic.co/t/what-are-list-of-data-types-that-we-have-to-mutate-in-the-pipeline-config-file-so-far-i-know-integer-and-float/123655 "2018-03-13T15:15:53Z")
Hello good folks, When I tried with mutate{convert=\>\["has-expired","bool"\]}, it didn't like. So far I was able to mutate only integer and float. mutate{convert=\>\["mileage","integer"\]} muta… --- ## [LOgstash 6.2 Template Mapping not able to write to index](https://discuss.elastic.co/t/logstash-6-2-template-mapping-not-able-to-write-to-index/123764)
**Author:** [@sri535](https://discuss.elastic.co/u/sri535)\ **Replies:** 1\ **Last updated:** [March 13, 2018, 3:32pm UTC](https://discuss.elastic.co/t/logstash-6-2-template-mapping-not-able-to-write-to-index/123764 "2018-03-13T15:32:44Z")
HI i am using 6.2 and i made changes to my template file as required but i am getting below error in logstash. Blockquote indent preformatted text by 4 spaces \[2018-03-13T06:55:55,748\]\[WARN \]\[logstash.outputs.elasti… --- ## [Logstash conf, multiline & multimatch](https://discuss.elastic.co/t/logstash-conf-multiline-multimatch/123611)
**Author:** [@adolfo.diaz](https://discuss.elastic.co/u/adolfo.diaz)\ **Replies:** 4\ **Last updated:** [March 13, 2018, 3:02pm UTC](https://discuss.elastic.co/t/logstash-conf-multiline-multimatch/123611 "2018-03-13T15:02:07Z")
Hi friends We are trying to process some files with this format: IMAGE 123-06-gestion 0 0 12 123-06-gestion\_1520509780 CNGN\_AS\_BROADSOFT 0 NULL root i\_dia\_bi 1 0 1520509780 276 1521114580 0 0 23554400 470 1 1 0 CNGN\_AS… --- ## [How to archive the messages from logstash](https://discuss.elastic.co/t/how-to-archive-the-messages-from-logstash/123719)
**Author:** [@Sunil\_Villuri](https://discuss.elastic.co/u/Sunil_Villuri)\ **Replies:** 4\ **Last updated:** [March 13, 2018, 2:38pm UTC](https://discuss.elastic.co/t/how-to-archive-the-messages-from-logstash/123719 "2018-03-13T14:38:40Z")
Hi All, i want to archive the old files for every month automatically.Could you please let us know the possibility and mechanism [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=801) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=803)