# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=956 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 957 --- ## [ESX logs -\> remote syslog -\> ELK : Timestamp problems](https://discuss.elastic.co/t/esx-logs-remote-syslog-elk-timestamp-problems/90396)
**Author:** [@kottapar](https://discuss.elastic.co/u/kottapar)\ **Replies:** 5\ **Last updated:** [June 28, 2017, 5:36am UTC](https://discuss.elastic.co/t/esx-logs-remote-syslog-elk-timestamp-problems/90396 "2017-06-28T05:36:33Z")
The syslog messages fromt the ESX server and being received on the remote syslog as below. Jun 22 05:51:19 esxserv01 Vpxa: info vpxa\[FFDF9B70\] \[Originator@6876 sub=vpxLro opID=PollQuickStatsLoop-655fae54-a2\] \[VpxLRO\] … --- ## [CSV output plugin](https://discuss.elastic.co/t/csv-output-plugin/88512)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 7\ **Last updated:** [June 28, 2017, 5:07am UTC](https://discuss.elastic.co/t/csv-output-plugin/88512 "2017-06-28T05:07:26Z")
Hi, I am trying to take the output of parsed input file into a csv file.I am using logstash 5.1.1 version. Here is my output configuration:- output { csv { path =\> "/root/output/logstash/csvdata.csv" fields =\> \[… --- ## [How do I avoid indexing through comma's between "\<\<\>\>" in my CSV files](https://discuss.elastic.co/t/how-do-i-avoid-indexing-through-commas-between-in-my-csv-files/90862)
**Author:** [@RajeshGarre](https://discuss.elastic.co/u/RajeshGarre)\ **Replies:** 3\ **Last updated:** [June 28, 2017, 3:37am UTC](https://discuss.elastic.co/t/how-do-i-avoid-indexing-through-commas-between-in-my-csv-files/90862 "2017-06-28T03:37:30Z")
I am new to logstash and I am trying to create a Conf File with CSV file as source. The CSV file contains field values within \<\>\> that are also comma separated which should be treated as one field. How do I avoid indexin… --- ## [SOLVED - Issue converting JSON after JDBC Streaming](https://discuss.elastic.co/t/solved-issue-converting-json-after-jdbc-streaming/90856)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 7\ **Last updated:** [June 28, 2017, 2:46am UTC](https://discuss.elastic.co/t/solved-issue-converting-json-after-jdbc-streaming/90856 "2017-06-28T02:46:13Z")
Hi Team, I have a logstash config file that takes input source of SQS. I then take this and apply a filter to convert the JSON message from SQS into a data structure (this is working fine) I then apply another filter w… --- ## [How to parse DNS log](https://discuss.elastic.co/t/how-to-parse-dns-log/91036)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 0\ **Last updated:** [June 28, 2017, 1:29am UTC](https://discuss.elastic.co/t/how-to-parse-dns-log/91036 "2017-06-28T01:29:50Z")
I have some log like (2)au(8)download(13)windowsupdate(3)com(0) (8)clients3(6)google(3)com(0) (3)v10(10)vortex-win(4)data(9)microsoft(3)com(0) Now i want it look like au.download.windowsupdate.com clients3.googl… --- ## [Datetime named parameter in jdbc input](https://discuss.elastic.co/t/datetime-named-parameter-in-jdbc-input/91012)
**Author:** [@zahodi](https://discuss.elastic.co/u/zahodi)\ **Replies:** 3\ **Last updated:** [June 27, 2017, 8:44pm UTC](https://discuss.elastic.co/t/datetime-named-parameter-in-jdbc-input/91012 "2017-06-27T20:44:58Z")
How do I pass a datetime as a named parameter in logstash jdbc input? Trying to query mysql based on the current date and time. --- ## [Logstash in .Netcore application](https://discuss.elastic.co/t/logstash-in-netcore-application/91013)
**Author:** [@sslv](https://discuss.elastic.co/u/sslv)\ **Replies:** 0\ **Last updated:** [June 27, 2017, 6:24pm UTC](https://discuss.elastic.co/t/logstash-in-netcore-application/91013 "2017-06-27T18:24:21Z")
Hi, I want to use logstash for aggregating logs in .Net core application. The reason I am using .Net core application is that, I would want it to run on Linux and windows. I am quite confused as in which is the better wa… --- ## [Aggregate Filter not working when aggregating Netflow data](https://discuss.elastic.co/t/aggregate-filter-not-working-when-aggregating-netflow-data/91019)
**Author:** [@Ankit\_Gupta](https://discuss.elastic.co/u/Ankit_Gupta)\ **Replies:** 0\ **Last updated:** [June 27, 2017, 7:27pm UTC](https://discuss.elastic.co/t/aggregate-filter-not-working-when-aggregating-netflow-data/91019 "2017-06-27T19:27:45Z")
I am getting the below data and trying to aggregate the events based on the prefix IP and add the traffic rate of all the events, but it is not working. Not sure what else I can try. { "netflow" =\> { "… --- ## [ELK stack on windows server with MSSQL db](https://discuss.elastic.co/t/elk-stack-on-windows-server-with-mssql-db/91010)
**Author:** [@lkapustina](https://discuss.elastic.co/u/lkapustina)\ **Replies:** 9\ **Last updated:** [June 27, 2017, 7:05pm UTC](https://discuss.elastic.co/t/elk-stack-on-windows-server-with-mssql-db/91010 "2017-06-27T19:05:49Z")
Hi, dear community. I am new in ELK, I have task to create Kibana dashboards for data visualization on Win server(aws) and I have to use MSSQL as data source. I created file db.conf with content below: input { jdbc { j… --- ## [Logstash and netflow filter problem / question with filter and if\[exp\]](https://discuss.elastic.co/t/logstash-and-netflow-filter-problem-question-with-filter-and-if-exp/90720)
**Author:** [@digitalpilot](https://discuss.elastic.co/u/digitalpilot)\ **Replies:** 1\ **Last updated:** [June 27, 2017, 6:37pm UTC](https://discuss.elastic.co/t/logstash-and-netflow-filter-problem-question-with-filter-and-if-exp/90720 "2017-06-27T18:37:28Z")
Hi, using latest versions of netflow / ipfix codec with logstash. I'm trying to create a filter that will match on the BGP ASN and set a new field with the human name for that ASN. At the moment I'm setting a tag,… --- ## [Logstash jdbc input timestamp parse issue](https://discuss.elastic.co/t/logstash-jdbc-input-timestamp-parse-issue/91002)
**Author:** [@Hoffin\_Ag](https://discuss.elastic.co/u/Hoffin_Ag)\ **Replies:** 4\ **Last updated:** [June 27, 2017, 6:07pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-timestamp-parse-issue/91002 "2017-06-27T18:07:36Z")
Previous version of logstash2.3.4. when we updated to 5.4.1 facing issue. please resolve ASAP. Provide fix. this is major issue in production. Logstash Version : 5.4.1 Date Field : format (dd:mm:yy hh:mm:ss:sssss) exce… --- ## [TimeStamp Preserve 6 digit ms](https://discuss.elastic.co/t/timestamp-preserve-6-digit-ms/91007)
**Author:** [@Hoffin\_Ag](https://discuss.elastic.co/u/Hoffin_Ag)\ **Replies:** 8\ **Last updated:** [June 27, 2017, 6:08pm UTC](https://discuss.elastic.co/t/timestamp-preserve-6-digit-ms/91007 "2017-06-27T18:08:57Z")
Hi , Time ms upto 6 digit is crucial in our case, how to use 6 digit ms. in elastic truncating ms after 4 digit. Transactional system each ms is very important. ex Format : yy:mm:dd hh:mm:ss:SSSSSSZ elastic format :… --- ## [Parsing complex log](https://discuss.elastic.co/t/parsing-complex-log/90875)
**Author:** [@Diya\_Ayaad](https://discuss.elastic.co/u/Diya_Ayaad)\ **Replies:** 2\ **Last updated:** [June 27, 2017, 6:05pm UTC](https://discuss.elastic.co/t/parsing-complex-log/90875 "2017-06-27T18:05:12Z")
Hello Everyone :slight\_smile: , I have the following log file 2017-06-13 13:00:01,494 - INFO \[Line: 48\]: Begin logging 2017-06-13 13:00:01,494 - DEBUG \[Line: 89\]: Writing to lockfile. Lockfile location: /etc/conf/the/b… --- ## [Filebeat -\> kafka - \> logstash no output](https://discuss.elastic.co/t/filebeat-kafka-logstash-no-output/90552)
**Author:** [@pashmina](https://discuss.elastic.co/u/pashmina)\ **Replies:** 2\ **Last updated:** [June 27, 2017, 6:01pm UTC](https://discuss.elastic.co/t/filebeat-kafka-logstash-no-output/90552 "2017-06-27T18:01:28Z")
Hi, I am trying to read apache-access log file via filebeat and send it to kafka topic. The kafka topic is used as input to logstash as follows. input { kafka { bootstrap\_servers =\> "localhost… --- ## [LogStash file plugin offline](https://discuss.elastic.co/t/logstash-file-plugin-offline/90511)
**Author:** [@ShashiK](https://discuss.elastic.co/u/ShashiK)\ **Replies:** 7\ **Last updated:** [June 27, 2017, 5:34pm UTC](https://discuss.elastic.co/t/logstash-file-plugin-offline/90511 "2017-06-27T17:34:43Z")
Hi, I am unable to install the logstash file input plugin. On checking the output i found that I need to provide a proxy with access to Internet. I do not have an Org proxy to download. Is it possible to download the fil… --- ## [Beats plugin failing](https://discuss.elastic.co/t/beats-plugin-failing/91004)
**Author:** [@daddyyankee](https://discuss.elastic.co/u/daddyyankee)\ **Replies:** 0\ **Last updated:** [June 27, 2017, 5:00pm UTC](https://discuss.elastic.co/t/beats-plugin-failing/91004 "2017-06-27T17:00:19Z")
Hello, Removed x-pack and uninstalled/ check that there should not be any x-pack settings enabled across my enviro. i forward logs using filebeats from several servers. The error i see now trying to run a config that… --- ## [Date filter offsetting timestamp from log file by 1 hr](https://discuss.elastic.co/t/date-filter-offsetting-timestamp-from-log-file-by-1-hr/90980)
**Author:** [@taweisse](https://discuss.elastic.co/u/taweisse)\ **Replies:** 2\ **Last updated:** [June 27, 2017, 4:07pm UTC](https://discuss.elastic.co/t/date-filter-offsetting-timestamp-from-log-file-by-1-hr/90980 "2017-06-27T16:07:07Z")
I just started using Logstash along with Elasticsearch and Kibana to visualize some log data coming from a database, and I am running into an issue with what I believe to be the date filter. Currently I am parsing the lo… --- ## [Problem With logstash output email plugin](https://discuss.elastic.co/t/problem-with-logstash-output-email-plugin/84379)
**Author:** [@Baco](https://discuss.elastic.co/u/Baco)\ **Replies:** 14\ **Last updated:** [June 27, 2017, 3:25pm UTC](https://discuss.elastic.co/t/problem-with-logstash-output-email-plugin/84379 "2017-06-27T15:25:17Z")
Could you help me with this error on my logstash 5.3 ? couldn't find any output plugin named 'email'. Thanks --- ## [Need help with the grok and the date filter for parsing logs](https://discuss.elastic.co/t/need-help-with-the-grok-and-the-date-filter-for-parsing-logs/86523)
**Author:** [@Gaurav\_Singh1](https://discuss.elastic.co/u/Gaurav_Singh1)\ **Replies:** 28\ **Last updated:** [June 27, 2017, 2:56pm UTC](https://discuss.elastic.co/t/need-help-with-the-grok-and-the-date-filter-for-parsing-logs/86523 "2017-06-27T14:56:54Z")
I am trying to parse logs from a CSV file . Issue 1\> I am using below mentioned conf file. input { file { path =\> "/tmp/test1.csv" type =\> "core2" start\_position =\> "beginning" } } filter { csv { … --- ## [Logstash 2 different input files into 2 different indices](https://discuss.elastic.co/t/logstash-2-different-input-files-into-2-different-indices/90956)
**Author:** [@Jalpesh1](https://discuss.elastic.co/u/Jalpesh1)\ **Replies:** 1\ **Last updated:** [June 27, 2017, 2:27pm UTC](https://discuss.elastic.co/t/logstash-2-different-input-files-into-2-different-indices/90956 "2017-06-27T14:27:33Z")
HI Team, We have 2 different input files to Logstash & we want feed this data to 2 different indices on same instance. Please suggest. Thanks & Regards, Jalpesh --- ## [Logstash 5.4 , i installed freshly in my ubuntu 16.04 server , after starting the logstash there is no logstash file in init.d directory](https://discuss.elastic.co/t/logstash-5-4-i-installed-freshly-in-my-ubuntu-16-04-server-after-starting-the-logstash-there-is-no-logstash-file-in-init-d-directory/89890)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 4\ **Last updated:** [June 27, 2017, 1:43pm UTC](https://discuss.elastic.co/t/logstash-5-4-i-installed-freshly-in-my-ubuntu-16-04-server-after-starting-the-logstash-there-is-no-logstash-file-in-init-d-directory/89890 "2017-06-27T13:43:50Z")
root@dev01:/etc/systemd/system# sudo /etc/init.d/logstash start sudo: /etc/init.d/logstash: command not found --- ## [Send to both host A and host B](https://discuss.elastic.co/t/send-to-both-host-a-and-host-b/90958)
**Author:** [@jcannell](https://discuss.elastic.co/u/jcannell)\ **Replies:** 1\ **Last updated:** [June 27, 2017, 12:55pm UTC](https://discuss.elastic.co/t/send-to-both-host-a-and-host-b/90958 "2017-06-27T12:55:30Z")
Hello, Whenever multiple "hosts" are specified in the elasticsearch output plugin, the request is load balanced. I'm not wanting to do this. Instead, whenever the hosts settings contains both host A and host B, I'm… --- ## [Field - Limit length](https://discuss.elastic.co/t/field-limit-length/90954)
**Author:** [@medley](https://discuss.elastic.co/u/medley)\ **Replies:** 3\ **Last updated:** [June 27, 2017, 12:54pm UTC](https://discuss.elastic.co/t/field-limit-length/90954 "2017-06-27T12:54:12Z")
Hello, Is it possible to limit the length of a field ? I am using Logstash 5.2.1. M. --- ## [Very slow processing of CDN logs](https://discuss.elastic.co/t/very-slow-processing-of-cdn-logs/90496)
**Author:** [@UnitedMarsupials](https://discuss.elastic.co/u/UnitedMarsupials)\ **Replies:** 5\ **Last updated:** [June 27, 2017, 12:52pm UTC](https://discuss.elastic.co/t/very-slow-processing-of-cdn-logs/90496 "2017-06-27T12:52:35Z")
I've configured logstash to parse CDN-uploaded logs and feed them into Graphite. Things work, but very slowly -- a log-file with 278K lines takes 6.5 minutes to process. The development instance of logstash copies prod… --- ## [Periodically run elasticsearch for new records since last run](https://discuss.elastic.co/t/periodically-run-elasticsearch-for-new-records-since-last-run/88700)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 6\ **Last updated:** [June 27, 2017, 11:32am UTC](https://discuss.elastic.co/t/periodically-run-elasticsearch-for-new-records-since-last-run/88700 "2017-06-27T11:32:11Z")
Hi, I've asked this before but no gotten a solution, I'm in need of a running an elasticsearch query periodically, every minute or so, and return a count that is based on new records since the last time the query was e… --- ## [How to implement to store Spring-boot embedded tomcat access logs?](https://discuss.elastic.co/t/how-to-implement-to-store-spring-boot-embedded-tomcat-access-logs/90562)
**Author:** [@Juneyoung\_Oh](https://discuss.elastic.co/u/Juneyoung_Oh)\ **Replies:** 3\ **Last updated:** [June 27, 2017, 10:19am UTC](https://discuss.elastic.co/t/how-to-implement-to-store-spring-boot-embedded-tomcat-access-logs/90562 "2017-06-27T10:19:33Z")
Hi. I am trying to store spring boot tomcat access log to logstash. So I implement org.springframework.boot.context.embedded.EmbeddedServletContainerCustomizer like below. It works fine, I can see the access-\* logs in K… --- ## [Logstash-filter-elasticsearch - Cannot get new connection from pool](https://discuss.elastic.co/t/logstash-filter-elasticsearch-cannot-get-new-connection-from-pool/90941)
**Author:** [@fpr](https://discuss.elastic.co/u/fpr)\ **Replies:** 0\ **Last updated:** [June 27, 2017, 10:13am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-cannot-get-new-connection-from-pool/90941 "2017-06-27T10:13:44Z")
Hi, logstash-filter-elasticsearch throws this error on high load: \[2017-06-27T11:58:03,714\]\[WARN \]\[logstash.filters.elasticsearch\] Failed to query elasticsearch for previous event {:index=\>"combine-inbound-2017.06.27", … --- ## [Grok Pattern For Access Log](https://discuss.elastic.co/t/grok-pattern-for-access-log/90626)
**Author:** [@Michael\_TRAN](https://discuss.elastic.co/u/Michael_TRAN)\ **Replies:** 2\ **Last updated:** [June 27, 2017, 9:30am UTC](https://discuss.elastic.co/t/grok-pattern-for-access-log/90626 "2017-06-27T09:30:09Z")
Hi, I have a issue with the grok pattern for access log. I try this grok filter %{COMBINEDAPACHELOG} but it's not working. I identify why and it's because of this variable "301148" between the date and GET 100.1.1.… --- ## [How to route logs to different Indexs](https://discuss.elastic.co/t/how-to-route-logs-to-different-indexs/89852)
**Author:** [@tomer](https://discuss.elastic.co/u/tomer)\ **Replies:** 8\ **Last updated:** [June 27, 2017, 9:12am UTC](https://discuss.elastic.co/t/how-to-route-logs-to-different-indexs/89852 "2017-06-27T09:12:27Z")
Hi, I am trying to route certain logs to index 1 and other logs to index 2. I know that this not the best practice but this will solve us many problems. One log looks like: /opt/mod/a.log The other looks like: /opt/… --- ## [Multiple Logstash conf files with the same Index](https://discuss.elastic.co/t/multiple-logstash-conf-files-with-the-same-index/90923)
**Author:** [@Vivin\_Pai](https://discuss.elastic.co/u/Vivin_Pai)\ **Replies:** 0\ **Last updated:** [June 27, 2017, 7:43am UTC](https://discuss.elastic.co/t/multiple-logstash-conf-files-with-the-same-index/90923 "2017-06-27T07:43:44Z")
Hello , I have created 2 conf files in /etc/logstash/conf.d folder as below \[root@elkstack-01 conf.d\]# cat gxmsg.conf input{ file{ path =\> "/advdata/tmp/Counters/cnt-gx.msg\*.csv" … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=955) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=957)