# Import Apache Log File from Win10 to Kibana/Logstash on Debian **URL:** \ **Category:** Logstash\ **Created:** [July 28, 2020, 5:16pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948 "2020-07-28T17:16:43Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 28, 2020, 5:16pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/1 "2020-07-28T17:16:43Z")
Hi Everyone, I am new to Kibana, I have recently uploaded Apache Log File through Filebeat (Installed on Win10) but when I check on Kibana Discovery it is displaying only single entry of entire logfile. My Setup: Filebeat on Windows10 --\> Logstash and Kibana Install on same machine (Debian) Below is the snapshot ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e51b8ac0b7851e4b2a8d42dadf0567d736525889.png) Can anyone help me? Regards Imran ---
**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\ **Post date:** [July 29, 2020, 8:06am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/2 "2020-07-29T08:06:50Z")
- have you put index-pattern? - what's your config in filebeat? Does it just send to logstash or directly to elasticsearch? (if it sends directly to elasticsearch, it will be in filebeat\* pattern) ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 29, 2020, 2:04pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/3 "2020-07-29T14:04:33Z")
Hi Kelk, I am really thankful for the response, in filebeat I have just send to logstash. I have no idea of any index pattern, could you please guide me? Regards Imran ---
**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\ **Post date:** [July 29, 2020, 2:42pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/4 "2020-07-29T14:42:50Z")
[![]( " - YouTube") ](https://www.youtube.com/watch?v=uBSRaUOgEz8) that should help in creating index pattern in Kibana ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 29, 2020, 3:28pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/5 "2020-07-29T15:28:32Z")
Dear Kelk, Thanks for the quick response. I have already done this one but, i am suspecting something i have to do it with logstash, as i have forwarded the log file but haven't done anything on the logstash. Is there anything I have to performed on logstash.conf file to accept and parse the logs to elasticsearch? Regards Imran ---
**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\ **Post date:** [July 29, 2020, 3:34pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/6 "2020-07-29T15:34:11Z")
ok. My approach will be to do baby steps 1.configure filebeat 2.send to logstash and print-it out onto console to see if it reached logstash 3.If yes, then put the filters and then again print to console 4.if all good, then try sending to elasticsearch & then create index pattern Step2 would be something like.. ```auto # logstash input { beats { port => 5044 } } # just output to logstash console output { stdout { codec => rubydebug } } ``` ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 29, 2020, 4:16pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/7 "2020-07-29T16:16:15Z")
Dear Kelk, could you please let me know how to run this file in linux to test the logstash? Regards Imran ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 29, 2020, 4:18pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/8 "2020-07-29T16:18:39Z")
Dear Kelk, Below Error I am receiving ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/e/fed246e821f4b7b7b09086cfe16618c4d0669558.png) ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 29, 2020, 4:27pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/9 "2020-07-29T16:27:06Z")
Dear Kelk, It seems configuration is k, as per below result: ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/e/6ef0d752a1b4633c3abb3a2703b88804d39da5d6.png) Please have a look and let me know the next step Regards Imran ---
**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\ **Post date:** [July 29, 2020, 4:35pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/10 "2020-07-29T16:35:49Z")
this is NOT an error. just a warning for some variable/modules. you can ignore them So just run and see as normal and you should see the output in stdout/console Also you could put your config here once you sanitise for any sensitive configs ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [July 29, 2020, 5:06pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/11 "2020-07-29T17:06:35Z")
Below is the configuration as requested... input { beats { port =\> "5044" } } filter { grok { match =\> { "message" =\> "%{COMBINEDAPACHELOG}"} } geoip { source =\> "clientip" } } output { elasticsearch { hosts =\> ["localhost:9200"] } } ---
**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\ **Post date:** [July 29, 2020, 7:39pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/12 "2020-07-29T19:39:13Z")
The config seems simple and you should receive the data. Try basic step and see if you can see data arriving at logstash (and output to console) ```auto input { beats { port => "5044" } } output { stdout { codec => rubydebug } } ``` ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [August 1, 2020, 10:58am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/13 "2020-08-01T10:58:54Z")
Hi Kelk, Hope your are doing well. I figured it out the issue. It is related with filebeat.yml and logstash config file. I have attached both the file configuration. The major issue is with logstash as it is deleting all the entries except one line due to "document\_id =\> "%{logstash\_checksum}"" this is not working with Apache logs. Now I have changed to match filtering with @timestamp it is displaying all the logs but i need to figure out unique value in apache as it shall not delete duplicate lines based on timestamp. Do u have any idea of apache unique ID ? ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/0/809a19ce24ba02bcba1e9bb2bc2351c71ed3a60b.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0b78eb6b4e88a3330824cd444f5bbecd080e7628.png) Regards Imran ---
**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\ **Post date:** [August 1, 2020, 6:24pm UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/14 "2020-08-01T18:24:09Z")
strange on you getting the issue. which version of filebeat you using? You don't need to do the filters in logstash as the filebeat apache module is quite sufficient and does all the magic. Please check in the filebeat, if you got - `module/apache/access/manifest.yml` (check contents and files are correct location & permission) - `module/apache/access/ingest/pipeline.yml` (check within it and you will see all the grok and logic) - in modules.d just change from `modules.d/apache.yml.disabled` to `modules.d/apache.yml` After that restart/reload the filebeat and no need to put filters in logstash.. just incoming data and output and all fields should be good (you can print stdout in logstash to see if all fields are coming) ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [August 2, 2020, 2:25am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/15 "2020-08-02T02:25:17Z")
Hi there, I am using 7.8 FileBeat Version. What about this issue "document\_id =\> "%{logstash\_checksum}"" - this is the major problem, if i am not changing the checksum to @timestamp it is overwriting on single log (deleting all logs only single single log displayed as per below image) ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/2/c26909562cb2256b8a5e1f55e6ef52b493d2b272.png) Check first row, it has now docs.count 27xxx with this setting in logstash "document\_id =\> "%{logstash\_checksum}"", I have only one count and remaining in docs.deleted. I will check suggested solution and let you know. Regards Imran ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [August 2, 2020, 3:10am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/16 "2020-08-02T03:10:24Z")
Where should that `logstash_checksum` field have come from? I don't see you creating it anywhere. So so wouldn't be surprised if your ES `_id` was always the same if that sprintf placeholder could not be filled and kept the value `%{logstash_checksum}` for every single document. Have I overlooked something? ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [August 2, 2020, 3:27am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/17 "2020-08-02T03:27:42Z")
Hi Jenni, It was in the output of logstash by default, just i have modified @timestamp for testing purpose. ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [August 2, 2020, 3:37am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/18 "2020-08-02T03:37:27Z")
When you look at your documents in Kibana, is there really a field called `logstash_checksum`? It if isn't, the data that you are trying to use as an ID doesn't exist. ---
**Author:** ![imransarkar](https://avatars.discourse-cdn.com/v4/letter/i/c6cbf5/32.png) [@imransarkar](https://discuss.elastic.co/u/imransarkar)\ **Post date:** [August 2, 2020, 3:40am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/19 "2020-08-02T03:40:28Z")
I am complete new to kibana and as I said it was default, I was surprised in the begining to have such error, anyhow I have resolve by following solution from "Kelk". Now, I want to use built-in filebeat modules for Apache logs (sending offline file) to Kibana. Do you have any idea? where should I send the logs to Elastic search or Logstash? ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [August 2, 2020, 4:20am UTC](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948/20 "2020-08-02T04:20:20Z")
I made those comments _because_ you are new. Just wanted to make sure that you understand why you kept overwriting the same document before you move on to another solution 🙂 @kelk probably has more knowledge about this than I do. But I think if you are using the Apache module and there are no changes to the data structure that you'd like to make, you can send the data directly to ES using the ingest pipeline. Otherwise you might want to [use Logstash](https://www.elastic.co/guide/en/logstash/current/filebeat-modules.html) (or a customized ingest pipeline maybe). [Next page](https://discuss.elastic.co/t/import-apache-log-file-from-win10-to-kibana-logstash-on-debian/242948.md?page=2)