# Problem with Fake Log in Logshash **URL:** \ **Category:** Logstash\ **Created:** [March 6, 2018, 9:41am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685 "2018-03-06T09:41:22Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![just666](https://avatars.discourse-cdn.com/v4/letter/j/b487fb/32.png) [@just666](https://discuss.elastic.co/u/just666)\ **Post date:** [March 6, 2018, 9:41am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/1 "2018-03-06T09:41:22Z")
1. I install Elastic (no problem) 2. I install Kibana (no problem) 3. I run Logshtast with argument: ./logstash -f apache\_config.conf ## apache\_config.conf > input { > file { > path =\> "/var/log/my\_apache\_logs/\*.log" > } > } > filter { > grok { > match =\> { "message" =\> "%{COMBINEDAPACHELOG}"} > } > geoip { > source =\> "clientip" > add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ] > add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ] > } > > mutate { > convert =\> { "[geoip][coordinates]" =\> "float" } > } > } > output { > stdout { codec =\> rubydebug } > elasticsearch { > hosts =\> ["localhost:9200"] > index =\> "logstash-apache-%{+YYYY.MM.dd}" > } > } * * * In /var/log/my\_apache\_logs/\*.log i generated Fake Log from * * * > [https://github.com/kiritbasu/Fake-Apache-Log-Generator](https://github.com/kiritbasu/Fake-Apache-Log-Generator) * * * ## Serwer respond: > Sending Logstash's logs to /data/dg\_vbox/Logstash/logstash-6.2.2/logs which is now configured via log4j2.properties > [2018-03-06T10:34:39,179][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/data/dg\_vbox/Logstash/logstash-6.2.2/modules/netflow/configuration"} > [2018-03-06T10:34:39,192][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/data/dg\_vbox/Logstash/logstash-6.2.2/modules/fb\_apache/configuration"} > [2018-03-06T10:34:39,613][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified > [2018-03-06T10:34:40,081][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.2"} > [2018-03-06T10:34:40,393][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} > [2018-03-06T10:34:43,162][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50} > [2018-03-06T10:34:43,558][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://localhost:9200/](http://localhost:9200/)]}} > [2018-03-06T10:34:43,567][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://localhost:9200/](http://localhost:9200/), :path=\>"/"} > [2018-03-06T10:34:43,714][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://localhost:9200/](http://localhost:9200/)"} > [2018-03-06T10:34:43,760][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>nil} > [2018-03-06T10:34:43,763][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} > [2018-03-06T10:34:43,781][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil} > [2018-03-06T10:34:43,796][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}} > [2018-03-06T10:34:43,831][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]} > [2018-03-06T10:34:43,979][INFO][logstash.filters.geoip] Using geoip database {:path=\>"/data/dg\_vbox/Logstash/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"} > [2018-03-06T10:34:44,251][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline\_id=\>"main", :thread=\>"#\"} > [2018-03-06T10:34:44,339][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]} * * * I can't read that fake log from Kibana CLI (menagement). I can't read that fake log from localhost:9200 (no password) X-pack is not installed What's a problem?? PS I will be grateful for your help ---
**Author:** ![tag\_v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tag_v/32/47466_2.png) [@tag\_v](https://discuss.elastic.co/u/tag_v)\ **Post date:** [March 6, 2018, 9:50am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/2 "2018-03-06T09:50:45Z")
Seems no error with conf file. If input size is huge, logstash take some time to push pipeline events. > [@just666](#): > > ./logstash -f apache\_config.conf try running logstash in debug mode. `./logstash -f apache_config.conf --debug` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 6, 2018, 9:53am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/3 "2018-03-06T09:53:01Z")
Logstash is tailing the log file you generated. Look into the `start_position` option for the file input and read the file input documentation in general to understand how it maintains the current state of the monitored files. ---
**Author:** ![just666](https://avatars.discourse-cdn.com/v4/letter/j/b487fb/32.png) [@just666](https://discuss.elastic.co/u/just666)\ **Post date:** [March 6, 2018, 9:53am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/4 "2018-03-06T09:53:12Z")
It's only 100 events. I try with --debug mode - not help ---
**Author:** ![just666](https://avatars.discourse-cdn.com/v4/letter/j/b487fb/32.png) [@just666](https://discuss.elastic.co/u/just666)\ **Post date:** [March 6, 2018, 9:57am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/5 "2018-03-06T09:57:35Z")
Can you explain exactly? I starting using Elastic and don't know how. ---
**Author:** ![just666](https://avatars.discourse-cdn.com/v4/letter/j/b487fb/32.png) [@just666](https://discuss.elastic.co/u/just666)\ **Post date:** [March 6, 2018, 9:58am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/6 "2018-03-06T09:58:26Z")
Respond from Kibana: GET /logstash/\_search { "query": { "match\_all": {} } } { > "error": { > "root\_cause": [ > { > "type": "index\_not\_found\_exception", > "reason": "no such index", > "resource.type": "index\_or\_alias", > "resource.id": "logstash", > "index\_uuid": "_na_", > "index": "logstash" > } > ], > "type": "index\_not\_found\_exception", > "reason": "no such index", > "resource.type": "index\_or\_alias", > "resource.id": "logstash", > "index\_uuid": "_na_", > "index": "logstash" > }, > "status": 404 > } ---
**Author:** ![tag\_v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tag_v/32/47466_2.png) [@tag\_v](https://discuss.elastic.co/u/tag_v)\ **Post date:** [March 6, 2018, 10:14am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/7 "2018-03-06T10:14:18Z")
``` input { file { path => "/var/log/my_apache_logs/*.log" sincedb_path => "/dev/null" start_position => "beginning" } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ] } mutate { convert => { "[geoip][coordinates]" => "float" } } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["localhost:9200"] index => "logstash-apache-%{+YYYY.MM.dd}" } } ``` ---
**Author:** ![just666](https://avatars.discourse-cdn.com/v4/letter/j/b487fb/32.png) [@just666](https://discuss.elastic.co/u/just666)\ **Post date:** [March 6, 2018, 11:11am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/8 "2018-03-06T11:11:15Z")
[https://pastebin.com/BV7mF06a](https://pastebin.com/BV7mF06a) it's read and return to cli ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 3, 2018, 11:11am UTC](https://discuss.elastic.co/t/problem-with-fake-log-in-logshash/122685/9 "2018-04-03T11:11:39Z")
Closed on April 3, 2018, 11:11am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.