# Developers would like to see some log files from they applications **URL:** \ **Category:** Elasticsearch\ **Created:** [June 4, 2020, 4:35pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814 "2020-06-04T16:35:40Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![jacobzh](https://avatars.discourse-cdn.com/v4/letter/j/5e9695/32.png) [@jacobzh](https://discuss.elastic.co/u/jacobzh)\ **Post date:** [June 4, 2020, 4:35pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/1 "2020-06-04T16:35:40Z")
I created elk project(elasticsearch, logstash, kibana and filebeat) for production environment. It’s contained 1 server and 5 filebeat clients. Our developers would like to see some log files from they applications. I used system module, because is no other module close to they application. Kibana in logs view shows the paths to they log files, but no content. I checked syslog and auth.log files, same story. Only message, no content. Maybe Elasticsearch works like this? Can you clarify this, please? Or I have to change something? Below, what I can see in LOG stream for one of my filebeat client: ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/0/4055544929baf07982fa2fdcb64f751c639fe0ad.png) My Index management: ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/1/613a272ea5579110f0c588ceb8f880034999a90e.png) ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [June 4, 2020, 7:53pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/2 "2020-06-04T19:53:30Z")
So you have collected 475 million log entries. As a next step: 1. Can you add one of your Filebeat configs? 2. Are you getting the logs from your application? Probably the "awsapi02" search isn't what you want — what happens if you search for `not event.dataset:system.log`? ---
**Author:** ![jacobzh](https://avatars.discourse-cdn.com/v4/letter/j/5e9695/32.png) [@jacobzh](https://discuss.elastic.co/u/jacobzh)\ **Post date:** [June 4, 2020, 9:01pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/3 "2020-06-04T21:01:39Z")
Hi Xeraa, This my filebeat.yml file on awsapi02: #=========================== Filebeat inputs ============================= filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. # paths: paths: /var/log/awsapi/localhost.log /var/log/awsapi/server.log\_yyyy-MM-ddTHH-mm-ss #- c:\programdata\elasticsearch\logs\* # Exclude lines. A list of regular expressions to match. It drops the lines that are # matching any regular expression from the list. #exclude\_lines: ['^DBG'] # Include lines. A list of regular expressions to match. It exports the lines that are # matching any regular expression from the list. #include\_lines: ['^ERR', '^WARN'] #============================= Filebeat modules =============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/\*.yml # Set to true to enable config reloading reload.enabled: false # Period on which files under path should be checked for changes #reload.period: 10s #==================== Elasticsearch template setting ========================== setup.template.settings: index.number\_of\_shards: 3 #index.codec: best\_compression #\_source.enabled: false #============================== Kibana ===================================== # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API. # This requires a Kibana endpoint configuration. setup.kibana: host: "10.70.53.201:5601" # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: [http://localhost:5601/path](http://localhost:5601/path) # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 #host: "localhost:5601" # Kibana Space ID # ID of the Kibana Space into which the dashboards should be loaded. By default, # the Default Space will be used. #space.id: #-------------------------- Elasticsearch output ------------------------------ output.elasticsearch: # Array of hosts to connect to. hosts: ["10.70.53.201:9200"] # Protocol - either `http` (default) or `https`. #protocol: "https" # Authentication credentials - either API key or username/password. #api\_key: "id:api\_key" #username: "elastic" #password: "changeme" Rest of the file is commented out. When I tried to search for `not event.dataset:system.log, I got:``` `` ![image001.jpg](https://us1.discourse-cdn.com/elastic/original/3X/5/0/507aa5292eb37ccc9faf693785742a79229f5e10.jpeg) It’s nothing what I am looking for. ![image005.jpg](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d2f0c0ccae06b37b7a20cfbb60a5c81f6722407e.jpeg) ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [June 4, 2020, 10:05pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/4 "2020-06-04T22:05:32Z")
Please format your code for better readability. The part with `/var/log/awsapi/localhost.log` looks good. Change the filter not in the highlights but in the KQL field (where it says `awsapi02` in your screenshot — top left). ---
**Author:** ![jacobzh](https://avatars.discourse-cdn.com/v4/letter/j/5e9695/32.png) [@jacobzh](https://discuss.elastic.co/u/jacobzh)\ **Post date:** [June 4, 2020, 11:51pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/5 "2020-06-04T23:51:09Z")
Which part of the code you want to see? I change the filter in KQL field: ![image002.jpg](https://us1.discourse-cdn.com/elastic/original/3X/2/0/206717ade37ee94b1c68d7944318dde283b1edc4.jpeg) ![image005.jpg](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d2f0c0ccae06b37b7a20cfbb60a5c81f6722407e.jpeg) ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [June 5, 2020, 1:19am UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/6 "2020-06-05T01:19:09Z")
Using properly formated code was just a general statement. The Filebeat config looked ok. And the query should be `not event.dataset:system.syslog`, sorry. I want to exclude the syslog logs so we can see your own logs; though there will be multiple ways to do that. Alternatively filter on the `log.file.path` to the ones you're interested in. ---
**Author:** ![jacobzh](https://avatars.discourse-cdn.com/v4/letter/j/5e9695/32.png) [@jacobzh](https://discuss.elastic.co/u/jacobzh)\ **Post date:** [June 5, 2020, 1:28pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/7 "2020-06-05T13:28:52Z")
When I tried to look for /var/log/awsapi/localhost.log, it shows message not content: ![image001.png](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0bd8ea2921528ae12f5766feae58e6f5451a4684.png) And for /var/log/awsapi/server.log\_yyyy-MM-ddTHH-mm-ss, same story, just message: ![image003.png](https://us1.discourse-cdn.com/elastic/original/3X/9/7/971146bba1eb460e171d859cbbb2b8d1956b5267.png) ![image002.jpg](https://us1.discourse-cdn.com/elastic/original/2X/2/2485d26919316b1d80aa60f5ca8e2cb5bc0275fe.jpg) ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [June 8, 2020, 4:58am UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/8 "2020-06-08T04:58:13Z")
1. I don't think `/var/log/awsapi/server.log_yyyy-MM-ddTHH-mm-ss` will do much, since this should be replaced by the actual timestamp. In your Filebeat config this should probably be `/var/log/awsapi/server.log_*`. 2. Can you refresh the page and make sure your search is being applied? Because if there are no search hits, it should look like this: ![Screenshot 2020-06-08 at 06.55.08](https://us1.discourse-cdn.com/elastic/original/3X/f/a/fa2271b4a28a786031d44046900080fa11ae2e5d.png) 1. If I search for `not log.file.path : "/var/log/syslog"` it excludes those, but shows the results from other files. ---
**Author:** ![jacobzh](https://avatars.discourse-cdn.com/v4/letter/j/5e9695/32.png) [@jacobzh](https://discuss.elastic.co/u/jacobzh)\ **Post date:** [June 8, 2020, 5:26pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/9 "2020-06-08T17:26:26Z")
I changed PATH in filebeat config file to `/var/log/awsapi/server.log_*. ``Restart filebeat service.``` `After this I executed: filebeat setup -e -E output.logstash.enabled=false -E output.elasticsearch.hosts=['10.70.53.201:9200'] -E setup.kibana.host=10.70.53.201:5601.``` `Refresh KDL field with /var/log/awsapi/server.log_yyyy_MM-ddTHH-mm-ss, same in Highlights and got:``` `` ![image003.jpg](https://us1.discourse-cdn.com/elastic/original/3X/9/9/99f69abeb26409ffea240636108a13104ffb476d.jpeg) Without Highlights I got: ![image004.jpg](https://us1.discourse-cdn.com/elastic/original/3X/1/8/18d0dbf6a371775550ae08cfcf7d6fe187be8e2a.jpeg) ![~WRD000.jpg](https://us1.discourse-cdn.com/elastic/original/3X/5/2/52d5f1c08649212fe6768d2cacdcf8df711810d9.jpeg) ![image009.jpg](https://us1.discourse-cdn.com/elastic/original/3X/7/2/72fef9cf0bea17b9310bf6cb04e6832ec02811bf.jpeg) ![image005.jpg](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d2f0c0ccae06b37b7a20cfbb60a5c81f6722407e.jpeg) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2020, 5:26pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814/10 "2020-07-06T17:26:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.