# Missing Available Fields after Update **URL:** \ **Category:** Beats\ **Created:** [December 31, 2018, 8:04pm UTC](https://discuss.elastic.co/t/missing-available-fields-after-update/162536 "2018-12-31T20:04:53Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![bfrd](https://avatars.discourse-cdn.com/v4/letter/b/919ad9/32.png) [@bfrd](https://discuss.elastic.co/u/bfrd)\ **Post date:** [December 31, 2018, 8:04pm UTC](https://discuss.elastic.co/t/missing-available-fields-after-update/162536/1 "2018-12-31T20:04:53Z")
Hello, Recently upgraded elastic stack and filebeat on clients from around 6.5.1 to 6.5.3. We use the system filebeat module. In Kibana \> Discover I can still search using terms like "system.auth.user: root" but system.auth.user is no longer showing up as an available field. When I unhide missing fields it still exists and I can add it but the field data comes up as "-" (no data). ![Hits](https://us1.discourse-cdn.com/elastic/original/3X/5/9/59bf54bd652e4ef6485940e99ba328ee7873ee6d.png) ![Fields](https://us1.discourse-cdn.com/elastic/original/3X/5/8/58d24db1e744b8f46f41f3f861b5202e0f543b72.png) If I expand the document, from the table tab, system has a "?" to the left which I think indicates unknown type. The content of system is json. ``` { "auth": { "ssh": { "event": "Accepted", "method": "publickey", "signature": "", "geoip": {}, "port": "38508", "ip": "" }, "hostname": "", "pid": "27374", "user": "", "timestamp": "Dec 31 11:12:01" } } ``` Also under available fields, if I select "? system", it expands and below reads "Analysis is not available for object fields" Under "t tags" it shows the top 5 values per usual. ![AnalysisNotAvailable](https://us1.discourse-cdn.com/elastic/original/3X/5/3/531169c4372183ff6294b8a3bcf34e9da2798a64.png) Forgive me for my crude explanation and ignorance. This may be posted to the wrong section, I only noticed the problem after upgrading the shippers but it may be a logstash, elasticsearch, or kibana issue. Thanks for your help! ---
**Author:** ![bfrd](https://avatars.discourse-cdn.com/v4/letter/b/919ad9/32.png) [@bfrd](https://discuss.elastic.co/u/bfrd)\ **Post date:** [January 3, 2019, 5:34pm UTC](https://discuss.elastic.co/t/missing-available-fields-after-update/162536/2 "2019-01-03T17:34:14Z")
I checked my filters just in case, I am using the one from the 6.5 Reference found [here](https://www.elastic.co/guide/en/logstash/current/logstash-config-for-filebeat-modules.html#parsing-system). I added tagging, they are coming through with success tag. ``` filter { if [fileset][module] == "system" { if [fileset][name] == "auth" { grok { match => { "message" => [ "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} %{DATA:[system][auth][ssh][method]} for (invalid user )?%{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]} port %{NUMBER:[system][auth][ssh][port]} ssh2(: %{GREEDYDATA:[system][auth][ssh][signature]})?", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} user %{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: Did not receive identification string from %{IPORHOST:[system][auth][ssh][dropped_ip]}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sudo(?:\[%{POSINT:[system][auth][pid]}\])?: \s*%{DATA:[system][auth][user]} :( %{DATA:[system][auth][sudo][error]} ;)? TTY=%{DATA:[system][auth][sudo][tty]} ; PWD=%{DATA:[system][auth][sudo][pwd]} ; USER=%{DATA:[system][auth][sudo][user]} ; COMMAND=%{GREEDYDATA:[system][auth][sudo][command]}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} groupadd(?:\[%{POSINT:[system][auth][pid]}\])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} useradd(?:\[%{POSINT:[system][auth][pid]}\])?: new user: name=%{DATA:[system][auth][user][add][name]}, UID=%{NUMBER:[system][auth][user][add][uid]}, GID=%{NUMBER:[system][auth][user][add][gid]}, home=%{DATA:[system][auth][user][add][home]}, shell=%{DATA:[system][auth][user][add][shell]}$", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} %{DATA:[system][auth][program]}(?:\[%{POSINT:[system][auth][pid]}\])?: %{GREEDYMULTILINE:[system][auth][message]}" ] } pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)*" } remove_field => "message" add_tag => ["_grok_system_auth_success"] tag_on_failure => ["_grok_system_auth_nomatch"] } date { match => ["[system][auth][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] } geoip { source => "[system][auth][ssh][ip]" target => "[system][auth][ssh][geoip]" } } else if [fileset][name] == "syslog" { grok { match => { "message" => [ "%{SYSLOGTIMESTAMP:[system][syslog][timestamp]} %{SYSLOGHOST:[system][syslog][hostname]} %{DATA:[system][syslog][program]}(?:\[%{POSINT:[system][syslog][pid]}\])?: %{GREEDYMULTILINE:[system][syslog][message]}" ] } pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)*" } remove_field => "message" add_tag => ["_grok_system_syslog_success"] tag_on_failure => ["_grok_system_syslog_nomatch"] } date { match => ["[system][syslog][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] } } } } ``` Also, in case it would help I am running up-to-date Debian 9 Stretch with elastic-6.x stable repo. Installed package versions here: ``` elasticsearch 6.5.2 kibana-oss 6.3.0 logstash 1:6.5.2-1 ``` ---
**Author:** ![bfrd](https://avatars.discourse-cdn.com/v4/letter/b/919ad9/32.png) [@bfrd](https://discuss.elastic.co/u/bfrd)\ **Post date:** [January 3, 2019, 8:11pm UTC](https://discuss.elastic.co/t/missing-available-fields-after-update/162536/3 "2019-01-03T20:11:00Z")
I've downloaded and installed \*-oss-6.5.4 packages ``` elasticsearch-oss 6.5.4 kibana-oss 6.5.4 logstash 1:6.5.4-1 ``` Was hoping the old Kibana was the problem but the problem persists. ---
**Author:** ![bfrd](https://avatars.discourse-cdn.com/v4/letter/b/919ad9/32.png) [@bfrd](https://discuss.elastic.co/u/bfrd)\ **Post date:** [January 3, 2019, 10:14pm UTC](https://discuss.elastic.co/t/missing-available-fields-after-update/162536/4 "2019-01-03T22:14:11Z")
After upgrading everything and waiting a while it seems as though the problem as been solved. Now fields are populated in Kibana. ![2019-01-03_1054x878](https://us1.discourse-cdn.com/elastic/original/3X/1/9/1986671198f769b5d06178acd95b0f871ccbe15e.png) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 1, 2019, 12:14am UTC](https://discuss.elastic.co/t/missing-available-fields-after-update/162536/5 "2019-02-01T00:14:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.