Mapping definition for [body] has unsupported parameters

Zorkmid · December 21, 2018, 12:13am

Hello All,

I've got the follow setup installed:
packetbeat-6.5.4-1.x86_64 (installed on 2 different downstream servers)
elasticsearch-6.5.4-1.noarch (installed on 6 physical servers - setup as 3 data and 3 masters)
kibana-6.5.4-1.x86_64 ( installed on one of the master servers)

The config for packetbeat is simple. Simple defined as pointing the beat at the ES server thats running Kibana and edit the beat name and change nothing else. The beat starts as expected.

2018-12-20T14:26:32.305-0800 INFO instance/beat.go:278 Setup Beat: packetbeat; Version: 6.5.4
2018-12-20T14:26:35.307-0800 INFO add_cloud_metadata/add_cloud_metadata.go:319 add_cloud_metadata: hosting provider type not detected.
2018-12-20T14:26:35.307-0800 INFO elasticsearch/client.go:163 Elasticsearch url: http://10.1.249.31:9200
2018-12-20T14:26:35.307-0800 INFO [publisher] pipeline/module.go:110 Beat name: hercules
2018-12-20T14:26:35.308-0800 INFO procs/procs.go:91 Process watcher disabled
Config OK
[ OK ]

However no index is created for either downstream beat. The following is sample from the one of the beat instances:

2018-12-20T15:33:10.292-0800 WARN elasticsearch/client.go:521 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xf79bb20, ext:63680945589, loc:(*time.Location)(0x22d2700)}, Meta:common.MapStr(nil), Fields:common.MapStr{"status":"Error", "client_port":0xc6d7, "client_server":"", "client_proc":"", "dns":common.MapStr{"op_code":"QUERY", "answers_count":0, "flags":common.MapStr{"truncated_response":false, "recursion_desired":true, "recursion_available":false, "authentic_data":false, "checking_disabled":false, "authoritative":false}, "opt":common.MapStr{"ext_rcode":"Unknown 15", "do":true, "version":"0", "udp_size":0xfa0}, "id":0x5bd, "question":common.MapStr{"etld_plus_one":"akadns.net.", "name":"v10-win.vortex.data.microsoft.com.akadns.net.", "type":"A", "class":"IN"}, "additionals_count":0, "response_code":"NOERROR", "authorities_count":0}, "ip":"8.8.8.8", "port":0x35, "proc":"", "host":common.MapStr{"name":"usherlsec01.bio-rad.com", "os":common.MapStr{"platform":"centos", "version":"6.8 (Final)", "family":"redhat", "codename":"Final"}, "containerized":true, "architecture":"x86_64"}, "client_ip":"10.3.122.224", "query":"class IN, type A, v10-win.vortex.data.microsoft.com.akadns.net.", "transport":"udp", "notes":"Another query with the same DNS ID from this client was received so this query was closed without receiving a response", "beat":common.MapStr{"version":"6.5.4", "name":"hercules", "hostname":"usherlsec01.bio-rad.com"}, "type":"dns", "resource":"v10-win.vortex.data.microsoft.com.akadns.net.", "bytes_in":73, "method":"QUERY", "server":""}, Private:interface {}(nil)}, Flags:0x0} (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [doc]: Mapping definition for [body] has unsupported parameters: [ignore_above : 1024]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [body] has unsupported parameters: [ignore_above : 1024]"}}

Thoughts of where to go to chase this down and correct?

Thanks
TimW

shaunak · January 2, 2019, 10:27pm

For starters, lets see if we can identify the index that has the bad mapping. Could you run the following ES query in Console and post the output here (might be large)?

GET */_mapping?filter_path=**.body.ignore_above

leprovokateur · January 4, 2019, 10:07am

Hi,

I have the same problem. The command you suggested gives me metricbeat indices, not packetbeat indices as I would expect:

{
  "metricbeat-6.5.4-2019.01.04": { },
  "metricbeat-6.5.2-2018.12.31": { },
  "metricbeat-6.5.2-2018.12.10": { },
  "metricbeat-6.5.2-2018.12.12": { },
  "metricbeat-6.5.2-2018.12.22": { },
  "metricbeat-6.5.4-2019.01.03": { },
  "metricbeat-6.5.1-2018.12.19": { },
  "metricbeat-6.5.1-2018.12.26": { },
  "metricbeat-6.5.1-2018.12.12": { },
  "metricbeat-6.5.1-2018.12.29": { },
  "metricbeat-6.5.1-2018.12.11": { },
  "metricbeat-6.5.1-2018.12.27": { },
  "metricbeat-6.5.1-2018.12.09": { },
  "metricbeat-6.5.1-2019.01.03": { },
  "metricbeat-6.5.1-2018.12.22": { },
  "metricbeat-6.5.1-2018.12.25": { },
  "metricbeat-6.5.2-2018.12.11": { },
  "metricbeat-6.5.1-2018.12.21": { },
  "metricbeat-6.5.2-2018.12.26": { },
  "metricbeat-6.5.2-2019.01.02": { },
  "metricbeat-6.5.1-2018.12.14": { },
  "metricbeat-6.5.2-2018.12.25": { },
  "metricbeat-6.5.1-2018.12.06": { },
  "metricbeat-6.5.2-2018.12.23": { },
  "metricbeat-6.5.2-2018.12.20": { },
  "metricbeat-6.5.2-2018.12.27": { },
  "metricbeat-6.5.1-2018.12.28": { },
  "metricbeat-6.5.1-2018.12.17": { },
  "metricbeat-6.5.1-2018.12.30": { },
  "metricbeat-6.5.1-2018.12.31": { },
  "metricbeat-6.5.1-2018.12.20": { },
  "metricbeat-6.5.2-2019.01.01": { },
  "metricbeat-6.5.2-2018.12.16": { },
  "metricbeat-6.5.2-2018.12.18": { },
  "metricbeat-6.5.2-2018.12.19": { },
  "metricbeat-6.5.1-2019.01.04": { },
  "metricbeat-6.5.1-2018.12.18": { },
  "metricbeat-6.5.1-2018.12.08": { },
  "metricbeat-6.5.2-2018.12.17": { },
  "metricbeat-6.5.1-2018.12.15": { },
  "metricbeat-6.5.1-2018.12.24": { },
  "metricbeat-6.5.1-2018.12.07": { },
  "metricbeat-6.5.2-2018.12.21": { },
  "metricbeat-6.5.2-2018.12.28": { },
  "metricbeat-6.5.2-2018.12.15": { },
  "metricbeat-6.5.2-2018.12.29": { },
  "metricbeat-6.5.1-2018.12.23": { },
  "metricbeat-6.5.2-2018.12.30": { },
  "metricbeat-6.5.1-2019.01.02": { },
  "metricbeat-6.5.1-2018.12.10": { },
  "metricbeat-6.5.2-2018.12.13": { },
  "metricbeat-6.5.2-2018.12.14": { },
  "metricbeat-6.5.2-2018.12.24": { },
  "metricbeat-6.5.1-2018.12.16": { },
  "metricbeat-6.5.2-2019.01.03": { },
  "metricbeat-6.5.1-2019.01.01": { },
  "metricbeat-6.5.1-2018.12.13": {
    "mappings": {
      "doc": {
        "properties": {
          "http": {
            "properties": {
              "request": {
                "properties": {
                  "body": {
                    "ignore_above": 1024
                  }
                }
              },
              "response": {
                "properties": {
                  "body": {
                    "ignore_above": 1024
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

I am highly interested in a solution.

Best regards,
Robert

shaunak · January 4, 2019, 1:01pm

Hi @leprovokateur,

Could you post the exact command you ran to generate the Mapping definition for [body] has unsupported parameters: [ignore_above : 1024]` error and any log output that resulted from that command, like the OP did?

leprovokateur · January 4, 2019, 1:18pm

Hi,

I ran packetbeat -e

The logs packetbeat generates are the following:

2019-01-04T14:11:14.602+0100    INFO    instance/beat.go:592    Home path: [/usr/share/packetbeat] Config path: [/etc/packetbeat] Data path: [/var/lib/packetbeat] Logs path: [/var/log/packetbeat]
2019-01-04T14:11:14.610+0100    INFO    instance/beat.go:599    Beat UUID: ab3a7ecf-9e1b-426a-801c-252fbbab13c1
2019-01-04T14:11:14.610+0100    INFO    [seccomp]       seccomp/seccomp.go:116  Syscall filter successfully installed
2019-01-04T14:11:14.610+0100    INFO    [beat]  instance/beat.go:825    Beat info       {"system_info": {"beat": {"path": {"config": "/etc/packetbeat", "data": "/var/lib/packetbeat", "home": "/usr/share/packetbeat", "logs": "/var/log/packetbeat"}, "type": "packetbeat", "uuid": "ab3a7ecf-9e1b-426a-801c-252fbbab13c1"}}}
2019-01-04T14:11:14.610+0100    INFO    [beat]  instance/beat.go:834    Build info      {"system_info": {"build": {"commit": "6da316ebb3ba6ed57725b7fd7c21e598522855bf", "libbeat": "6.5.3", "time": "2018-12-06T19:09:48.000Z", "version": "6.5.3"}}}
2019-01-04T14:11:14.611+0100    INFO    [beat]  instance/beat.go:837    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":6,"version":"go1.10.3"}}}
2019-01-04T14:11:14.648+0100    INFO    [beat]  instance/beat.go:841    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-12-11T08:19:18+01:00","containerized":true,"name":"WUM96178","ip":["127.0.0.1/8","::1/128","192.168.96.178/24","fe80::250:56ff:fe81:6bf7/64"],"kernel_version":"3.10.0-957.1.3.el7.x86_64","mac":["00:50:56:81:6b:f7"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":6,"patch":1810,"codename":"Core"},"timezone":"CET","timezone_offset_sec":3600,"id":"3499004948574b048702e521999c7fde"}}}
2019-01-04T14:11:14.649+0100    INFO    [beat]  instance/beat.go:870    Process info    {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/root", "exe": "/usr/share/packetbeat/bin/packetbeat", "name": "packetbeat", "pid": 39246, "ppid": 39197, "seccomp": {"mode":"filter"}, "start_time": "2019-01-04T14:11:13.640+0100"}}}
2019-01-04T14:11:14.649+0100    INFO    instance/beat.go:278    Setup Beat: packetbeat; Version: 6.5.3
2019-01-04T14:11:14.678+0100    INFO    add_cloud_metadata/add_cloud_metadata.go:319    add_cloud_metadata: hosting provider type not detected.
2019-01-04T14:11:14.678+0100    INFO    elasticsearch/client.go:163     Elasticsearch url: http://localhost:9200
2019-01-04T14:11:14.679+0100    INFO    [publisher]     pipeline/module.go:110  Beat name: WUM96178
2019-01-04T14:11:14.679+0100    INFO    procs/procs.go:91       Process watcher disabled
2019-01-04T14:11:14.681+0100    INFO    [monitoring]    log/log.go:117  Starting metrics logging every 30s
2019-01-04T14:11:14.681+0100    INFO    instance/beat.go:400    packetbeat start running.
2019-01-04T14:11:16.634+0100    INFO    pipeline/output.go:95   Connecting to backoff(elasticsearch(http://localhost:9200))
2019-01-04T14:11:16.637+0100    INFO    elasticsearch/client.go:713     Connected to Elasticsearch version 6.5.4
2019-01-04T14:11:16.639+0100    INFO    template/load.go:129    Template already exists and will not be overwritten.
2019-01-04T14:11:16.639+0100    INFO    pipeline/output.go:105  Connection to backoff(elasticsearch(http://localhost:9200)) established
2019-01-04T14:11:17.868+0100    WARN    elasticsearch/client.go:521     Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x112eba38, ext:63682204275, loc:(*time.Location)(0x22d2160)}, Meta:common.MapStr(nil), Fields:common.MapStr{"path":"ff02::1", "client_ip":"2001:67c:6c4:1803::1", "ip":"ff02::1", "beat":common.MapStr{"hostname":"WUM96178", "version":"6.5.3", "name":"WUM96178"}, "host":common.MapStr{"architecture":"x86_64", "os":common.MapStr{"codename":"Core", "platform":"centos", "version":"7 (Core)", "family":"redhat"}, "id":"3499004948574b048702e521999c7fde", "containerized":true, "name":"WUM96178"}, "status":"OK", "bytes_in":24, "icmp":common.MapStr{"version":0x6, "request":common.MapStr{"message":"NeighborAdvertisement(0)", "type":0x88, "code":0x0}}, "type":"icmp"}, Private:interface {}(nil)}, Flags:0x0} (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [doc]: Mapping definition for [body] has unsupported parameters:  [ignore_above : 1024]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [body] has unsupported parameters:  [ignore_above : 1024]"}}

Regards,
Robert

shaunak · January 4, 2019, 8:05pm

Thanks @leprovokateur. I think the metricbeat mappings might be a red herring here.

Could you also post the output of these two ES queries?

GET _cat/indices/*beat?v
GET _template/*packetbeat*

Zorkmid · January 4, 2019, 9:32pm

Hello,

Ran "GET */_mapping?filter_path=**.body.ignore_above" and which returned 2 empty brackets {}.

However in rereading the error message "Mapping definition for [body] has unsupported parameters: [ignore_above : 1024]" it appears to be generated from the packetbeat template on the ES cluster since I've elect not to override the default templates in my config.

Running "GET /_template/packetbeat-6*" in the ES console shows the various packetbeat templates present in the system and the only area within the template that has the keyword "body" with the keyword "ingnore_above" is the http portion of the template.

Note: the WinlogBeat is still running as expected.

"GET /_cat/templates/beat"
packetbeat-6.4.3 [packetbeat-6.4.3-] 1
packetbeat [packetbeat-] 0
.management-beats [.management-beats] 0 65000
winlogbeat-6.3.0 [winlogbeat-6.3.0-] 1
.monitoring-beats [.monitoring-beats-6-] 0 6050399
packetbeat-6.5.4 [packetbeat-6.5.4-] 1
packetbeat-6.3.2 [packetbeat-6.3.2-] 1
winlogbeat-6.3.1 [winlogbeat-6.3.1-*] 1

"GET /_cat/indices/beat"
green open winlogbeat-6.3.1-2018.12.26 P7BaylgWQT-hJbONXYdlRg 5 1 6 0 132.6kb 66.3kb
green open winlogbeat-6.3.0-2018.12.29 QQT-RnsXSf6he-fhk38BRw 5 1 633 0 940.3kb 480.7kb
green open winlogbeat-6.3.0-2018.12.28 Xg4nCigNR52MKsS4lNYrIQ 5 1 595 0 1.1mb 563.4kb
green open winlogbeat-6.3.1-2019.01.03 OsmkqQurT5S7l_V5C329jw 5 1 255 0 1mb 529.3kb
green open winlogbeat-6.3.0-2018.12.31 4rq_b33KTxa5bavSXp7wHw 5 1 585 0 1mb 567.7kb
green open winlogbeat-6.3.0-2018.12.26 6shjpRPSTjq8nkzNqO9HlQ 5 1 567 0 1mb 548.6kb
green open winlogbeat-6.3.1-2018.12.30 pfRDrz-uSEK0tOzgB8Ot1g 5 1 186 0 920.8kb 460.3kb
green open winlogbeat-6.3.0-2019.01.03 SI_v0F1tRwGvBZijHA8n9w 5 1 595 0 1017kb 477.6kb
green open winlogbeat-6.3.0-2018.12.30 tjFz5feuQ0i3nuu8oiuYFg 5 1 589 0 1mb 587.2kb
green open winlogbeat-6.3.1-2018.12.28 g9XwZb1pSLKE9gM_Qz3dyw 5 1 194 0 1mb 508.5kb
green open winlogbeat-6.3.1-2018.12.27 dZ7cRSjLR8i6WF3uUNoG6w 5 1 189 0 697kb 341.3kb
green open winlogbeat-6.3.1-2018.12.29 8HIT4cj9Ry6QrnaiXAcb7w 5 1 332 0 949.6kb 469.6kb
green open winlogbeat-6.3.1-2019.01.01 9pdvPCUCSlGYRrXhxVi0jQ 5 1 198 0 1mb 537kb
green open winlogbeat-6.3.1-2018.12.31 JQnEJUE1TQyyKmRvDF78gg 5 1 253 0 854kb 422.7kb
green open winlogbeat-6.3.0-2019.01.01 2edl6HsfQiKXQLbvL9wAxA 5 1 631 0 1.2mb 689.4kb
green open winlogbeat-6.3.1-2019.01.02 aPLb_mVXQKCd56IBUyzTNw 5 1 187 0 784.6kb 392.3kb
green open winlogbeat-6.3.0-2018.12.27 Vs9A7_tlSEi2TB4kVw64ag 5 1 654 0 1.1mb 565.5kb
green open winlogbeat-6.3.1-2019.01.04 DRjd5swuQ2qGhufbzSQbbg 5 1 190 0 965.6kb 467kb
green open winlogbeat-6.3.0-2019.01.04 GlGla39_Raej_HtM8UI3fA 5 1 572 0 1000.3kb 563.1kb
green open winlogbeat-6.3.0-2019.01.02 61vh_HK6RHu58TD1MO0Cqw 5 1 582 0 1mb 525.4kb

Regards
TimW

Zorkmid · January 5, 2019, 12:03am

Hello,

My problem seems to be solved -

In the ES console I hunted down the various templates that related to the problem.
Found a template from 5.4 version of Packetbeat plus a few others.
In the ES console I did a "DELETE _template/<packetbeat_template>" on each errant template that I did not want.
Lastly I restarted the remote beats and voilà, the correct template for 6.5.4 is used and the indices are built with event data - yeah me!!

Cheers and thanks for the sparking an idea.
TimW

system · February 2, 2019, 12:03am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.