Failed to parse mapping [_doc]: unknown parameter [properties] on mapper [error] of type [text]

Elastic Stack Elasticsearch
1

I am running filebeat aws module to fetch AWS cloudtrail logs, since 1st August I am getting this error and data is not getting indexed. I am using custom index name and using copied template and pipeline from Filebeat index template and pipeline.

{"type": "server", "timestamp": "2021-08-03T12:00:33,517Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "my_cluster_name", "node.name": "node1", "message": "failed on parsing mappings on index creation [cloud-audit-aws-2021.08]", "cluster.uuid": "Q081W-QcSJK7J9N-nkH_1A", "node.id": "Gf8quOkhSi--sNJwRfx3fA" ,
"stacktrace": ["org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [_doc]: unknown parameter [properties] on mapper [error] of type [text]"

Can anyone help asap, getting this issue in the production environment. ?

2

I tried sending data to a whole new index, but still the same error !!

can anyone help? I am stuck :frowning_face:

3

It would help if you actually shared your configuration files.

To me it looks like perhaps you have a bad mapping or index template.

Also Where did you define the pipeline in the filebeat configs.

Also just an observation/suggestion is that I would suggest perhaps using the defaults first and understand how that works before customizing but that's just an observation.

4

Thanks for your response @stephenb !!

my logstash configuration file:

input
{
        beats 
	{
		port => "5044"
	}
}
output
{

	
	if [service][type] == "gsuite"
	{
		elasticsearch 
		{
		hosts => [ "ES_HOST:9200" ]
		index => "gsuite"
		ssl => true
		ssl_certificate_verification => false
		user => 'elastic'
		password => 'password'
		}
	}
	if [input][type] == "s3"
        {
                elasticsearch
                {
                hosts => [ "ES_HOST:9200" ]
                index => "cloud-audit-aws-%{+YYYY.MM}"
	            pipeline => "signals-aws-cloudtrail"
                ssl => true
                ssl_certificate_verification => false
                user => 'elastic'
                password => 'password'
                }
        }

	if [input][type] == "azure-eventhub"
        {
                elasticsearch
                {
                hosts => [ "ES_HOST:9200" ]
                index => "cloud-audit-azure-%{+YYYY.MM}"
	            pipeline => "signals-azure-activitylogs-pipeline"
                ssl => true
                ssl_certificate_verification => false
                user => 'elastic'
                password => 'password'
                }
        }


	stdout { codec => rubydebug }
}

here is my ingest_pipeline :

Following is the mapping from July's index

5

I defined pipeline in Logstash config.

6

Hi @ankitdevnalkar

With respect to the Mapping I did not want to see the mapping from an existing index I would want to see your new Template there could be an issue there that seems to be what the error indicates perhaps it is malformed.

The error indicates unknown parameter [properties] that there is a properties field in the wrong place

Also what happens if you just try to post an empty document to you new index does it work?

POST cloud-audit-aws-2021-09-test/_doc
{
}
7

Hey @stephenb

I just resolved the issue. I had a template having two index patterns cloud-audit-duo* and cloud-audit*. So the cloud-audit* was conflicting with cloud-audit-aws, I removed cloud-audit* and it worked. :slightly_smiling_face:

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.