As the final mapping would have more than 1 type ERROR

Elastic Stack Elasticsearch
1

I had data going to ES from filebeats with no issues and then yesterday, I get tons of these errors for most of our rest-api services.

i looked at the filebeat debug output and see metadata type as doc. I looked at logstash in debug and I dont know were this other type is coming from. Has anybody else seent this with json documents?

[2018-01-14T20:59:36,323][DEBUG][o.e.a.b.TransportShardBulkAction] [application-2018.01.14][4] failed to execute bulk item (index) BulkShardRequest [[application-2018.01.14][4]] containing [index {[application-2018.01.14][doc][7P549mABJPWY86CHb3z9], source[{"request":{"agent":"java-sdk,0.1.0-dev.5709.uncommitted+sprint.d39eb72;Linux,3.10.0-514.16.1.el7.x86_64,amd64;,,;SBM,0.1.0-dev.5709.uncommitted+sprint.d39eb72","route":"/organization//pipeline//account//state/","method":"GET","size":"","resource":"/organization/xxxxxxxxxxx/pipeline/89xxxxx92-a4e8-/account/5-/state/featureExtractionState","addr":"xxxxxxx"},"offset":192733520,"log":"","level":"INFO","source":"/var/log/timeseries-service/rest-api/timeseries_info","logsource":"timeseries-api@rest-api","version":"LATEST","tags":["leveled","leveled2"],"logtype":"application","@timestamp":"2018-01-14T20:59:34.618Z","response":{"size":165110,"status":200},"beat":{"name":"pre-prod-20171229-timeseries-api-02","hostname":"pre-prod-20171229-timeseries-api-02","version":"6.0.0"},"@version":"1","time":{"total":77},"index_prefix":"application","levelcode":3}]}]
java.lang.IllegalArgumentException: Rejecting mapping update to [application-2018.01.14] as the final mapping would have more than 1 type: [doc, bebf52d2-7d9d-4b4b-a31b-3779dc9d1c8a]

my filebeat file is as follows,

filebeat.prospectors:

  • input_type: log
    paths:
    • /var/log/java-gateway/java-gateway.log
      fields:
      logsource: java-gateway
      logtype: application
      fields_under_root: True
      json.keys_under_root: True
      json.message_key: log
      json.overwrite_keys: True
  • input_type: log
    paths:
    • /var/log/timeseries-service/rest-api/*
      fields:
      logsource: timeseries-api@rest-api
      logtype: application
      version: LATEST
      fields_under_root: True
      json.keys_under_root: True
      json.message_key: log
      json.overwrite_keys: True
      output.redis:

    Array of hosts to connect to.

    hosts: ["lxxxxx:6379"]
    ssl.enabled: false
2

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

It sounds like you are trying to send a weird doc type or that you already created it: bebf52d2-7d9d-4b4b-a31b-3779dc9d1c8a.

Run a GET application-2018.01.14/_mapping and see what you have yet.

May be a wrong index template.

3

thanks David,

sorry about the code formatting...

i did a Get on _mapping on that index and you are correct i am getting an incorrect mapping but i'm trying to find out how. I dont set that in my template, Bascially I took the filebeat template and cloned it a few times one for services in our infrastructure and one for all the applications logging into our infrastructure. Looks like this particular application is sending json logs and I dont see how that is getting applied at the logstash layer or at the filebeat layer. It was also sending logs before and just stopped working.

{
  "application-2018.01.14": {
    "mappings": {
      "bebf52d2-7d9d-4b4b-a31b-3779dc9d1c8a": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
4

I can't tell you what. But for sure something is creating this mapping.

You can see it in elasticsearch logs. You should see either the index creation or the update mapping message. Then you'll get a date and that might give you a clue.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.