# How to get the sum in time of values in Lens

**URL:** <https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735>\
**Category:** Kibana\
**Created:** [July 27, 2022, 10:06am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735 "2022-07-27T10:06:06Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 27, 2022, 10:06am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/1 "2022-07-27T10:06:06Z")

</div>

I collect logs in Elasticsearch and keep track of certain values in the logs with fields.  
Since the values are numeric, I would like to calculate the sum over a certain time interval and see the transition.

However, when I select the field in the `Vertical axis` of the Lens, I cannot select `Sum`.  
I can only select `Differences`, `Moving average` and `Unique count`.  
Why is this?

Is there a problem with the field settings in Elasticsearch?  
Any advice would be appreciated. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![ghudgins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ghudgins/32/138532_2.png) [@ghudgins](https://discuss.elastic.co/u/ghudgins)\
**Post date:** [July 27, 2022, 12:09pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/2 "2022-07-27T12:09:14Z")

</div>

Can you send me an example of the field's mapping? It should allow a sum...what happens when you click "sum" and you select this field?

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 27, 2022, 1:07pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/3 "2022-07-27T13:07:27Z")

</div>

Thanks for answering my question.

The field is assigned an integer value. Specifically, it is the amount of access log transfer.

If I select `Sum` in `Select a function`, `Select a field` can only select `log.offset`.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 27, 2022, 1:32pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/4 "2022-07-27T13:32:42Z")

</div>

> [@its-ogawa](#):
>
> The field is assigned an integer value.

Please show the mapping and tell us which field you are trying to sum, Lens is acting like is type `keyword`

The common reason for the behavior that you're seeing is an incorrect data mapping.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 27, 2022, 1:50pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/5 "2022-07-27T13:50:23Z")

</div>

> Please show the mapping and tell us which field you are trying to sum,

How can we get this across to you?

I will send you the screen I am looking at.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/e/2e67d4382e02f1ad5bb53754b742b4365b231bce.png)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 27, 2022, 2:07pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/6 "2022-07-27T14:07:31Z")

</div>

Hi @its-ogawa

So first yes this is your exact problem you `bytes` field is not a numeric type.  
 ![Screen Shot 2022-07-27 at 7.00.25 AM](https://us1.discourse-cdn.com/elastic/original/3X/4/8/48857ccb871752dd9cbadd7c566ecf682d516ffa.png)

So first question how did you import the data? and it looks like perhaps it is apache or ngnix logs? What kind of data is it?

Short story you need to define the mapping / data types to properly use your data...

It looks like you important data without defining a [mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html) (think schema) so a default mapping was created automatically and the type is not correct.

Did you create a [template](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html) when you imported the data (that will specify the mapping) for all matching indices

You can get the mappings by going to  
Kibana -\> Dev Tools

`GET my-index-name`

Answer a few questions and perhaps we can help.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 27, 2022, 2:48pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/7 "2022-07-27T14:48:26Z")

</div>

I am getting the following data. You are right, it is an nginx log.

```auto
XXX.XXX.XXX.XXX.XXX - - [27/Jul/2022:23:23:06 +0900] "GET /favicon.ico HTTP/1.0" 200 417 "https://XXXX/sp/item.php?pn=20994&fbclid=PAAaYp1mNWdhHcfJxyDJn-lFuhNqPPBK0BEGXusOa3sGNH3p5vKH76nI-S9J0" "Mozilla/5.0 (Linux; Android 11; SO-41A Build/59.1.B.1.226; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/103.0.5060.129 Mobile Safari/537.36 Instagram 244.1.0.19.110 Android (30/11; 420dpi; 1080x2394; Sony/docomo; SO-41A; SO-41A; qcom; ja_JP; 384108453)"
XXX.XXX.XXX.XXX.XXX - - [27/Jul/2022:23:23:02 +0900] "GET /api/mailmag_open_notify.php?mgid=6210504 HTTP/1.0" 200 - "-" "Outlook-Express/7.0 (MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0E; .NET4.0C; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729; YTB730; McAfee; MALNJS; TmstmpExt)"
... snip ...

```

I imported this log into Elasticsearch via Filebeat and Logstash.  
I have the following filter settings in the Logstash conf file.

```auto
grok {
    match => {
      #"message" => '%{IPV4:ip-address}\s-\s-\s\s[%{DATA:timestamp}\]\s"%{DATA}"\s%{DATA}\s%{DATA}\s"%{DATA:url}"\s"%{DATA}"'
      "message" => '%{IP:ip-address}\s%{DATA:username}\s%{DATA:user}\s\s[%{DATA:timestamp}\]\s"%{DATA:action}\s%{DATA:path}\s%{DATA:protocol}"\s%{DATA:status}\s%{DATA:bytes}\s"%{DATA:url}"\s"%{DATA:user_agent}"'
    }
}

```

Also, before indexing, the following index template is created.

```auto
$ curl -X PUT "localhost:9200/_index_template/its_index_template?pretty" -H 'Content-Type: application/json' -d'
{
  "index_patterns" : [... snip ... "kin-*" ... snip ...],
  "priority" : 1,
  "template": {
    "settings" : {
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  }
}
'

```

I then ran the command in Dev Tools.

```auto
... snip ...
        "bytes" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
... snip ...

```

I think I finally understand what mapping is here.

`"type" : "text",`!!!

I want this place to be numeric.  
How can I get him to recognize it as a number?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 27, 2022, 4:15pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/8 "2022-07-27T16:15:56Z")

</div>

> [@its-ogawa](#):
>
> ```auto
> "bytes" : {
> "type" : "text",
> "fields" : {
> "keyword" : {
> "type" : "keyword",
> "ignore_above" : 256
> }
> }
> 
> ```

In the template create / add a mapping section with all the [correct mapping types.](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html)

Create the template with the correct mapping types already in..

```auto
 "bytes" : {
          "type" : "integer",
          }

```

BTW there is an nginx filebeat module that does all this for you... you would not need to use logstash just Filebeat -\> Elasticsearch and the data would all be nicely parse and typed and there are built in dashboards etc.

> **[Nginx module | Filebeat Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-nginx.html)**

Quicks start just do the ngnix module which the quickstart show how to do

> **[Filebeat quick start: installation and configuration | Filebeat Reference...](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html)**

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 28, 2022, 1:46am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/9 "2022-07-28T01:46:20Z")

</div>

Ah, so you need to set the `type` in the index template.  
That would mean I need to have a template for each type of log, right?  
Since I already collect 10 different logs, does that mean I need to prepare 10 different index templates?

> [Nginx module | Filebeat Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-nginx.html)  
> So the information you gave me will be of great help to me.

One thing I am concerned about is what will happen to the index names.  
I am using nginx with 10 different services.  
Can I name them for each service?

Currently, I set `index_name` as the name in Filebeat and explicitly specify the name of the index in Logstash.

For example,

```auto
- type: log
  enabled: true
  paths:
    - /var/log/www/httpd-access_log
  fields: /var/log/www/httpd-access_log
    index_name: kin-www-access

```

```auto
if([fields][index_name]) {
  elasticsearch {
    hosts => ["localhost"].
    index => "%{[fields][index_name]}-%{+YYYYY.MM.dd}"
  }
}

```

Can I do something similar to this?

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 29, 2022, 1:33am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/10 "2022-07-29T01:33:05Z")

</div>

> ```auto
> "bytes" : {
> "type" : "integer",
> }
> 
> ```

Following the above advice, I created the following index template.

```auto
curl -X PUT "localhost:9200/_template/httpd-access-template?pretty" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["*-www-access-*", "*-httpd-access-*"],
  "mappings": {
    "_source": {
      "enabled": false
    },
    "properties": {
      "bytes": {
        "type": "integer"
      }
    }
  }
}
'

```

Then I get an error that timestamp is not found and it is not displayed by Kibana.

I just want to change the type of a given property.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 29, 2022, 3:10am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/11 "2022-07-29T03:10:09Z")

</div>

You need to add all the fields not just that one...with the correct types etc

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 29, 2022, 4:07am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/12 "2022-07-29T04:07:32Z")

</div>

Eh. That's a hassle...

I tried to get all the types and set them again.

```auto
curl -X PUT "localhost:9200/_template/httpd-access-template?pretty" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["*-www-access-*", "*-httpd-access-*"],
  "mappings" : {
    "properties" : {
      "@timestamp" : {
        "type" : "date"
      },
      "@version" : {
        "type" : "text"
      },
      "action" : {
        "type" : "text"
      },
      "agent" : {
        "properties" : {
          "ephemeral_id" : {
            "type" : "text"
          },
          "hostname" : {
            "type" : "text"
          },
          "id" : {
            "type" : "text"
          },
          "name" : {
            "type" : "text"
          },
          "type" : {
            "type" : "text"
          },
          "version" : {
            "type" : "text"
          }
        }
      },
      "bytes" : {
        "type" : "integer"
      },
      "ecs" : {
        "properties" : {
          "version" : {
            "type" : "text"
          }
        }
      },
      "fields" : {
        "properties" : {
          "index_name" : {
            "type" : "text"
          }
        }
      },
      "host" : {
        "properties" : {
          "architecture" : {
            "type" : "text"
          },
          "containerized" : {
            "type" : "boolean"
          },
          "hostname" : {
            "type" : "text"
          },
          "ip" : {
            "type" : "text"
          },
          "mac" : {
            "type" : "text"
          },
          "name" : {
            "type" : "text"
          },
          "os" : {
            "properties" : {
              "codename" : {
                "type" : "text"
              },
              "family" : {
                "type" : "text"
              },
              "kernel" : {
                "type" : "text"
              },
              "name" : {
                "type" : "text"
              },
              "platform" : {
                "type" : "text"
              },
              "type" : {
                "type" : "text"
              },
              "version" : {
                "type" : "text"
              }
            }
          }
        }
      },
      "input" : {
        "properties" : {
          "type" : {
            "type" : "text"
          }
        }
      },
      "ip-address" : {
        "type" : "text"
      },
      "log" : {
        "properties" : {
          "file" : {
            "properties" : {
              "path" : {
                "type" : "text"
              }
            }
          },
          "offset" : {
            "type" : "long"
          }
        }
      },
      "message" : {
        "type" : "text"
      },
      "path" : {
        "type" : "text"
      },
      "protocol" : {
        "type" : "text"
      },
      "status" : {
        "type" : "text"
      },
      "tags" : {
        "type" : "text"
      },
      "timestamp" : {
        "type" : "text"
      },
      "url" : {
        "type" : "text"
      },
      "user" : {
        "type" : "text"
      },
      "user_agent" : {
        "type" : "text"
      },
      "username" : {
        "type" : "text"
      }
    }
  },
  "settings" : {
    "number_of_shards" : "1",
    "number_of_replicas" : "0"
  }
}'

```

However, the type is not changed. Why?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 29, 2022, 4:51am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/13 "2022-07-29T04:51:55Z")

</div>

> [@its-ogawa](#):
>
> However, the type is not changed. Why?

Not sure what that means... Have to tell us /show what you're seeing. Probably means the templates not being applied.

> [@its-ogawa](#):
>
> Eh. That's a hassle...

Hence why we make modules... That do a it all... Understand the desire to name all your indexes It's typically a bit overrated... You can put them all in the filebeat indices and then just put a tag at the source for the type and you'll always be able to filter with one simple filter You could argue. It's easier than changing the index pattern... Kind of the initial idea.

But I understand the desire but data into different index names... There are some good reasons to do that.

You could use the module of then copy the mappings into your own template..

BTW most of those `text` fields should be type `keyword`

Bottom line if you want to do it all into your own indexes then you need to create the templates which are the schema.

Could call the ngnix pipeline ...

Bottom line if you want to do your own indexes you need to create a template and a pipeline and an ILM policy if you want it to all be taken care of.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 29, 2022, 5:46am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/14 "2022-07-29T05:46:15Z")

</div>

Sorry. I will explain exactly.

I created the index template in the above way.  
And got the following result

```auto
curl -X GET "localhost:9200/_template/httpd-access-template?pretty"
{
  "httpd-access-template" : {
    "order" : 0,
    "index_patterns" : [
      "*-www-access-*",
      "*-httpd-access-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "0"
      }
    },
    "mappings" : {
      "properties" : {
  ... snip ...
        "bytes" : {
          "type" : "integer"
        },
  ... snip ...
      }
    }
  }
}

```

However, I found that the type of index created has not been changed.

```auto
curl -X GET "localhost:9200/kin-www-access-2022.07.29?pretty"
{
  "kin-www-access-2022.07.29" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
  ... snip ...
        "bytes" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
  ... snip ...
      }
    }
  }
}

```

Does it not change the type of the index already created?

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 29, 2022, 6:01am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/15 "2022-07-29T06:01:27Z")

</div>

If by creating an index template, it will be applied to indexes that are created in the future (after tomorrow), then that is fine.  
I would also like to change the type for indexes that have already been obtained.

For example, I executed the following command

```auto
curl -X PUT "localhost:9200/kin-www-access-2022.07.29/_mapping?pretty" -H 'Content-Type: application/json' -d'
{
  "properties" : {
    "@timestamp" : {
      "type" : "date"
    },
    "@version" : {
      "type" : "text"
    },
    "action" : {
      "type" : "text"
    },
    "agent" : {
      "properties" : {
        "ephemeral_id" : {
          "type" : "text"
        },
        "hostname" : {
          "type" : "text"
        },
        "id" : {
          "type" : "text"
        },
        "name" : {
          "type" : "text"
        },
        "type" : {
          "type" : "text"
        },
        "version" : {
          "type" : "text"
        }
      }
    },
    "bytes" : {
      "type" : "integer"
    },
    "ecs" : {
      "properties" : {
        "version" : {
          "type" : "text"
        }
      }
    },
    "fields" : {
      "properties" : {
        "index_name" : {
          "type" : "text"
        }
      }
    },
    "host" : {
      "properties" : {
        "architecture" : {
          "type" : "text"
        },
        "containerized" : {
          "type" : "boolean"
        },
        "hostname" : {
          "type" : "text"
        },
        "ip" : {
          "type" : "text"
        },
        "mac" : {
          "type" : "text"
        },
        "name" : {
          "type" : "text"
        },
        "os" : {
          "properties" : {
            "codename" : {
              "type" : "text"
            },
            "family" : {
              "type" : "text"
            },
            "kernel" : {
              "type" : "text"
            },
            "name" : {
              "type" : "text"
            },
            "platform" : {
              "type" : "text"
            },
            "type" : {
              "type" : "text"
            },
            "version" : {
              "type" : "text"
            }
          }
        }
      }
    },
    "input" : {
      "properties" : {
        "type" : {
          "type" : "text"
        }
      }
    },
    "ip-address" : {
      "type" : "text"
    },
    "log" : {
      "properties" : {
        "file" : {
          "properties" : {
            "path" : {
              "type" : "text"
            }
          }
        },
        "offset" : {
          "type" : "long"
        }
      }
    },
    "message" : {
      "type" : "text"
    },
    "path" : {
      "type" : "text"
    },
    "protocol" : {
      "type" : "text"
    },
    "status" : {
      "type" : "text"
    },
    "tags" : {
      "type" : "text"
    },
    "timestamp" : {
      "type" : "text"
    },
    "url" : {
      "type" : "text"
    },
    "user" : {
      "type" : "text"
    },
    "user_agent" : {
      "type" : "text"
    },
    "username" : {
      "type" : "text"
    }
  }
}'

```

However, it does not work.  
I get the following error

```auto
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "mapper [bytes] cannot be changed from type [text] to [integer]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "mapper [bytes] cannot be changed from type [text] to [integer]"
  },
  "status" : 400
}

```

Can I achieve what I want to do?

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 29, 2022, 10:54am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/16 "2022-07-29T10:54:15Z")

</div>

> [@stephenb](#):
>
> BTW there is an nginx filebeat module that does all this for you... you would not need to use logstash just Filebeat -\> Elasticsearch and the data would all be nicely parse and typed and there are built in dashboards etc.
> 
> [Nginx module | Filebeat Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-nginx.html)

Filebeat's nginx module feature looks very attractive.

However, the index name seems to default to a form like `filebeat-7.12.0-2022.07.29-000001`.

On nginx, I have `access.log` and `error.log`, but I can't distinguish between them, so I want to change the index name.  
How can I do this?

I found the following article but cannot set the index name properly

> **[Configure the Elasticsearch output | Filebeat Reference \[7.0\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.0/elasticsearch-output.html#indices-option-es)**

I have added the following statement

```auto
# vi /etc/filebeat/modules.d/nginx.yml
- module: nginx
   access:
     enabled: true
     var.paths: ["/var/log/nginx/access.log"]

```

```auto
# vi /etc/filebeat/filebeat.yml
... snip ...
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]
  indices:
    - index: "else-httpd-access-%{+yyyy.MM.dd}"
      when.equals:
        event.module: "nginx"
    - default: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
... snip ...

```

However, I get the following error.

```auto
# tail /var/log/filebeat/filebeat
... snip ...
2022-07-29T19:27:17.106+0900 ERROR instance/beat.go:971 Exiting: error initializing publisher: missing output.elasticsearch.indices.1.index
... snip ...

```

I would like to have index names such as `{my service name}-httpd-access-{date}`, `{my service name}-httpd-error-{date}` for each `access.log` and `error.log`, and for each outgoing server.

For example, kin-httpd-access-2022.07.29, kin-httpd-error-2022.07.29, els-httpd-access-2022.07.29, els-httpd-error-2022.07.29, etc ...

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 29, 2022, 4:19pm UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/17 "2022-07-29T16:19:37Z")

</div>

> [@its-ogawa](#):
>
> Does it not change the type of the index already created?

Only go forward index

> [@its-ogawa](#):
>
> I would also like to change the type for indexes that have already been obtained.

That can not be done.

> [@its-ogawa](#):
>
> However, the index name seems to default to a form like `filebeat-7.12.0-2022.07.29-000001`.
> 
> On nginx, I have `access.log` and `error.log`, but I can't distinguish between them, so I want to change the index name.  
> How can I do this?

This is actually how filebeat modules are designed to work.... by intention 🙂

I often ask.... What are you actually trying to accomplish with separating the index name?... AGAIN that is fine but often it does not actually do much other allowing the users to _ **see** _ 2 index names... it is generally not more efficient etc, no storage space, more to manage etc and I would argue unless you do a good job of naming etc could make analysis more challenging etc...

With the module Yes both the data sets goes into the same index (in 8.x same Data Stream) ... but every doc is tagged with the access / error so any and all queries, visualizations, data etc...etc can _always_ show one and the other or both etc. and since they are in the same index you can do aggregation

Example here are the dashboards etc using the modules... [here](https://demo.elastic.co/app/dashboards#/view/55a9e6e0-a29e-11e7-928f-5dbe6f6f5519-ecs?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-4h,mode:quick,to:now)))

> [@its-ogawa](#):
>
> so I want to change the index name.

To do this : This is what folks do for custom data, it is a very common approach.  
Create your own template (or copy / modify the filebeat template)  
Create your own pipeline / parsing (or copy / modify the ngnix one provided)  
Create or reuse and ILM policy  
Then set / use your own names etc..

This is what happens with using the module

 ![Screen Shot 2022-07-29 at 9.58.02 AM](https://us1.discourse-cdn.com/elastic/original/3X/e/2/e292f05b6ef528265e4d6e38b1a543f587074462.png)

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [August 1, 2022, 4:21am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/18 "2022-08-01T04:21:50Z")

</div>

> [@stephenb](#):
>
> I often ask.... What are you actually trying to accomplish with separating the index name?

Hmmm. That's because you are monitoring the services of multiple companies.  
For example, if you load-balance Company A's service with 5 servers and Company B's service with 20 servers, it would be possible to have a single index of the access logs from Company A's 5 servers (which is exactly what we are doing now).  
However, it is not very desirable to have the access logs of Company A's server and Company B's server mixed together.  
This is because although the purpose of monitoring access logs is the same, the timing we want to monitor is completely different between the two.

There are more or less the same number of servers to be load averaged, but the problem is that there are several access logs to be distinguished.

One way to do this is through what we call separate indexes.  
Is there a way to solve the above problem if the indexes are grouped?  
For example, putting a tag in the FIELD that distinguishes each service?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 1, 2022, 5:51am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/19 "2022-08-01T05:51:26Z")

</div>

Ahhh Multi-Tenancy 🙂

Short answer yes, you can add as many fields or tags to the filebeat data at the source by just adding them into the filebeat.yml using `add_fields` or `add_tags` processor. The more advanced users do this via automation when deploying filebeat... Even more sophisticated Use cloud metadata to tag and then use.

So you have many options on the table.

One end of the spectrum is everything into a single index with tags or identifiers for company and services etc. Then all separation happens at query time.

The other end of the spectrum is an index for each company and service. Which is totally valid.

I know it seems a bit hard When you first get started but setting up a template, an ILM policy and reusing the pipeline is really only a couple hours work.

Either these are fine. It really depends on what your Multi-Tenancy requirements are Perhaps it's some combination of the two.

Example: I had a use case with 2,500 customers... It was a big use case and complex and my solution was moderately complex as well.

My top 10 customers all got their own indices because they created a lot of volume and they were high touch and I needed to do charge back and all those wonderful things.

My next 500 customers went into a common index.

And then the last 2000 small customers went into an index.

This afforded me a nice balance of control flexibility versus complexity, etc.

Everything was tagged for customer and service along the way.

You have lots of choices... your solution will depend on what your requirements are.

Let us know your thoughts.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [August 1, 2022, 10:46am UTC](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/20 "2022-08-01T10:46:12Z")

</div>

I don't have that many clients, but the examples you show are very useful to me.

I would like to create two Dashboards for each company you demoed.

- [Filebeat Nginx Overview ECS](https://demo.elastic.co/app/dashboards#/view/55a9e6e0-a29e-11e7-928f-5dbe6f6f5519-ecs?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-4h,mode:quick,to:now)))
- [Filebeat Nginx Access and error logs ECS](https://demo.elastic.co/app/dashboards#/view/55a9e6e0-a29e-11e7-928f-5dbe6f6f5519-ecs?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-4h,mode:quick,to:now)))

For this purpose, we think it is better to create an index for each company rather than defining fields and tags.  
That is because I want to create one for each company in Discover as well as Dashboard.

I have two problems.

The first is that when using Filebeat and Logstash to define index names, I need to create a template with all property types mapped.  
This requires knowing all properties in advance. Also, when a property changes, you have to enumerate all the properties again. This is a hassle, and at the same time, it is not possible to change the type for an already created index.  
To top it off, in my environment, the type is still text in Kibana even though I changed the type of the template.

```auto
# curl -X GET "localhost:9200/_template/httpd-access-template?pretty"
{
  "httpd-access-template" : {
    "mappings" : {
      "properties" : {
        "bytes" : {
          "type" : "integer"
        },

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb88562fc79434ef485f80f77b897c0cae6474f6.png)

Second, when using Filebeat's nginx module, the index name is `filebeat-7.12.0`. This is the default index name, but I can't figure out how to change it.  
The method I have tried is the one shown in [#16](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735/16), but it doesn't seem to work.  
It would be very nice to be able to pre-determine the index name in Filebeat's nginx module.

Is there a solution for these?

[Next page](https://discuss.elastic.co/t/how-to-get-the-sum-in-time-of-values-in-lens/310735.md?page=2)