ElasticSearch Index Mapping for Dynamic

Elastic Stack Elasticsearch
1

I have Nodejs Application and I want to send request and response to elastic, I have an issue with mapping, so basically my code looks like this:

static sendLogToElastic = (req, res, next) => {
    const self = this;
    const originalSend = res.send;

    res.send = async function (responseBody) {
      if (typeof responseBody === "object") {
        console.log(responseBody);
        if (req.body?.password) delete req.body.password;
        if (req.body?.passwordConfirmation)
          delete req.body.passwordConfirmation;

        self.esClient
          .index({
            index: process.env.ELASTIC_REQUEST_RESPONSE_INDEX,
            document: {
              user_id: req.user?.id || responseBody?.data?.user?.id,
              date: new Date(),
              request: {
                originalUrl: req.originalUrl,
                params: req.params,
                query: req.query,
                body: req.body,
                clientIp: req.headers["x-client-ip"],
                deviceId: req.headers["x-device-id"],
              },
              response: {
                body: responseBody,
              },
            },
          })
          .then((value) => {
            logger.info(JSON.stringify(value));
          })
          .catch((err) => {
            logger.error(
              `request: ${req.originalUrl}, sendLogToElastic error: ${err}`
            );
          });
      }
      originalSend.apply(res, arguments);
    };
    next();
  };

This sends all the requests and responses to the elastic but in the response object, the data field sometimes is an object or a number, or just a string. But Elastic doesn't accept when it's a number or string. I got the following error:
sendLogToElastic error: ResponseError: document_parsing_exception
Root causes:
document_parsing_exception: [1:213] object mapping for [response.body.data] tried to parse field [data] as object, but found a concrete value

2

Every field in Elasticsearch need to have a consistent mapping, so a field can not sometimes be an object and other times a string or number, as this as per the error you showed results in a mapping conflict. When you use dynamic mapping, the mapping for the field is defined the first time it is encountered and all subsequent documents need to adhere to the mapping.

If you want to index this you will either need to change the structure of the document or map the field as not indexed (which means you can not search on it or anything underneath it).

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.