# Can not get any logs after create a datastream

**URL:** https://discuss.elastic.co/t/can-not-get-any-logs-after-create-a-datastream/291629
**Category:** Elasticsearch
**Tags:** ilm-index-lifecycle-management, datastreams
**Created:** [December 13, 2021, 10:51am UTC](https://discuss.elastic.co/t/can-not-get-any-logs-after-create-a-datastream/291629 "2021-12-13T10:51:34Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![bilgicsin](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@bilgicsin](https://discuss.elastic.co/u/bilgicsin)
#### Post date: [December 13, 2021, 10:51am UTC](https://discuss.elastic.co/t/can-not-get-any-logs-after-create-a-datastream/291629/1 "2021-12-13T10:51:34Z")

</div>

Hi Everyone!

I am trying to change my data to the datastream. Normally I can get logs, I can see indice first. After I create a datastream I could not get any logs even though logs keep coming. I followed this steps:

> **[Set up a data stream | Elasticsearch Guide \[7.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/set-up-a-data-stream.html)**

I created a data stream like below:

```auto
{
  "data_streams" : [
    {
      "name" : "git-demo-topic",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-git-demo-topic-2021.12.13-000001",
          "index_uuid" : "y9bC8QZoQgijDKfp1zKO6Q"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "git-demo-topic",
      "hidden" : false,
      "system" : false
    }
  ]
}

```

This is the template a created:

```auto
{
  "index_templates" : [
    {
      "name" : "git-demo-topic",
      "index_template" : {
        "index_patterns" : [
          "git-demo-topic*"
        ],
        "template" : {
          "mappings" : {
            "properties" : {
              "city" : {
                "type" : "text"
              },
              "log_level" : {
                "type" : "text"
              },
              "detail" : {
                "type" : "text"
              },
              "ts" : {
                "type" : "text"
              }
            }
          }
        },
        "composed_of" : [],
        "data_stream" : {
          "hidden" : false
        }
      }
    }
  ]
}

```

This is the indice that created automatically, but it does not get any documents:

```auto
{
  ".ds-git-demo-topic-2021.12.13-000001" : {
    "aliases" : { },
    "mappings" : {
      "_data_stream_timestamp" : {
        "enabled" : true
      },
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "city" : {
          "type" : "text"
        },
        "detail" : {
          "type" : "text"
        },
        "log_level" : {
          "type" : "text"
        },
        "ts" : {
          "type" : "text"
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_hot"
            }
          }
        },
        "hidden" : "true",
        "number_of_shards" : "1",
        "provided_name" : ".ds-git-demo-topic-2021.12.13-000001",
        "creation_date" : "1639392538811",
        "number_of_replicas" : "1",
        "uuid" : "-LOWaaddSyWXXwiXfVk6GA",
        "version" : {
          "created" : "7150299"
        }
      }
    },
    "data_stream" : "git-demo-topic"
  }
}

```

Thank you in advance!

---

<div class="post-metadata">

### Author: ![bilgicsin](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@bilgicsin](https://discuss.elastic.co/u/bilgicsin)
#### Post date: [December 13, 2021, 12:39pm UTC](https://discuss.elastic.co/t/can-not-get-any-logs-after-create-a-datastream/291629/2 "2021-12-13T12:39:11Z")

</div>

I found the problem. I had to add "action-\> create" line to my logstash out conf:

```auto
  stdout {  
        codec => json_lines  
    } 
  elasticsearch {
      hosts => ["elk:9200"]
      index => "git-demo-topic"
	  action => "create"
  }
}`

```

---

<div class="post-metadata">

### Author: ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)
#### Post date: [December 14, 2021, 8:48am UTC](https://discuss.elastic.co/t/can-not-get-any-logs-after-create-a-datastream/291629/3 "2021-12-14T08:48:11Z")

</div>

The bulk and index api will fail with a client errors if the operation type is not create.  
When using the bulk or index api directly this is immediately visible, but I guess this  
is not easily visible when other components are indexing into a data stream.

When indexing data into a data stream only append-only writes are allowed and  
the way this is enforced is by checking whether the operation type is create.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 11, 2022, 8:49am UTC](https://discuss.elastic.co/t/can-not-get-any-logs-after-create-a-datastream/291629/4 "2022-01-11T08:49:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
