# Data in index not populating in the data view or kibana

**URL:** <https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366>\
**Category:** Elastic Search\
**Created:** [July 18, 2024, 4:51pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366 "2024-07-18T16:51:34Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zachary\_Joyner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_joyner/32/135770_2.png) [@Zachary\_Joyner](https://discuss.elastic.co/u/Zachary_Joyner)\
**Post date:** [July 18, 2024, 4:51pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/1 "2024-07-18T16:51:34Z")

</div>

Hello,  
I have a pipeline to update my documents with parsed url information via the uri\_parts processor. However when I look at the datastream or in kibana those addition fields are absent. I checked the data in the backing indexes and it is present. I tried updating the index template in the datastream but that doesn't seem to help and I have mappings for the fields.

Any ideas on how to get that data to show up?

---

<div class="post-metadata">

**Author:** ![jessgarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessgarson/32/129841_2.png) [@jessgarson](https://discuss.elastic.co/u/jessgarson)\
**Post date:** [July 18, 2024, 5:09pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/2 "2024-07-18T17:09:57Z")

</div>

Thanks so much for reaching out, @Zachary_Joyner. Do you have a code sample you can provide? Can you also give some further details about how you set up your data view or dashboards?

Here are some similar posts that may be helpful, too:

- [Issues with viewing index data in kibana](https://discuss.elastic.co/t/issues-with-viewing-index-data-in-kibana/279452)
- [Kibana not updating index in Discover - #7 by zaeemmasood](https://discuss.elastic.co/t/kibana-not-updating-index-in-discover/330885/7)
- [Data not appearing in dashboards - #2 by jsanz](https://discuss.elastic.co/t/data-not-appearing-in-dashboards/317582/2)

---

<div class="post-metadata">

**Author:** ![Zachary\_Joyner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_joyner/32/135770_2.png) [@Zachary\_Joyner](https://discuss.elastic.co/u/Zachary_Joyner)\
**Post date:** [July 18, 2024, 5:24pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/3 "2024-07-18T17:24:49Z")

</div>

This is my pipeline.

```auto
{
  "description": "Updates the data for use with watchdog.",
  "processors": [
    {
     "json": {
      "field": "message",
      "add_to_root": true
    },
      "uri_parts": {
        "field": "httpRequest.requestUrl",
        "target_field": "httpRequest.url",
        "keep_original": true,
        "remove_if_successful": false,
        "ignore_missing": true
      },
    "gsub": {
      "field": "httpRequest.latency",
      "pattern": "s$",
      "replacement": ""
    }
  }
  ]
}

```

I found an issue with my index management. I didn't update the pipeline there, which is now fixed with:

```auto
{
  "index": {
    "default_pipeline": "logs@default-pipeline",
    "final_pipeline": "watchdog_macro_pipeline"
  }
}

```

However I am seeing this new error in my ingress which is the elastic provided dataflow job:

```auto
java.io.IOException: Error writing to Elasticsearch, some elements could not be inserted: Document id tBzMxpABx8HgWJ22SKrq: [1:4025] failed to parse field [httpRequest.url] of type [text] in document with id 'tBzMxpABx8HgWJ22SKrq'. Preview of field's value: '{path=/api/sites/subdomains/gingafc, original=https://api.leagueapps.io/api/sites/subdomains/gingafc, scheme=https, domain=api.leagueapps.io}' (document_parsing_exception) Caused by: Can't get text on a START_OBJECT at 1:3872 

```

I think the issue is with the indexes that back the data stream but I am still digging there.

---

<div class="post-metadata">

**Author:** ![jessgarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessgarson/32/129841_2.png) [@jessgarson](https://discuss.elastic.co/u/jessgarson)\
**Post date:** [July 18, 2024, 5:33pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/4 "2024-07-18T17:33:46Z")

</div>

Thanks for following up, @Zachary_Joyner. Typically this error occurs when new requests were disreaged because the queue was full.

Here are some further resources that could be helpful here:

- [java - Error writing to Elasticsearch, some elements could not be inserted from dataflow - Stack Overflow](https://stackoverflow.com/questions/59588598/error-writing-to-elasticsearch-some-elements-could-not-be-inserted-from-dataflo)
- [Ingest data with GCP Dataflow - #6 by gustafkisi](https://discuss.elastic.co/t/ingest-data-with-gcp-dataflow/294046/6)

---

<div class="post-metadata">

**Author:** ![Zachary\_Joyner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_joyner/32/135770_2.png) [@Zachary\_Joyner](https://discuss.elastic.co/u/Zachary_Joyner)\
**Post date:** [July 18, 2024, 7:04pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/5 "2024-07-18T19:04:07Z")

</div>

I'm not sure this is a queue issue. Looking at the timing, this started when I created the new pipeline.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/d/ed24047671a20c56b58171800ad60453887cbc1d.png)  
I also see this in the error message:

```auto
{path=/api/sites/subdomains/cityside208, original=https://api.leagueapps.io/api/sites/subdomains/cityside208, scheme=https, domain=api.leagueapps.io}

```

Which is the text I would expect to get added to the document.  
I ran some tests with an index not managed by the data stream. And the pipeline was working as expected.

---

<div class="post-metadata">

**Author:** ![Zachary\_Joyner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_joyner/32/135770_2.png) [@Zachary\_Joyner](https://discuss.elastic.co/u/Zachary_Joyner)\
**Post date:** [July 18, 2024, 7:05pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/6 "2024-07-18T19:05:44Z")

</div>

Also in the performance tab I am not seeing pegged CPUs or memory.

---

<div class="post-metadata">

**Author:** ![Zachary\_Joyner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_joyner/32/135770_2.png) [@Zachary\_Joyner](https://discuss.elastic.co/u/Zachary_Joyner)\
**Post date:** [July 18, 2024, 8:37pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/7 "2024-07-18T20:37:48Z")

</div>

The issue was with the mapping I had an object field set as a text field in the index template.

---

<div class="post-metadata">

**Author:** ![jessgarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessgarson/32/129841_2.png) [@jessgarson](https://discuss.elastic.co/u/jessgarson)\
**Post date:** [July 19, 2024, 6:14pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/8 "2024-07-19T18:14:34Z")

</div>

Thanks so much for your follow up here and sharing your solution, @Zachary_Joyner.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 16, 2024, 6:15pm UTC](https://discuss.elastic.co/t/data-in-index-not-populating-in-the-data-view-or-kibana/363366/9 "2024-08-16T18:15:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.