# Empty record in ElasticSearch

**URL:** <https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481>\
**Category:** Elasticsearch\
**Created:** [August 15, 2018, 9:32am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481 "2018-08-15T09:32:15Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 9:32am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/1 "2018-08-15T09:32:15Z")

</div>

Hello,

I don't understand why my logstash config don't save data in ES. My data comes from an xml file and here is my pipeline conf :

> input {  
> beats {  
> port =\> "5044"  
> host =\> "localhost"  
> client\_inactivity\_timeout =\> 3600  
> }  
> }  
> filter  
> {  
> xml  
> {  
> source =\> "message"  
> remove\_namespaces =\> "true"  
> xpath =\> ["/ImportLogs/@LogResult", "logResult",  
> "/ImportLogs/@FileName", "fileName",  
> "/ImportLogs/@DateImport", "importDate",  
> "/ImportLogs/@ImportType", "importType",  
> "/ImportLogs/@ImportFormat", "importFormat",  
> "/ImportLogs/@username", "username"  
> #"/ImportLogs/LineLogs/linelog/@lineNumber", "lineNumber",  
> #"/ImportLogs/LineLogs/linelog/@lineState", "lineState",  
> # "/ImportLogs/LineLogs/linelog/log/message/text()", "logMessage"  
> ]  
> store\_xml =\> false  
> target =\> "message"  
> }  
> mutate {  
> replace =\> { "logResult" =\> "%{logResult[0]}" }  
> replace =\> { "fileName" =\> "%{fileName[0]}" }  
> replace =\> { "importType" =\> "%{importType[0]}" }  
> replace =\> { "username" =\> "%{username[0]}" }  
> replace =\> { "importDate" =\> "%{importDate[0]}" }  
> replace =\> { "importFormat" =\> "%{importFormat[0]}" }  
> }  
> }  
> output {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> index =\> "myWorkIndex"  
> }  
> stdout { codec =\> rubydebug }  
> }  
> My ES picture's
> 
> ![emptyDataOnES](https://us1.discourse-cdn.com/elastic/original/3X/3/4/34aabc2fbadcf08e87dfc5bf632a43699ab2fc4c.jpeg)  
> And then my logstash Output :  
> ![LogstashOutputCapture](https://us1.discourse-cdn.com/elastic/original/3X/a/f/af71a79b8001647149988a8b770bb377135eaf2d.jpeg)  
> Can you help me please?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 9:34am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/2 "2018-08-15T09:34:55Z")

</div>

> [@czome](#):
>
> index =\> "myWorkIndex"

Index names must be lower case. You should be able to see something about this in the Elasticsearch logs.

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 9:38am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/3 "2018-08-15T09:38:04Z")

</div>

Thank for your reactivity, but even if i don't specify index name ,no change. It will create default index (logstash-$date) but with no data.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 9:53am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/4 "2018-08-15T09:53:24Z")

</div>

What is the output of the [cat indices API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-indices.html)?

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 9:59am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/5 "2018-08-15T09:59:41Z")

</div>

![LogstashOutputCapture2](https://us1.discourse-cdn.com/elastic/original/3X/9/1/91b4d25e24720afee57a41ae8821efa5edf4ddeb.jpeg)

As you see my index was created, but when i request this index , i see no hits.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 10:03am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/6 "2018-08-15T10:03:48Z")

</div>

There is data in the index, so it looks like your query may be wrong. Try `q=response:200` instead.

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 10:06am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/7 "2018-08-15T10:06:54Z")

</div>

As i sended in this picture , you will see no "hits". no data in my hits collection :

 ![LogstashOutputCapture3](https://us1.discourse-cdn.com/elastic/original/3X/a/c/accf8985a1da7ad8cd353b4a0402ab296f6bb1d7.jpeg)

I'm sure that something is wrong in my filebeat config or logstash config.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 10:18am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/8 "2018-08-15T10:18:07Z")

</div>

Your query is still `q=response=200` which is not the example I gave.

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 10:27am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/9 "2018-08-15T10:27:23Z")

</div>

![LogstashOutputCapture4](https://us1.discourse-cdn.com/elastic/original/3X/6/8/68d93359351b2be6ea4b0e7b12453cff84c59ec9.jpeg)  
No change.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 10:40am UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/10 "2018-08-15T10:40:39Z")

</div>

Do you even have a field named `response` in your data? Can you show a full document from that index?

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 12:06pm UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/11 "2018-08-15T12:06:42Z")

</div>

No i haven't field "response", here is my origin xml document :

```
<?xml version="1.0" encoding="utf-8"?>
<ImportLogs LogResult="SUCCESS" ExecTime="00:00:02.424" FileName="filename.csv" ignored="17" valid="62" alert="0" error="0" DateImport="2018-06-13T09:03:24Z" ImportMode="InsertUpdate" user="Tutu Toto" username="tutu.qiqi@gmail.com" ImportType="Organisation" ImportFormat="Organisations">
	<GeneralLog />
	<LineLogs>
		<linelog lineNumber="11" lineState="Ignored">
			<log type="Warning">
				<message>L'individu "tutu franck" n'existe pas</message>
			</log>
		</linelog>
		<linelog lineNumber="14" lineState="Ignored">
			<log type="Warning">
				<message>L'individu "Juju.Huhu.23544.20382807" n'existe pas</message>
			</log>
		</linelog>
	</LineLogs>
</ImportLogs>
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 12:07pm UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/12 "2018-08-15T12:07:56Z")

</div>

That is what your query is searching for, so it is then not surprising it does not find any results.

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 3:48pm UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/13 "2018-08-15T15:48:03Z")

</div>

Sorry, can you explain your answer please ?  
In my logstash output console i have :

 ![logstashScreenShot](https://us1.discourse-cdn.com/elastic/original/3X/f/9/f9668c869d6b848691618d28f1aa8227d72628bf.jpeg)  
Why this output don't flush in ElasticSearch ?  
Here is my elasticSearch screen shot :  
 ![ESScreenShot](https://us1.discourse-cdn.com/elastic/original/3X/a/e/aeab673198e216a5f56ef5719fd7b9992583da32.jpeg)  
And my xml is :

```
<?xml version="1.0" encoding="utf-8"?>
<ImportLogs LogResult="SUCCESS" ExecTime="00:00:02.424" FileName="ORG.csv" ignored="17" valid="62" alert="0" error="0" DateImport="2018-06-13T09:03:24Z" ImportMode="InsertUpdate" user="Tutu Kiki" username="tutu.kiki@gmail.com" ImportType="Organisation" ImportFormat="Organisations">
	<GeneralLog />
	<LineLogs>
		<linelog lineNumber="11" lineState="Ignored">
			<log type="Warning">
				<message>L'individu "Tutu.Franck.31966.20381925" n'existe pas</message>
			</log>
		</linelog>
		<linelog lineNumber="14" lineState="Ignored">
			<log type="Warning">
				<message>L'individu "Laura.Beffa.23544.20382807" n'existe pas</message>
			</log>
		</linelog>
	</LineLogs>
</ImportLogs>

```

What's wrong in my logstash conf ? or filebeat.yml conf please ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 15, 2018, 4:40pm UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/14 "2018-08-15T16:40:31Z")

</div>

There is not necessarily anything wrong with your Logstash or Filebeat config. It is your query that is wrong as you are searching for something that does not match any documents. Try with a query that matches all documents in Elasticsearch: `q=*:*`

---

<div class="post-metadata">

**Author:** ![czome](https://avatars.discourse-cdn.com/v4/letter/c/e95f7d/32.png) [@czome](https://discuss.elastic.co/u/czome)\
**Post date:** [August 15, 2018, 5:04pm UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/15 "2018-08-15T17:04:06Z")

</div>

Very good, it was my problem. Sorry, i'm starting on ELK.  
Thank's for your help. Now i will try to display some graphic on kibana.  
see you soon

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 12, 2018, 5:04pm UTC](https://discuss.elastic.co/t/empty-record-in-elasticsearch/144481/16 "2018-09-12T17:04:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.