# Lostash problem

**URL:** <https://discuss.elastic.co/t/lostash-problem/112555>\
**Category:** Logstash\
**Created:** [December 20, 2017, 6:28am UTC](https://discuss.elastic.co/t/lostash-problem/112555 "2017-12-20T06:28:13Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![narasiman1986](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@narasiman1986](https://discuss.elastic.co/u/narasiman1986)\
**Post date:** [December 20, 2017, 6:28am UTC](https://discuss.elastic.co/t/lostash-problem/112555/1 "2017-12-20T06:28:13Z")

</div>

My logstash file code as follows

```
input {

jdbc {

jdbc_driver_library => "D:\mysql-connector-java-5.1.44\mysql-connector-java-5.1.44\mysql-connector-java-5.1.44-bin.jar"

jdbc_driver_class => "com.mysql.jdbc.Driver"

 
jdbc_connection_string => "jdbc:mysql://localhost:3306/sample"

  
jdbc_user => "root"

jdbc_password => "root"

jdbc_fetch_size => 10000

    schedule => "* * * * *"
    statement => "SELECT * from sample"

codec => "json"

  }

}

output {

elasticsearch { 
hosts => ["localhost:9200"] 
manage_template => false
index => "clinical"
document_id => "%{RRH_MR_NUM}" }
stdout { codec => rubydebug }
}

```

when i see the output in kibana as follows

cpa\_addr\_2: - cpa\_addr\_area: - mcs\_crt\_uid:1104158 cpa\_addr\_1:GALSI- rrh\_mr\_num:3439829 cpa\_pin\_code: - rrh\_pat\_sex:Male mcs\_crt\_dt:2015-04-18T  
mcs\_case\_summary: MRD No: 3439829 Patient Name: MR. AKKASH S K Gender: Male Age: 59 Year(s) ReportDate: 2015-04-18 Examined By: MOUPIYA DAS Consultant: External File Uploaded - Patient InfoInvestigation Report - FUN

when i click the refresh button in top right side of kibana discover record refreshing in the above record but count (hits) shows 1 only.

when i click refresh button count shows one only. but records changing.

is there any problem in logstash?

please let me know

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 20, 2017, 6:32am UTC](https://discuss.elastic.co/t/lostash-problem/112555/2 "2017-12-20T06:32:02Z")

</div>

> [@narasiman1986](#):
>
> RRH\_MR\_NUM

Does that field map to a column in the database table?

---

<div class="post-metadata">

**Author:** ![narasiman1986](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@narasiman1986](https://discuss.elastic.co/u/narasiman1986)\
**Post date:** [December 20, 2017, 6:44am UTC](https://discuss.elastic.co/t/lostash-problem/112555/3 "2017-12-20T06:44:41Z")

</div>

in My sql table query as follows

select \* from sample

gives output as follows

RRH\_MR\_NUM cpa\_addr cpa\_addr\_area mcs\_crt\_uid cpa\_addr\_1 cpa\_pin\_code rrh\_pat\_sex mcs\_Crt\_dt mcs\_case\_summary

1104158 - - 1104158 GALSI- 3439829 - male 2015-04-18T MRD No: 3439829 Patient Name: MR. AKKASH S K Gender: Male Age: 59 Year(s) ReportDate: 2015-04-18 Examined By: MOUPIYA DAS Consultant: External File Uploaded - Patient InfoInvestigation Report - FUN

in kibana screen shows field as follows

RRH\_MR\_NUM  
CPA\_ADDR  
CPA\_ADDR\_AREA  
MCS\_CRT\_UID  
CPA\_ADDR\_1  
CPA\_Pin\_CODE  
RRH\_PAT\_SEX  
MCS\_CRT\_DT  
MCS\_CASE\_SUMMARY

yes the field mapped to column in the database.

Any information required to you.

imam importing csv file. from that csv i am displaying into mysql table. that table i mentioned above sample.

already i mentioned from the elastic search and logstash display those table name sample details in to kibana,

---

<div class="post-metadata">

**Author:** ![narasiman1986](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@narasiman1986](https://discuss.elastic.co/u/narasiman1986)\
**Post date:** [December 20, 2017, 8:51am UTC](https://discuss.elastic.co/t/lostash-problem/112555/4 "2017-12-20T08:51:11Z")

</div>

yes that field is mapped into column of database table

do you want any more information.

is there any mistake in my logstash confi file code for removing duplicates

output {

elasticsearch {  
hosts =\> ["localhost:9200"]  
manage\_template =\> false  
index =\> "clinical"  
document\_id =\> "%{RRH\_MR\_NUM}" }  
stdout { codec =\> rubydebug }  
}  
i

i sent the screen shot as follows

 ![screenshot](https://us1.discourse-cdn.com/elastic/original/3X/e/d/ed06864494b451be5c8b6fd05956fe86af36b8e8.png)

in that screen shot RRH\_MR\_NUM column is mapped.

please let me know how to solve this avoiding duplicates in logstash

or in kibana we can avoid duplicates or in the elastic search we can avoid duplicates

---

<div class="post-metadata">

**Author:** ![narasiman1986](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@narasiman1986](https://discuss.elastic.co/u/narasiman1986)\
**Post date:** [December 20, 2017, 9:29am UTC](https://discuss.elastic.co/t/lostash-problem/112555/5 "2017-12-20T09:29:00Z")

</div>

any one can you please tell me how to solve that error

please let me know

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [December 27, 2017, 1:07am UTC](https://discuss.elastic.co/t/lostash-problem/112555/6 "2017-12-27T01:07:46Z")

</div>

just a guess: maybe it's case sensitive? have you tried using `${rrh_mr_num}` instead?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 24, 2018, 1:07am UTC](https://discuss.elastic.co/t/lostash-problem/112555/7 "2018-01-24T01:07:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.