XML to Elasticsearch via Logstash

Elastic Stack Logstash
42

What does the event look like? Use a stdout { codec => rubydebug } output.

43

Hi wwalker,
Unfortunately I'm unable to apply the conversion (I get the same error so it never proceeds to that step). I think the way I'm referencing the variable is wrong - see post 39!

44

The output I've provided you with is actually whats produced when I have stdout { codec => rubydebug } set haha, so I'm a bit unsure of what exactly you're after?
Some of the data is pushed successfully to elasticsearch so it's displayed in the normal JSON-esque format in the output, however when there's an issue, the error msg I pasted in post 39 is what appears

45

so I'm a bit unsure of what exactly you're after?

I am obviously interested in seeing the event that ES rejects. What you posted earlier doesn't include the LISTED_ON field that it complains about.

Some of the data is pushed successfully to elasticsearch so it's displayed in the normal JSON-esque format in the output, however when there's an issue, the error msg I pasted in post 39 is what appears

So comment the elasticsearch output so you always get the raw event dumped to stdout. Until we can get the LISTED_ON field in shape the elasticsearch output is an irrelevant distraction.

46

Ah my bad! I didn't realise you called them events haha. It looks something like this:

<INDIVIDUAL>
    <DATA>1212121</DATA>
    <VERSION>1</VERSION>
    <FIRST_NAME>John Doe</FIRST_NAME>
    <LIST_TYPE>AAA</LIST_TYPE>
    <REFERENCE_NUMBER>111</REFERENCE_NUMBER>
    <LISTED_ON>2015-03-27-04:00</LISTED_ON>
    <COMMENTS1>Just some guy</COMMENTS1>
    <LIST_TYPE>
        <VALUE>List One</VALUE>
    </LIST_TYPE>
    <LAST_DAY_UPDATED>
        <VALUE/>
    </LAST_DAY_UPDATED>
    <NICKNAME>
        <QUALITY/>
        <VALUE/>
    </NICKNAME>
    <ADDRESS>
        <CITY>nyc</CITY>
        <STATE_PROVINCE>ny</STATE_PROVINCE>
        <COUNTRY>usa</COUNTRY>
    </ADDRESS>
    <DOB>
        <TYPE_OF_DATE>EXACT</TYPE_OF_DATE>
        <YEAR>1901</YEAR>
    </DOB>
    <POB>
        <COUNTRY>London</COUNTRY>
    </POB>
</INDIVIDUAL>
47

No! Come on! I asked for stdout { codec => rubydebug } output. Last chance now before I give up.

48

I'm not quite sure what you mean by that, could you give me an example? I thought what I provided in post 39 was the output?

49

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.