Xml PARSING IN LOGSTASH & filebeat

[Puneeth_S_B_Gowda1 (Puneeth S B Gowda)]

Hi

I got new task in ELK and i never worked on ELK

My task is take xml data from mysql and store it in notepad and forward it through filebeat and pars it from logstash

my xml i need BedType as saparate value in the column will be D1K column from this in Kibana

Please help me with logstash and filebeat configuration

I saved this xml in notepad

<HotelCompleteAvailability_14>
<HotelCompleteAvailability>
<HotelType>
<HtlTypeInd>I</HtlTypeInd>
</HotelType>
<HotelPropHeader>
<City>LBA</City>
<HtlRefPt/>
<RefPtPostalInd/>
<State/>
<MileKmInd/>
<CityCodeRefMatch/>
</HotelPropHeader>
<HtlAvailHeader>
<CatInd/>
<StartDt>20200916</StartDt>
<EndDt>20200917</EndDt>
<NumNights>1</NumNights>
<NumPersons>1</NumPersons>
<PrefPropCnt/>
<NeutralAvailInd/>
</HtlAvailHeader>
<HotelPropertyRecord>
<Vnd>DV</Vnd>
<RMID>89337</RMID>
<PropName>VILLAGE LEEDS SOUTH</PropName>
<ShortAddr/>
<Dist>0</Dist>
<Dir/>
<Locn/>
<Transportation/>
<InsideLinkInd>E</InsideLinkInd>
<AAARating/>
<Advertiser/>
<NTMRating/>
<NetTransCommissionInd/>
<SponsoredSearchInd/>
<TokenValue/>
<LineNum>0</LineNum>
<TripAdvisorID/>
<TripAdvisorScore/>
<TripAdvisorNumOfReview/>
</HotelPropertyRecord>
<HotelVendorMarketing>
<VMsg/>
</HotelVendorMarketing>
<HotelRate>
<StoredCrncy>GBP</StoredCrncy>
<StoredNumDec>2</StoredNumDec>
<MoreRatesInd>N</MoreRatesInd>
<IARatesOnlyInd/>
<AvailNeedInd/>
<TaxInd/>
<RateChgInd>N</RateChgInd>
<RoomByReq>N</RoomByReq>
<RateCatInd/>
<DispRate>DBL</DispRate>
<VStoredRateAmt>9810</VStoredRateAmt>
<BIC>DBLDTI</BIC>
<RateGuarInd/>
<PricingType/>
<TotAmt/>
<IndexNum>0</IndexNum>
</HotelRate>
<HotelCorporateDiscount>
<CorporateDiscCodeAry>
<CorporateDiscCodeItem>
<CorporateDiscCode>DIRECTTRAVEL</CorporateDiscCode>
<Delimiter>@</Delimiter>
<MultiLevelRateCode>DTI</MultiLevelRateCode>
</CorporateDiscCodeItem>
</CorporateDiscCodeAry>
<IndexNum>0</IndexNum>
</HotelCorporateDiscount>
<HotelPricing>
<IndexNum>0</IndexNum>
<PricingAry>
<PricingItem>
<Type>PN</Type>
<Amt>9810</Amt>
<NightCnt>01</NightCnt>
</PricingItem>
<PricingItem>
<Type>ST</Type>
<Amt>9810</Amt>
<NightCnt/>
</PricingItem>
<PricingItem>
<Type>AT</Type>
<Amt>9810</Amt>
<NightCnt/>
</PricingItem>
</PricingAry>
</HotelPricing>
<HtlIndexedRateDesc>
<IndexNum>0</IndexNum>
<DescriptionAry>
<Description>DIRECT TRAVEL INC ROOM ONLY</Description>
<Description>DOUBLE ROOM SEALY KING BED FREE WIFI</Description>
<Description>TEA AND COFFEE MAKING FACILITIES SAT TV</Description>
</DescriptionAry>
</HtlIndexedRateDesc>
<HotelSurchargeTax>
<IndexNum>0</IndexNum>
<Total>000000000</Total>
<TotTax/>
<TotSurcharges/>
<TaxbreakoutAry>
<TaxbreakoutItem>
<Amt/>
<Description/>
</TaxbreakoutItem>
</TaxbreakoutAry>
</HotelSurchargeTax>
<HotelMatch>
<IndexNum>0</IndexNum>
<RateCatMatch/>
<RoomCntMatch>Y</RoomCntMatch>
<AdultCntMatch>Y</AdultCntMatch>
<ChildCntMatch/>
<BedMatch>Y</BedMatch>
<AdultRollawayMatch/>
<ChildRollawaysMatch/>
<CribMatch/>
<RateCat/>
<RoomCnt>1</RoomCnt>
<AdultCnt>1</AdultCnt>
<ChildCnt/>
<BedType>D1K</BedType>

you can call me on 9632096056

You can email me on puneeth.cpg@gmail.com

Regards,

Puneeth

[Puneeth_S_B_Gowda1 (Puneeth S B Gowda)]

indent preformatted text by 4 spaces
With this logstash configuration able to parse required value as column.

Sample Logstash configuration for creating a simple

Beats -> Logstash -> Elasticsearch pipeline.

input {
beats {
port => 5044
}
}

filter {
csv {
separator => "@$@"
columns => ["date","est_cd","hotelhubcode","sessionid","searchid","requestid","request","response","parentmethodname","starttime","endtime","duration","exception","notes","channelpropertyid"]
}

xml {
store_xml => "false"
source => "response"
remove_namespaces => "true"

                     xpath =>[
                            "/HotelCompleteAvailability_14/HotelCompleteAvailability/HotelPreformatted textMatch/BedType/text()","BedType",

"//HotelCompleteAvailability_14/HotelCompleteAvailability/HtlIndexedRateDesc[1]/DescriptionAry/Description[1]/text()","Description"
]
}

}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "4jun9"
#user => "elastic"
#password => "changeme"
}
}

indent preformatted text by 4 spaces