Hi Team,
I am facing an error related with date in elasticsearch. Below is the detail information, can you please help advice what's the problem? I googled this issue and there is do similar issue, but tried many times with the possible solution, but all is failed and the error keeps occurring.
1. I have below CSV file which contains a date information "COMMIT_DATE".
GIT_ORG,GIT_REPOS,COMMIT_SHA1,COMMIT_AUTHOR,COMMIT_DATE,COMMIT_COMMENT,JIRA_KEY,JIRA_ISSUE_TYPE,JIRA_ISSUE_CATEGORY,JIRA_PRIORITY,JIRA_COMPONENTS,JIRA_DEFECT_TYPE,JIRA_TESTCASE_ID,JIRA_FIND_BY_AUTO bizx,idl-analytics-api,1be44f52f25f6b540f284eb17e8cee5838826cb9,ssheriff,3/30/2017,RPT-7954: Cross domain reports are failing with Birt Exception,RPT-7954,Defect,Internal Issue,2-High,|Report Builder|,Regression,ENT200002079CrossDomainReport,Auto bizx,au-recruiting,d88e03469576f30079c366850fc952864815fb65,Kaderjan Ilghar,1/24/2018,fixing SML-710\, SML-714\,SML-708\,SML-709,,,,,,,, bizx,au-V4,f54abe7a0516407a8f6e2334ce8b1a1c229dd6aa,wwang,9/19/2016,TFT- DTCE new common component change ( Change FeatureEnum type),,,,,,,, bizx,au-V4,da5ab4deecd598e514bc3848d1dd0204643ccc0a,I848428,3/1/2017,PLT-56858: Remove the \field-permissions\ level message key for field-permissions.,PLT-56858,Defect,Internal Issue,3-Medium,|Z. DO NOT USE Permission|,New,,No bizx,au-V4,d08b994151f39412bea199ddcf45a3aa6d45ba1a,hkumar,11/19/2014,JIRA Defect Fixes;|PAE-1335 - Groups with same name as existing in manage employee groups can be shared by another user.|Crucible Code review Link : http://crucible.successfactors.com/cru/PAE-613|Reviewers: Vikram S,,,,,,,, bizx,au-homepage3,a82db7f431bf5083a4bd4b7885d8c0122aa80d70,I303180,9/22/2016,DESC:UI-7047 B1611HP3:Tile Name is changing automatically while creating a tour|JIRA:http://jira.successfactors.com/browse/UI-7047|REVIEW:http://crucible.successfactors.com/cru/PLT-6638,PLT-6638,Defect,,3-Medium,|Z. DO NOT USE User Mgmt - Manage Users|,New,,
- Logstash log in debug mode looks like parsing as expected.
> { > "FILE_TYPE" => "java", > "GIT_ORG" => "bizx", > "JIRA_PRIORITY" => "2-High", > "JIRA_TESTCASE_ID" => "ENT200002079CrossDomainReport", > "path" => "C:/elkstack/elasticsearch-6.5.1/logs/less.csv", > "host" => "PVGN50859047A", > "COMMIT_SHA1" => "1be44f52f25f6b540f284eb17e8cee5838826cb9", > "COMMIT_COMMENT" => "RPT-7954: Cross domain reports are failing with Birt Exception", > "@timestamp" => 2019-03-07T08:48:50.231Z, > "FILE_METHOD" => "PermissionMetaDataGenerator.populateMissingDataInLojMap", > "COMMIT_DATE" => "3/30/2017", > "JIRA_ISSUE_TYPE" => "Defect", > "JIRA_ISSUE_CATEGORY" => "Internal Issue", > "COMMIT_AUTHOR" => "ssheriff", > "message" => "bizx,idl-analytics-api,1be44f52f25f6b540f284eb17e8cee5838826cb9,ssheriff,3/30/2017,RPT-7954: Cross domain reports are failing with Birt Exception,RPT-7954,Defect,Internal Issue,2-High,|Report Builder|,Regression,ENT200002079CrossDomainReport,Auto\r", > "GIT_REPOS" => "idl-analytics-api", > "JIRA_KEY" => "RPT-7954", > "FILE_CHANGED" => "idl-analytics-api-service/src/main/java/com/successfactors/analytics/permission/PermissionMetaDataGenerator.java", > "@version" => "1", > "JIRA_COMPONENTS" => "|Report Builder|", > "JIRA_DEFECT_TYPE" => "Regression", > "JIRA_FIND_BY_AUTO" => "Auto" > }
-
But in elasticsearch, there's an exception saying that " Caused by: java.lang.IllegalArgumentException: Invalid format: "1/24/2018" is malformed at "/24/2018" "
-
My elasticserch index template is
{
"template": "git_template", "index_patterns":["logstash-git"], "mappings": { "doc": { "properties": { "GIT_ORG": { "type": "text" }, "GIT_REPOS": { "type": "text" }, "COMMIT_SHA1": { "type": "text" }, "COMMIT_AUTHOR": { "type": "text" }, "COMMIT_DATE": { "type": "date","format":"MM/dd/yyyy" }, "COMMIT_COMMENT": { "type": "text" }, "JIRA_KEY": { "type": "text" }, "JIRA_ISSUE_TYPE": { "type": "text" }, "JIRA_ISSUE_CATEGORY": { "type": "text" }, "JIRA_PRIORITY": { "type": "text" }, "JIRA_COMPONENTS": { "type": "text" }, "JIRA_DEFECT_TYPE": { "type": "text" }, "JIRA_TESTCASE_ID": { "type": "text" }, "JIRA_FIND_BY_AUTO": { "type": "text" } } } } }