Logstash fails to load

geriberi · December 18, 2018, 10:36am

Hi,

i just tried to set up ELK on my win10 64 bit:
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
The packages for Elasticsearch, Kibana and Logstash are from the Elasticsearch download portal, obviously the .zip version.

After loading correctly all the .jar files from the shipped library if fails for org.logstash.Logstash with the error at the end:

C:\Users<USER>\Documents\ELK\logstash>"C:\Program Files (x86)\Common Files\Oracle\Java\javapath\java.exe" -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit.threshold=0 -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -cp ""C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\animal-sniffer-annotations-1.14.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\commons-codec-1.11.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\commons-compiler-3.0.8.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\error_prone_annotations-2.0.18.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\google-java-format-1.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\gradle-license-report-0.7.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\guava-22.0.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\j2objc-annotations-1.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\jackson-annotations-2.9.5.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\jackson-core-2.9.5.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\jackson-databind-2.9.5.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\jackson-dataformat-cbor-2.9.5.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\janino-3.0.8.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\jruby-complete-9.1.13.0.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\jsr305-1.3.9.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\log4j-api-2.9.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\log4j-core-2.9.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\log4j-slf4j-impl-2.9.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\logstash-core.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.commands-3.6.0.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.contenttype-3.4.100.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.expressions-3.4.300.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.filesystem-1.3.100.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.jobs-3.5.100.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.resources-3.7.100.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.core.runtime-3.7.0.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.equinox.app-1.3.100.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.equinox.common-3.6.0.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.equinox.preferences-3.4.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.equinox.registry-3.5.101.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.jdt.core-3.10.0.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.osgi-3.7.1.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\org.eclipse.text-3.5.101.jar";"C:\Users<USER>\Documents\ELK\logstash\logstash-core\lib\jars\slf4j-api-1.7.25.jar"" org.logstash.Logstash -f C:\Users<USER>\Documents\ELK\logstash\config\logstash.config
[ERROR] 2018-12-18 10:51:39.982 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) no such file to load -- bundler

in logstash.bat i found the corresponding line:
%JAVA% %JAVA_OPTS% -cp "%CLASSPATH%" org.logstash.Logstash %*

When the program gets to this point the variable CLASSPATH is empty (see the double quotes after the last jar file).
I already tried to download and set it up again, but got the same result.
Any ideas what else could be done? As far as i had seen, this topics was already raised a couple of times, but never resolved.

Ty,
Geri

balumurari1 · December 20, 2018, 11:29am

Hello,

Open your command prompt and check
java -version.

It must display version 1.8.0_191

If it is fine, check how you were executing the code.

geriberi · December 20, 2018, 6:25pm

Hi,

yes the java version is correct, as i wrote at the begining
what do you mean by how I execute the code? If in terms of privilege, it doesn´t matter if i run it as my local user or with administrative privileges, I get the same result.

geriberi · December 27, 2018, 9:23am

anyone who was able to set it up on win10?

balumurari1 · December 27, 2018, 9:43am

Hello,

I have setup in win10 only, and it works fine.
Give me your input code and the steps how you were trying to run the logstash.
Hope this helps to resolve the issue

geriberi · December 27, 2018, 10:01am

Downloading from https://www.elastic.co/downloads/logstash (the zip version)
Unpacking it into the same folder as elasticsearch and kibana.
created my own config file (logstash.config)
switching directories to "C:\Users<USER>\Documents\ELK\logstash"
execute bin\logstash -f config\logstash.config
then as per the logs, all the jar files are loaded then it fails with the bundler not found error.

balumurari1 · December 27, 2018, 10:07am

can you send me the content what you have written in the file

geriberi · December 27, 2018, 10:22am

input {
file{
	path => "Users/Gergö/Elk/data/misp.csv"
	start_position => "beginning"
	sincedb_path => "/dev/null"
}
} # /input
filter{
	csv{
		# Define separator from the csv
		separator => ","
		# Define the columns in the csv
		columns => [ "uuid", "event_id", "category", "type", "value", "comment", 
		"to_ids", "date", "object_relation", "attribute_tag", "object_uuid", 
		"object_name", "object_meta_category", "event_info", "event_member_org", 
		"event_source_org", "event_distribution", "event_threat_level_id", 
		"event_analysis", "event_date", "e vent_tag" ]
	}
	# Define number formats
	mutate {
		convert => ["event_id","integer"]
		convert => ["to_ids","integer"]
		convert => ["event_threat_level_id","integer"] 
	}
	
	date {
		match => ["date", "yyy MM dd"]
		match => ["event_date", "yyy MM dd"]
	}
} # /filter
output{
	elasticsearch => "localhost"
	index => "misp_input"
	document_type => "CSL_KM"
} # /output

balumurari1 · December 27, 2018, 10:25am

Case1

case2

switching directories to "C:\Users<USER>\Documents\ELK\logstash"

in case 1: it is like linux
in case 2: it is windows.

which machine were you using..??????????
Be clear....,

The code seems to be like you have just copied from somewhere and trying to run the logstash.

Make sure you start with samples and then move ahead..,

geriberi · December 27, 2018, 10:29am

i am running it on windows, i just corrected the path. Still the same result. No, it was created by me according to the format which is needed.
Okay, I´ll try it with the sample data first.

balumurari1 · December 27, 2018, 10:32am

here is the sample, copy in conf file and check,

Ex;

input {
heartbeat{
type => "heartbeat"
interval => 10
}
}
output {
elasticsearch { hosts => ["localhost:9200"] index => "pulse" }
stdout { codec => rubydebug }
}

geriberi · December 27, 2018, 10:49am

i get the same result as in the first place.

balumurari1 · December 27, 2018, 11:04am

The error might be because,

Is your elasticsearch up and running..?? check with
localhost:9200
(or)
Were you using config file or text file...??

Keep the conf file in location " C:\Users<USER>\Documents\ELK\logstash"
and run the command
bin\logstash -f logstash.conf

geriberi · December 27, 2018, 11:24am

elasticsearch is up and running.

it is a text file with the extension .config (I just tried with .conf extension, got the same result)

The file is in the default "config" folder ( C:\Users<USER>\Documents\ELK\logstash\config) (I now tried with copying the .conf file into the logstash folder directly, got the same result)

balumurari1 · December 27, 2018, 11:34am

if you rename the text file with logstash.conf, it will not become conf file.

Download sample conf file from the internet and use it in your logstash folder.

or

you will have logstash-sample file in config folder. Use it

system · January 24, 2019, 11:34am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.