[CLOSED ]Logstash crash after service start logstash

Elastic Stack Logstash
1

hello, i'm using logstash on ubuntu (14.04LTS) with logstash_2.3.1-1_all.deb

After installation with dpkg, i'm trying to start logstash but logstash always stop after 30 secondes.
I already check my conf file, it's OK. If i start logstash with /opt/logstash/bin/, its OK.

sudo service logstash start

I think the probleme is here, /etc/init.d/logstash :

# Run the program!
nice -n ${LS_NICE} chroot --userspec $LS_USER:$LS_GROUP $EXTRA_GROUPS / sh -c "
cd $LS_HOME
ulimit -n ${LS_OPEN_FILES}
exec \"$program\" $args
" > "${LS_LOG_DIR}/$name.stdout" 2> "${LS_LOG_DIR}/$name.err" &

On /var/log, i have only :

:message=>"Pipeline main started"}
:message=>"Pipeline main has been shutdown"}
:message=>"stopping pipeline", :id=>"main"}

2

Why do you think that?

Can you run LS directly, ie call the full path and point to the config directory and see what happens?

3

Sorry for the delay.

I think that because when i comment this line for :
`

Run the program!

nice -n ${LS_NICE} chroot --userspec $LS_USER:$LS_GROUP $EXTRA_GROUPS / sh -c "

cd $LS_HOME

ulimit -n ${LS_OPEN_FILES}

exec "$program" $args

" > "${LS_LOG_DIR}/$name.stdout" 2> "${LS_LOG_DIR}/$name.err" &

$program $args >> "${LS_LOG_DIR}/$name.stdout" 2>> "${LS_LOG_DIR}/$name.err" &
`

that work fine. But its like if a start as root.

If i run LS directly (as root), that work too.

4

You can watch the end of strace here (not sure that help you)

  [pid 23470] futex(0x7f78340f9254, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f78340f9250, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} <unfinished ...>
[pid 23480] <... mmap resumed> )        = 0x7f781aa83000
[pid 23476] <... futex resumed> )       = 0
[pid 23480] rt_sigprocmask(SIG_SETMASK, [QUIT],  <unfinished ...>
[pid 23476] futex(0x7f78340f9228, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 23480] <... rt_sigprocmask resumed> NULL, 8) = 0
[pid 23470] <... futex resumed> )       = 1
[pid 23480] madvise(0x7f781aa83000, 2076672, MADV_DONTNEED <unfinished ...>
[pid 23470] futex(0x7f78340f9228, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 23476] <... futex resumed> )       = 0
[pid 23470] <... futex resumed> )       = 1
[pid 23480] <... madvise resumed> )     = 0
[pid 23476] futex(0x7f78340f9228, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 23480] _exit(0)                    = ?
[pid 23476] <... futex resumed> )       = 0
[pid 23480] +++ exited with 0 +++
[pid 23476] mprotect(0x7f783ac2a000, 4096, PROT_READ <unfinished ...>
[pid 23470] futex(0x7f783400c054, FUTEX_WAIT_PRIVATE, 787, NULL <unfinished ...>
[pid 23476] <... mprotect resumed> )    = 0
[pid 23476] mprotect(0x7f783ac2a000, 4096, PROT_READ|PROT_WRITE) = 0
[pid 23476] mprotect(0x7f783ac2b000, 4096, PROT_NONE) = 0
[pid 23476] unlink("/tmp/hsperfdata_logstash/23456") = 0
[pid 23476] exit_group(0)               = ?
[pid 23562] +++ exited with 0 +++
[pid 23548] +++ exited with 0 +++
[pid 23483] +++ exited with 0 +++
[pid 23482] +++ exited with 0 +++
[pid 23481] +++ exited with 0 +++
[pid 23478] +++ exited with 0 +++
[pid 23477] +++ exited with 0 +++
[pid 23476] +++ exited with 0 +++
[pid 23475] +++ exited with 0 +++
[pid 23474] +++ exited with 0 +++
[pid 23473] +++ exited with 0 +++
[pid 23479] +++ exited with 0 +++
[pid 23472] +++ exited with 0 +++
[pid 23471] +++ exited with 0 +++
[pid 23470] +++ exited with 0 +++
+++ exited with 0 +++
5

we actually rewrote the whole package thing for new 5.0 alpha3, I would also test with this version if upgrade is a good thing for you.

also can you share your java version?

thanks a lot and sorry for your pain.

1 Like
6

java version "1.7.0_101"
OpenJDK Runtime Environment (IcedTea 2.6.6) (7u101-2.6.6-0ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Thank you

7

Have you checked java privileged port permissions?
see: Running logstash as a deamon

8

Good idea, i check this permission :

sudo getcap /usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/bin/java
/usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/bin/java = cap_net_bind_service+ep

/usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/bin/java -version
java version "1.7.0_101"
OpenJDK Runtime Environment (IcedTea 2.6.6) (7u101-2.6.6-0ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Normally, its good for permission, but logstash deamons still stop.

9

I'm running with java 8 btw.

java -version
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

10

On Ubuntu 14.04LTS, java version "1.7.0_101" is the most recent.

I made manually update, still no change

openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-8u91-b14-0ubuntu4~14.04-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

11

I have no answer but i suppose its java permission problem.

We can close this ticket, thanks for help