Endian CE 2.5.1 stops responding

[Guest]

[logicasrl]

Hi all,

I've installed Endian on an custom appliance (with motherboard JETWAY J7F4K, CPU VIA C7 EDEN 1.2GHZ, 1GB RAM, HDD SATA 500GB): this hardware worked without problems for 3 years for internet access to an ADSL 1.2Mbps/256Kbps and Endian 2.3 and 2.4.
Now I've reinstalled it completely with Endian 2.5.1 and connect it to a fiber optic link at 10Mbps since a  of weeks.

From time to time (let's say a  of time per week) the appliance stop responding completely, but does NOT power off (it remains powered on): the web GUI is unreachable and even the SSH access is no more available.
I simply reboot it (in the hard way, with the power on button) and everything start working again.

The CPU is working at 10% and RAM at 5%: there are plenty of disk space... And nothing is logged (I have also taken a look to "/var/log").

The question is: there is a log file that I can analyse to understand the source of te problem???

Thank you very much,
Luca Z.

[bash3r]

This same exact issues is happening to me as well.  I have 5 different EFWs VMs installed and they all do it.. first I'll notice a VPN tunnel down, or can't login to the WebUI.. but routing between WAN/LAN is working fine.. can't get on the console either.. Then after awhile the whole thing stops functioning, so have to hard reset it.

Seems like there's definitely an issue going on.

[pwinterf]

This is probably unrelated

But what happens when you select reboot on console, and is your disk light on hard

Probably not related, but you initial  problems start out like mine 

Regards peter

[havoc3d]

I have a post about this too, but I wanted to give a "me too" on this thread so maybe it can get more attention.

It's like the logs stop and everything.  The whole OS just....stops.  For no discernible reason.  I have tested memory, network interfaces, cpu, everything tests fine in synthetic tests.

[kashifmax]

Jumping in
Now take it easy boys, EFW is working like a charm till now but any misconfiguration or any incompatibility even faulty hardware leads to such issues. Before joining here, mine EFW was taking 90% of RAM and than system got hanged. I don't know exactly what solved the issue but I was checking dmesg, message, etc., for any clues. So in my case I saw a strange entry.

Code:

dmesg | grep -i memory

At that time, I was unable to understand but I replaced all the ram modules with new one's.

Also I disabled all of the services, logs and I was just using FW for test, so it was working more than 2 weeks and consuming around 5% of RAM, than gradually I started other services.

The above is just a hint/example. In your scenario you have to dig it down.

Try to upgrade EFW...

Code:

efw-upgrade

[rosch]

Now take it easy boys, EFW is working like a charm till now but any misconfiguration or any incompatibility even faulty hardware leads to such issues.

Hm well I do want to take it easy 
But in my case there is not much traffic at all so EFW has no reason to mess up, hardware issues excluded.
When the freezes happened with the SSDs the EFW was still doing its job as a gateway but with the harddisks all services were down, as was internet access. Depending on the size of the network behind this can be quite annoying.

Try to upgrade EFW...

It is very rare that you see available community upgrades 

edit: I'm still on 2.4.1 but the issue might be the same or related.

[pwinterf]

Im agreeing here stinks like same problem, see if you can ssh in,
Cat  var log messages, and dmesg are you seeing ata errors, no good rebooting and checking, once it starts logs are not able to update on disk and so all traces are lost.

If its same problem.

Otherwise you should see the cause if its something else in the log after a reset.

Regards peter

[havoc3d]

It's been a while, but I have some new information. 

I should probably cover that after a hard reboot, the logs all just seem to stop.  I thought maybe it was some sort of power save trying to kick in, so I disabled ACPI in bios as well as any other power saving options I could find, but that didn't help anything.

What I have found, however, is that if I leave the console at a bash prompt, it never seems to have an issue.  I have had it running over a week just sitting at the command prompt and it hasn't had an issue.  Maybe that will give someone a little more knowledgeable than myself some clue as to what might be going on?  For now it looks like I will just have to do all reboots right at the router rather than a webgui so I can hit 0 then enter after the reboot to keep it from locking up.

EDIT: oh yeah, hardware testing wise I have given memtest86 a full 24hour run with no issues as well as run several full low levels and tests on both the 2GB SSD and 20GB spinning disk that I have been using for testing.  I also loaded Ubuntu just for funsies and it doesn't seem to ever have an issue either; ran it for a week straight with no issues.

[treviris]

The same issue is happening to me also.
I reported on it (in German) in: efw-forum.de/www/forum/viewtopic.php?f=73&t=1411
The hardware ( Jetway board and SSD) worked well in  2.3, but the feature to allow only fixed leases does not work in that release - so I performed an upgrade.
After upgrading to 2.5.1 and checking the box "Allow only fixed leases" I see stops as you described them.
Three days ago  I unchecked that box and since then, after a reboot, the device works without any problem.
Can anybody confirm that experience and is there a solution - may by a faster hardware or no SSD?

[mmiat]

me too, same problem with 2.5.1 and 2.5.2
I've 2 sata disk in raid1, no ssd, 4gb ram (HP ML110)
I've added a daily reboot and so it seems hangs more rarely, but it's a workaround and not a solution, and it not solved completely my problem

[mmiat]

again, 10 minutes ago. twice this morning

[mmiat]

in Endian 3.0.0 release notes:
  * [CORE-638]     - Freezings and reboots caused by out of memory
is it our problem?

[mmiat]

I tried with acpi=off option, but it still freezes

[mmiat]

I reinstalled 2.5.2, and after importing backup, just few second before rebooting, I can read these lines on console:

Jul 14 15:16:50 efw-1405350765 kernel: 3371]b0 ot1eh)etrn owrigsae<7>[  74191]eh:n P6ruespee                                          t<7>[   74.15902]b2 oIv otr rsn
Jul 14 15:17:02 efw-1405350765 kernel: [ 8.296 d d:rsn oe
Jul 14 15:17:02 efw-1405350765 kernel: 5987]RI1cn rnot

have they some meaning?

[mmiat]

again.... twice today
grrrrr
how I can add more details in messages to understand what happens?