I finally figured out why our firewall kept crashing at 3 AM

Last month I was on call and got paged at 2:47 AM because our Palo Alto firewall at the Denver office kept dropping connections. Turns out someone had set the log retention to max and the disk filled up completely, which made the firewall panic and reboot. I cleared 200 GB of old logs and set a 90 day retention policy instead of infinite. Has anyone else dealt with firewall reboots from full storage or is that just my bad luck?

2 comments

2 Comments

matthewg635d ago

Hang on though, isn't a firewall panic and reboot kind of a feature? I'd rather have it crash and recover on its own than silently start dropping packets or giving you bad data. Seems like a design choice, not a straight up bug.

anthony8835d ago

Wait, don't you want the firewall to tell you something went wrong instead of just hiding it? That's my take @matthewg63. If it crashes and reboots, you know there's a problem and can look into it. Silent packet loss is way worse because you don't know if your traffic is getting through until something breaks. I'd rather have a loud crash than quiet failure any day.