I will be keeping an eye on this thread to see what other people do, but what I have done in the past is to have a couple different health checking strategies.
- For web-accessible services I am running, I usually run something like Uptime Kuma or Gatus on a different box checking to make sure those web endpoints are available and performant. I lately have been really digging how Gatus can check more than just the response header, but also latency and certificate validity.
- For the host machine, you can set up custom alerts within netdata for stuff like cpu utilization and memory with custom thresholds. The only other solution I have used for this in the past is setting up alerts through my VPS provider (if it is a VPS that is).
- On really low-spec machines I have had trouble with netdata though, so I don’t have a good solution in those cases. Interested to see if there are less demanding options. Instead, I have resorted to just using dashdot as a PWA so that I can check it easily on my phone if I am on the go.
- For some custom services in the past that run on set schedules, I have used healthchecks.io (which you can selfhost) to send alerts in the case that they don’t run for some reason.
- As for the containers being restarted, I actually don’t have experience with that, so I am interested to see what others have done.
I feel like every time I see a version of this question online, I mostly see people talking about how HDHPs and HSAs are always the right choice. So, I want to provide the perspective of somebody that opted for a PPO rather than an HDHP.
My workplace offers both a PPO option and a HDHP with a company contribution. I opted for the PPO because the HDHP’s out of pocket max was so much higher than the PPO that it would be very difficult for us if something were to happen. My wife has a chronic illness that is manageable most of the time, but there is always that risk that we suddenly need to have a whole bunch of very expensive specialist visits and procedures done. Yes, if it is a good year, I probably came out behind compared to having a HDHP + HSA investments. However, I value the risk mitigation that having the PPO plan provides for the chance that it isn’t a good year. The situation can be very different for different people though.