HA Database corrupting on almost daily basis

Ched · Post by **Ched** » Sat May 13, 2023 7:57 pm

I am running HA on an old RPi2 and it was working fine at first but now it seems to corrupt the database on an almost daily basis.
Any ideas?
Could it be an iffy micro sd card or would I be better off running HA on an old laptop with Ubuntu on?

Thanks for any suggestions.

calum · Post by **calum** » Sat May 13, 2023 9:17 pm

HA is very hard on its storage, I'm guessing your SD card is on its way out.

I'd look for something else to run HA on, it's outgrown the RPi platform in my opinion.

Ched · Post by **Ched** » Sat May 13, 2023 10:55 pm

Cheers, I will have to look at the micro sd as I can't remember what one I used.
I just remembered its a RPi3 but still a bit weak but it uses such low power. Only other things I have kicking about are old laptops. Might give one of those a go.
Thanks.

calum · Post by **calum** » Sun May 14, 2023 11:04 am

Just make sure you're not running it off a laptop hard drive!

Ched · Post by **Ched** » Sun May 14, 2023 3:41 pm

calum wrote: ↑Sun May 14, 2023 11:04 am Just make sure you're not running it off a laptop hard drive!

Cheers. I was thinking of using the whole laptop which has a small SSD drive in. Think I will try an alternative microsd card first as a laptop will consume lots more power than a rpi3.
Cheers

calum · Post by **calum** » Sun May 14, 2023 7:07 pm

There's lots of stuff in forums about how to tune HA to be nicer to the SD card, mainly revolving around how much gets written to the database and how often. Worth a look if you want to persist with the platform.

Ched · Post by **Ched** » Mon May 15, 2023 10:38 pm

Cheers.
I've ordered a new microsd card - a Samsung PRO Endurance which is supposed to support cctv type recording so HA shouldn't be too bad with it. If that doesn't help I will look at cutting down the write frequency but it did seem a bit complex in that I have to decide what sensors to actually log so that might be a challenge.
There does also seem to be a fair bit of info on using ssd drives. So looks like I have a few options.
Thanks.

APfox · Post by **APfox** » Thu May 18, 2023 11:33 am

calum wrote: ↑Sat May 13, 2023 9:17 pm it's outgrown the RPi platform in my opinion.

Why exactly?
And what do you propose as an alternative then?

At the moment HA runs as a docker container on a rpi4 8GB with an 128GB Samsung 850 Pro. (Ubuntu 20.04 LTS)
I'm now on the look for something else as I want to run HAOS so i can make use of the Supervisor and the add-ons. I'm also thinking on moving from the default sqlite to MariaDB (my db exploded since using the foxess-modbus and foxess-em integrations)

I was thinking to use a RPi4 4GB with a new SSD for it , but now you say it's outgrown the RPi platform

canton7 · Post by **canton7** » Thu May 18, 2023 1:55 pm

> (my db exploded since using the foxess-modbus and foxess-em integrations)

There's a new experimental option for foxess_modbus under "CONFIGURE" to round and filter sensors slightly, which should help a lot with database size issues.

APfox · Post by **APfox** » Thu May 18, 2023 3:42 pm

canton7 wrote: ↑Thu May 18, 2023 1:55 pm > (my db exploded since using the foxess-modbus and foxess-em integrations)

There's a new experimental option for foxess_modbus under "CONFIGURE" to round and filter sensors slightly, which should help a lot with database size issues.

yeah saw that one and installed it already.
I've also started an analysis on which data actually is important enough to keep and how to set update intervals depending on the needs

calum · Post by **calum** » Thu May 18, 2023 5:19 pm

APfox wrote: ↑Thu May 18, 2023 11:33 am
calum wrote: ↑Sat May 13, 2023 9:17 pm it's outgrown the RPi platform in my opinion.
Why exactly?
And what do you propose as an alternative then?

At the moment HA runs as a docker container on a rpi4 8GB with an 128GB Samsung 850 Pro. (Ubuntu 20.04 LTS)
I'm now on the look for something else as I want to run HAOS so i can make use of the Supervisor and the add-ons. I'm also thinking on moving from the default sqlite to MariaDB (my db exploded since using the foxess-modbus and foxess-em integrations)

I was thinking to use a RPi4 4GB with a new SSD for it , but now you say it's outgrown the RPi platform

My feeling is that for something that is monitoring a key part of my house's infrastructure (namely, power), I want to be running something that is more performant, reliable and scaleable. Especially when you take into account things like long term analysis, logging to InfluxDB, running Grafana and other plugins - if you are using HA's capabilities, you end up having to deal with weaknesses of the platform like storage performance, which is really key when you get into substantial databases. They also don't work well with the small SSDs and SD cards that tend to be used with RPis.

So sure, it can be made to work, but constantly having to be wary of how hard I'm beating on the storage, and not being able to use what I see as the full capabilities of the device in a performant way, is my reason for saying that. As I said at the time, it's a personal opinion, not claiming it to be a universal truth

Ex-business uSFF PCs with low power x86 CPUs are a better bet IMO, although I acknowledge they come at a cost of somewhat higher power usage vs a RPi.

Ched · Post by **Ched** » Wed May 24, 2023 8:48 pm

Just to update:
The new Samsung MicroSD card has been running for a week and it's been fine. Fingers crossed it will stay OK for a few months at least.
Cheers for your help.