Restoring Coolify after failed update

Noticed today that one of my Coolify services on Hetzner was not working - I was getting Bad gateway message in browser, but the app itself was working without an issue.

First thing I did is check the status of coolify containers:

docker ps -a | grep -i coolify

But that returned almost no results. There were no coolify, coolify-db, coolify-redis or coolify-realtime containers.

Just in case, I checked disk and RAM, but that was fine, as I expected.

df -h
free -m

That means that Coolify containers stopped for some reason. I remember running an update, but I don't remember it finishing, so that was probably it.

I was hoping that I could restore the containers, instead of recreating everything from backup of, worse, from scratch, and for that I needed to check if there was still data and Coolify env files on disk still.

docker volume ls | grep coolify # check if volumes are still there
ls -la /data/coolify/source/ # check if there are env and compose files here
grep APP_KEY /data/coolify/source/.env # check if APP_KEY is there (you can kiss your instance goodbye if you loose or change this, as it's used for encryption)
docker images | grep coollabsio # check if coolify image is still there

Luckily, everything was still there. coolify-db and coolify-redis were intact, .env and APP_KEY as well, so I could just simply restore everything, get it up and running.

To do this, I had to run this:

cd /data/coolify/source # go to source folder, the files we need are there

Then recreate the containers:

docker compose --env-file /data/coolify/source/.env \
  -f docker-compose.yml -f docker-compose.prod.yml \
  up -d --remove-orphans --force-recreate

Then check if containers are up:

docker ps | grep -i coolify

And finally, curl the Coolify url to see if it's working, but this can be done in the browser as well (you know, by visiting the Coolify UI)

curl -I http://localhost:8000