Failed Proxmox 7 -> 8 upgrade. I forgot to substitute bullseye with bookworm in one of the repository lists!

They literally say in Proxmox wiki for 7 to 8 upgrade:

Ensure that there are no remaining Debian Bullseye specific repositories left, if you can use the # symbol at the start of the respective line to comment these repositories out. Check all files in the /etc/apt/sources.list.d/pve-enterprise.list and /etc/apt/sources.list and see Package_Repositories for the correct Proxmox VE 8 / Debian Bookworm repositories.

And I managed to forget the one in the /etc/apt/sources.list.d/pve-enterprise.list. I should not have had it there in the first place, at that time I didn't know Proxmox at all.

So after following the upgrade steps and having no errors after apt dist-upgrade I faced no Web interface being available as the first sign of my failure.

journalctl -u pveproxy showed:

Nov 13 14:24:08 pve62 pveproxy[314931]: ipcc_send_rec[3] failed: Connection refused

systemctl status pve-cluster showed:

○ pve-cluster.service
     Loaded: masked (Reason: Unit pve-cluster.service is masked.)
     Active: inactive (dead)

pveversion:

-bash: pveversion: command not found

Running apt update showed me that I have bullseye still somewhere in the sources.

Where?

List the repositories:

grep "^[^#]" /etc/apt/sources.list /etc/apt/sources.list.d/*

/etc/apt/sources.list:deb http://ftp.cz.debian.org/debian bookworm main contrib
/etc/apt/sources.list:deb http://ftp.cz.debian.org/debian bookworm-updates main contrib
/etc/apt/sources.list:deb http://security.debian.org bookworm-security main contrib
/etc/apt/sources.list.d/pve-enterprise.list:deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription

Fix:

vim /etc/apt/sources.list.d/pve-enterprise.list

What now? Let see how to put broken proxmox installation back on top of debian:

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm

Running:

wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg 

Now:

apt install proxmox-ve

Fails with error:

E: Failure running script /usr/share/proxmox-ve/pve-apt-hook

Here:

https://forum.proxmox.com/threads/upgrade-failed-pve-apt-hook-not-found.107950/post-464053

Quoting:

I did the following and that seems to have solved the problem.

mkdir -p /usr/share/proxmox-ve
touch /usr/share/proxmox-ve/pve-apt-hook
chmod +x /usr/share/proxmox-ve/pve-apt-hook

After that:

apt update && apt full-upgrade

apt dist-upgrade

And the last one:

apt install proxmox-ve

Web inteface is back, no errors so far. All containers are running as they should.