I was trying to issue ceph upgrade from 17 to 18.2.4, as outlined here

ceph orch upgrade start --ceph-version 18.2.4
Initiating upgrade to quay.io/ceph/ceph:v18.2.4

After this, however, the orchestrator no longer responds

ceph orch upgrade status
Error ENOENT: Module not found

Setting the backend back to orchestrator or cephadm fails, because the service appears as ‘disabled’. Ceph mgr swears instead that the service is on.

from what I can gather now I’m stuck with one mgr daemon running reef, while the rest of the cluster runs on quincy.

ceph versions
{
    "mon": {
        "ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable)": 5
    },
    "mgr": {
        "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)": 1
    },
    "osd": {
        "ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable)": 31
    },
    "mds": {
        "ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable)": 4
    },
    "overall": {
        "ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable)": 40,
        "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)": 1
    }
}

How do I reinstate the cluster to a healthy state?