Downgrade kops 1.34.1 to 1.33.1 will fails due to etcd v3.5.24: not found

[nickdallavalentina]

/kind bug

1. What kops version are you running? The command kops version, will display
this information.

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

currently kops 1.33.1 but the issue happened due to downgrade from 1.34.1
k8s version 1.33.7 (unchanged for both kops versions)

3. What cloud provider are you using?
AWS

4. What commands did you run? What is the simplest way to reproduce this issue?
kops update cluster --allow-kops-downgrade

5. What happened after the commands executed?

I think the downgrade is failing because kOps 1.33.1 doesn’t stage the etcd 3.5.24 binaries, but the cluster (and/or etcd-manager) is expecting v3.5.24, which results in:
the first control-plane will not join the cluster due to this error

W0305 20:54:22.281433    5589 etcdserver.go:115] error running etcd: unknown etcd version v3.5.24: not found in [/opt/etcd-v3.5.24 /opt/etcd-v3.5.24/usr/local/bin /tmp/etcd-v3.5.24 /tmp/etcd-v3.5.24/usr/local/bin]

Would it be possible to get a kOps 1.33.2 release (or a patched 1.33.x build) that cherry-picks the change for etcd v3.5.24 (see PR #17721), so users can downgrade from 1.34.x to 1.33.x without breaking the control plane?

As a workaround, I tried manually setting the configured etcd version back to an older value (e.g. 3.5.21) while using 1.31.1 kOps version. However, etcd-manager then attempts an in-place downgrade, which can cause version skew across etcd members. If not all members are healthy, etcd-manager refuses to proceed, reconciliation fails, and the etcd cluster remains unhealthy.

6. What did you expect to happen?

allow downgrade without any issue

**7. Please provide your cluster manifest. Execute
n/a

8. Please run the commands with most verbose logging by adding the -v 10 flag.
Paste the logs into this report, or in a gist and provide the gist link here.
n/a
9. Anything else do we need to know?
n/a