Overhaul buildbot worker setup documentation

[stratakis]

As I've recently had to setup again the aarch64 buildbots, I've taken a shot at re-reading and verifying the current docs. I've rewriten the Linux buildbot worker setup instructions with per-distro nested tabs, all of which I've verified in containers and VMs.

Also added recommended buildbot.tac settings, FreeBSD rc.d instructions, and pip/manual setup for distros without available buildbot packages.

Plus some minor MacOS fixups.

[read-the-docs-community]

Documentation build overview

📚 CPython devguide | 🛠️ Build #31776577 | 📁 Comparing 99320e9 against latest (5a4dd49)

🔍 Preview build

Show files changed (4 files in total): 📝 4 modified | ➕ 0 added | ➖ 0 deleted

File	Status
testing/index.html	📝 modified
versions/index.html	📝 modified
developer-workflow/psrt/index.html	📝 modified
testing/new-buildbot-worker/index.html	📝 modified

[stratakis]

Also another section that I believe warrants an update is the "Required resources" one as it still references a 2012 survey recommending e.g. 512 MB RAM. This is likely too low for modern CPython builds. python/cpython#133454 shows some tests now consume >1 gb, and github actions CI runners use 16 gb. However I don't have data on possible minimal specs, so I left it untouched, but maybe some higher baseline would be better.

[hugovk]

By the way, cstratak-fedora-stable-ppc64le is now failing with timeouts: https://buildbot.python.org/#/builders/435

• last pass: https://buildbot.python.org/#/builders/435/builds/6753
• first fail: https://buildbot.python.org/#/builders/435/builds/6754

[stratakis]

By the way, cstratak-fedora-stable-ppc64le is now failing with timeouts: https://buildbot.python.org/#/builders/435

* last pass: https://buildbot.python.org/#/builders/435/builds/6753

* first fail: https://buildbot.python.org/#/builders/435/builds/6754

I believe this is due to llvm/llvm-project#179695, we'll need to have clang fix it.

[stratakis]

@encukou btw should I follow the CPython conventions here for the PR? Aka every next fix to get its own commit and not rebase? Is everything squashed before a merge here?

[StanFromIreland]

Is everything squashed before a merge here?

Yes, please don't force push.

[encukou]

Everything's squashed.
Don't force push (so we don't need to re-review everything). And don't worry about rebasing before a push -- neatly split commits are only nice-to-have.

[encukou]

Looks good, thanks!
I'll merge later this week if there are no objections.

[stratakis]

Looks good, thanks! I'll merge later this week if there are no objections.

Thanks!

And since I got a nice taste of testing other OSes, I could do subsequest PRs for maybe adding other BSDs, Solaris, verifying the windows and Mac instructions. Would that be of interest?

[encukou]

Would that be of interest?

I'd definitely merge them, on the chance that they help someone.

I take the instructions as hints -- they'll inevitably go out of date and they'll inevitably not apply exactly to all environments (and the environments we want tests for are the weird ones :)
If someone goes through the setup process, it's very valuable to update the instructions. Going through the process just to update them might not be worth your time.

[hugovk]

This section: https://cpython-devguide--1761.org.readthedocs.build/testing/new-buildbot-worker/index.html#conventional-always-on-machines

Has a Unix tab with FreeBSD and OpenBSD subtabs:

---

But this section: https://cpython-devguide--1761.org.readthedocs.build/testing/new-buildbot-worker/index.html#service-management

Has a Unix tab with a single rc.d tab, but then separate FreeBSD and OpenBSD instructions:

Shall we get rid of the rc.d tab, because there's no other sibling tabs?

And instead have FreeBSD and OpenBSD subtabs?

[stratakis]

Shall we get rid of the rc.d tab, because there's no other sibling tabs?

And instead have FreeBSD and OpenBSD subtabs?

The categorization there is about service management types, so splitting via OSes doesn't make sense in accordance to the other tabs. rc.d is the same for OpenBSD and FreeBSD, just the commands are different. I get that a lone tab might seems odd there, but for example if Solaris is added, the SMF tab should be added next to rc.d there.