Skip to content

Fix ci#428

Open
Erotemic wants to merge 4 commits intomainfrom
fix_ci
Open

Fix ci#428
Erotemic wants to merge 4 commits intomainfrom
fix_ci

Conversation

@Erotemic
Copy link
Member

Bumping CI versions with xcookie to see if that avoids recent CI errors. This will drop 3.8 support, which is overdue anyway.

I think we may need to re-add the arm entries:

        - python-version: '3.13'
          install-extras: tests
          os: ubuntu-24.04-arm

The wheel path detection logic changed, but I think that's ok. Would be nice if xcookie could keep the comments in the CI yaml. I was having issues getting ruamel.yaml doing that. I may manually update if I can solve the CI issues.

@TTsangSC
Copy link
Collaborator

Some observations:

  • The common failure points on the jobs seem to consistently be tests/test_child_procs.py::test_running_multiproc_script, i.e. the tests created in FIX: kernprof to always run code in sys.modules['__main__']'s namespace #423 to guard against BUG: kernprof executed code inconsistent with sys.modules['__main__'], causing pickling issues #422. Which is strange – after all, the PR which introduced the tests also implemented the fix. The problem seems to be that somehow pip install isn't respecting the -f flag, pulling the wheel from PyPI instead of ./wheelhouse.

    • Given that the wheelhouse and a wheel of the matching version and architecture exist, IDK why this would happen... can't seem to reproduce this behavior on local with (vanilla) pipeither.
    • Unfortunately the equivalent uv command in the passing pipeline didn't produce enough output to indicate whence it pulled the line_profiler wheel, but I have to guess the wheelhouse and not PyPI judging from how the tests did pass.
  • The 3.9 [...] with tests-strict,runtime-strict jobs additionally fail the ref-counting in these four tests in tests/test_line_profiler.py:

    • test_double_decoration
    • test_function_decorator
    • test_async_gen_decorator[...]

    The cause is how the tests-strict extra pins coverage to 6.5 (see requirements/tests.txt); in my testing, anything below 7.3 consistently breaks these tests between Python 3.9 and 3.11. This bug seems consistent with what is described in CTracer fails to deallocate bound methods references coveragepy/coveragepy#1283 (reported on Python 3.8, coverage 6.1), where a couple refs are leaked when coverage is used; this however begs the question of why this adverse interaction didn't manifest when we had equivalent setups on Python 3.8? (Same discrepancy on local – 6.5 works with 3.8 but not any newer Pythons.)

  • Looking at the diffs, not only is the ARM-Linux stuff gone (regressing FIX: building Linux-ARM64 wheels #402), one of the test jobs on regular Ubuntu (3.13 on ubuntu-latest, arch=auto with tests) is also caught in the crossfire:

             - python-version: '3.13'
               install-extras: tests-strict,runtime-strict,optional-strict
               os: windows-11-arm
               arch: auto
    -        - python-version: '3.13'
    -          install-extras: tests
    -          os: ubuntu-latest
    -          arch: auto
    -        - python-version: '3.13'
    -          install-extras: tests
    -          os: ubuntu-24.04-arm
    -          arch: auto
             - python-version: '3.13'
               install-extras: tests
               os: macOS-latest
               arch: auto
  • Is there any reason why we're removing cp39-win_arm64 from cibw_skip?

    -        - '*-win32 cp3{9,10}-win_arm64 cp313-musllinux_i686'
    +        - '*-win32 cp3{10}-win_arm64 cp313-musllinux_i686'

    As described in FIX: building on Windows-ARM64 #391, we skipped these Python versions because we can't test them on GitHub's windows-11-arm runners. As far as I can tell, this constraint is still true:

    test script
    import json
    import subprocess  # I don't have requests
    
    
    # Format:
    # [{"version": ...,
    #   "stable": ...,
    #   "release_url": ...,
    #   "files": [{"filename": ...,
    #               "arch": ...,
    #               "platform": ...,
    #               "platform_version": ...,  # Optional
    #               "download_url": ...},
    #             ...]},
    #  ...]
    manifest = 'https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json'
    proc = subprocess.run(['curl', manifest], text=True, capture_output=True)
    versions = json.loads(proc.stdout)
    
    
    def is_available(python_version: str) -> bool:
        return any((version['version'].startswith(python_version)
                    and any((file['arch'].startswith('arm64')
                             and file['platform'] == 'win32')
                            for file in version['files']))
                   for version in versions)
    
    
    assert is_available('3.11')
    assert not is_available('3.10')
    assert not is_available('3.9')

    Not to mention: even if the intent was to specifically un-skip building 3.9 wheel for ARM-Windows, this form still causes issues as braces without multiple options therein isn't correctly parsed by cibuildwheel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants