Skip to content

Rosetta: amd64 process deadlocks on fork after opening a ZeroMQ socket #2806

Open

Description

@NoahGX19

Describe the bug

Problem: Rosetta: amd64 process holding a ZeroMQ socket deadlocks on subprocess/fork (main thread futex_wait, ZMQ thread futex_lock_pi → rt_mutex_schedule), which breaks VyOS in containerlab


Summary

Inside an arm64 OrbStack Linux machine, an amd64 process that has an open ZeroMQ socket deadlocks the first time it starts a child process when it runs under Rosetta. The same script runs correctly as arm64. The hung process never creates the child, uses no CPU, and ignores SIGTERM. Only SIGKILL stops it. It reproduces every time I've run it, twice so far, with identical thread states.

The hang reproduces in about 10 lines of Python with pyzmq, with nothing else involved. I first found it because it blocks VyOS (amd64) from booting in containerlab: VyOS's vyshim helper (a ZeroMQ client) deadlocks in the same thread state during its boot-time config commit.

Environment

   
Mac MacBook Pro, Apple M5 Pro
macOS 27.0.1 (build 26A434)
OrbStack 2.2.5 (2020500), commit bf842321f95fc3ca7cda3227bf0cd41cb188c471
Linux machine Ubuntu 24.04 (noble), arm64, created with orb create ubuntu:24.04 clab
Machine kernel 7.2.12-orbstack-00394-gbecdd6130068
Container runtime (inside the machine) Docker CE 27.5.1 and containerd 2.4.2, installed with containerlab's setup script. This is not OrbStack's built-in Docker engine
Intel emulation Rosetta enabled in OrbStack settings. docker run --rm --platform linux/amd64 alpine uname -m returns x86_64
Rosetta in the hung process Mapped from /mnt/rv/[rosetta] at 0x800000000000 (see below)
Rosetta version on macOS Couldn't determine: pkgutil --pkg-info com.apple.pkg.RosettaUpdateAuto reports no receipt, and pkgutil --pkgs | grep -i rosetta returns nothing on this macOS version

binfmt_misc inside the machine (ls /proc/sys/fs/binfmt_misc/) shows three Rosetta entries: rosetta, rosetta followed by one zero-width space (U+200B), and rosetta followed by two. All three are enabled, with interpreter [rosetta] and the same magic and mask. Their flags differ: POCF!, POCF! and OCF!. They seem to appear after toggling Rosetta and restarting OrbStack. They may be unrelated, but I'm noting them in case they matter.

Steps to reproduce

All commands run inside the arm64 Ubuntu machine.

1. Create the test script

mkdir -p ~/zmqtest && cat > ~/zmqtest/t.py <<'EOF'
import zmq, subprocess
ctx = zmq.Context()
s = ctx.socket(zmq.REQ)
s.connect("ipc:///tmp/x")
for i in range(20):
    subprocess.run(["true"])
    print("spawn", i + 1, "ok", flush=True)
print("ALL OK")
EOF

2. Run it natively as arm64 (control)

timeout 300 docker run --rm --platform linux/arm64 -v ~/zmqtest:/t python:3.12-slim \
  sh -c 'pip install -q pyzmq && python /t/t.py'; echo "exit=$?"

3. Run it as amd64 (Rosetta)

timeout 300 docker run --rm --platform linux/amd64 -v ~/zmqtest:/t python:3.12-slim \
  sh -c 'pip install -q pyzmq && python /t/t.py'; echo "exit=$?"

Expected result

Both runs print spawn 1 ok through spawn 20 ok, then ALL OK, and exit with code 0.

Actual result

arm64: works.

spawn 1 ok
...
spawn 20 ok
ALL OK
exit=0

amd64 (Rosetta): pip install completes, then the script hangs on the first subprocess.run(). No spawn line is ever printed. After 300 seconds, timeout sends SIGTERM, which has no effect: docker run keeps waiting indefinitely. Only docker kill (SIGKILL) from a second shell ends it. Only then does timeout return exit=124, minutes after its own 300-second deadline.

Diagnostics from the hung process

Captured by running the amd64 case detached (docker run -d --name zmqhang ...) and inspecting it after 120 seconds. Container output at that point was empty: no spawn lines.

Process status (main thread)

State:	S (sleeping)
Threads:	3
SigBlk:	fffffffe7ffbfaef
SigIgn:	0000000001001000
SigCgt:	0000000100000412

SigBlk has almost every signal blocked, including SIGTERM. That's consistent with the main thread being stuck inside the signal-blocked critical section of process creation (CPython's subprocess spawn path), and it would explain why SIGTERM never takes effect.

Threads and wait channels

python: futex_wait
ZMQbg/Reaper: __arm64_sys_epoll_pwait
ZMQbg/IO/0: rt_mutex_schedule

Kernel stacks

-- python
[<0>] futex_wait+0x1a4/0x370
[<0>] __arm64_sys_futex+0x11c/0x320
[<0>] do_el0_svc+0x70/0xe8
[<0>] el0_svc+0x28/0x78
[<0>] el0t_64_sync_handler+0x40/0x100
[<0>] el0t_64_sync+0x154/0x158
-- ZMQbg/Reaper
[<0>] __arm64_sys_epoll_pwait+0x600/0x910
[<0>] do_el0_svc+0x70/0xe8
[<0>] el0_svc+0x28/0x78
[<0>] el0t_64_sync_handler+0x40/0x100
[<0>] el0t_64_sync+0x154/0x158
-- ZMQbg/IO/0
[<0>] rt_mutex_schedule+0x498/0x1270
[<0>] futex_lock_pi+0x2dc/0x998
[<0>] __arm64_sys_futex+0x130/0x320
[<0>] do_el0_svc+0x70/0xe8
[<0>] el0_svc+0x28/0x78
[<0>] el0t_64_sync_handler+0x40/0x100
[<0>] el0t_64_sync+0x154/0x158

So the main thread is in a plain FUTEX_WAIT, and the ZMQ I/O thread is blocked acquiring a priority-inheritance futex (futex_lock_pi). The Reaper thread is idle, which is normal.

Rosetta mappings in the hung process (confirms Rosetta is the translator, not QEMU)

800000000000-800000027000 r--p 00000000 00:25 2   /mnt/rv/[rosetta]
800000027000-80000009a000 r-xp 00027000 00:25 2   /mnt/rv/[rosetta]
8000000a0000-8000001a6000 rw-p 000a0000 00:25 2   /mnt/rv/[rosetta]

Children of the hung process: none. The child process is never created.

Kernel log (dmesg): nothing relevant around the hang; only routine docker0/veth messages.

Process tree from the first run (same result):

root  1004   977  0 09:19 ?  00:00:00  \_ sh -c pip install -q pyzmq && python /t/t.py
root  1029  1004  0 09:19 ?  00:00:00      \_ python /t/t.py

Real-world impact: VyOS in containerlab

VyOS (Stream 2026.03 and 1.5-stream-2025-Q1, both amd64 only, built per containerlab's vyosnetworks_vyos docs) never finishes booting under Rosetta. Its boot-time config commit hangs here:

python3 /usr/libexec/vyos/vyos-boot-config-loader.py /opt/vyatta/etc/config/config.boot
 \_ /opt/vyatta/sbin/my_commit
     \_ sudo sh -c /usr/sbin/vyshim /usr/libexec/vyos/conf_mode/system_host-name.py
         \_ /usr/sbin/vyshim /usr/libexec/vyos/conf_mode/system_host-name.py   (no children)
  • vyshim is a ZeroMQ client that talks to the vyos-configd daemon. It sits in futex_wait with no child processes, the same state as the Python repro.
  • vyos-configd and vyos-hostsd sit idle in ppoll. vyos-configd only ever logs Received message: {"type": "init"}, so vyshim stalls right after its first handshake, at the point where it would start a child process.
  • The result: the config never loads, the admin user is never created, and containerlab deploy waits forever.

Native arm64 network images (Arista cEOS-arm, Nokia SR Linux) work perfectly in the same machine.

Not yet tested

  • The same repro against OrbStack's built-in Docker engine from macOS, outside the Linux machine.
  • The same repro with Rosetta disabled (QEMU), to separate Rosetta-specific behaviour from emulation in general. (VyOS under QEMU fails differently and earlier: systemd inside the container reports "Transport endpoint is not connected".)

I'm happy to run either if it would help.

Possibly related

If this is a duplicate of one of the above, feel free to merge it there. I'm happy to run any extra diagnostics or test builds.

Activity

  1. orbstack-bot commented on Nov 17, 2026

    @orbstack-bot

    Hi! Please upload a diagnostic report from the OrbStack menu > Help > Upload Diagnostics (or run orb report) and then paste the output here. The extra info makes it easier for us to debug issues.

    It should look like:

    System info:
    [...]
    
    Full report: orbstack.dev/[...].zip

    (It's safe to post the link publicly — only the OrbStack team can access it.)

  2. NoahGX19 commented on Nov 17, 2026

    @NoahGX19
    Author
    OrbStack info:
      Version: 2.2.5
      Commit: bf842321f95fc3ca7cda3227bf0cd41cb188c471 (v2.2.5)
    
    System info:
      macOS: 27.0.1 (26A434)
      CPU: arm64, 18 cores
      CPU model: Apple M5 Pro
      Model: Mac17,8
      Memory: 24 GiB
    
    Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2026-11-17T18-10-43.694929Z.zip
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    t/bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions