test_codecs: IconvTest.test_encode_shift_state_flush() does crash on FreeBSD15

[vstinner]

$ ./python -m test -v test_codecs -m IconvTest
== CPython 3.16.0a0 (tags/v3.15.0b1-2175-ga7b5fc50e98:a7b5fc50e98, Oct 6 2026, 00:32:32) [Clang 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)]
== FreeBSD-15.1-RELEASE-amd64-64bit-ELF little-endian
== Python build: debug
== cwd: /home/vstinner/python/main/build/test_python_worker_21546æ
== CPU count: 6
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1356424329
0:00:00 load avg: 1.43 mem: 28.5 MiB Run 1 test sequentially in a single process
0:00:00 load avg: 1.43 mem: 28.5 MiB [1/1] test_codecs
test_decode_errors (test.test_codecs.IconvTest.test_decode_errors) ... ok
test_empty (test.test_codecs.IconvTest.test_empty) ... ok
test_encode_errors (test.test_codecs.IconvTest.test_encode_errors) ... ok
test_encode_errors_unencodable_replacement (test.test_codecs.IconvTest.test_encode_errors_unencodable_replacement) ... ok
test_encode_kinds (test.test_codecs.IconvTest.test_encode_kinds) ... ok
test_encode_shift_state_flush (test.test_codecs.IconvTest.test_encode_shift_state_flush) ...

Abort trap                 (core dumped)./python -m test -v test_codecs -m IconvTest

gdb traceback:

test_encode_shift_state_flush (test.test_codecs.IconvTest.test_encode_shift_state_flush) ... 
Program received signal SIGABRT, Aborted.
Sent by kill() from pid 21550 and user 1001.
kill () at kill.S:4
warning: 4	kill.S: No such file or directory
(gdb) where
#0  kill () at kill.S:4
#1  0x00000008009e5010 in __fail (msg=0x8008e617b "stack overflow detected; terminated") at /usr/src/lib/libc/secure/libc_stack_protector.c:119
#2  0x00000008009e4f80 in __stack_chk_fail () at /usr/src/lib/libc/secure/libc_stack_protector.c:126
#3  0x0000000802e88e7b in _ISO2022_sputwchar (ei=0x800e09b00, wc=1090541136 L'\x41005650', string=string@entry=0x7fffffffc1fa "\347\002\b", n=6, 
    result=result@entry=0x7fffffffc1f0, psenc=0x800e3c108, nresult=nresult@entry=0x7fffffffc1e8)
    at /usr/src/lib/libiconv_modules/ISO2022/citrus_iso2022.c:1115
#4  0x0000000802e87aa8 in _citrus_ISO2022_wcrtomb_priv (ei=0x542e, wc=6 L'\006', psenc=0x0, nresult=0x7fffffffc2a8, s=<optimized out>, n=<optimized out>)
    at /usr/src/lib/libiconv_modules/ISO2022/citrus_iso2022.c:1214
#5  _citrus_ISO2022_stdenc_cstomb (ce=<optimized out>, s=0x802c8b943 '\377' <repeats 23 times>, "\335", '\377' <repeats 176 times>..., n=23, 
    csid=<optimized out>, idx=<optimized out>, ps=0x0, nresult=0x7fffffffc2a8, hooks=<optimized out>) at /usr/src/lib/libc/iconv/citrus_stdenc_template.h:154
#6  0x0000000802e6a1a1 in _citrus_stdenc_cstomb (ce=0x542e, s=0x6 <error: Cannot access memory at address 0x6>, n=0, csid=12386186, idx=0, ps=0x0, 
    nresult=0x7fffffffc2a8, hooks=<optimized out>) at /usr/src/lib/libc/iconv/citrus_stdenc.h:88
#7  cstombx (se=0x800e3c0c0, s=0x6 <error: Cannot access memory at address 0x6>, n=0, csid=12386186, idx=0, nresult=0x7fffffffc2a8, hooks=<optimized out>)
    at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:122
#8  _citrus_iconv_std_iconv_convert (cv=<optimized out>, in=0x7fffffffc388, inbytes=0x7fffffffc350, out=0x7fffffffc3f0, outbytes=0x7fffffffc358, flags=0, 
    invalids=0x7fffffffc320) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:587
#9  0x0000000800970734 in _citrus_iconv_convert (cv=0x542e, cv@entry=0x800fce150, in=0x6, inbytes=0x0, inbytes@entry=0x7fffffffc350, 
    out=0x800bcff8a <getpid+10>, out@entry=0x7fffffffc3f0, outbytes=0x0, outbytes@entry=0x7fffffffc358, flags=0, nresults=0x7fffffffc320)
    at /usr/src/lib/libc/iconv/citrus_iconv.h:61
#10 __bsd_iconv (handle=0x542e, handle@entry=0x800fce150, in=0x6, szin=0x0, szin@entry=0x7fffffffc350, out=0x800bcff8a <getpid+10>, 
    out@entry=0x7fffffffc3f0, szout=0x0, szout@entry=0x7fffffffc358) at /usr/src/lib/libc/iconv/bsd_iconv.c:117
#11 0x0000000800970c1a in iconv_compat (a=0x542e, a@entry=0x800fce150, b=0x6, c=0x0, c@entry=0x7fffffffc350, d=0x800bcff8a <getpid+10>, 
    d@entry=0x7fffffffc3f0, e=0x0, e@entry=0x7fffffffc358) at /usr/src/lib/libc/iconv/iconv_compat.c:63
#12 0x00000000005ea63b in _PyUnicode_EncodeIconv (encoding=encoding@entry=0x802c4b2b8 "ISO-2022-CN-EXT", unicode=unicode@entry='ABC中DEF', errors=0x0)
    at Objects/unicodeobject.c:8544
#13 0x0000000000774d1a in _codecs_iconv_encode_impl (encoding=0x802c4b2b8 "ISO-2022-CN-EXT", str='ABC中DEF', errors=0x0, module=<optimized out>)
    at ./Modules/_codecsmodule.c:995
#14 _codecs_iconv_encode (module=<optimized out>, args=<optimized out>, nargs=2) at ./Modules/clinic/_codecsmodule.c.h:2795
(...)

Only test_encode_shift_state_flush() does crash, other tests are fine. This test is new in Python 3.16 (Python 3.15 and older are not affected).

This program reproduces the crash:

import codecs
enc = 'ISO-2022-CN-EXT'
text = 'ABC中DEF'
res = codecs.iconv_encode(enc, text)
print(repr(res))

Example:

vstinner@freebsd$ ./python x.py 
Abort trap                 (core dumped)./python x.py

cc @serhiy-storchaka

[vstinner]

Oh, I was confused since I ran tests on FreeBSD 15 and on FreeBSD 16. I seen this bug on FreeBSD 15. I cannot reproduce the bug on FreeBSD 16. It can be a bug in FreeBSD.