Unlimited loop on trying to import the module of the filesystem encoding if iconv() is used

[vstinner]

If I run the Python test suite with the UTF-8 mode disabled and a locale which is implemented with the iconv codec, Python enters an unlimited loop trying to import encodings.<encoding> module (ex: import encodings.cp1131).

Example:

LC_ALL=be_BY.CP1131 ./python -u -X utf8=0 -m test test_sys -m xxx -v

In gdb, I get a traceback like that:

Traceback (most recent call first):
  <built-in method __import__ of module object at remote 0x8018b0710>
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 99, in search_function
    mod = __import__('encodings.' + modname, fromlist=_import_tail,
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  <built-in method __import__ of module object at remote 0x8018b0710>
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 99, in search_function
    mod = __import__('encodings.' + modname, fromlist=_import_tail,
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  <built-in method __import__ of module object at remote 0x8018b0710>
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 99, in search_function
    mod = __import__('encodings.' + modname, fromlist=_import_tail,
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  <built-in method __import__ of module object at remote 0x8018b0710>
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 99, in search_function
    mod = __import__('encodings.' + modname, fromlist=_import_tail,
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  <built-in method __import__ of module object at remote 0x8018b0710>
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 99, in search_function
    mod = __import__('encodings.' + modname, fromlist=_import_tail,
(...)

I was trying to run test_codecs.LocaleCodecTest on every supported locale on FreeBSD:

for loc in $(locale -a); do echo "== locale $loc =="; LC_ALL=$loc LANG=$loc ./python -X utf8=0 -m test -v test_codecs -m LocaleCodecTest || break; done

[vstinner]

I can reproduce this issue on FreeBSD 15 and FreeBSD 16.

Output:

$ LC_ALL=be_BY.CP1131 ./python -u -X utf8=0 -m test test_sys -m xxx -v
== CPython 3.16.0a0 (heads/main:85c03d47a24, Oct 6 2026, 00:36:37) [Clang 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)]
== FreeBSD-15.1-RELEASE-amd64-64bit-ELF little-endian
== Python build: debug
== cwd: /home/vstinner/python/main/build/test_python_worker_1567�
== CPU count: 6
== encodings: locale=CP1131 FS=cp1131
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 748962200
0:00:00 load avg: 0.16 mem: 27.3 MiB Run 1 test sequentially in a single process
0:00:00 load avg: 0.16 mem: 27.3 MiB [1/1] test_sys
Warning -- Unraisable exception
Exception ignored in the internal traceback machinery:
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/traceback.py", line 185, in _print_exception_bltin
  File "/home/vstinner/python/main/Lib/traceback.py", line 174, in print_exception
  File "/home/vstinner/python/main/Lib/traceback.py", line 1148, in __init__
  File "/home/vstinner/python/main/Lib/traceback.py", line 553, in _extract_from_extended_frame_gen
  File "/home/vstinner/python/main/Lib/traceback.py", line 422, in line
  File "/home/vstinner/python/main/Lib/traceback.py", line 400, in _set_lines
  File "/home/vstinner/python/main/Lib/linecache.py", line 26, in getline
  File "/home/vstinner/python/main/Lib/linecache.py", line 41, in getlines
  File "/home/vstinner/python/main/Lib/linecache.py", line 138, in updatecache
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function

(...... many lines ......)

  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 96, in search_function
  File "<frozen importlib._bootstrap>", line 1344, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1292, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1277, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1251, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1367, in find_spec
  File "<frozen importlib._bootstrap_external>", line 155, in _path_stat
  File "/home/vstinner/python/main/Lib/encodings/__init__.py", line 82, in search_function
RecursionError: maximum recursion depth exceeded

[vstinner]

Ah, I can reproduce the issue on Linux (Fedora 44) using:

LC_ALL=hy_AM.armscii8 LANG=hy_AM.armscii8 ./python -X utf8=0 -m test -v test_codecs -m LocaleCodecTest

So the issue is not specific to FreeBSD.

[vstinner]

cc @serhiy-storchaka

[vstinner]

On Linux, I can reproduce the issue with these locales:

• hy_AM.armscii8
• ka_GE
• ka_GE.georgianps

On FreeBSD 15, I can reproduce the issue with these locales:

• be_BY.CP1131
• hi_IN.ISCII-DEV
• hy_AM.ARMSCII-8