Skip to content

gh-153603: Fix out-of-bounds read in the ISO-2022 decoder for an unknown charset - #153604

Merged
serhiy-storchaka merged 1 commit into
python:mainfrom
tonghuaroot:iso2022-setstate-oob
Aug 17, 2026
Merged

gh-153603: Fix out-of-bounds read in the ISO-2022 decoder for an unknown charset#153604
serhiy-storchaka merged 1 commit into
python:mainfrom
tonghuaroot:iso2022-setstate-oob

Conversation

@tonghuaroot

@tonghuaroot tonghuaroot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

The ISO-2022 decoders' designation-table scan compiled its dsg->mark != '\0'
terminator only under Py_DEBUG, so a release build walked off the table for an
unknown charset set via setstate(). Make the terminator unconditional and
report the byte as undecodable when the charset is unknown.

@tonghuaroot
pythongh-153603: Fix out-of-bounds read in the ISO-2022 decoder for a…
d4a3777 
…n unknown charset

The designation-table scan compiled its terminator only under Py_DEBUG, so a
release build walked off the table for an unknown charset set via setstate().
Make the terminator unconditional and report the byte as undecodable.
@tonghuaroot
tonghuaroot requested a review from corona10 as a code owner July 12, 2026 02:04
@bedevere-app bedevere-app Bot mentioned this pull request Jul 12, 2026
Closed
@serhiy-storchaka serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Aug 17, 2026
serhiy-storchaka
serhiy-storchaka approved these changes Aug 17, 2026
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

@serhiy-storchaka
serhiy-storchaka merged commit f40043e into python:main Aug 17, 2026
67 checks passed
@miss-islington-app

miss-islington-app Bot commented Aug 17, 2026

Copy link
Copy Markdown

Thanks @tonghuaroot for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Aug 17, 2026

Copy link
Copy Markdown

GH-155959 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 17, 2026
@bedevere-app

bedevere-app Bot commented Aug 17, 2026

Copy link
Copy Markdown

GH-155960 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Aug 17, 2026
@bedevere-app

bedevere-app Bot commented Aug 17, 2026

Copy link
Copy Markdown

GH-155961 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Aug 17, 2026
serhiy-storchaka pushed a commit that referenced this pull request Aug 17, 2026
@miss-islington @tonghuaroot
[3.13] gh-153603: Fix out-of-bounds read in the ISO-2022 decoder for …
f0e236e 
…an unknown charset (GH-153604) (GH-155961)

The designation-table scan compiled its terminator only under Py_DEBUG, so a
release build walked off the table for an unknown charset set via setstate().
Make the terminator unconditional and report the byte as undecodable.
(cherry picked from commit f40043e)

Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
serhiy-storchaka pushed a commit that referenced this pull request Aug 17, 2026
@miss-islington @tonghuaroot
[3.14] gh-153603: Fix out-of-bounds read in the ISO-2022 decoder for …
a59c5bb 
…an unknown charset (GH-153604) (GH-155960)

The designation-table scan compiled its terminator only under Py_DEBUG, so a
release build walked off the table for an unknown charset set via setstate().
Make the terminator unconditional and report the byte as undecodable.
(cherry picked from commit f40043e)

Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
@bedevere-bot

bedevere-bot commented Aug 17, 2026

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable 3.13 (tier-3) has failed when building commit f0e236e.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1658/builds/1086) and take a look at the build logs.
  4. Check if the failure is related to this commit (f0e236e) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1658/builds/1086

Summary of the results of the build (if available):

Click to see traceback logs 
Note: switching to 'f0e236e50e1b7a7752fbb80441ceec7034fd8718'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f0e236e50e1b7 [3.13] gh-153603: Fix out-of-bounds read in the ISO-2022 decoder for an unknown charset (GH-153604) (GH-155961)
Switched to and reset branch '3.13'

In file included from ./Include/Python.h:14,
                 from ./Modules/_testexternalinspection.c:53:
./pyconfig.h:1985:9: warning: ‘_POSIX_C_SOURCE’ redefined
 1985 | #define _POSIX_C_SOURCE 200809L
      |         ^~~~~~~~~~~~~~~
In file included from /usr/include/bits/libc-header-start.h:33,
                 from /usr/include/stdint.h:26,
                 from /usr/lib/gcc/s390x-redhat-linux/16/include/stdint.h:11,
                 from /usr/include/elf.h:24,
                 from ./Modules/_testexternalinspection.c:4:
/usr/include/features.h:319:10: note: this is the location of the previous definition
  319 | # define _POSIX_C_SOURCE        202405L
      |          ^~~~~~~~~~~~~~~
./pyconfig.h:2006:9: warning: ‘_XOPEN_SOURCE’ redefined
 2006 | #define _XOPEN_SOURCE 700
      |         ^~~~~~~~~~~~~
/usr/include/features.h:234:10: note: this is the location of the previous definition
  234 | # define _XOPEN_SOURCE  800
      |          ^~~~~~~~~~~~~
ar: libpython3.13d.a: error reading Objects/enumobject.o: No space left on device
make: *** [Makefile:1065: libpython3.13d.a] Error 1

@bedevere-bot

bedevere-bot commented Aug 17, 2026

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable 3.14 (tier-3) has failed when building commit a59c5bb.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1797/builds/1301) and take a look at the build logs.
  4. Check if the failure is related to this commit (a59c5bb) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1797/builds/1301

Summary of the results of the build (if available):

Click to see traceback logs 
Note: switching to 'a59c5bb023fd74a3c482fc29bd6cefc32c56b531'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at a59c5bb023fd7 [3.14] gh-153603: Fix out-of-bounds read in the ISO-2022 decoder for an unknown charset (GH-153604) (GH-155960)
Switched to and reset branch '3.14'

/tmp/ccMV0MYw.s: Assembler messages:
/tmp/ccMV0MYw.s: Fatal error: Python/Python-ast.o: No space left on device
make: *** [Makefile:3306: Python/Python-ast.o] Error 1
make: *** Waiting for unfinished jobs....
/tmp/cc5Fgl7X.s: Assembler messages:
/tmp/cc5Fgl7X.s: Fatal error: can't write 3952 bytes to section .debug_info of Python/sysmodule.o: 'No space left on device'
/tmp/cc5Fgl7X.s: Fatal error: Python/sysmodule.o: No such file or directory
make: *** [Makefile:2136: Python/sysmodule.o] Error 1
/tmp/ccdMWq9Z.s: Assembler messages:
/tmp/ccdMWq9Z.s: Fatal error: can't write 1020 bytes to section .text of Objects/unicodeobject.o: 'No space left on device'
/tmp/ccdMWq9Z.s: Fatal error: Objects/unicodeobject.o: No such file or directory
make: *** [Makefile:3306: Objects/unicodeobject.o] Error 1
Assembler messages:
Fatal error: can't create Python/ceval.o: No space left on device
make: *** [Makefile:3316: Python/ceval.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:3398: clean-retain-profile] Error 1 (ignored)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes
@corona10 corona10 Awaiting requested review from corona10 corona10 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tonghuaroot @bedevere-bot @serhiy-storchaka