Skip to content

unicodedata: Fix name, unmask test_name - #8599

Draft
joshuamegnauth54 wants to merge 1 commit into
RustPython:mainfrom
joshuamegnauth54:unicodedata-fix-test_name
Draft

unicodedata: Fix name, unmask test_name#8599
joshuamegnauth54 wants to merge 1 commit into
RustPython:mainfrom
joshuamegnauth54:unicodedata-fix-test_name

Conversation

@joshuamegnauth54

@joshuamegnauth54 joshuamegnauth54 commented Aug 26, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor
  • Closes #xxxx

One of checkbox below must be checked.

  • I did not use AI to write the code of this patch.
  • This PR follows our AI policy

Summary

  • Fix unicodedata.name and unmask its test

Summary by CodeRabbit

  • Bug Fixes
    • Unicode character lookups and name resolution now respect the selected Unicode database version.
    • Legacy Unicode views now correctly identify characters introduced after Unicode 3.2.
    • Existing error handling and default behavior remain unchanged.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 31d17f0c-85e6-4162-a447-df37c98e9a86

📥 Commits

Reviewing files that changed from the base of the PR and between 7931cb0 and 014d345.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_unicodedata.py is excluded by !Lib/**
📒 Files selected for processing (2)
  • crates/stdlib/src/unicodedata.rs
  • crates/unicode/src/data.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Walkthrough

The Unicode layer adds version-aware membership checks. The standard library now validates lookup and name results against the active Unicode database while preserving existing error and default behavior.

Changes

Unicode membership validation

Layer / File(s) Summary
Version-aware membership API
crates/unicode/src/data.rs		 membership_3_2 is publicly exported with cold-path attributes. Ucd::membership uses Unicode 3.2 data for legacy views and accepts modern Unicode characters.
Standard library membership checks
crates/stdlib/src/unicodedata.rs		 lookup and name now require active-database membership before returning results. Existing undefined-name and default handling remains in use.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 014d3

This localized change fixes Unicode name handling and restores its test coverage; no actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: youknowone, shaharnaveh

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main changes: fixing unicodedata.name and enabling the related test.
Docstring Coverage ✅ Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ShaharNaveh

ShaharNaveh commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

github is having problems again, I've tried to cancel some of the workflows to rerun them but couldn't re-run them, only cancel them.

Hopefully I'll remember to re-run them once github fixes it. ping me incase I don't:)

@ShaharNaveh ShaharNaveh reopened this Aug 26, 2026
@ShaharNaveh

ShaharNaveh commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

turning it off and on again worked:)

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] test: cpython/Lib/test/test_unicodedata.py (TODO: 27)
[x] test: cpython/Lib/test/test_unicode_file.py
[x] test: cpython/Lib/test/test_unicode_file_functions.py
[x] test: cpython/Lib/test/test_unicode_identifiers.py (TODO: 1)
[x] test: cpython/Lib/test/test_ucn.py (TODO: 4)

dependencies:

dependent tests: (no tests depend on unicode)

[ ] lib: cpython/Lib/urllib
[ ] test: cpython/Lib/test/test_urllib.py
[ ] test: cpython/Lib/test/test_urllib2.py
[x] test: cpython/Lib/test/test_urllib2_localnet.py
[x] test: cpython/Lib/test/test_urllib2net.py
[x] test: cpython/Lib/test/test_urllibnet.py
[ ] test: cpython/Lib/test/test_urlparse.py
[x] test: cpython/Lib/test/test_urllib_response.py
[ ] test: cpython/Lib/test/test_robotparser.py

dependencies:

  • urllib (native: _scproxy, email.utils, http.client, http.cookiejar, math, sys, time, unicodedata, urllib.error, urllib.parse, urllib.request, urllib.response, winreg)
    • tempfile (native: _thread, errno, sys)
    • warnings (native: _contextvars, _thread, _warnings, builtins, sys)
    • base64, bisect, collections, contextlib, email, fnmatch, ftplib, functools, hashlib, io, ipaddress, mimetypes, os, re, socket, ssl, string, types

dependent tests: (81 tests)

  • urllib: test_genericalias test_http_cookiejar test_http_cookies test_httpservers test_logging test_pathlib test_pydoc test_robotparser test_sax test_site test_sqlite3 test_ssl test_ucn test_urllib test_urllib2 test_urllib2_localnet test_urllib2net test_urllib_response test_urllibnet test_urlparse
    • email.utils: test_email test_smtplib
      • http.server: test_xmlrpc
      • logging.handlers: test_concurrent_futures test_pkgutil
      • smtplib: test_smtpnet
    • http: test_docxmlrpc test_hashlib test_httplib test_unicodedata test_wsgiref test_xml_dom_xmlbuilder
      • pydoc: test_enum
    • mimetypes: test_mimetypes
    • pathlib: test_ast test_dbm_sqlite3 test_ensurepip test_importlib test_json test_launcher test_os test_pathlib test_peg_generator test_pyrepl test_runpy test_tarfile test_tempfile test_tomllib test_tools test_traceback test_unparse test_venv test_winapi test_zipapp test_zipfile test_zoneinfo test_zstd
      • compileall: test_compileall
      • importlib: test_asdl_parser test_bdb test_cmd_line_script test_codecs test_ctypes test_doctest test_external_inspection test_frozen test_importlib test_inspect test_linecache test_modulefinder test_multiprocessing_main_handling test_py_compile test_pyclbr test_reprlib test_sundry test_support test_unittest test_zipfile test_zipimport test_zoneinfo
      • zipapp: test_pdb

[x] lib: cpython/Lib/dataclasses.py
[ ] test: cpython/Lib/test/test_dataclasses (TODO: 1)

dependencies:

  • dataclasses

dependent tests: (98 tests)

  • dataclasses: test__colorize test_copy test_ctypes test_enum test_genericalias test_patma test_pprint test_pydoc test_regrtest test_typing test_zoneinfo
    • pprint: test_htmlparser test_ssl test_sys_setprofile test_unittest
      • pdb: test_pdb
      • pickle: test_annotationlib test_argparse test_array test_ast test_asyncio test_bool test_builtin test_bytes test_bz2 test_codecs test_collections test_concurrent_futures test_configparser test_coroutines test_csv test_ctypes test_decimal test_defaultdict test_deque test_descr test_dict test_dictviews test_email test_enumerate test_exceptions test_fractions test_functools test_generators test_http_cookies test_importlib test_inspect test_io test_ipaddress test_iter test_itertools test_list test_logging test_lzma test_memoryio test_memoryview test_minidom test_opcache test_operator test_ordered_dict test_os test_pathlib test_pickle test_picklebuffer test_pickletools test_platform test_plistlib test_positional_only_arg test_posix test_random test_range test_re test_set test_shelve test_slice test_socket test_statistics test_str test_string test_structseq test_super test_time test_trace test_tuple test_turtle test_type_aliases test_type_params test_types test_unittest test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
    • pstats: test_profile test_pstats

[ ] test: cpython/Lib/test/test_descr.py (TODO: 31)
[ ] test: cpython/Lib/test/test_descrtut.py (TODO: 3)

dependencies:

dependent tests: (no tests depend on descr)

[x] test: cpython/Lib/test/test_dict.py (TODO: 4)
[x] test: cpython/Lib/test/test_dictcomps.py (TODO: 1)
[ ] test: cpython/Lib/test/test_dictviews.py (TODO: 1)
[x] test: cpython/Lib/test/test_userdict.py
[ ] test: cpython/Lib/test/mapping_tests.py

dependencies:

dependent tests: (no tests depend on dict)

[ ] test: cpython/Lib/test/test_enumerate.py

dependencies:

dependent tests: (no tests depend on enumerate)

[x] lib: cpython/Lib/compileall.py
[ ] test: cpython/Lib/test/test_compileall.py (TODO: 2)

dependencies:

  • compileall

dependent tests: (1 tests)

  • compileall: test_compileall

[x] lib: cpython/Lib/calendar.py
[ ] test: cpython/Lib/test/test_calendar.py

dependencies:

  • calendar

dependent tests: (29 tests)

  • calendar: test_calendar test_imaplib test_strftime
    • http.cookiejar: test_http_cookiejar test_urllib2
      • urllib.request: test_pathlib test_pydoc test_sax test_site test_ssl test_urllib test_urllib2_localnet test_urllib2net test_urllibnet
    • mailbox: test_genericalias test_mailbox
    • ssl: test_asyncio test_ftplib test_httplib test_httpservers test_logging test_poplib test_venv test_xmlrpc
      • asyncio.selector_events: test_asyncio
      • logging.handlers: test_concurrent_futures test_pkgutil
      • smtplib: test_smtplib test_smtpnet

[x] lib: cpython/Lib/io.py
[x] lib: cpython/Lib/_pyio.py
[ ] test: cpython/Lib/test/test_io.py (TODO: 10)
[x] test: cpython/Lib/test/test_bufio.py
[x] test: cpython/Lib/test/test_fileio.py (TODO: 1)
[ ] test: cpython/Lib/test/test_memoryio.py (TODO: 3)

dependencies:

  • io

dependent tests: (108 tests)

  • io: test__colorize test_android test_argparse test_ast test_asyncio test_base64 test_buffer test_bufio test_builtin test_bz2 test_calendar test_cmd test_cmd_line_script test_codecs test_compile test_compileall test_compiler_assemble test_concurrent_futures test_configparser test_contextlib test_csv test_dbm_dumb test_descr test_dis test_email test_enum test_file test_fileinput test_fileio test_ftplib test_generated_cases test_getpass test_gzip test_hashlib test_http_cookiejar test_httplib test_httpservers test_importlib test_inspect test_io test_json test_largefile test_logging test_lzma test_mailbox test_marshal test_memoryio test_memoryview test_mimetypes test_minidom test_multibytecodec test_optparse test_pathlib test_pdb test_peg_generator test_pickle test_pickletools test_platform test_plistlib test_pprint test_print test_profile test_pstats test_pty test_pulldom test_pydoc test_pyexpat test_pyrepl test_quopri test_regrtest test_robotparser test_sax test_shlex test_shutil test_site test_smtplib test_socket test_socketserver test_subprocess test_support test_sys test_tarfile test_tempfile test_threadedtempfile test_timeit test_tokenize test_traceback test_types test_typing test_unittest test_univnewlines test_urllib test_urllib2 test_uuid test_wave test_webbrowser test_winconsoleio test_wsgiref test_xml_dom_xmlbuilder test_xml_etree test_xml_etree_c test_xmlrpc test_xpickle test_zipapp test_zipfile test_zipimport test_zoneinfo test_zstd

[x] lib: cpython/Lib/collections
[x] lib: cpython/Lib/_collections_abc.py
[x] test: cpython/Lib/test/test_collections.py
[x] test: cpython/Lib/test/test_deque.py (TODO: 2)
[x] test: cpython/Lib/test/test_defaultdict.py
[ ] test: cpython/Lib/test/test_ordered_dict.py (TODO: 7)

dependencies:

  • collections

dependent tests: (331 tests)

  • collections: test_annotationlib test_array test_asyncio test_bisect test_builtin test_c_locale_coercion test_call test_collections test_configparser test_contains test_context test_copy test_csv test_ctypes test_defaultdict test_deque test_descr test_dict test_dictviews test_embed test_enum test_exception_group test_file test_fileinput test_fileio test_frame test_funcattrs test_functools test_genericalias test_hash test_httpservers test_inspect test_io test_ipaddress test_iter test_iterlen test_json test_logging test_math test_monitoring test_ordered_dict test_pathlib test_patma test_pickle test_plistlib test_pprint test_pydoc test_random test_reprlib test_richcmp test_set test_shelve test_sqlite3 test_statistics test_string test_struct test_sys test_traceback test_tuple test_types test_typing test_unittest test_urllib test_userdict test_userlist test_userstring test_weakref test_weakset test_with
    • ast: test_ast test_compile test_compiler_codegen test_dis test_fstring test_future_stmt test_peepholer test_peg_generator test_site test_ssl test_type_comments test_ucn test_unparse
      • annotationlib: test_grammar test_type_annotations test_type_params
      • dbm.dumb: test_dbm_dumb
      • inspect: test_abc test_argparse test_asyncgen test_buffer test_clinic test_code test_coroutines test_decimal test_generators test_ntpath test_operator test_posixpath test_signal test_turtle test_yield_from test_zipimport test_zipimport_support test_zoneinfo
      • pyclbr: test_pyclbr
      • traceback: test_asyncio test_code_module test_contextlib test_contextlib_async test_dictcomps test_exceptions test_http_cookiejar test_importlib test_listcomps test_pyexpat test_setcomps test_socket test_subprocess test_threadedtempfile test_threading test_unittest
    • asyncio: test_asyncio test_external_inspection test_os test_pdb
    • concurrent.futures._base: test_concurrent_futures
    • dbm.sqlite3: test_dbm_sqlite3
    • difflib: test_difflib test_profile test_sys_settrace
    • dis: test__opcode test_compiler_assemble test_dtrace test_opcache test_positional_only_arg test_type_cache
      • bdb: test_bdb
      • modulefinder: test_importlib test_modulefinder
      • trace: test_trace
    • email.feedparser: test_email
    • http.client: test_docxmlrpc test_hashlib test_unicodedata test_urllib2 test_wsgiref test_xmlrpc
      • urllib.request: test_sax test_urllib2_localnet test_urllib2net test_urllibnet
    • idlelib: test_idle
    • importlib.metadata: test_importlib
    • inspect:
      • cmd: test_cmd
      • dataclasses: test__colorize test_ctypes test_regrtest
      • pkgutil: test_pkgutil test_pyrepl test_runpy
      • rlcompleter: test_pyrepl test_rlcompleter
    • logging: test_support
      • hashlib: test_hmac test_smtplib test_tarfile
      • multiprocessing.util: test_compileall test_concurrent_futures
      • venv: test_venv
    • multiprocessing: test_fcntl test_memoryview test_multiprocessing_main_handling test_re
    • platform: test__locale test__osx_support test_baseexception test_cmath test_ctypes test_mimetypes test_platform test_posix test_shutil test_strptime test_sysconfig test_time test_winreg
    • pprint: test_htmlparser test_sys_setprofile
      • pickle: test_bool test_bytes test_bz2 test_codecs test_concurrent_futures test_ctypes test_email test_enumerate test_fractions test_http_cookies test_itertools test_list test_lzma test_memoryio test_minidom test_picklebuffer test_pickletools test_range test_slice test_str test_structseq test_super test_type_aliases test_unittest test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
    • queue: test_android test_dummy_thread test_sched
    • selectors: test_selectors
      • socket: test_epoll test_exception_hierarchy test_ftplib test_httplib test_imaplib test_kqueue test_largefile test_mailbox test_mmap test_poplib test_pty test_smtpnet test_socketserver test_stat test_timeout test_urllib_response
      • subprocess: test_atexit test_audit test_cmd_line test_cmd_line_script test_ctypes test_faulthandler test_file_eintr test_gc test_gzip test_json test_launcher test_msvcrt test_osx_env test_peg_generator test_poll test_py_compile test_pyrepl test_quopri test_repl test_script_helper test_select test_tempfile test_unittest test_utf8_mode test_wait3 test_webbrowser test_zipfile
    • shlex: test_shlex
    • shutil: test_filecmp test_glob test_importlib test_string_literals test_unicode_file
      • ctypes.util: test_ctypes
      • ensurepip: test_ensurepip
      • pathlib: test_importlib test_pathlib test_tomllib test_tools test_winapi test_zipapp test_zstd
      • tempfile: test_cprofile test_doctest test_generated_cases test_importlib test_linecache test_pkg test_pstats test_pyrepl test_tabnanny test_termios test_tokenize test_winconsoleio test_zipfile64
      • zipfile: test_zipfile
    • statistics:
      • random: test_complex test_devpoll test_email test_float test_grp test_heapq test_int test_long test_numeric_tower test_pow test_pwd test_queue test_sort test_strtod test_thread
    • string: test_email test_fnmatch test_pyrepl test_secrets test_string
    • threading: test_concurrent_futures test_ctypes test_fork1 test_importlib test_ioctl test_pyrepl test_robotparser test_syslog test_threading_local
      • dummy_threading: test_dummy_threading
      • sysconfig: test_asdl_parser test_tools
    • traceback:
      • timeit: test_timeit
    • tracemalloc: test_tracemalloc
    • urllib.parse: test_urlparse
    • wave: test_wave

[x] lib: cpython/Lib/codeop.py
[ ] test: cpython/Lib/test/test_codeop.py (TODO: 2)

dependencies:

  • codeop

dependent tests: (104 tests)

  • codeop: test_codeop
    • code:
      • pdb: test_pdb
      • sqlite3.main: test_sqlite3
    • traceback: test_asyncio test_builtin test_code_module test_contextlib test_contextlib_async test_coroutines test_dictcomps test_exceptions test_http_cookiejar test_importlib test_iter test_listcomps test_pyexpat test_setcomps test_socket test_ssl test_subprocess test_sys test_threadedtempfile test_threading test_traceback test_unittest test_with test_zipimport
      • concurrent.futures.process: test_compileall test_concurrent_futures
      • http.cookiejar: test_urllib2
      • logging: test_asyncio test_decimal test_genericalias test_hashlib test_logging test_pkgutil test_support test_unittest test_urllib2net
      • multiprocessing: test_asyncio test_concurrent_futures test_fcntl test_memoryview test_multiprocessing_main_handling test_re
      • py_compile: test_argparse test_cmd_line_script test_importlib test_modulefinder test_py_compile test_pydoc test_runpy
      • pydoc: test_enum
      • site: test_site
      • socketserver: test_imaplib test_socketserver test_wsgiref
      • threading: test_android test_asyncio test_bytes test_bz2 test_code test_concurrent_futures test_context test_ctypes test_docxmlrpc test_email test_external_inspection test_fork1 test_frame test_ftplib test_functools test_gc test_httplib test_httpservers test_importlib test_inspect test_io test_ioctl test_itertools test_largefile test_linecache test_opcache test_pathlib test_poll test_poplib test_pyrepl test_queue test_robotparser test_sched test_signal test_smtplib test_sqlite3 test_super test_syslog test_termios test_threading_local test_time test_urllib2_localnet test_weakref test_winreg test_xmlrpc test_zstd
      • timeit: test_timeit

[x] lib: cpython/Lib/random.py
[ ] test: cpython/Lib/test/test_random.py

dependencies:

  • random

dependent tests: (141 tests)

  • random: test_asyncio test_bisect test_buffer test_builtin test_bz2 test_collections test_complex test_context test_dbm_dumb test_decimal test_deque test_descr test_devpoll test_dict test_dummy_thread test_email test_float test_functools test_grp test_heapq test_hmac test_importlib test_int test_io test_itertools test_logging test_long test_lzma test_math test_mmap test_numeric_tower test_ordered_dict test_poll test_posixpath test_pow test_pprint test_pwd test_queue test_random test_regrtest test_richcmp test_selectors test_set test_shutil test_signal test_socket test_sort test_statistics test_strtod test_struct test_sys test_tarfile test_thread test_threading test_tokenize test_traceback test_unparse test_uuid test_weakref test_zipfile test_zlib test_zstd
    • email.generator: test_email
      • mailbox: test_genericalias test_mailbox
      • smtplib: test_smtplib test_smtpnet
    • email.utils: test_httpservers test_urllib2
      • http.server: test_robotparser test_urllib2_localnet test_xmlrpc
      • logging.handlers: test_concurrent_futures test_pkgutil
      • urllib.request: test_http_cookiejar test_pathlib test_pydoc test_sax test_site test_ssl test_urllib test_urllib2net test_urllibnet
    • imaplib: test_imaplib
    • secrets: test_secrets
    • tempfile: test_argparse test_ast test_asyncio test_bytes test_cmd_line test_compile test_compileall test_concurrent_futures test_contextlib test_cprofile test_csv test_ctypes test_dis test_doctest test_embed test_ensurepip test_faulthandler test_filecmp test_fileinput test_generated_cases test_hashlib test_importlib test_inspect test_launcher test_linecache test_modulefinder test_ntpath test_os test_peg_generator test_pickle test_pkg test_posix test_pstats test_py_compile test_pyrepl test_runpy test_shlex test_string_literals test_subprocess test_support test_sys_settrace test_tabnanny test_tempfile test_termios test_threadedtempfile test_tomllib test_turtle test_urllib_response test_venv test_winconsoleio test_zipapp test_zipfile64 test_zoneinfo
      • ctypes.util: test_ctypes
      • pdb: test_pdb
    • uuid:
      • wave: test_wave

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@joshuamegnauth54

joshuamegnauth54 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Oh wow, so it turns out that there are algorithmically derived names that are not in unicode_names2. It looks like I'll have to do some parsing.

@joshuamegnauth54
joshuamegnauth54 marked this pull request as draft August 26, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@joshuamegnauth54 @ShaharNaveh