Skip to content

unicodedata: Removed unneeded UnicodeVersion struct - #8131

Merged
youknowone merged 1 commit into
RustPython:mainfrom
joshuamegnauth54:ucd-const-version
Jun 24, 2026
Merged

unicodedata: Removed unneeded UnicodeVersion struct#8131
youknowone merged 1 commit into
RustPython:mainfrom
joshuamegnauth54:ucd-const-version

Conversation

@joshuamegnauth54

@joshuamegnauth54 joshuamegnauth54 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary

unicodedata's version can be const evaluated and doesn't need to allocate.

Summary by CodeRabbit

  • Refactor
    • Simplified unicode data handling by replacing version-based branching with a modern/legacy flag mechanism for improved clarity and maintainability.

@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

unicodedata.rs replaces the UnicodeVersion struct and UNICODE_VERSION constant with a modern: bool field on Ucd. Ucd::new now takes a bool, lookup_numeric_val accepts modern: bool, all UCD query methods branch on self.modern, and unidata_version derives its return value from that flag.

Changes

Unicode version flag refactor

Layer / File(s) Summary
Ucd struct and lookup_numeric_val signature refactor
crates/stdlib/src/unicodedata.rs
UnicodeVersion type and UNICODE_VERSION constant are removed; fmt imports are dropped; lookup_numeric_val signature changes from UnicodeVersion to modern: bool; Ucd field changes from unic_version to modern: bool; Ucd::new takes modern: bool; module_exec constructs Ucd::new(true) and ucd_3_2_0 constructs Ucd::new(false).
UCD query methods and unidata_version updated to self.modern
crates/stdlib/src/unicodedata.rs
category, bidirectional, east_asian_width, mirrored, combining, decomposition, numeric_type_matches, digit (hardcoded true), decimal, and numeric all replace unic_version comparisons with self.modern; unidata_version returns the char::UNICODE_VERSION triple when modern and "3.2.0" when legacy; module-level unidata_version uses char::UNICODE_VERSION directly.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 A boolean hops where structs once stood,
modern: bool does what version could.
No more UnicodeVersion to track,
Just true or false, no looking back!
The rabbit refactors, clean and bright,
One flag to rule the legacy plight. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.16% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main refactoring: removing the UnicodeVersion struct and replacing it with a simpler modern bool flag.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/stdlib/build.rs`:
- Around line 659-664: The escaped quotes in the println! macro's format string
are causing literal quote characters to be embedded in the RUST_UNICODE_VERSION
environment variable value. Remove the `\"` escape sequences from the format
string so that the version string contains only the numeric value without
literal quotes. The format string should output the version directly as `15.1.0`
rather than `"15.1.0"`.

In `@crates/stdlib/src/unicodedata.rs`:
- Line 486: The digit() method at line 486 is hardcoding true as the modern flag
in the lookup_numeric_val(ch, true) call, while the decimal() and numeric()
methods both use lookup_numeric_val(ch, self.modern). This inconsistency means
digit() will always use modern tables regardless of the Unicode version. Change
the hardcoded true to self.modern in the digit() method's lookup_numeric_val()
call to match the consistent behavior used in decimal() and numeric().
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: fca68cc1-521d-46c8-961b-917baeb34e36

📥 Commits

Reviewing files that changed from the base of the PR and between fe2a7db and 5a42103.

📒 Files selected for processing (2)
  • crates/stdlib/build.rs
  • crates/stdlib/src/unicodedata.rs

Comment thread crates/stdlib/build.rs Outdated
Comment thread crates/stdlib/src/unicodedata.rs
@joshuamegnauth54

Copy link
Copy Markdown
Contributor Author

I'm not sure the Windows test failed on something completely unrelated to this patch. 🤔

@ShaharNaveh

Copy link
Copy Markdown
Contributor

I'm not sure the Windows test failed on something completely unrelated to this patch. 🤔

I've restarted it

@ShaharNaveh ShaharNaveh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this cleanup!

tysm:)

Comment thread crates/stdlib/build.rs Outdated

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! reverting the build system part will make it perfect

@joshuamegnauth54 joshuamegnauth54 changed the title unicodedata: Const, embedded version unicodedata: Removed unneeded UnicodeVersion struct Jun 23, 2026
@joshuamegnauth54

Copy link
Copy Markdown
Contributor Author

Reverted. I renamed the PR to match the change as well.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/stdlib/src/unicodedata.rs (1)

533-558: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Deduplicate the version-string formatting.

The X.Y.Z formatting is repeated three times: the modern branch here, the module-level unidata_version (Line 552), and they share identical logic differing only in the version tuple. Extract the differing tuple, then format once.

♻️ Proposed refactor

Add a small helper:

fn unicode_version_string(modern: bool) -> String {
    let (major, minor, patch) = if modern {
        char::UNICODE_VERSION
    } else {
        (3, 2, 0)
    };
    format!("{major}.{minor}.{patch}")
}

Then collapse both call sites:

         #[pygetset]
         fn unidata_version(&self) -> String {
-            if self.modern {
-                format!(
-                    "{}.{}.{}",
-                    char::UNICODE_VERSION.0,
-                    char::UNICODE_VERSION.1,
-                    char::UNICODE_VERSION.2
-                )
-            } else {
-                "3.2.0".into()
-            }
+            unicode_version_string(self.modern)
         }
     #[pyattr]
     fn unidata_version(_vm: &VirtualMachine) -> String {
-        format!(
-            "{}.{}.{}",
-            char::UNICODE_VERSION.0,
-            char::UNICODE_VERSION.1,
-            char::UNICODE_VERSION.2
-        )
+        unicode_version_string(true)
     }

As per coding guidelines: "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/stdlib/src/unicodedata.rs` around lines 533 - 558, The version string
formatting logic in `X.Y.Z` format is duplicated in both the `modern` branch and
the `unidata_version` function. Create a helper function that accepts a boolean
parameter to determine whether to use modern (char::UNICODE_VERSION) or legacy
(3, 2, 0) version tuples, then format the version string once within that
helper. Replace both the conditional formatting in the `modern` branch and the
formatting in the `unidata_version` function body with calls to this new helper
to eliminate the duplication.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/stdlib/src/unicodedata.rs`:
- Around line 533-558: The version string formatting logic in `X.Y.Z` format is
duplicated in both the `modern` branch and the `unidata_version` function.
Create a helper function that accepts a boolean parameter to determine whether
to use modern (char::UNICODE_VERSION) or legacy (3, 2, 0) version tuples, then
format the version string once within that helper. Replace both the conditional
formatting in the `modern` branch and the formatting in the `unidata_version`
function body with calls to this new helper to eliminate the duplication.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 5a03474b-5f3e-4f17-8174-8cf85eb8471d

📥 Commits

Reviewing files that changed from the base of the PR and between 6315476 and 3930c0a.

📒 Files selected for processing (1)
  • crates/stdlib/src/unicodedata.rs

@youknowone
youknowone merged commit 39a2fda into RustPython:main Jun 24, 2026
26 checks passed
@joshuamegnauth54
joshuamegnauth54 deleted the ucd-const-version branch June 24, 2026 16:45
@coderabbitai coderabbitai Bot mentioned this pull request Jul 6, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants