6
$\begingroup$

Occasionally, a new version of Unicode comes out, with new characters. This could introduce, for example, new whitespace characters, which may pose a problem for existing programs, for a number of reasons:

  1. The program may contain a previously unused Unicode character, which now has a meaning which changes how the program is parsed (e.g., it becomes a valid identifier, or a whitespace character). This seems like a small issue since undefined codepoints would likely be an error and thus not included in existing programs
  2. Programs may expect the whitespace character class, or similar, to only consist of certain characters (which could fail to be the case if the compiler isn't smart enough to downgrade its Unicode version for older programs)
  3. Programs which can properly handle updates to Unicode classes might want to receive updated Unicode lists without having to be rewritten to be compatible with a newer compiler edition (which would conflict with the previous possible issue). For example, a chat program which is designed not to allow whitespace in account names might fail to do that

What are some ways this could be worked around?

$\endgroup$
2
  • 1
    $\begingroup$ This is a real problem in Swift, where Strings are Unicode-aware and iterating over them yields extended grapheme clusters. It’s the reason Character still isn’t Codable: a valid Character to one version of Swift might look like multiple to another. $\endgroup$ Commented May 16, 2023 at 20:43
  • $\begingroup$ Problem 1 is easily solved. Simply make it a syntax error for source code to contain undefined code points. $\endgroup$ Commented May 16, 2023 at 23:16

1 Answer 1

Reset to default
9
$\begingroup$

Unicode classes are a library

For example, methods like String::to_lower_case or String::is_white_space could depend on some sort of library which provides Unicode class information in a standardized format. Thus, to use these methods, you'd need to specify which version of unicode you're using somewhere, which would allow it to be updated (or left the same) independently of the compiler or standard library version.

$\endgroup$
1
  • $\begingroup$ This seems like a good use of the Rust version scheme. Programs compiled with the 2021 version would behave towards Unicode as if none of the charpoints added after 2021 exist. $\endgroup$ Commented May 16, 2023 at 20:58

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.