Detecting malicious Unicode (daniel.haxx.se)
from floofloof@lemmy.ca to programming@programming.dev on 19 May 2025 13:46
https://lemmy.ca/post/44345489

#programming

threaded - newest

Boomkop3@reddthat.com on 19 May 2025 14:27 next collapse

I have a bit of a radical approach preventing these issues: don’t allow unicode unless explicitly indicated or in contexts where it’s required or harmless

[deleted] on 19 May 2025 14:28 next collapse

.

floofloof@lemmy.ca on 19 May 2025 14:28 next collapse

That sounds good, but when you start thinking about how to implement this practically, it seems like it would either be unfeasible or would fail to really address the problem.

Maybe the rule would be: Unicode is allowed only in resource files. It would make code comments awkward for many non-English-speaking programmers. But suppose you did it, then since URLs can include Unicode, it would become normal to put URLs in resource files. If the VCS flagged up Unicode commits in source code, it would have to give resource files a pass. So in any case where you’re not hardcoding a URL it wouldn’t flag up Unicode URL abuses like the one illustrated here. You wouldn’t really have fixed the problem, just hidden it in a different way. You’d still need to flag up ambiguous Unicode characters in resource files.

Boomkop3@reddthat.com on 19 May 2025 22:31 collapse

A linter can get quite a bit more in depth than a file-to-file approach. At the very least you can use an ambigious character block list for function names

Bogasse@lemmy.ml on 20 May 2025 06:09 collapse

That’s exactly the solution from the article 👌

In the curl git repository most files and most content are plain old ASCII so we can “easily” whitelist a small set of UTF-8 sequences and some specific files, the rest of the files are simply not allowed to use UTF-8 at all as they will then fail the CI job and turn up red.

dormedas@lemmy.dormedas.com on 19 May 2025 15:05 collapse

This appears to have prompted GitHub to warn on ambiguous Unicode characters in diffs like other source control programs (namely Gitea from the article) already do. Love it.

Kissaki@programming.dev on 20 May 2025 13:21 collapse

github.blog/…/2025-05-01-github-now-provides-a-wa… since may 1st (OP is from may 16th)

<img alt="" src="https://github.com/user-attachments/assets/5da3da50-642f-485e-926d-b8c7d8741ef3">

dormedas@lemmy.dormedas.com on 20 May 2025 15:44 collapse

Ah, I see. GitHub reached out to Daniel saying “hey we’re actually doing that” not because it was a reaction to the post.

Kissaki@programming.dev on 21 May 2025 07:10 collapse

It’s possible their communication happened before that, and it just took a while to write and post OP post. 16 days is not that long or delayed. I haven’t checked the dates of the commits in question though.