aho-corasick (and thus the regex crate too) now uses SIMD on aarch64 (e.g., Apple silicon) to greatly accelerate some searches
(github.com)
from burntsushi@programming.dev to rust@programming.dev on 18 Sep 2023 21:19 +0000
https://programming.dev/post/3181902
from burntsushi@programming.dev to rust@programming.dev on 18 Sep 2023 21:19 +0000
https://programming.dev/post/3181902
#rust
Cross-posting from reddit:
The PR has more details, but here are a few ad hoc benchmarks using ripgrep on my M2 mac mini while searching a 5.5GB file.
This one is just a case insensitive search. A case insensitive regex expands to something like (ignoring Unicode)
[Ss][Hh][Ee][Rr]…
, which means that it has multiple literal prefixes. In fact, you can enumerate them! As long as the set is small enough, this is something that the new SIMD acceleration onaarch64
can handle (and has done for a long time onx86-64
):And of course, using multiple literals explicitly also uses this optimization:
And it doesn’t just work for prefixes, it also works for inner literals too:
If you’re curious about how the SIMD stuff works, you can read my description of Teddy here. I ported this algorithm out of the Hyperscan project several years ago, and it has been one of the killer ingredients for making ripgrep fast in a lot of common cases. But it only worked on
x86-64
. With the rise and popularity ofaarch64
and Apple silicon, I was motivated to port it over. I just recently finished analogous work for thememchr
crate as well.This sounds really great and will probably have quite an impact on a lot of users. So, nice work!