[Help] Is there a way to detect all structs in the current crate that implement a certain trait?
from thevoidzero@lemmy.world to rust@programming.dev on 10 Sep 2024 17:00
https://lemmy.world/post/19615564

Hi all.

I want to develop a plugin system within my program, and I have a trait that functions defined by plugins should implement.

Currently, my code gets all the functions in a HashMap and then calls them by their name. Problem is, I have to create that hashmap myself by inserting every function myself.

I would really appreciate it if there was a way to say, suppose, all pub members of mod functions:: that implement this trait PluginFunction call register(hashmap) function. So as I add more functions as mod in functions it’ll be automatically added on compile.

Pseudocode:

Files:

src/
├── attrs.rs
├── functions
│   ├── attrs.rs
│   ├── export.rs
│   └── render.rs
├── functions.rs
├── lib.rs

Basically, in mod functions I want:

impl AllFunctions{
    pub fn new() -> Self {
       let mut functions_map = HashMap::new();[[
       register_all!(crate::functions::* implementing PluginFunction, &mut functions_map);
       Self { function_map }
  }
}

Right now I’m doing:

impl AllFunctions{
    pub fn new() -> Self {
       let mut functions_map = HashMap::new();[[
       crate::functions::attrs::PrintAttr{}.register(&mut functions_map);
       crate::functions::export::ExportCSV{}.register(&mut functions_map);
       crate::functions::render::RenderText{}.register(&mut functions_map);
       // More as I add more functions
       Self { function_map }
  }
}

#rust

threaded - newest

Giooschi@lemmy.world on 10 Sep 2024 18:05 next collapse

No, macros can see only the tokens you give them. They have no notion of the fact that crate::functions is a module, that PluginFunction is a trait and that functions_map is a variable. Not even the compiler may know those informations when the macro is expanded.

If you really really want to do this, you can use something like inventory. Note that inventory uses a special linker section to do this, which some consider a hack. This is also not supported on WASM if you want to target it.

thevoidzero@lemmy.world on 10 Sep 2024 18:19 collapse

Thank you. I just put the call with !, I don’t necessarily want a macro solution. Any solution is acceptable, my requirement is that I can just keep adding more mods with functions in src/functions/ and not have to register each function.

Inventory seems like the solution I am looking for. Although in my case, instead of collecting different values of the same type, I want to collect different types that all have same trait. But maybe I can make a temporary struct with Box<dyn _> member to collect it if trying to collect it directly doesn’t work. I do not plan to support WASM. I am planning to make C/C++ and Python API for the libraries though, so if it has problems with them, then I might have a problem.

taladar@sh.itjust.works on 11 Sep 2024 10:13 collapse

Maybe keep maintaining the HashMap you have now and use one of these less portable mechanisms in a test to alert you when you forgot to register one?

thevoidzero@lemmy.world on 11 Sep 2024 17:37 collapse

That seems like a good compromise if I don’t find something better. Thank you.

I’m hoping to make it easy for people to add more functions, that’s why I want minimal code change required to add more functions.

VegOwOtenks@lemmy.world on 11 Sep 2024 08:13 next collapse

You could wrap the entirety of your file in a monster macro but you’d still have to assign the macro result to a variable you need to register, which doesn’t sound viable to me at least.

Maybe you can use a script that would extract all the trait implementations and create the boilerplate glue code for you, something like this:

grep --recursive --only-matching "impl PluginFunction for \w*" functions/ | sed --quiet "s/functions\/\(.*\)\.rs:impl PluginFunction for \(\w*\)/crate::functions::\1::\2{}.register(\&mut functions_map)/p"

I tried to recreate your situation locally but it may not match perfectly, maybe you’ll have to adjust it a little. When I run it on my file tree which looks like this

functions
├── attr.rs
├── export.rs
└── render.rs

1 directory, 3 files

where every file has a content like this

// comment

pub struct MyAttrStructName {}

impl PluginFunction for MyAttrStructName {

}

Then I receive the following output:

crate::functions::attr::MyAttrStructName{}.register(&mut functions_map)
crate::functions::export::MyExportStructName{}.register(&mut functions_map)
crate::functions::render::MyRenderStructName{}.register(&mut functions_map)
thevoidzero@lemmy.world on 11 Sep 2024 17:40 collapse

Thank you. Yeah, something like this would work for me as I can add in a script and run it before compiling. But it won’t be a cross platform solution and windows/mac users are probably not going to be able to do anything. Maybe if I do the same thing but from build.rs it’ll work. I’ll try that.

trurl@lemmy.sdf.org on 11 Sep 2024 12:19 next collapse

Getting metadata like this requires adding an extension to the language, either directly in code (as with a macro) or by messing with the build process (running another pass through your code to extract the metadata).

If you’re not adverse to using a global, you could write a macro that you use instead of impl for these traits. So, for example:

register_plugin!(impl MyTrait for MyStruct {
  /* regular impl stuff goes here */
});

This macro would populate the global with metadata about the association between MyStruct and MyTrait. You could then pass a copy of it to AllFunctions::new.

Alternatively, write a macro that just does the registration, which is like what you’re already doing:

impl MyTrait for MyStruct { ... }

register_plugin!(MyTrait, MyStruct);

Another option would be to write something funky in build.rs that does whatever rustdoc does to discover impls and pass that to the rest of your code as const data. As a hack, you could have it invoke rustdoc and then parse the output.

How dynamic is this plugin system, though? If you can only change the registered plugins by rebuilding, then automatic discovery doesn’t seem so warranted. Invoking a simple registration macro at the bottom of each file would save a lot of complexity budget.

thevoidzero@lemmy.world on 11 Sep 2024 17:35 collapse

Thank you for your detailed response.

I am ok using macros. But even proc macro only get the tokens and using in on the whole mod is unstable unless you use use it on mod sth{…} instead of code being on in a different file (sth.rs).

The plug-in system is dynamic in a sense that my plans for it are loading them through shared libraries (.dll, .so) compiled separately by users. But I also have internally provided core plugins that come with the program. But rust ABI system is not that stable, so in worst case I might have to ask users to just add plugin code to some directory and re-compile program instead of loading from shared libraries. That’s why I’m trying to make it as simple as possible. Asking users to modify the rust code somewhere else yo register the plugin might be met with resistance.

I was thinking that using build script to parse the source code and generating those codes could work, but that seemed hacky. So I was trying to see if there are better solutions, as it felt like a problem people might have come across themselves.

PlexSheep@infosec.pub on 11 Sep 2024 16:16 collapse

How about parsing the source files?

thevoidzero@lemmy.world on 11 Sep 2024 17:43 collapse

Yeah, doing that in build.rs seems like a reasonable solution. Since we can’t do that using proc macros.

I don’t know if language server will keep giving me errors if the files are only linked through build.rs.