Inversion of Control with plugins?
-
@pie_flavor said in The Official Status Thread:
@Tsaukpaetra program A has plugin compatibility. so any DLLs in a particular folder, including DLL B, get loaded at runtime and a particular exported symbol is called. This is an inverse of how DLLs are supposed to work; the program provides the library functions to the DLL instead of the other way around. This is fine on non-Windows systems, because externally-defined functions can be referenced even if you don't specify the DLL name as long as the function exists at runtime. Windows however does not have an 'amorphous blob' model of functions, and requires that you say what DLL each function comes from. This means that this plugin compatibility model is impossible in Windows.
Unless of course you pass a struct to that particular exported symbol containing all your library function pointers. Which is the solution I'm currently running with.Taking this out of the Status Thread because I'm genuinely curious.
So what you're saying is, your program loads a DLL, and that DLL is normally expected to be told what functions the loading program is allowed to call it? Or...?
Because I'm incredibly confused still.
When I was making a plugin system for my defunct IRC chatbot, it simply loaded the DLL, checked if it had implemented any classes that implemented the plugin interface, and (since it knew therefore that the classes implemented the interface) could then call the functions it expects.
In here, it seems like the main program is expected to call arbitrary functions at will and discover functionality at runtime?
-
@Tsaukpaetra I'm writing a Rust plugin for a C program.
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra I'm writing a Rust plugin for a C program.
Short, simple, and completely opaque. Sweet!
Sooo......?
-
@Tsaukpaetra So you're missing the part where you link the functions that the plugin is able to call.
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra So you're missing the part where you link the functions that the plugin is able to call.
I guess? I don't know?
What are you linking against? I'm still utterly confused, and you're evasion isn't helping.
-
@Tsaukpaetra Nothing. That's the point. There is no way to say 'these functions will exist when an executable dynamically loads me'.
-
@pie_flavor said in Inversion of Control with plugins?:
Nothing. That's the point.
@pie_flavor said in Inversion of Control with plugins?:
There is no way to say 'these functions will exist when an executable dynamically loads me'.
So, you do know what the functions are, and where to call them, but are... what? Trying to leave the actual location unknown for the purposes of....?
I'm still utterly confused. Maybe stop repeating yourself and try explaining it a different way, please?
-
@Tsaukpaetra foo.exe loads bar.dll. For bar.dll to effectively plug in to foo.exe, foo.exe must provide an API. Yes?
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra foo.exe loads bar.dll. For bar.dll to effectively plug in to foo.exe, foo.exe must provide an API. Yes?
Not necessarily.
-
@Tsaukpaetra Then what the hell is the plugin doing?
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra Then what the hell is the plugin doing?
Whatever the loading program asks it to. And reacts to whatever the loading program tells it. And responds to whatever the loading program asks it.
edit: I did mention this in the OP, by the way.
-
@Tsaukpaetra How does it do anything, though, if it has no API to call?
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra How does it do anything, though, if it has no API to call?
I don't understand the question.
-
@Tsaukpaetra let's say foo.exe is an email program. If bar.dll can't do something like load email, what's the point of plugins in the first place? So foo.exe must provide a way for bar.dll to read an email.
-
@pie_flavor said in Inversion of Control with plugins?:
If bar.dll can't do something like load email
And why can't it? What is preventing this functionality from happening?
@pie_flavor said in Inversion of Control with plugins?:
So foo.exe must provide a way for bar.dll to read an email.
Sure. bar.dll knows it should implement a reception function (I'll call it a function delegate) for foo.exe to call when interaction with an email is needed. Since bar.dll knows the structure and other data that will be provided it, it can read the email just fine when foo.exe calls the delegate. (crude example, natch).
If you desire more, then foo.exe provide a pointer that bar.dll can use to access known functions (i.e. from an included header file) from.
This doesn't seem strange to me.
-
@Tsaukpaetra said in Inversion of Control with plugins?:
If you desire more, then foo.exe provide a pointer that bar.dll can use to access known functions (i.e. from an included header file) from.
Right. Now, the problem here, is that the known functions must then be linked against. And you can't do that without the library name on Windows.
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra said in Inversion of Control with plugins?:
If you desire more, then foo.exe provide a pointer that bar.dll can use to access known functions (i.e. from an included header file) from.
Right. Now, the problem here, is that the known functions must then be linked against. And you can't do that without the library name on Windows.
My question is thusly: Why don't you know? How would you call a function you don't know exists?
-
@Tsaukpaetra said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra said in Inversion of Control with plugins?:
If you desire more, then foo.exe provide a pointer that bar.dll can use to access known functions (i.e. from an included header file) from.
Right. Now, the problem here, is that the known functions must then be linked against. And you can't do that without the library name on Windows.
My question is thusly: Why don't you know? How would you call a function you don't know exists?
Why don't I know what?
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra said in Inversion of Control with plugins?:
If you desire more, then foo.exe provide a pointer that bar.dll can use to access known functions (i.e. from an included header file) from.
Right. Now, the problem here, is that the known functions must then be linked against. And you can't do that without the library name on Windows.
My question is thusly: Why don't you know? How would you call a function you don't know exists?
Why don't I know what?
@pie_flavor said in Inversion of Control with plugins?:
the library name
-
@Tsaukpaetra There is no library name. It's foo.exe. You can't link functions against an executable.
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra There is no library name. It's foo.exe. You can't link functions against an executable.
No, you link functions as provided in the header file for the program you're supposedly having load your plugin.
Again, what am I missing here?
-
@Tsaukpaetra said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra There is no library name. It's foo.exe. You can't link functions against an executable.
No, you link functions as provided in the header file for the program you're supposedly having load your plugin.
Again, what am I missing here?
You are missing that linking isn't something you do to a header file. It's something that gets done to object code. foo_plugin.h has the functions you're looking for, but a header file isn't a link target. You cannot link against a function on Windows unless you specify what DLL you're loading it from. In this instance, there is no DLL - the function is being provided by the executable.
-
@pie_flavor said in Inversion of Control with plugins?:
the function is being provided by the executable.
How though? By what mechanism? The dll is loading is somehow (as you just said). How is it to load it?
Unless, as I said, the plugin is provided a pointer (let's call it an entry point) to the loading program, which we know the locations of functions of (thanks to the header file) and therefore can call them.
You're still not telling me how this magic of "providing" is happening, by the way.
-
@Tsaukpaetra said in Inversion of Control with plugins?:
which we know the locations of functions of (thanks to the header file) and therefore can call them.
No.
You are missing that linking isn't something you do to a header file
A header file tells you the signature of a function. It doesn't tell you it will be at address 0xDEADB33F in the final assembled executable.
-
@Tsaukpaetra said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
the function is being provided by the executable.
How though? By what mechanism? The dll is loading is somehow (as you just said). How is it to load it?
Unless, as I said, the plugin is provided a pointer (let's call it an entry point) to the loading program, which we know the locations of functions of (thanks to the header file) and therefore can call them.
You're still not telling me how this magic of "providing" is happening, by the way.
Once more with feeling!
I am not talking about the plugin being passed control. That is very easy. I am talking about the plugin calling functions from the program.// plugin.h char **foo_read_email(); void foo_msg_box(const char *msg);On Linux, my plugin can simply be built using this header, and then when the program loads the plugin, it can call those functions. On Windows, my plugin cannot be built using this header because the linker demands to know what DLL those functions are in, and there isn't one.
-
@Zecc said in Inversion of Control with plugins?:
A header file tells you the signature of a function. It doesn't tell you it will be at address 0xDEADB33F in the final assembled executable.
Hmm. Okay. Obviously I don't c very well.
@pie_flavor said in Inversion of Control with plugins?:
then when the program loads the plugin, it can call those functions.
BUT HOWWWWW
@pie_flavor said in Inversion of Control with plugins?:
because the linker demands to know what DLL those functions are in, and there isn't one.
So stub one out for appearances sake? After all, if the name it's supposed to import from is (apparently) unnecessary, then having a stub library to fake out the linker should be easy enough, no?
-
@Tsaukpaetra said in Inversion of Control with plugins?:
@Zecc said in Inversion of Control with plugins?:
A header file tells you the signature of a function. It doesn't tell you it will be at address 0xDEADB33F in the final assembled executable.
Hmm. Okay. Obviously I don't c very well.
@pie_flavor said in Inversion of Control with plugins?:
then when the program loads the plugin, it can call those functions.
BUT HOWWWWW
what the fuck are you talking about
@pie_flavor said in Inversion of Control with plugins?:
because the linker demands to know what DLL those functions are in, and there isn't one.
So stub one out for appearances sake? After all, if the name it's supposed to import from is (apparently) unnecessary, then having a stub library to fake out the linker should be easy enough, no?
Right, and then it'll say the stub library isn't present so the plugin library couldn't be loaded.
-
Random people who have found my library intrinsically understand it better than @Tsaukpaetra who develops on Windows for a living and I've been trying to explain this to for ten posts.
<Arnavion> pie_flavor: Is the "damn you MSVC" commit because MSVC's linker complains that it can't find the the hexchat functions to link to?
-
@pie_flavor said in Inversion of Control with plugins?:
who develops on {0} for a living
Understanding has never been a prerequisite for that.
Honestly, I've read this thread from either end to another... and neither can I.
-
@pie_flavor The way you do it is this:
Provide a function (with some sort of well known name and signature) in bar.dll that foo.exe can call to provide it with the ABI it can call. The ABI it can call is described via a pointer to a struct of function pointers. Once foo.exe has loaded bar.dll, it looks up that function and calls it with the table of functions it wants to expose, and that lets bar.dll do what it needs to without binding library names or shit like that. In practice, you write a bit of tooling to make this stuff easier so that you don't have to write out declarations of things multiple times, and maybe you make a small static library that is used to hide the details of provisioning the table and calling through it.
There is software out there that does this. It's a bit more complex, but makes plugins work well even on Windows. (The complexity isn't so needed on more POSIXy platforms, where dynamic libraries can export symbols to the global namespace. But it's still a good idea for them to work via the mechanism described above.)
-
@dkf Yes, exactly. That's the solution I said I was using.
-
@pie_flavor said in Inversion of Control with plugins?:
@dkf Yes, exactly. That's the solution I said I was using.
I don't recall you ever saying that.
Thanks @dkf, I finally understand what they were talking about!
-
@kt_ it was in another thread.
-
This thread takes me back to a job that had 3 hour long "scrum" meetings full of these kind of discussions. Only the CTO had a short fuse so there was a lot of loud screaming.
I hated that job.
-
-
@kt_ said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
@dkf Yes, exactly. That's the solution I said I was using.
I don't recall you ever saying that.
Thanks @dkf, I finally understand what they were talking about!
He mentioned it in a one-liner in the status thread.
-
@dkf said in Inversion of Control with plugins?:
The way you do it is this:
This is what I was expecting. I'm still asking @pie_flavor how his idea is supposed to work, and have yet to receive a comprehensible answer.
-
@kt_ said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
@dkf Yes, exactly. That's the solution I said I was using.
I don't recall you ever saying that.
You have to have experience and a good understanding of dynamic linking on Windows to have connected the dots.
Thanks @dkf, I finally understand what they were talking about!
Yes, his explanation included a lot more explicit detail.
-
@pie_flavor said in Inversion of Control with plugins?:
On Windows, my plugin cannot be built using this header because the linker demands to know what DLL those functions are in, and there isn't one.
You do realize you can use GetProcAddress on an EXE...
-
@dcon said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
On Windows, my plugin cannot be built using this header because the linker demands to know what DLL those functions are in, and there isn't one.
You do realize you can use GetProcAddress on an EXE...
That will give you the address at runtime, but what @pie_flavor wants is for the address to be known at compile time akin to linking in a static library.
From a quick search I found this: https://stackoverflow.com/a/30475042/1959975
-
I don't want to read Abbot and Costello at 80pt, so is it fair to assume that what @pie_flavor is complaining about is that most Windows compilers neither emit a .lib/.def file nor put entries in the export table when building .EXEs? And that all the other dynamic dispatch systems such as COM, RMI, and good old pass-a-pointer-to-a-callback-function-or-vtable aren't available in the Rust universe?
-
@TwelveBaud in a word: no.
Edit: still waiting on that answer from @pie_flavor
-
@pie_flavor That's how I understand it. In my program (from the Square reader thread), I have plug-ins for taxes and fees. The program provides an interface. The plug-in DLLs provide implementations of that interface. At runtime, the program scans the plug-in folder for DLLs, looks for the interface via reflection, and returns objects that match. I thought that was how plug-ins worked (although I've never done it in C where it's probably far more difficult).
-
@Zenith said in Inversion of Control with plugins?:
@pie_flavor That's how I understand it. In my program (from the Square reader thread), I have plug-ins for taxes and fees. The program provides an interface. The plug-in DLLs provide implementations of that interface. At runtime, the program scans the plug-in folder for DLLs, looks for the interface via reflection, and returns objects that match. I thought that was how plug-ins worked (although I've never done it in C).
That's how I understand plugins to work. But somehow, @pie_flavor has a different method he hasn't explained to me that's not what you describe.
-
@Tsaukpaetra said in Inversion of Control with plugins?:
@TwelveBaud in a word: no.
Edit: still waiting on that answer from @pie_flavor
I already responded to you.
-
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra said in Inversion of Control with plugins?:
@TwelveBaud in a word: no.
Edit: still waiting on that answer from @pie_flavor
I already responded to you.
Responding to a request with a request is not really an acceptable response.
Let's redo from start:
You claim the following (paraphrased):
- A plugin can define a pointer to a pointer to a function in itself.
- The loading program doesn't pass control to the plugin.
- {Magic}
- The plugin can use the pointer-to-a-pointer-to-a-function and call it at will, and this somehow leads back to the loading program to do things.
I'm asking you to tell me what the {magic} is.
You've said, "the function is being provided by the executable." to which I asked "How though? By what mechanism?", which you then replied with, "when the program loads the plugin, it can call those functions." which didn't answer the question whatsoever.
It's like if I asked "How does the garage door open?" and you say "The way the garage door gets opened."
Are you comprehending the question now?
-
@Tsaukpaetra said in Inversion of Control with plugins?:
@pie_flavor said in Inversion of Control with plugins?:
@Tsaukpaetra said in Inversion of Control with plugins?:
@TwelveBaud in a word: no.
Edit: still waiting on that answer from @pie_flavor
I already responded to you.
Responding to a request with a request is not really an acceptable response.
Let's redo from start:
You claim the following (paraphrased):
- A plugin can define a pointer to a pointer to a function in itself.
I have not said this.
- The loading program doesn't pass control to the plugin.
I have not said this.
- {Magic}
I have not said this.
- The plugin can use the pointer-to-a-pointer-to-a-function and call it at will, and this somehow leads back to the loading program to do things.
I have not said this.
I'm asking you to tell me what the {magic} is.
Beats me. What magic?
You've said, "the function is being provided by the executable." to which I asked "How though? By what mechanism?", which you then replied with, "when the program loads the plugin, it can call those functions." which didn't answer the question whatsoever.
It's like if I asked "How does the garage door open?" and you say "The way the garage door gets opened."
Are you comprehending the question now?
Do you understand what a provided function is? Are you familiar with the concept that Thing A can declare
int foo()and Thing B can then callfoo()?
-
@pie_flavor never become a teacher
-
@pie_flavor said in Inversion of Control with plugins?:
I have not said this.
@Tsaukpaetra said in Inversion of Control with plugins?:
(paraphrased)
Fuck you.
@pie_flavor said in Inversion of Control with plugins?:
Do you understand what a provided function is?
No! I've asked no less than four times for you to tell me!
Are you familiar with the concept that Thing A can declare
int foo()and Thing B can then callfoo()?Sure. And I already explained (twice, if I'm counting correctly) how I expect this to be done. But this does not match what you say is being done. Hence my request for more information.
-
@Tsaukpaetra if I understand correctly, he has a plug-in that, in order to provide whatever functionality it is supposed to provide, uses functionality of the executable itself.
So, you have the function frobnicate in the executable, which the executable happily calls whenever it needs to frobnicate something. The plug-in apparently also needs to frobnicate things, and instead of writing it's own version of the function wants to use the version in the executable.
So then at runtime, the executable loads the plug-in, tells the plug-in to do its thing, and the plug-in calls frobnicate. I find this horrible, since it means there's a cyclical dependency between the plug-in and the executable, as they depends on each other, and also the plug-in has knowledge of the internals of the executable, but it's not technically wrong. I might order this by extracting frobnicate into a separate shared library that both the plug-in and the executable depend on but it might not be possible for his project.
The thing is, in Unix land an elf executable and shared library are basically the same thing, so you can link to an executable just as easily as you can to a shared library. Normally, if you write a shared library that depends on another, and you get loaded by a process, the OS will look to see if the library is already loaded, load it ahead of you if not, then load your library and you're done. If you depend on an executable and the executable itself loads you, the OS does the same thing, only obviously the executable is already loaded.
Funnily enough, if you depend on executable A and executable B loads you, executable A will also get loaded. But you better hope the executable was compiled as a relocatable executable, or bad things may happen.
Apparently on Windows this is not so easy. I'm not familiar enough with linking on Windows to know what step is broken there. But then, Windows never dealt with shared libraries very well.
