IMHO this is one of the longstanding feature requests for Dolphin Smalltalk (the other being Unicode support :-)) .
Up to now it was not possible to create normal DLLs in Dolphin. The only thing you could do in Dolphin was creating COM DLLs. But where is the difference between a COM DLL and a normal DLL.
Normal DLL
A normal DLL is a PE module which exports functions. This means its export section contains a list of export function names which point to function ordinals - and these ordinals point to (imagebase-relative) function addresses. After loading a DLL you can get it's address by GetProcAddress(). GetProcAddress does nothing else than trying to find the provided name in the export names table and - if successfull - resolves the address by using the ordinal reference and add the base address (to get an absolute address). However ...
... Dolphin is not able to create those export tables. This is (IMHO) due to the fact that the export table is part of the DLL stub that Dolphin provides. And except for a few minor changes this stub remains unchanched during deployment. Even if Dolphin could change the export table you'd still have the problem, that the "functions" would actually be Dolphin Callbacks ... and the address of those callbacks are not guaranteed to remain constant.
So what is Dolphin able to do?
COM DLL
Dolphin is able to create so called "COM Server DLLs". These COM DLLs are DLLs which "export" COM Objects by using a few predefined regular DLL export functions:
DllRegisterServer(), DllUnregisterServer(), DllCanUnloadNow(), DllGetClassObject().
The first two functions are called to (un-)register the control(s) (and type libs) in the registry. The important stuff is the last function. This is the base to get a COM "Class Factory" which is able to create the COM Instances in the DLL. On the image side this is realized by the IIPDolphin interface (implemented by IPDolphin) and the IIPPlughole Interface.
Dolphin COM DLL/Image interaction
If a process loads a Dolphin COM DLL the embedded image is not started automatically. This only happens once the first COM Call hits the DLL (i.e. via DllGetClassObject()). In this case the image is started and (after some setup) the current IPDolphin's #onStartup method is called. This method queries the current DLL's IIPPlugHole interface. This IPPlugHole Interface is then used to inform the DLL about the current IPDolphin instance.
At this stage the DLL has a (COM Interface) Pointer to the IPDolphin instance. This is then used to forward all the COM stuff into the image and thus allow the image to do most of the COM stuff.
The nice thing about this scheme is that except for a few very low level things most of the COM Implementation is in the image - not in the VM.
You said Regular DLLs! I want them!
So how to do "real" DLLs in Smalltalk. As we already saw it not possible to change the export table - but without export table no "real" DLL.
The trick is to alter the export table after the DLL has been loaded. I found an article dealing with un-exporting a function after the DLL has been called. So I thought it must also be possible to do it the other way around.
To make a long story short: It works.
A bit longer: You have to find the module base address (not that easy ... unless you know how :-) ) and from there walk through the different PE headers until you finally hit a IMAGE_EXPORT_DIRECTORY. This is a structure which basically points to arrays of names, ordinals and (relative) function addresses. So all you have to do is to build new new arrays (including your own names and their associated ExternalCallbacks) and change the struct ... and as I said. It works .... kind of.
What do you mean with "works .... kind of"?
There are two things to be aware of:
When does the export table gets modified?
The export Table of the DLL, as long as it is stored on the disk, does not contain the modified Export Table. It's only modified once the image (at a very early stage) has run. However it seems (PLEASE CORRECT ME if I'm wrong!!!!) that the image is not started until the first COM call hits the DLL. And you can't expect third party apps to first do a COM call to your library before resolving exported functions.
So let me state this as clear as possible: If you know a way to execute Smalltalk code in a DLL w/o any COM stuff going on or even better execute Smalltalk code in any event once the lib gets loaded ... PLEASE tell me!!!! I'll do/did the rest.
Dolphin Smalltalk X6.1 beta does not support DLL deployment!
The current beta is missing the DLL Deployment capability (DLL Stubs are missing). So I did all the testing in 6.0.
Summary
Modifing a modules export table is the way to go. This should cover all use cases of DLL symbols being resolved dynamically (via GetProcAddress()). However I'm quite sure this even works if the dynamic linker does it. And the best thing is ... it's ready!!!!
So if someone can provide me with a way to get my Smalltalk code running in a DLL in any case as early as possible we will be able to have regular DLLs in Smalltalk. And the great thing is that it's all in Smalltalk - no VM suport needed ("it's turtles all the way down ...").
Up to now it was not possible to create normal DLLs in Dolphin. The only thing you could do in Dolphin was creating COM DLLs. But where is the difference between a COM DLL and a normal DLL.
Normal DLL
A normal DLL is a PE module which exports functions. This means its export section contains a list of export function names which point to function ordinals - and these ordinals point to (imagebase-relative) function addresses. After loading a DLL you can get it's address by GetProcAddress(). GetProcAddress does nothing else than trying to find the provided name in the export names table and - if successfull - resolves the address by using the ordinal reference and add the base address (to get an absolute address). However ...
... Dolphin is not able to create those export tables. This is (IMHO) due to the fact that the export table is part of the DLL stub that Dolphin provides. And except for a few minor changes this stub remains unchanched during deployment. Even if Dolphin could change the export table you'd still have the problem, that the "functions" would actually be Dolphin Callbacks ... and the address of those callbacks are not guaranteed to remain constant.
So what is Dolphin able to do?
COM DLL
Dolphin is able to create so called "COM Server DLLs". These COM DLLs are DLLs which "export" COM Objects by using a few predefined regular DLL export functions:
DllRegisterServer(), DllUnregisterServer(), DllCanUnloadNow(), DllGetClassObject().
The first two functions are called to (un-)register the control(s) (and type libs) in the registry. The important stuff is the last function. This is the base to get a COM "Class Factory" which is able to create the COM Instances in the DLL. On the image side this is realized by the IIPDolphin interface (implemented by IPDolphin) and the IIPPlughole Interface.
Dolphin COM DLL/Image interaction
If a process loads a Dolphin COM DLL the embedded image is not started automatically. This only happens once the first COM Call hits the DLL (i.e. via DllGetClassObject()). In this case the image is started and (after some setup) the current IPDolphin's #onStartup method is called. This method queries the current DLL's IIPPlugHole interface. This IPPlugHole Interface is then used to inform the DLL about the current IPDolphin instance.
At this stage the DLL has a (COM Interface) Pointer to the IPDolphin instance. This is then used to forward all the COM stuff into the image and thus allow the image to do most of the COM stuff.
The nice thing about this scheme is that except for a few very low level things most of the COM Implementation is in the image - not in the VM.
You said Regular DLLs! I want them!
So how to do "real" DLLs in Smalltalk. As we already saw it not possible to change the export table - but without export table no "real" DLL.
The trick is to alter the export table after the DLL has been loaded. I found an article dealing with un-exporting a function after the DLL has been called. So I thought it must also be possible to do it the other way around.
To make a long story short: It works.
A bit longer: You have to find the module base address (not that easy ... unless you know how :-) ) and from there walk through the different PE headers until you finally hit a IMAGE_EXPORT_DIRECTORY. This is a structure which basically points to arrays of names, ordinals and (relative) function addresses. So all you have to do is to build new new arrays (including your own names and their associated ExternalCallbacks) and change the struct ... and as I said. It works .... kind of.
What do you mean with "works .... kind of"?
There are two things to be aware of:
When does the export table gets modified?
The export Table of the DLL, as long as it is stored on the disk, does not contain the modified Export Table. It's only modified once the image (at a very early stage) has run. However it seems (PLEASE CORRECT ME if I'm wrong!!!!) that the image is not started until the first COM call hits the DLL. And you can't expect third party apps to first do a COM call to your library before resolving exported functions.
So let me state this as clear as possible: If you know a way to execute Smalltalk code in a DLL w/o any COM stuff going on or even better execute Smalltalk code in any event once the lib gets loaded ... PLEASE tell me!!!! I'll do/did the rest.
Dolphin Smalltalk X6.1 beta does not support DLL deployment!
The current beta is missing the DLL Deployment capability (DLL Stubs are missing). So I did all the testing in 6.0.
Summary
Modifing a modules export table is the way to go. This should cover all use cases of DLL symbols being resolved dynamically (via GetProcAddress()). However I'm quite sure this even works if the dynamic linker does it. And the best thing is ... it's ready!!!!
So if someone can provide me with a way to get my Smalltalk code running in a DLL in any case as early as possible we will be able to have regular DLLs in Smalltalk. And the great thing is that it's all in Smalltalk - no VM suport needed ("it's turtles all the way down ...").
11 comments:
You must be proud of your achievements!
But what is the point making dll without speed boost?
The reason (for me) is the same as for COM DLLs - the ability to provide "embeddable" Smalltalk solutions.
Take license generators/checkers for an example. I have all the licensing stuff in Smalltalk. The CRM system we are using is able to autogenerate licenses once a order comes in. However it expects one DLL per product with a specified exported function.
Post a Comment