Static data in a shared library



  • Hi all,

     I'm writing a theming engine (shared library) and I want the loaded bitmaps to be shared among all programs that use this library. I'm writing this library in C++ (don't worry, I use "extern "C"{}" for exported functions).

     
    Can anyone give any pointers or hints on this task?

    P.S: I googled around and the closest thing was a post in a comp.lang.COBOL!



  • @CapitalT said:

    I want the loaded bitmaps to be shared among all programs that use this library.

    First question: why?

    You might consider your platform of choice's shared memory segment APIs.  On linux and unix, that's mmap and friends; on Windows, you'll have to dig through MSDN.  Static data in shared libraries is frequently per-program by default, and making it per-application can introduce unexpected security problems.  On Windows, consider http://blogs.msdn.com/oldnewthing/archive/2004/08/04/208003.aspx as a warning against shared variables and in favour of explicit shared-segment management.


  • It's a theming engine, a button bitmap is the same in all applications. And loading it for each one is a very huge waste of resources.



  • Have you measured how huge?

    I'm not disagreeing with you, exactly.  It's been my experience that guessing and estimating what parts of an application are slow or wasting memory is much, much harder than it appears.  Even very smart people get it completely wrong on a regular basis.  So before you add a bunch of complexity to your library for managing shared segments, make sure it's actually worth it, not just "obviously" worth it.
     



  • Let's say that the whole operating system use it.



  • Sharing data between multiple processes is why they are called "shared libraries" in the first place. Compile the relevant data into a shared library and load it. Note that it must be shareable, which (for a library) is the opposite of writeable - on ELF platforms, that means .rodata rather than .data (other platforms have equivalents). Persuading your compiler how to do this reliably is a bit subtle and far too long-winded for this forum.

    Generally this can be accomplished by judicious use of a bitmap-to-C compiler and a wizard. Be warned that meddling in the affairs of wizards is inadvisable, because it makes them soggy and hard to light. 



  • The bitmaps are dynamically loaded, they are not static. I just want all the applications importing the drawing functions to use the already loaded and formatted bitmaps.


     



  • @CapitalT said:

    The bitmaps are dynamically loaded, they are not static. I just want all the applications importing the drawing functions to use the already loaded and formatted bitmaps.

    Well, you better look out for race conditions then. Assuming you want to do this for Syllable, and also assuming that Syllable is closer to Unix than to Windows, you might want to use shared memory for storage, and semaphores to manage concurrency. And of course, as already mentioned, you have to consider security aspects - you don't want a faulty (or even evil) application to render the whole system unusable.



  • Actually I'm "trying" to make one for Syllable, as I'm not an experienced programmer. Don't count on anything.

    But I really like the system and its goals, you should check it sometime. You can't help but try to help.

     

    Anyway thanks for the tips. 



  • It occurs to me that, at least under Linux, if you map the same file, in read-only mode (PROT_READ), into memory with mmap in multiple processes, the kernel VM subsystem is free to use a single physical segment and to load the file exactly once even though each application sees it as its own logical segment that it loaded.  So you may not need to explicitly share the file, just map it once in each process and forget about it.  Since this is very close to what you'd have to do to set up a shared memory area, you may have discovered this already yourself.

    It's also worth noting that if you watch a program's startup with strace(1), you can see it mmap'ing (PROT_READ | PROT_EXEC) various shared libraries.  That's how the ELF loader works.
     



  • @Angstrom said:

    It occurs to me that, at least under Linux, if you map the same file, in read-only mode (PROT_READ), into memory with mmap in multiple processes, the kernel VM subsystem is free to use a single physical segment and to load the file exactly once even though each application sees it as its own logical segment that it loaded.  So you may not need to explicitly share the file, just map it once in each process and forget about it.  Since this is very close to what you'd have to do to set up a shared memory area, you may have discovered this already yourself.

    It's also worth noting that if you watch a program's startup with strace(1), you can see it mmap'ing (PROT_READ | PROT_EXEC) various shared libraries.  That's how the ELF loader works.

    In theory this is possible. In practice it is subtle, and compiling the data into shared libraries is far, far easier than setting it up by hand. The shared library system is already designed to do exactly this while taking care of all the subtleties.

    However, he seems to want writeable shared data, and that's far harder. I doubt it will really accomplish anything, since the cost of all the code to manage it is likely to exceed any time and space gains from not recomputing the images. It reeks of premature optimisation to me. 



  • @CapitalT said:

    Let's say that the whole operating system use it.

    Have you measured how huge a "waste" it is?

    @asuffield said:

    It reeks of premature optimisation to me.

    Quite. 


Log in to reply