Centralized Application Configuration Management



  • Hi all,

    My boss would like to be able to store all of our configuration data for all of our applications in a database so that we don't have to rely on our network people whenever there is a change in an application's configuration.  She would like us to do this for all of our applications--console, desktop, and web.  If you have had any experience with this kind of thing, could you tell me about it?  Did it work, have mixed results, or become TDWTF-worthy?



  • Be sure to avoid this anti-pattern:

    How much WTF potential this idea of your boss has depends on the volume of data you'd have to keep in that database. Also, the amount of work it'd take to change each application to get their configuration from a database instead of a file, for example.

    Think about how you're going to maintain that database. You might have to update the code in each application when you change something like the name of the database server, or when you change the configuration tables schema (you could have to change the length of some field in order to set up some new application, for example). In the long run, recoding many applications due to a small change in the config database may end up being too costly.

    You should also have some understanding of the technologies used in your company's apps. Take ASP.NET applications, for example. It's easy to store all their configuration in files. Almost always you'll have one in the root folder of the app, and sometimes you'll have another one in other folders. If these files have a key name ("web.config"), the application will automatically use them. If you don't use these files, and use the database idea proposed by your boss, you'll have to hard-code a connection string in your application. You'll then have to connect to that database and programatically set everything the application might need (themes, authentication and authorization settings, role management, other connection strings it should use etc.). That is a hell lot of work, and people use configuration files so that they don't lose productivity having to do write code just to read the application settings. Some of these settings are actually pretty hard to set programatically without a huge level of WTF (culture settings in pages, for example).



  •  It really depends on what kind of information you keep in your configuration files.  What kind of data do you plan on storing in the DB?  B2B connection information / login params?  File names?  URL / DNS information? 



  • LIke me to work on your database design? I'm available, and my rates are very reasonable. Please. 



  • .@amischiefr said:

     It really depends on what kind of information you keep in your configuration files.  What kind of data do you plan on storing in the DB?  B2B connection information / login params?  File names?  URL / DNS information? 

     

    The idea is to store such things as database connection strings, log storage paths, timeouts, and the like--everything that one normally finds in an app.config or web.config file.  That would include the stuff you mention above



  • OIC... Your IT development costs will skyrocket if you walk that path, then.



  • @slavdude said:

    The idea is to store such things as database connection strings, log storage paths, timeouts, and the like--everything that one normally finds in an app.config or web.config file.  That would include the stuff you mention above

    If what you're wanting to store centrally can be put into a file that is the same across multiple hosts you could use something like cfengine or puppet, if your configuration is also stored in the registry there is radmind. I've only experience with these systems on unix-like machines, but as far as I can tell they do what you want.



  • [quote user="Renan "C#" Sousa"]OIC... Your IT development costs will skyrocket if you walk that path, then.[/quote]

     

     Do you mean the costs of retrofitting our existing applications?  What about maintenance costs?



  • Both will depend on how you design your software. A good design would allow you to write some library to deal with the configuration part, abstracting that for your applications. Existing applications will have to be retrofitted one by one, but new applications may be built upon that library from start. Good management of said library's releases will keep maintenance costs under control. However, I insist on the point that this won't be simple, and may not be cheap (in terms of manhours).

    Now, think about it. Instead of having a connection string in a file where the application's framework expects it, you'll have it in a database. But you'll need to connect to that database in order to fetch that string. So you either have a connection string in a config file anyway, just to connect to the centralized configuration database (which makes you question the usefulness of the centralized solution), or you hard-code the connection string to the centralized base - and then, whenever the connection string changes, you'll have to recode a piece of the application to get it working again.

    It'll also have some impact on your apps' performance. ASP.NET pages, for example, don't keep their state between posts. The common way of obtaining configuration info is from .config files, and I think the framework does some optimizations like keeping some configuration info in a cache until the web.config file is rewritten, or the app is taken down (someone please verify this, as I'm not sure about it). Now, if you fetch configuration programatically, you'll add the cost of a series of queries to the config database for everytime a page is loaded. That is way more costly, both in processor time and in bandwidth usage, than having that read from a local file. Unless you store the configuration for an app in its cache, but that seems like a hack, and depending lots of other factors this would be bound to fail if IIS decided to reclaim some of cache memory (for other applications it might be running, for example).



  • Renan, while I agree with you in principal, I think you're exaggerating the downsides. Your point that the central config is going to have to be configured for each app is true and .NET's unfortunate unwillingness to do an include to a config file outside it's webroot is enough to bring tears to my eyes (Had a look for a simple solution once before but couldn't find any simple way to do it and came to that very conclusion - that any system to maintain it would be more overhead than it was worth when so many things already integrate so well with app/web.config files).

    But on the caching point so long as you use decent threadsafe caching code you can reliably maintain a cache of an external config in the memory of a .NET app, reloading it if IIS decides to clear it out, with a bare minimum overhead. If you couldn't do that then the caching code would be entirely useless.

    Your info on config caching is close to correct, .NET itself does cache the standard config files until they are edited or the app pool recycles, but it caches it basically in the same methods it exposes to you to allow you to cache other things (Including the ability to hook up a cache so that if a file is edited it is cleared - this is built-in to the available .NET cache code). If you had your own config file which you deserialized into an object then cached the object at a high priority with a dependency on the file then you'd essentially have rebuilt that functionality in code that could be easily done in 30 minutes (Obviously web.config will likely still have more capabilities than your custom config, but the parts to do with caching are easy to reproduce). For non-web apps they just read it when they first start then hold it in RAM the whole time, which is easy enough to achieve. The fact you'll still have to store where to find that config file and the fact you'll have to hook it into all your existing code is still a pretty big argument against doing this though.

    Slavdude, I'm curious, is your goal to merge the configs so 1 change affects them all, or just to keep all the configs in 1 spot so they're easy to edit? Also are these applications serverside or are we talking apps with configs stored on end-user machines?



  • @fyjham said:

    But on the caching point so long as you use decent threadsafe caching code you can reliably maintain a cache of an external config in the memory of a .NET app, reloading it if IIS decides to clear it out, with a bare minimum overhead. If you couldn't do that then the caching code would be entirely useless.

    Your info on config caching is close to correct, .NET itself does cache the standard config files until they are edited or the app pool recycles, but it caches it basically in the same methods it exposes to you to allow you to cache other things (Including the ability to hook up a cache so that if a file is edited it is cleared - this is built-in to the available .NET cache code). If you had your own config file which you deserialized into an object then cached the object at a high priority with a dependency on the file then you'd essentially have rebuilt that functionality in code that could be easily done in 30 minutes (Obviously web.config will likely still have more capabilities than your custom config, but the parts to do with caching are easy to reproduce). For non-web apps they just read it when they first start then hold it in RAM the whole time, which is easy enough to achieve. The fact you'll still have to store where to find that config file and the fact you'll have to hook it into all your existing code is still a pretty big argument against doing this though.

    Thanks =)



  • "you'll have to hard-code a connection string in your application. You'll then have to connect to that database and programatically set everything the application might need (themes, authentication and authorization settings, role management, other connection strings it should use etc.). "

    where can i get the sample of hard codes that you're saying?please reply asap



  • @angelafaye29 said:

    "you'll have to hard-code a connection string in your application. You'll then have to connect to that database and programatically set everything the application might need (themes, authentication and authorization settings, role management, other connection strings it should use etc.). "

    where can i get the sample of hard codes that you're saying?please reply asap


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.