Virtual Private Headache



  • So I've got an e-commerce website with around a thousand visitors daily. It runs on a LAMP setup with a webservice pulling data from our supplier, nothing too fancy. Recently I've been having some trouble with my regular hosting provider, so I decided to go for a VPS. A VPS would let me set things up the way I want, and then I'd have my own process and "my own" machine and all that. More control, less interference = better, right? VPS > regular hosting, right?

    With this naive thought, I picked out the slightly above regular option at a local host and hired the service. Now, the service clearly advertised 128MB of memory dedicated to my hosting, and I thought that this was a bit low, I mean, I always heard about servers going with huge 4GB+ banks of memory and stuff, but I imagined that the hosting company would know better than me, a newbie in the world of dedicated hosting. Maybe they need all that horsepower to host a couple of hundred accounts, which eventually just use less than the 128MB on average? Or at least that the main services are already accounted for and they mean 128MB of useful memory. Or maybe Linux really is that lightweight. Still, if they offer it, then it must work, right?

    Anyway, it took them a whole week to set up the VPS because of the cPanel licence (a WTF in itself), then I finally got access. I checked the memory, the OS was reporting 2GB. That semed good, means the memory the 128MB was just a theorethical limit.  I set things up normally, and everything was working fine and smoothly, until I pointed the DNS to it and the visitors started to come in. That's when I started getting messages in my email about services failing and refusing to start back up. Turns out I had hit the memory ceiling, and allocations were failing all over the place. SSHing into the shell revealed that even things like 'ls' were throwing 'fork: Cannot allocate memory' errors.

    A day of frantic tweaking and research about low memory apache/mysql configurations (the services that seem to take up the most) hasn't helped at all to alleviate the problem. I can't even run cPanel itself anymore, which takes down email with it. FTP crashes all the time, and the website can't reach the database 1 out of 5 times.

    So maybe I'm just a complete newbie at this and am completely missing some obvious glaring thing here, but what exactly is the point of offering VPS web hosting, an apparently superior solution that boasts the ability of higher levels of control, and the ability to host several websites and accounts, if just the bare minimal, with one website, already takes up more memory that what's normally offered by the host, leaving me with a very dedicated unusable system?



  • Of the half-dozen VPS/dedicated hosting I've used over the past few years none of them have had anything close to a workable web server setup. They usually just give a default config file and expect you to figure it out on your own...which is good in some ways as it forces you to learn things but sometimes you just want something that works. Perhaps I've just been unlucky with providers and other services have more sane setups.

     

    For example, my current dedicated host seemed to just use the default config file for Apache 1.3. MaxClients has set to 255 when the 512MB of server RAM could only support 50 clients at the max. KeepAlive was on and set to an extremely high value of 10 (seconds) when it should be 1 or 2 at the most or even off on high traffic sites. What would happen is the high KeepAlive kept sessions going for far too long which in turn caused more clients to be spawned to handle incoming connections. The server quickly ran out of RAM (within an hour) and then would start swapping like crazy and performance dropped through the floor. By researching and tweaking the config I was able to improve the traffic the server was capable of handling by a factor of 100 or so.
     

    I might guess that something similar is happening to your server, although with your relatively low traffic it may be something else. If you're running Apache 1.3 I'd check your the settings mentioned above and tweak them. There should be something similar for Apache 2.0 or other web servers. Google should turn up a bunch of articles related to tuning performance of Apache. You can also use the 'top' command to see exactly what is taking up memory...it may well be a bunch of other things that you don't necessarily need. I know one VPS I had had dozens of optional things running on it that I could have removed if needed.

     



  • One thing I'd like to mention: Do NOT run cPanel on a 128 MB VPS! It just won't work properly... You'll need at least 512 MB of RAM for cPanel to work properly. I'd suggest to run something more lightweight like ISP-Config on a low-memory VPS. If you do not use a control panel at all (or use something extremely lightweight like Webmin), you'll be able to run quite a few sites properly (you'd need to heavily optimise MySQL, though, as it will often need more than 100 MB itself).

    Of course, you could always go dedicated, and then you'll have a whole system to yourself. My dedicated server is from http://dedicatedbox.net/, one of the cheapest dedicated server providers. I've got a AMD Athlon 2000+, 1 GB RAM, 120 GB hard drive, 100 Mb/s port for US$45 per month (special offer, is usually US$50 per month). If you want to check the ping times or whatever, http://www.daniel15.com/ and http://www.dansoftaustralia.net/ are both hosted on my server.

    . I know one VPS I had had dozens of optional things running on it that I could have removed if needed.

    Mine was exactly how I wanted it - A plain Linux installation (I chose Debian), with nothing but Apache running. The initial installation was using only ~12 MB out RAM, out of 1 GB :)

    Of the half-dozen VPS/dedicated hosting I've used over the past few years none of them have had anything close to a workable web server setup. They usually just give a default config file and expect you to figure it out on your own...which is good in some ways as it forces you to learn things but sometimes you just want something that works. Perhaps I've just been unlucky with providers and other services have more sane setups.

    If you want the provider to do most things for you, you'd be looking for a "managed" dedicated server. Most servers are unmanaged, and you need to do most things yourself.



  • @DaveHumphrey said:

    The server quickly ran out of RAM (within an hour) and then would start swapping like crazy and performance dropped through the floor. By researching and tweaking the config I was able to improve the traffic the server was capable of handling by a factor of 100 or so.

    I might guess that something similar is happening to your server, although with your relatively low traffic it may be something else. If you're running Apache 1.3 I'd check your the settings mentioned above and tweak them. There should be something similar for Apache 2.0 or other web servers.

    Yes, I've gone through a lot of tweaking of the Apache settings (and gained a lot of insight on how Apache works in the process, the only satisfying thing I've managed to extract from this situation). Several articles on low memory configs helped a lot here. It didn't do much, every instance of Apache still weighs 20MB each, 17 of which is shared memory. I've also looked for things I can disable, but everything I'm running is necessary.

    One thing I find strange is that the system has tons of swap space, but it just doesn't use it. I'm not versed in Linux, but I believe swap should be the Linx eqquivalent of Virtual Memory, ie, the memory it would use when it runs out of physical memory. Yet, here it is: 

    top - 15:06:00 up 23:41,  1 user,  load average: 0.01, 0.13, 0.15
    Tasks:  24 total,   1 running,  23 sleeping,   0 stopped,   0 zombie
    Cpu(s):  0.1% us,  0.0% sy,  0.0% ni, 99.9% id,  0.0% wa,  0.0% hi,  0.0% si
    Mem:   2070776k total,  2051568k used,    19208k free,    71808k buffers
    Swap:  4192944k total,   502920k used,  3690024k free,   747344k cached

    Tons of swap just sitting there, while it takes me 3 tries to get "top" to allocate memory. I wouldn't even mind the slowness so much, as long as it would at least work. Eventually.

    @Daniel15 said:

    One thing I'd like to mention: Do NOT run cPanel on a 128 MB VPS! It just won't work properly... You'll need at least 512 MB of RAM for cPanel to work properly. I'd suggest to run something more lightweight like ISP-Config on a low-memory VPS. If you do not use a control panel at all (or use something extremely lightweight like Webmin), you'll be able to run quite a few sites properly (you'd need to heavily optimise MySQL, though, as it will often need more than 100 MB itself).

    Of course, you could always go dedicated, and then you'll have a whole system to yourself. My dedicated server is from http://dedicatedbox.net/, one of the cheapest dedicated server providers. I've got a AMD Athlon 2000+, 1 GB RAM, 120 GB hard drive, 100 Mb/s port for US$45 per month (special offer, is usually US$50 per month). If you want to check the ping times or whatever, http://www.daniel15.com/ and http://www.dansoftaustralia.net/ are both hosted on my server.

    Alas, I have extremely little experience in dealing with Linux, and I'm not brave enough to wander too much outside of the safety net of a control panel that knows what it's doing. Nor I have the knowhow to uninstall cPanel and do things from scratch. Hopefully I will one day, but so far I learned most of what I know about Linux by inspecting the effects that cPanel creates. This is my favorite kind of learning. :)

    I also have the problem with the fact that I hail from Portugal, and I need a local server because of the ridiculous international traffic restrictions of the local ISPs. Portugal isn't very developed at all in terms of webhosting; the company I'm with right now is one of the very few that actually has servers located in the country, most of the others just outsource to datacenters elsewhere. Portugal is also notorious in terms of lazy work practices. I'd much rather learn things myself.

    In the meantime, I'm having them upgrade me to a plan with 512MB, which is going to cost me double of what your fully dedicated box costs you. Just to show what things are like here.



  • The OS is reporting that the server has 2GB of RAM. You only get 128MB of that.   There is a way to get your VPS' ram usage but it's specific to your system.

    Anyway, a box with 128MB is an el cheapo box to begin with. Then you threw apache, an RDBMS, AND cpanel with all its WTF-ish junk on there? You're crazy.

    A VPS provides you with max configurability, not performance.



  • I figured that since they sell a 128MB box with cPanel preinstalled on it, then it would be actually feasible to run cPanel and basic web stuff on a 128MB box. It's not a mistake I'll repeat.



  • @Sunstorm said:

    @Daniel15 said:

    One thing I'd like to mention: Do NOT run cPanel on a 128 MB VPS! It just won't work properly... You'll need at least 512 MB of RAM for cPanel to work properly. I'd suggest to run something more lightweight like ISP-Config on a low-memory VPS. If you do not use a control panel at all (or use something extremely lightweight like Webmin), you'll be able to run quite a few sites properly (you'd need to heavily optimise MySQL, though, as it will often need more than 100 MB itself).

    Alas, I have extremely little experience in dealing with Linux, and I'm not brave enough to wander too much outside of the safety net of a control panel that knows what it's doing.

    Don't mistake cPanel for something that knows what it is doing. It is unbelievably crap. To simplify and clarify the above advice: do NOT run cPanel. Hire a blind emu to admin your server, it'll do a better job.

    (I find the entire concept of VPS hosting highly amusing and can't imagine why anybody would want to pay for it - it's not like colo servers are all that expensive)



  • Well I think it really depends on your needs (and also your provider ;) ).

    I've started off with 256 MB RAM on a confixx controlled v-server for 7€ a month, which worked quite well. Now I've upgraded to 512 MB RAM (only 10€ per month) as i used to hit the limit and it's working like a charm. Ok, it's only serving something like 200 unique visitors a day and a few emailaccounts, but I think the value for money is unbeaten.
    And the support is also doing a really good job (didn't really expect that for the money :) ).

    However if you need more power, a dedicated server will be the only option - really depends.


    PS: While tuning apache is a good idea, you might want check out lighttpd, http://www.lighttpd.net/, which should be really nice for v-servers. But others use it as well, for example sourceforge.net



  • @xeraa said:

    PS: While tuning apache is a good idea, you might want check out lighttpd, http://www.lighttpd.net/, which should be really nice for v-servers. But others use it as well, for example sourceforge.net

    While I'm sure lighttpd is a fine piece of software, "SourceForge uses it" is not exactly a ringing endorsement.



  • @SomeGuy said:

    @xeraa said:

    PS: While tuning apache is a good idea, you might want check out lighttpd, http://www.lighttpd.net/, which should be really nice for v-servers. But others use it as well, for example sourceforge.net

    While I'm sure lighttpd is a fine piece of software, "SourceForge uses it" is not exactly a ringing endorsement.


    LightTPD is excellent, and heaps of sites use it (YouTube is one of them)... Unfortunately, not many control panels support it at the moment.

    Tons of swap just sitting there

    You're already using 500 MB of swap... That's already quite excessive!
    Once MySQL starts swapping, expect your sites to go very very slowly.

    Alas, I have extremely little experience in dealing with Linux, and I'm not brave enough to wander too much outside of the safety net of a control panel that knows what it's doing. Nor I have the knowhow to uninstall cPanel and do things from scratch. Hopefully I will one day, but so far I learned most of what I know about Linux by inspecting the effects that cPanel creates. This is my favorite kind of learning. :)

    There's heaps of good tutorials on how to do this. As an example, take a look at http://www.howtoforge.com/perfect_setup_debian_etch (the data centre do the initial setup, so you'd skip the first two pages of that tutorial).

    Don't mistake cPanel for something that knows what it is doing. It is unbelievably crap.

    Yeah, definitely...
    My server provider gives a free 30-domain Plesk license with every server. Plesk is so much better than cPanel - it does not take control of the system like cPanel does. With Plesk, you can use whatever version of Apache and PHP you like (I'm using the Debian packages), whereas cPanel forces you to use their "easyapache" tool to compile Apache.



  • Somebody earlier mentioned Webmin

    The Webmin developers now have a commercial product called Virtualmin Pro.

    Based on the GPL virtualmin which runs under Webmin as a module, Virtualmin Pro has a bash installer that totally sets up a LAMP system and more.

    Fantastic for those who are new to all of this.

    You get (in my opinion) the best most feature rich server admin panel out there and Jamie, Joe and the entire development team are extremely helpful.

    Commercial but reasonably priced for the extremes you get.
    www.virtualmin.com

    GPL www.webmin.com
    the GPL version of virtualmin is under modules at the webmin site but I stronly suggest the commercial version of less experienced users.

    Webmin is the one saving grace I've had in that it really took me by the hand and at the same time helped me understand the inner workings of a linux server.

    I guarantee you  will be very impressed.
     



  • Not sure what I did wrong but the above sites I created links for some how go to the forums. They are typed ok but lead to the wrong site.

    You will have to copy and paste them into your address bar manually.

     



  • Edit the post and add http:// to the beginning of the address. Believe it or not, links can't tell the difference between a site called www.abc.com and a subfolder of the current site called www.abc.com. The http:// also indicates that it's a http (webpage) link, as opposed to https:// (http secure), ftp:// (ftp), mailto: (email) or any number of others.

    Most forums now automatically add http:// to any links that don't already have a protocol, and browsers add it to any addresses typed into the address bar, because so many people have grown up thinking that "www.abc.com" is the whole address, and that the internet is nothing but websites.



  • I learn something new everyday.

    I would correct it however I get:

    Permission Expired: Post Edit Permission Expired

    The administrator has configured this forum to have a time limit for editing messages. If you still require a message to be edited, please check with a forum moderator.
     
    Ha Ha.... Curious...
     

     


Log in to reply