It's like Three Card Monte with source control, Part 1



  • The early morning email from the fellow developer, "What platform was the the application I'm working on developed in?" Huh? The email clarified: One of the other developers, I'll call him Dev Hero, had taken the code she was working on and tried to upgrade it to the latest version of .NET so we could work on the code in VS 2013 instead of VS 2010. Well, that had broken it so she wanted to try and downgrade the version of .NET it was using in VS 2013. Why didn't she just get an older version out of source code and look?

    Because we didn't have source control. Well, we did, but you couldn't get to it from here anymore.

    When I got there, at this job, we didn't even have source control. The Dev lead would get on our case if there wasn't a new zip file up on the file server at least once a week, or so. That was about a decade ago but things had changed for the better. It took a lot of work but a year or so ago we were at a stage where we could go to a web page and click a link and the code would compile and be deployed to the Dev, Test, or Pre-production web servers. Deployment Integration (DI) is so cool. Database changes still required one of the DB admins to log in and run a script but the database structure was stable enough it wasn't much of an issue for us. Most saw the value. Others, well, didn't. We were fat, dumb, and happy. (Pick two if you want to be more realistic)

    Super SA had done a lot of the work to get us there. He loved Linux and the RaspberryPI. He was the kind of guy that wanted to show you how he could log into his entertainment server at home to change the the TV channel from his smartphone. He also liked to use free software to run our enterprise sized organization. He was the guy that everyone went to for a miracle. We needed and still need those because of the overall environment we were working in. (I'll not go into this now.)

    Eventually, Super SA got tired of his position as Miracle Max with us and left to go play in the woods with the Merry Men. A coworker turned to me the next day and said, "Everyone said that when Super SA left everything would fall apart. Well, it's all still running isn't it." I shook my head as he went back to browsing the Fox "News" website.

    Then came the edict: All desktop machines will stop using XP and servers will stop using Windows 2003. The two replacement SAs (Yes, Super SA was replaced with two people) were working like crazy. Software was moved to new machines with new(er) operating systems, databases were upgraded, and most everything was locked down so the developers couldn't get into it.

    Neither could the Deployment Integration server. First, it was locked out of the web servers so it couldn't deploy. At least we still had automatic builds we thought. While the web server access was (supposed to be) getting worked on by the new SAs the version control software quit working. Drastic measures were taken and a whole lot of hours were spent by Dev Hero and myself getting it up and running again. And again. And again. Nothing was stable. Machine names were changing. Permissions were changing. Patches were being installed and rolled back. You see, the plan to move things wasn't very well planned so machine names and permissions changed every couple of weeks or so for a period of about six months. Notifications were handled using the interrupt technique. When you noticed that your website went down you had been notified.

    I mentioned that developers weren't allowed into the servers anymore. This included the ability to setup and administer the Constant Integration server. It didn't matter because of what happened to the sever with SVN version control software on it.

    The SAs were finally told to get the final Windows 2003 machine off of the network. It was a special machine. It had free virtual machine software running on it with an installation of Linux running on that with our SVN version control software running on that. Because they couldn't figure out the configuration of this server they put the old machine in an enclave. As far as I know, and enclave is a place on the network where no one can get to it if you don't have permissions and know the right magic incantations and have an adult virgin that isn't yourself to sacrifice. They had trouble with that move because some production applications were running on it that no one knew about. So it was moved out of the enclave for a couple of weeks while that production software was moved to another development server and our precious version control server was put in the enclave.

    In the mean time, the New Guy saw our plight and with the blessings of the new Dev Lead he setup a windows version of SVN, the version control software, on his desktop machine. All was great and good with that except none of the old code was on it, it wasn't getting backed up anywhere, he either restarted the machine many times a day or was remoting into it and shut off the version control software because it was slowing down the machine. The final nail in that coffin was when the network powers that be uninstalled the version control software because it wasn't approved for the network, even though it was.

    End of Part 1



  • @SteamBoat said:

    One of the other developers, I'll call him Dev Hero, had taken the code she was working on and tried to upgrade it to the latest version of .NET so we could work on the code in VS 2013 instead of VS 2010.

    Wow, it's like sentence number 2 and already crazy with WTF value.



  • @SteamBoat said:

    wasn't very well planned so machine names and permissions changed every couple of weeks or so for a period of about six months. Notifications were handled using the interrupt technique. When you noticed that your website went down you had been notified.

    And who the fuck is in charge? Are people not complaining?



  • @swayde said:

    And who the fuck is in charge? Are people not complaining?

    Production servers moved pretty well. It was only the dev crew that was experiencing the outages.



  • ...and here I thought my work environment where our Dev server (running Windows 2003) also hosted our SVN server was a huge wtf,

    (and then that box was down for like a week to due a Windows 2003 to Windows 2008 "upgrade" (read: backup, then wipe and reinstall))

    On a side note, I'm now the Subversion Administrator at our office or something. At least until they finally get around to doing the SVN to TFS import they've been talking about for the last 2+ years.



  • Suggestion: Go to Git on TFS if you have any input. Version control on native TFS requires a constant connection that will bite you in the butt.



  • Can sympathize.

    We also had a superstar admin.
    We also had two jokers replace him.
    Unlike your guys, our admins generally do nothing. Don't even have an idea what's out there.
    The network is rotting away instead of thrown into chaos.
    Results are similar.



  • @SteamBoat said:

    @swayde said:
    And who the fuck is in charge? Are people not complaining?

    Production servers moved pretty well. It was only the dev crew that was experiencing the outages.

    People, obviously, means developers here.

    Unless your managers are TRWTF (in which case they are, obviously, TRWTF and the best solution may be to jump the ship) they will understand if the developers complain that they can't do any progress because they were cut off from important data.

    They don't need technical details, they just need to know that there was something that significantly affects productivity on the enclaved server and make it a priority for the administrators to connect it or implement other replacement.

    Also any automatic deployment is also a thing that affects productivity. Not so much, but if you tell sane manager that before a deployment took 10 minutes and now it takes an hour and because of decreased confidence there are often problems, they should request the admins to restore or replace the functionality.

    And then you need to come with some options. Subversion repositories can be copied, so you could simply move the repository elsewhere. Or you could import it to Git or Mercurial (both have import tools) for improved resilience against network changes.



  • @Bulb said:

    Unless your managers are TRWTF (in which case they are, obviously, TRWTF

    Obviously.



  • Sadly, it's not my choice.



  • @SteamBoat said:

    Suggestion: Go to Git on TFS if you have any input. Version control on native TFS requires a constant connection that will bite you in the butt.

    Hasn't been an issue for me. TFS is a nice system.

    Git … well, let's just say that I avoid Git when possible.



  • Because it bogs down after you keep adding your zip files to it?


  • Discourse touched me in a no-no place

    And all his binaries too. After all, builds are precious so they go in source control…



  • @SteamBoat said:

    Constant Integration server

    Continuous Integration is what CI means.



  • @swayde said:

    who the fuck is in charge?

    Colonel Kurtz, by the sound of it.


  • Fake News

    @s73v3r said:

    Because it bogs down after you keep adding your zip files to it?

    I'll bite: if TFS, SVN or god forbid Perforce works for you, don't use Git as the central repo. Just make sure that central repository is backed up like one should do with any production data, because that is what it is.

    We've had some discussion before about the sense and nonsense of Git, or at least Blakey did:
    https://what.thedailywtf.com/t/well-reasoned-anti-git-article/8567


  • Discourse touched me in a no-no place

    @flabdablet said:

    Colonel Kurtz, by the sound of it.

    Sounds more like Colonel Sandurz to me…



  • @JBert said:

    I'll bite: if TFS, SVN or god forbid Perforce works for you, don't use Git as the central repo. Just make sure that central repository is backed up like one should do with any production data, because that is what it is.

    Well, yeah. Any critical data should be covered by some sort of backup-recovery plan. The central repo for the code of your business critical applications is no different.

    Filed Under: Well, duh



  • @SteamBoat said:

    Version control on native TFS requires a constant connection that will bite you in the butt.

    Only on TFS <= 2010. Since 2012 TFS has 'local workspaces', which is a blessing.



  • @flabdablet said:

    @swayde said:
    And who the fuck is in charge?

    Colonel Kurtz, by the sound of it.

    We have a Colonel Kurtz mask for the times when you have to go up the river.

    It replaced the cowboy hat.



  • If you take the amount of time I've had my current public IP address (3 hours) and divide it by the amount of time my server has been running without rebooting (), the number you get is about an order of magnitude above how much sense the new system admins' decisions make.



  • @RTapeLoadingError said:

    We have a Colonel Kurtz mask for the times when you have to go up the river.

    Never get off the goddamned boat.


  • kills Dumbledore

    Did you get bored of posting ridiculously large images and decide to go the other way?


  • BINNED

    @Jaloopa said:

    Did you get bored

    He plays DF ...



  • @Luhmann said:

    @Jaloopa said:
    Did you get bored

    He plays DF ...

    And uses Milwaukee PC …



  • This sounds just like Pdfium. Some droob is currently working their way through the (C++) code changing all occurrences of "0" to "nullptr". They were up to number 208 as at 30 June. We now have boolean expressions like "ptr == NULL", "ptr == 0", and "ptr == nullptr" in the same file.

    So now I can create my own mini-fork and create a new header: "#define nullptr 0", Alternately I can upgrade to VS2013 and g++ 5.whatever.

    Naturally the Google style guide says in essence not to change existing code.



  • @Unixwolf said:

    Some droob is currently working their way through the (C++) code changing all occurrences of "0" to "nullptr".

    for (int i = (int)((bool)nullptr); i <= 9; i++)
    

  • Discourse touched me in a no-no place

    Oh, that's nasty. /me takes notes…


Log in to reply