Ideas for batch checking in project folders



  • So i have decided to join a WTF company for a short while and sort stuff out.
    They have a VB6 application that has developed over many years.
    Since 2007 the version control has been :

    1. Copy the project folder
    2. Rename it and increment the version number. Example 'ProjectNameV5_35_12'
    3. Open project in VB6 and increment the project version to match the folder name.
    4. Within the project folder is a word document that gets changelog notes added.

    Now i have around 700 folders, and have selected to use Mercurial (HG) for various raisins.
    Any one have good ideas or suggestions of tools to use to automate:

    1. Start with a initail checkin
    2. Finding the next folder in the sequence
    3. Check it in
    4. Tag the changeset with the version number from the folder name.

    It would be nice to grab the word document into the changeset comments.... but not going to happen.


  • Winner of the 2016 Presidential Election

    @Helix said in Ideas for batch checking in project folders:

    So i have decided to join a WTF company for a short while and sort stuff out.

    I'm pretty sure one of our Lounge threads started out exactly this way.

    @Helix said in Ideas for batch checking in project folders:

    Any one have good ideas or suggestions of tools to use to automate:

    Start with a initail checkin
    Finding the next folder in the sequence
    Check it in
    Tag the changeset with the version number from the folder name.

    It would be nice to grab the word document into the changeset comments.... but not going to happen.

    I'm assuming you're on Windows, so I can't offer any detailed help yet, but (I'm working out a path in my own mind to try doing, so please don't take it as talking down to you):

    • PowerShell could probably be used to do what you want.
    • As long as all the folders have the exact same naming convention (or within suitable variation) you could use a regex to extract the version number (or you could extract it from wherever VB6 stores it inside the project).
    • Somehow you'll want to order them based on the regex result (or maybe the sort will figure out the pattern automatically? Windows seems to prefer at least trying for a smart sort.)
    • Then run down the sorted list of (regex'd version number)+(folder) pairs doing:
      • Delete current contents of Hg repo, being sure not to hit any of the important parts (like .hg)
      • Copy current folder contents into Hg repo folder
      • Add all changes and commit
      • Tag version
      • Move to next version+folder pair
      • Repeat

    Now I wonder if powershell can read word docs..


  • Winner of the 2016 Presidential Election

    Initial results:

    PowerShell displays as sorted correctly:

    Windows PowerShell
    Copyright (C) 2015 Microsoft Corporation. All rights reserved.
    
    PS C:\Users\Dreikin> cd A:
    PS A:\> cd .\WTDWTF_Hg_checkin\
    PS A:\WTDWTF_Hg_checkin> ls
    
    
        Directory: A:\WTDWTF_Hg_checkin
    
    
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----       2016-06-02     06:14                ProjectNameV1_38_12
    d-----       2016-06-02     06:14                ProjectNameV5_1_1
    d-----       2016-06-02     06:14                ProjectNameV5_1_12
    d-----       2016-06-02     06:14                ProjectNameV5_34_12
    d-----       2016-06-02     06:14                ProjectNameV5_35_1
    d-----       2016-06-02     06:14                ProjectNameV5_35_11
    d-----       2016-06-02     06:14                ProjectNameV5_35_12
    d-----       2016-06-02     06:14                ProjectNameV5_35_13
    d-----       2016-06-02     06:13                repo
    
    
    PS A:\WTDWTF_Hg_checkin>
    

    (Files were created out of order temporally, so it's not that.)



  • @Dreikin
    Had my coffee now. Glad i got your thought on this, as i was going to just copy with overwrite into the working HG repo. But you are right, i should delete all and recopy to take into account files that are deleted or renamed between versions. hg addremove i think is the thing to use here.

    My other thought is that i forget trying to build the version from the filename string, and just iterate though all possible version numbers and check if the folder exists, otherwise move on to the next version number. I will have a little manual check to see if the folders conform to the naming. I haven't had much experience with automating HG so i need to read up on that. Also to manipulate the check in time/date to match the project workspace time/date.

    As for to the word doc, most of it has been written as a table, so i could try and convert it to something more useful. Although i suspect the version number formats entered here are mushy.


  • Winner of the 2016 Presidential Election

    Current status:

    PS A:\WTDWTF_Hg_checkin>`
    >> Get-ChildItem -dir old `
    >> | Sort-Object `
    >> | ForEach-Object {
    >>     $match = $_ -match "ProjectNameV(\d+)_(\d+)_(\d+)";
    >>     $version = $matches[1] + '.' + $matches[2] + '.' + $matches[3];
    >>     Add-Member -InputObject $_ -MemberType NoteProperty -Name Version -Value $version;
    >>     return $_; } `
    >> | Format-Table Name,Version
    
    Name                Version
    ----                -------
    ProjectNameV1_38_12 1.38.12
    ProjectNameV5_1_1   5.1.1
    ProjectNameV5_1_12  5.1.12
    ProjectNameV5_34_12 5.34.12
    ProjectNameV5_35_1  5.35.1
    ProjectNameV5_35_11 5.35.11
    ProjectNameV5_35_12 5.35.12
    ProjectNameV5_35_13 5.35.13
    
    
    PS A:\WTDWTF_Hg_checkin>
    

    Getting somewhere. Probably not in the best way possible, but at least now we've got folder names and versions as two properties of this object.

    Edit: Made the one-liner easier to read.


  • Winner of the 2016 Presidential Election

    Success! Well, for my limited example, anyway.

    Warnings:

    • I do not guarantee this will work. It works on my test sample, and that's all I know it works on.
    • You need to customize the regex for your own use.
    • WORK ON COPIES. You don't want to lose the old repos because you or I made a stupid mistake. Copy, don't move, the old versions into the appropriate location.
    • You really shouldn't be using scripts you find on the internet without understanding what they do. At least not outside a VM you're perfectly okay with getting totally destroyed.

    Layout:

    working_folder
    ├ .hg
    ├ .hgtags
    └ old_versions
      ├ ProjectNameV1_38_12
      ├ ProjectNameV5_1_1
      ├ ProjectNameV5_1_12
      ├ ProjectNameV5_34_12
      ├ ProjectNameV5_35_1
      ├ ProjectNameV5_35_11
      ├ ProjectNameV5_35_12
      └ ProjectNameV5_35_13
    

    The Script:

    (Don't try using this until you've read all the instructions.)

    Get-ChildItem -dir `
    | Sort-Object `
    | ForEach-Object {
        $match = $_ -match "ProjectNameV(\d+)_(\d+)_(\d+)";
        $version = $matches[1] + '.' + $matches[2] + '.' + $matches[3];
        Add-Member -InputObject $_ -MemberType NoteProperty -Name Version -Value $version;
        return $_; } `
    | ForEach-Object {
        $parent = Get-Location;
        $gp = Resolve-Path ..\;
        Move-Item ..\.hg $_.Name;
        Move-Item ..\.hgtags $_.Name;
        Set-Location $_.Name;
        hg addremove;
        hg commit -m $_.Version;
        hg tag $_.Version;
        mv .hg $gp;
        mv .hgtags $gp;
        Set-Location $parent; }
    

    The Instructions:

    Creating the working folders:

    1. Create an empty working directory (here called working_folder).
    2. Inside that directory, create a directory named old_versions.

    Copying the old versions over:

    1. Use explorer to COPY, not move, the old project version folders into old_versions.

    Creating .hg and .hgtags:

    1. Open PowerShell and navigate to working_folder
    2. Do hg init
    3. Do New-Item .hgtags -ItemType file

    Modifying the script:

    1. Look over the script and make sure it will work for your setup.
    2. Take notice of the regex expression on the fourth line. Make sure it matches your actual project name pattern.

    Running the script:

    1. cd into old_versions.
    2. Paste the script into PowerShell
      • Don't know how to paste in PowerShell? Do this:
      1. Right click on the title bar of the PowerShell window.
      2. Move down to the Edit sub-menu.
      3. Click Paste.
    3. If it is not already running, press Enter or Return.

    What to do if you see something wrong:

    If it's doing something that needs to be stopped:
    1. Hit Ctrl+C
    2. Continue on to the next part:
    Anything else:
    1. Take note of the error text, if any. It should show up as red text on a black background. Copy it somewhere if you need to.
    2. Try seeing if you can figure it out just by looking at what happened, the error text (if present), and the script.
    3. If you can't, try googling the error text or problem and seeing if anything that shows up helps.
    4. If that doesn't work, try asking friends or colleagues for help diagnosing the problem.
    5. If that doesn't help, try coming back to this thread and posting the error text (if present), a description of the problem (if needed), and a request for help.
    6. Be patient, it may be a while before anyone who can help is available and notices.
    7. If these options don't work, well ¯\_(ツ)_/¯ you are, once again, on your own.

    Edit: Evidence of success:
    0_1464874789031_upload-4618af5c-27f3-4a77-b5f1-b3c012beeb2b


  • Winner of the 2016 Presidential Election

    @Helix said in Ideas for batch checking in project folders:

    @Dreikin
    Had my coffee now. Glad i got your thought on this, as i was going to just copy with overwrite into the working HG repo. But you are right, i should delete all and recopy to take into account files that are deleted or renamed between versions. hg addremove i think is the thing to use here.

    My other thought is that i forget trying to build the version from the filename string, and just iterate though all possible version numbers and check if the folder exists, otherwise move on to the next version number. I will have a little manual check to see if the folders conform to the naming. I haven't had much experience with automating HG so i need to read up on that. Also to manipulate the check in time/date to match the project workspace time/date.

    As for to the word doc, most of it has been written as a table, so i could try and convert it to something more useful. Although i suspect the version number formats entered here are mushy.

    While I was building the script above (I didn't see your post until after I'd posted that), I realized it's actually just as easy to copy .hg and .hgtags to each folder, do addremove, and then commit. (Actually easier in PowerShell, since I couldn't get it to ignore the .hg directory and contents on delete).

    For the datetime, I don't know what you want to set that from, but hg commit has a --date flag.

    I'm currently trying out the ability to interact with Word, which is actually pretty easy:

    PS A:\WTDWTF_Hg_checkin> $word = New-Object -com word.application
    PS A:\WTDWTF_Hg_checkin> $word.visible = $True
    PS A:\WTDWTF_Hg_checkin> $doc = $word.Documents.Open((Resolve-Path text.docx).toString())
    PS A:\WTDWTF_Hg_checkin> $doc.Paragraphs | ForEach-Object { $_.Range.Text }
    Commit Title
    
    Commit Message
    
    Table Title
    Column1
    Column2
    
    Row1
    R1C1
    R1C2
    
    Row2
    R2C1
    R2C2
    

    From a doc that looks like:
    0_1464873934394_upload-fa4baf58-3fa8-42f1-a0c0-f573ae240fa9



  • @Helix Is that a valuable use of your time? How often do you expect needing to go back to versions from years ago?

    Just check-in a copy about 5-10 revisions back, and manually do the changes from there, and call it good.



  • @blakeyrat
    Well i have been fixing bugs with code that was added 2012, and amended around 2014.. and it's poorly commented. So I am trying to understand why/what feature it was added for. The team is two guys who tell me it's pretty organised with the folder names (!) and all I have to do is use beyond compare to find when it was added :facepalm:

    Since there is 700 or so copies from 2007 on average there is 77 folders per year. Although in practice there is a ton of releases when there is a flurry of features added.

    Not only will this be a tool for my own sanity it will be a way of demonstration why 'folder name' version control has to go.



  • Well you might keep in mind that:

    1. VB6 has been dead and buried for like over a decade at this point
    2. The best migration path is to convert the project to VB.net
    3. VB.net's IDE, Visual Studio (alas) works much better with TFS or Git than with Mercurial
    4. These idiot programmers, if forced to move to VB.net or C# or anything even remotely modern, are going to need that move to have as little friction as possible. Because there's going to be kicking and screaming already with just the language change

    What I'm saying is that while we're all know Mercurial is a better product than Git, I'm not sure it's better for your purposes.



  • @blakeyrat
    Gosh i did not know that Vb6 is discontinued and replaced by Vb.Net



  • @Helix Is that some kind of sarcasm or...?

    What I'm saying is there's only a matter of time before the VB6 framework has some horrible security bug and Microsoft says, "look, we wash our hands of this shit. Deal with it yourself." Unless you're just freezing time and are going to run the application in a Windows 2000 VM from now until the end of the universe, I suppose.



  • 0_1464877989052_VB.NET Shop.png

    ? 🎣



  • @blakeyrat said in Ideas for batch checking in project folders:

    @Helix Is that some kind of sarcasm or...?

    :whoosh:

    What I'm saying is there's only a matter of time before the VB6 framework has some horrible security bug and Microsoft says, "look, we wash our hands of this shit. Deal with it yourself." Unless you're just freezing time and are going to run the application in a Windows 2000 VM from now until the end of the universe, I suppose.

    Yes.... i thought they have done that already. We are lucky that runs in Win10.
    So you want me to explain:

    1. Planning the migration to vb.net but have a load of third party graphics controls, integration into odd ball niche hardware comms library and databases to sort out. It's in the planning stage but the resources are making the migration slow.
    2. New applications are built using .Net anyhow, so they are getting sharper in this regard.
    3. The guys are EEs who work on embedded software in other packages, we will need to use standalone VC tools to manage these.
    4. If they are happy with 'folders', Git will blow them away and move back to folders.
    5. TFS was ok but not a good fit for the embedded stuff. TFS or Git i wasn't really keen on the inbuilt support in VS2015 anyway.


  • @Helix If they can't finish the migration from VB6 to a .net language in literally a decade, ... why are you working there again?


Log in to reply