[Words in Brackets] about that migration thing



  • That readonly thing that just happened was me backing up Discourse just in case the migration fucks it up. In about 5 seconds, I'm going to start the migration. If it fucks everything up, I'll restore the backup, so either we'll be getting a shitton of posts or we'll be losing a dozen.

    Also, why is the backup step named "gzipping archive" about five lines after "creating archive: what-the-daily-wtf-*.tar.gz"?

    Holy fuck, the backup is like 25% of the server's disk space.



  • So should we stop posting until we know what's going on? (Like I'm doing now...)


  • sockdevs

    well i know the sql backup is coming up on a gig or so (according to @PJH) so i can imagine a full backup is rather more.

    did you take a full backup or a data only backup?



  • Ok, so it looks like I can't back up the forum because there's not enough disk space. Not sure what to do.



  • Welcome to my life, except for builds instead of backups.


  • sockdevs

    hmm... you just have shell access, not admin. you're probably trying to take a full backup.

    see if there's a data only backup opetion or delete the original import backup (that'll still be on the old server just in case) ?



  • I wonder if @apapadimoulis can take a backup of the server through the DigitalOcean control panel. That wouldn't be constrained by the size of the server, so it should work. I just want this backup so there's something to fall back on if everything goes to shit.



  • Actually, I can probably just delete the local backups, since they're stored redundantly offsite.


  • mod

    @ben_lubar said:

    Actually, I can probably just delete the local backups, since they're stored redundantly offsite.

    Are you sure?

    ARE YOU SURE‽



  • Wow...I remember looking at the admin dashboard, and backups were listed as...uh...something...but I remember there being, like 9.5GB free. Anyways, a lot more than there used to be before the weekend migration. Now I see this:

    uploads 2.34 GB (4.98 GB free) backups 10.5 GB (4.97 GB free)

    So it looks like about 5GB more in backups since then. I think @PJH has been manually backing stuff up, since automated ones aren't working.



  • @abarker said:

    Are you sure?

    $ rm -rf *.bak

    Wait, what do you mean, sure?


  • Winner of the 2016 Presidential Election

    @ben_lubar said:

    Not sure what to do.

    Any access to any decent storage anywhere else? Old droplet? Or is that just replaced?

    You could maybe mount a network drive if you have access to something like that.

    No, for the love of all that is holy, don't use your hard drive... We all know your connection... Restoring would take years...


  • Discourse touched me in a no-no place

    @abarker said:

    Are you sure?

    [pjh@sofa backups]$ aws s3 ls s3://wtdwtf-back 
    <37 lines deleted>
    2015-03-08 03:11:56 3574568716 what-the-daily-wtf-2015-03-08-030000.tar.gz
    2015-03-08 04:05:56  352771843 what-the-daily-wtf-2015-03-08-040000.tar.gz
    2015-03-09 03:13:14 3574535476 what-the-daily-wtf-2015-03-09-030000.tar.gz
    2015-03-09 04:06:52  353594957 what-the-daily-wtf-2015-03-09-040000.tar.gz
    2015-03-10 14:48:57  356877896 what-the-daily-wtf-2015-03-10-144141.tar.gz
    2015-03-11 04:06:59  358150822 what-the-daily-wtf-2015-03-11-040000.tar.gz
    2015-03-12 04:07:03  361503025 what-the-daily-wtf-2015-03-12-040000.tar.gz
    2015-03-13 04:07:09  363732858 what-the-daily-wtf-2015-03-13-040000.tar.gz
    [pjh@sofa backups]$ 
    

  • Discourse touched me in a no-no place

    @ben_lubar said:

    Not sure what to do.

    Are you taking a backup at the moment?



  • Yeah, it's gzipping right now.

    <snip 891 lines>
    Archiving uploads...
    tar: uploads/default: file changed as we read it
    Gzipping archive...
    


  • @ben_lubar said:

    ```
    tar: uploads/default: file changed as we read it

    
    
    Misquoted! :scream_cat:

  • Winner of the 2016 Presidential Election

    @tar-czf notify.tgz ./t/words-in-brackets-about-that-migration-thing/9191/17



  • @Onyx said:

    @tar-czf notify.tgz ./t/words-in-brackets-about-that-migration-thing/9191/16

    WHAT'S WITH ALL THE GIBBERISH AROUND HERE‽



  • Must be Linux hardware!


  • Grade A Premium Asshole

    @boomzilla said:

    WHAT'S WITH ALL THE GIBBERISH AROUND HERE‽

    I know!

    We need more memes as a counterbalance.


  • Grade A Premium Asshole

    @loopback0 said:

    Must be Linux hardware!


  • Discourse touched me in a no-no place

    @accalia said:

    well i know the sql backup is coming up on a gig or so (according to @PJH) so i can imagine a full backup is rather more.

    [pjh@sofa backups]$ gzip -l what-the-daily-wtf-2015-03-16-040000.tar.gz /tmp/what-the-daily-wtf-2015-03-14-030000.tar.gz 
             compressed        uncompressed  ratio uncompressed_name
              365976910          1579714560  76.8% what-the-daily-wtf-2015-03-16-040000.tar
             3874910013          1315713024 -194.5% /tmp/what-the-daily-wtf-2015-03-14-030000.tar
             4240886923          2895427584 -46.5% (totals)
    [pjh@sofa backups]$ 
    

    Apart from two days, not sure why the apparent disparity.

    The first is the database only, the second a full backup.



  • What exactly does -194.5% compression ratio mean?


  • Discourse touched me in a no-no place

    Ah. Corrupt download - I think you deleted that while I was downloading it...

    I'll try again with the current backup.


  • sockdevs

    @PJH said:

    365976910

    ~350MiB

    @PJH said:

    1579714560

    ~1.4GiB

    @PJH said:

    3874910013

    ~3.6GiB

    @PJH said:

    1315713024

    ~1.2 GiB

    :wtf: the full backup compresses with a negative compression ratio‽



  • ok, backup's done.



  • @accalia said:

    the full backup compresses with a negative compression ratio‽

    Discozipping™

    Yes, I saw the posts with the real explanation. Facts, barriers to jokes etc.



  • loading existing groups...
    loading existing users...
    loading existing categories...
    loading existing posts...
    loading existing topics...


  • sockdevs

    @loopback0 said:

    Yes, I saw the posts with the real explanation. Facts barriers to jokes etc.

    so did i. after i posted they appeared below my post then vanished to reappear above it.



  • The Discostreaming™ of posts has seemed a bit.... iffy today.



  • @ben_lubar said:

    loading existing groups...
    loading existing users...
    loading existing categories...
    loading existing posts...
    loading existing topics...

    /usr/local/lib/ruby/2.0.0/csv.rb:1851:in `block (2 levels) in shift': Unquoted fields do not allow \r or \n (line 104282). (CSV::MalformedCSVError)
            from /usr/local/lib/ruby/2.0.0/csv.rb:1816:in `each'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1816:in `block in shift'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1776:in `loop'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1776:in `shift'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1718:in `each'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1732:in `to_a'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1732:in `read'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1323:in `block in read'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1266:in `open'
            from /usr/local/lib/ruby/2.0.0/csv.rb:1323:in `read'
            from communityserver.rb:13:in `execute'
            from /var/www/discourse/script/import_scripts/base.rb:78:in `perform'
            from communityserver.rb:134:in `<main>'
    

    Welp, back to asking @InedoJohn to fix the csv file.


  • Discourse touched me in a no-no place

    Did that not crop up in testing? o_O


  • Winner of the 2016 Presidential Election

    @PJH said:

    Did that not crop up in testing? o_O



  • It did, and I thought they fixed it. Thankfully, my script can resume from a failed import, and since the file that was corrupt was the user list, it didn't actually change anything in the database.


  • Grade A Premium Asshole

    If ever there were perfect timing on a meme...that was it. You win these internets.


  • Winner of the 2016 Presidential Election

    :bow:



  • @Polygeekery said:

    You win these internets.




  • Grade A Premium Asshole

    @loopback0 said:

    <img src="/uploads/default/16693/68b0f8abec0e3383.png" width="591" height="500">

    Just as long as it is not a @ben_lubar internet. If it is, that is not much of a prize.



  • It says © 2005. Ain't no way @ben_lubar's Internet is that new.



  • Oh come on, just delete some old posts, nobody will miss them.

    Or that thread


  • Winner of the 2016 Presidential Election

    @anonymous234 said:

    Or that thread

    I hope you're not thinking what I think you're thinking :frystare.bak:



  • @ben_lubar said:

    Thankfully, my script can resume from a failed import, and since the file that was corrupt was the user list, it didn't actually change anything in the database.

    Wonder who was the genius that put a \r in his user info.


  • Winner of the 2016 Presidential Election

    @boomzilla said:

    Wonder who was the genius that put a \r in his user info.

    You must be new here.



  • I'm just saying, you could simply "encode" the likes thread as a few parameters for a markov chain, then recreate it afterwards.

    Hmm, that could use some research. You could compress entire websites like this, at imperceptible levels of loss.


  • Winner of the 2016 Presidential Election

    @anonymous234 said:

    I'm just saying, you could simply "encode" the likes thread as a few parameters for a markov chain, then recreate it afterwards.

    The compression ratio on Dwarf Fortress posts will be amazing! Do it!



  • @boomzilla said:

    uploads 2.34 GB

    Silly question... the unused pictures (that people upload, possibly accidentally, but don't post) are being pruned right?

    Because I think I've personally uploaded twice as many files than I've posted.



  • It wasn't inside a record, unless sql server somehow allowed a user to put an unescaped quote in the CSV export.



  • @anonymous234 said:

    Silly question... the unused pictures (that people upload, possibly accidentally, but don't post) are being pruned right?

    Yeah....I have no clue what's in that number.


  • Discourse touched me in a no-no place

    @anonymous234 said:

    Silly question... the unused pictures (that people upload, possibly accidentally, but don't post) are being pruned right?

    I presume it's this:


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.