Cross-platform zips
-
yeah. that's not the same....
okay what's going wrong in the transfer then?
-
I've been holding off mentioning this because it seems so utterly stupid it can't possibly be this, but this isn't another case of line endings, is it? Maybe the ZIP is being transferred as a text file instead of binary? It would explain both the corruption Windows finds and the altered MD5, but then again, if 7-Zip can open it...
Anyway, something to check when you get to the straw-grasping stage.
Edit: OK, so it turns out I'm not the only one
going insanethinking it might be text transfer
-
Silly question: how are you transfering the file? You sure that it's not undergoing something weird like for example EOL-conversion because whatever is responsible for doing the transfer thinks it's text?
Edit: 'd
-
but then again, if 7-Zip can open it...
7-Zip might just be more resilient to stuff like that.
-
User error. I have like fifteen of these in my folder by now, I md5'd the wrong one.
77-0D-C4-A9-B3-E2-7F-36-1F-21-89-24-62-8E-83-28
vs770dc4a9b3e27f361f218924628e8328
on the working version76-E7-13-8C-1A-2A-03-06-67-53-59-EE-2B-A3-63-81
vs76e7138c1a2a0306675359ee2ba36381
on the broken one
-
User error. I have like fifteen of these in my folder by now, I md5'd the wrong one.
77-0D-C4-A9-B3-E2-7F-36-1F-21-89-24-62-8E-83-28
vs770dc4a9b3e27f361f218924628e8328
on the working version76-E7-13-8C-1A-2A-03-06-67-53-59-EE-2B-A3-63-81
vs76e7138c1a2a0306675359ee2ba36381
on the broken oneoh.... well it's not the transfer then....
what else is different?
/me is rapidly running out of ideas
are you sure using 7zip portable isn't an option?
-
So they're sending correctly...
-
Looks like the only option is trial-and-error i.e. build the zip folder-by-folder until it breaks.
-
I think my solution is to walk into the meeting today and say "This is stupid and I can't fix it and I refuse to let this hold up the process when we already got approval to put 7zip on the machines (and hell, I think they already did). Ideology be damned, it's fine to use 7zip."
-
+
-
Ah, the 'let's just get this shit done' approach.
I approve ;)
-
Maybe a stupid suggestion, but maybe Linux zip header doesn't match Windows expected zip headers, so it crashes and burns partway into the open file?
But I still support 7z being used. It's better compression choices, cross platform, password possible, encryption on file names, and has support for pretty much every major language to programmatically create and decompress files.
-
Thanks to ya'll in this thread I have documentation of my efforts to fix the issue, so I gave it a reasonable effort and came up with "not feasible" :D
-
maybe Linux zip header doesn't match Windows expected zip headers
That would likely be the issue, except I can get it working just fine in one server and not the other, and I'm still not sure what the difference is. Probably some library Zip depends on being a different version or something?
-
Have you tried upgrading / downgrading your sandbox zip program?
-
It wouldn't match prod then; they're identical versions as far as I can tell.
-
Apt get reinstall to force dependency reinstall?
Shrug.
I still recommend 7z
-
Agreed. My working theory is that Zip depends on, say,
lib-make-zip-work-on-windows
which is version0.notsucks
on the sandbox and0.suckshardcore
on prod. Or something. I'm not going to be able to solve that anytime soon.
-
I mean, you can, you just need to be willing to employ arson.
-
Ooh! I can set one particular guy on fire, and then his opinion doesn't count anymore because he's on fire, so I win, right?
-
I was more thinking about inside the box, but I hear outside the box solves problems too.
-
Though you'll show up scummy to any investigators...
-
Nah, I have Ninja :D
-
TBH I'm not sure how to calculate the MD5 sum on Windows
In addition to the powershell solution there's an executable, FCIV, you can download from the MS web site, that will calculate both MD5 and SHA-1 sums.
-
(A bit late to the party). But here's a thing: Is there a possibility that one or more of the files you want to zip has an excessively long path/file name. Before you reach for your flamethrower ignition switch consider this: In a production environment, the branch the files are in, are closer to the top of the tree (to bugger a metaphor). Whereas a dev / sandbox will often "bury" it (so much for the tree analogy) in sub-folders like "my-dev/sandbox/this-version etc.
Also, the question is based on my recent experiencing with transferring "archives" and an observed (possibly new) tendency for Windows to over analyze possible problems. The case in point: My Windows 8.1 throws a serious wobbly when it tries to install 32bit Apps form a time when 32bit was cutting edge and will not get past the banner that tells me it can't . Yet, if I copy an installed version, it works fine. Extensive research has led me to understand that the reason is because 64 bit windows "copes" with 32 bit apps by installing them in program file(x86) and anally invoking the contents of "system32" rather than those of "syswow64" - Old 32bit apps know nothing about this and want to install in program files. Because of the speed of the response I am guessing that Windows "emulates" what it is about to do and barfs at the result.
I would not be surprised if a similar "anti-pattern" is being used here. I.e. It has decided that there will be an issue in doing this and therefore barfs. The "corruption" may come from the fact that it cannot "read" a file.
Just asking / suggesting is all...
-
Is there a possibility that one or more of the files you want to zip has an excessively long path/file name.
Yes
In a production environment, the branch the files are in, are closer to the top of the tree
Nope. They're set up the same: /var/atlassian/bamboo/xml-data/build-dir/something/something/components/repoGoesHere
It has decided that there will be an issue in doing this and therefore barfs. The "corruption" may come from the fact that it cannot "read" a file.
Very likely :)
-
For curiosity's sake, since 7-Zip can open the broken files just fine, can you open one of them and post the results of Test and Info?
-
9 posts were split to a new topic: Windows compatibility mode
-
I'm not going to be able to solve that anytime soon.
Did they accept 7-zip as a solution, or do you still need help figuring this out?
I was reading a bit more about zip archive formats, and Windows 7+ should be able to open a Zip64 archive, so I think that whole conversation was a red herring.
node
:running:So... node version differences? The working one was installed with 0.10.36, while the non-working one was installed via 0.12.4....
Seriously though - if you can manage to switch node versions on your staging box, maybe you can get a repro.
If you do mange to get a repro that way, it suggests that it really is one file corrupting the archive and you can hopefully bisect the tree to figure out which one. I'm not sure what you could do with that info since presumably you need that file, but at least you'd know.
That still leaves the possibility that it's a super-hard-to-trigger bug in either
zip
or Windows' zip lib that won't bisect - you'll split the tree in half and be able to open both zips. If that happens I'll buy you some
-
Note that Zip64 and Deflate64 are different Zip compression algorithm. I think Windows only have native Deflate64 support.
-
-
Before Windows Explorer will show you what's inside a Zip file, it actually unzips the whole thing under %TEMP%. If it can't do that, it might plausibly declare the file corrupt. So before you give up entirely, you might want to run
find components -regex '.*[?<>\:*|"].*'
against the directory you're zipping up on the Linux side, just to make sure the helpful Node folks haven't used file or directory names that Windows can't create.
-
The relevant compression algorithm was going to be my first suggestion.