God's GIF to programming

[Zemm]

I have an interesting story today. A few months ago I was a new hire and trolling through the codebase I noticed that an image generating function would send out a the content-type http headers "text/html" instead of "image/gif". I flagged it and suggested a fix but the lead programmer shot it down saying that "sometimes it sends a PNG" and we shouldn't change something that works. My suggestion - that it doesn't really work (looking at mojibake on my screen) and at least using image/anything would be closer to the expected - fell on deaf ears. So I let it go.

Until today. All of our clients had a large part of their websites blocked by our own WAF due to a "GIF obfuscation backdoor". Whoops! So we had to scramble to whitelist etc. But I also learned something: my original fix would have only worked in a quarter of the cases. I spent about three hours this afternoon trying to figure why the others are not working. I'm on my way home right now and I still haven't figured it out. Something about how an ancient cpan module interacts with an ancient part of the codebase.

[bode]

I learned the word "mojibake" today, thanks.

[Guest]

@Zemm said:

(looking at mojibake on my screen)

I've had the same thing happen to me when I come across something that makes me laugh and I spill my lunch on my monitor.

[Ben L.]

If you had written it in Go, you wouldn't have had this problem.

[MiffTheFox]

@Ben L. said:

If you had written it in Go, you wouldn't have had this problem.

Or he could use Perl's solution.

[Sutherlands]

@Ben L. said:

If you had written it in Go, you wouldn't have had this problem.

Really?  Are you kidding me?  First off, you have no idea what language this is in.  Second, you act as though this is exclusive to Go.  Third, you think that a group of programmers who return text/html for a generic MIME type are smart enough to actually look up how to solve a problem.

[drurowin]

@Sutherlands said:

@Ben L. said:

If you had written it in Go, you wouldn't have had this problem.

Really?  Are you kidding me?  First off, you have no idea what language this is in.  Second, you act as though this is exclusive to Go.  Third, you think that a group of programmers who return text/html for a generic MIME type are smart enough to actually look up how to solve a problem.

I usually just send application/octet-stream and hope for the best.

[error]

I'm pretty well shocked that no one has pointed out the only Right Way is to keep the original mimetype stored along with the rest of the file metadata.

[MiffTheFox]

@Sutherlands said:

you have no idea what language this is in. 

@Zemm said:

an ancient cpan module

It's Perl.

[drurowin]

@joe.edwards said:

I'm pretty well shocked that no one has pointed out the only Right Way is to keep the original mimetype stored along with the rest of the file metadata.

My idea is more like telling the user agent "Here, it's YOUR problem now, asshole!  BAHAHAHAHA."

[spamcourt]

@drurowin said:

@joe.edwards said:

I'm pretty well shocked that no one has pointed out the only Right Way is to keep the original mimetype stored along with the rest of the file metadata.

My idea is more like telling the user agent "Here, it's YOUR problem now, asshole!  BAHAHAHAHA."

And that's the correct way according to linux. Metadata? I'll just look inside every file and check against my enormous magic string database to figure out the type, thank you very much.

[dkf]

@spamcourt said:

Metadata? I'll just look inside every file and check against my enormous magic string database to figure out the type, thank you very much.

It'd be OK (if wasteful) if it always got it right. But where would the fun be in that? Luckily, there's plenty of scope for confusion...

[Zemm]

@joe.edwards said:

I'm pretty well shocked that no one has pointed out the only Right Way is to keep the original mimetype stored along with the rest of the file metadata.

These are dynamically created images. Graphs etc. The new system uses hicharts so avoids this problem but the legacy area still generates these couple on the server and send them as gif or PNG, depending on the mood of the original programmer. I'm changing everything to png and figuring out how to print the header properly. For some reason the print "Content-type: image/png\n\n" is being output after the image data even though it's the first thing in that file.

[Severity_One]

@Zemm said:

For some reason the print "Content-type: image/png\n\n" is being output after the image data even though it's the first thing in that file.

That's because ou're doing it wrong.

It wouldn't have anything to do with flushing buffers, right?

[pjt33]

@spamcourt said:

@drurowin said:

@joe.edwards said:

I'm pretty well shocked that no one has pointed out the only Right Way is to keep the original mimetype stored along with the rest of the file metadata.

My idea is more like telling the user agent "Here, it's YOUR problem now, asshole!  BAHAHAHAHA."

And that's the correct way according to linux. Metadata? I'll just look inside every file and check against my enormous magic string database to figure out the type, thank you very much.

I'm not sure why you're singling out Linux here, given that IE wilfully ignores the Content-Type supplied by the server if its own magic tests get a match.

[blakeyrat]

@pjt33 said:

I'm not sure why you're singling out Linux here, given that IE wilfully ignores the Content-Type supplied by the server if its own magic tests get a match.

Wow this Statis Sleep Timepod technology is getting better and better. You just referred to something in IE 4 as if it has any relevance at all to the year 2013. Which, by the way, is where you woke up-- WELCOME TO THE WORLD OF TOMORROW! You must have been under for a full decade at least. Amazing.

[Sutherlands]

@blakeyrat said:

@pjt33 said:

I'm not sure why you're singling out Linux here, given that IE wilfully ignores the Content-Type supplied by the server if its own magic tests get a match.

Wow this Statis Sleep Timepod technology is getting better and better. You just referred to something in IE 4 as if it has any relevance at all to the year 2013. Which, by the way, is where you woke up-- WELCOME TO THE WORLD OF TOMORROW! You must have been under for a full decade at least. Amazing.

Well, it says "and later"... at the bottom of the page, there's a link saying it changed in XP SP2.  If you follow that link, that page also says it does MIME type sniffing with a link back to the original page.  That page also lists some registry directories for MIME handling, of which my work Windows 7 machine has the directory.

So, no time machine needed.

[PJH]

@blakeyrat said:

@pjt33 said:

I'm not sure why you're singling out Linux here, given that IE wilfully ignores the Content-Type supplied by the server if its own magic tests get a match.

Wow this Statis Sleep Timepod technology is getting better and better. You just referred to something in IE 4 [...]

... and later. The fact that the date on that article is only 8 months ago would indicate it still has some relevance. Of course, the fact that people still have problems in IE9 and IE10 because of MIME sniffing even in 2013, you'll claim have no relevance on the matter I'm sure.

[pjt33]

@blakeyrat said:

You just referred to something in IE 4 as if it has any relevance at all to the year 2013.

Some of the limitations of MS-DOS are still pissing off everyone who uses PowerShell in 2013, and that predates IE 4 by 16 years.

[Ronald]

@drurowin said:

@joe.edwards said:

I'm pretty well shocked that no one has pointed out the only Right Way is to keep the original mimetype stored along with the rest of the file metadata.

My idea is more like telling the user agent "Here, it's YOUR problem now, asshole!  BAHAHAHAHA."

I totally agree. That's why I usually not only send as text/html but I also make sure there is no extension or part of the url that could be used as a hint. Here, take this: /html/a0117c46-38cc-4d3a-8146-7a9fcad6732c?2=b



for the novice: the query string is there only to mess with CDNs

[MiffTheFox]

@spamcourt said:

And that's the correct way according to linux. Metadata? I'll just look inside every file and check against my enormous magic string database to figure out the type, thank you very much.

Except on Ubuntu (at least in 2008 when I last used it). "I'm not letting you open this file because I think it's a C++ file and it has the extension .js!" (Any time you tried to open a javascript file that started with /*.)

[morbiuswilters]

@MiffTheFox said:

@spamcourt said:

And that's the correct way according to linux. Metadata? I'll just look inside every file and check against my enormous magic string database to figure out the type, thank you very much.

Except on Ubuntu (at least in 2008 when I last used it). "I'm not letting you open this file because I think it's a C++ file and it has the extension .js!" (Any time you tried to open a javascript file that started with /*.)

Really? That bites.

[GNU_Pepper]

@Zemm said:

I noticed that an image generating function would send out http headers

 

Ladies and gentlemen, I give you: TRWTF. Sending HTTP headers from an image generation function makes about as much sense as if I used the rest of this post to write a letter to my grandmother.

Dear Grandmother,

I hope this letter reaches you well. Has your leg gotten any better since the fall? I heard the swelling had gone down and I was glad. You should take more care of yourself, anyway! You can't hold your liquor like you used to any more.

We're all fine here, and we're looking forward to seeing you at Christmas. Darryl's hockey team made it to the finals after all, but they lost. Darryl sulked for a day! Poor kid! Oh, and I didn't get that promotion at work. Never mind, eh?

Yours sincerely,

Content-type: text/html

PS: There needs to be a book about programming that companies are legally required to give to new employees that have never worked as programmers before. The book should be about 10 - 20 pages long in large bold print, and it should say things like "ONLY DO ONE THING IN EACH FUNCTION, AND MAKE SURE THE NAME OF THE FUNCTION TELLS YOU WHAT THAT ONE THING IS CLEARLY". Because apparently word is having trouble getting out.

 

[Ronald]

@GNU Pepper said:

Sending HTTP headers from an image generation function makes about as much sense as if I used the rest of this post to write a letter to my grandmother.

Suppose that you want to serve images that are resized on the fly (try this and change the 234 value to test - this is written in .Net but you'll get the idea) what do you suggest should be sent?

[Guest]

@Zemm said:

I have an interesting story today. A few months ago I was a new hire and
trolling through the codebase I noticed that an image generating serving
function would send out a the content-type http headers "text/html"
instead of "image/gif".

Fixed!

[morbiuswilters]

@GNU Pepper said:

Dear Grandmother,

I hope this letter reaches you well. Has your leg gotten any better since the fall? I heard the swelling had gone down and I was glad. You should take more care of yourself, anyway! You can't hold your liquor like you used to any more.

Your grandmother sounds swell!

@GNU Pepper said:

Filed under: Content-type: text/racist-joke

Well, at least it's less bloated than application/xml+racist-joke

[morbiuswilters]

@Zecc said:

@Zemm said:

I have an interesting story today. A few months ago I was a new hire and
trolling through the codebase I noticed that an image generating serving
function would send out a the content-type http headers "text/html"
instead of "image/gif".

Fixed!

Zecc vs. Zemm: The battle of the century!

[Severity_One]

@GNU Pepper said:

There needs to be a book about programming [...]

I've been thinking of writing such a book for years. A book that doesn't go into such things like polymorphism, recursion or other such complex subjects, but instead focusses on common sense, which unfortunately is very uncommon when it comes to programming.

How's your grandmother, by the way?

 

[Guest]

@morbiuswilters said:

Zecc vs. Zemm: The battle of the century!

It's funny because we're almost polar opposites:

Zecc (a West coast)	N 38° 42′ 49.72″	W 9° 8′ 21.79″
	S 28° 1′ 0″	E 153° 24′ 0″	Zemm (an East coast

[Guest]

)

[Zemm]

@GNU Pepper said:

TRWTF. Sending HTTP headers from an image generation function

I give you this: http://search.cpan.org/~chartgrp/Chart/Chart.pod#CGI_and_Chart. This is the exact module the 3/4 uses. But it doesn't work on our system: HTTP headers have to be sent out or it will 500 Internal Server Error, and they have to be sent prior to including the specific file that does the work. I didn't have time to figure how cgi_png() bypasses the buffer, but at least it does now send out the correct content-type...

@GNU Pepper said:

ONLY DO ONE THING IN EACH FUNCTION, AND MAKE SURE THE NAME OF THE FUNCTION TELLS YOU WHAT THAT ONE THING IS CLEARLY

I used the word function not as a "sub" (perl syntax) or method or whatever - but as an overall "thing the system does". The graph is shown using something like <img src="/cgi-bin/dashboard.cgi?fn=graph_something&foo=123&bar=321"> which, assuming fn=function, makes "image generating" a "function". (This is why so much was blocked by the WAF: all "dashboard" URLs are the same before the question mark). This is from the system from before even the lead programmer, so I can't completely blame him!

[Zemm]

@Zecc said:

It's funny because we're almost polar opposites:

Wow you got within 2km of where I used to live.

@morbiuswilters said:

Zecc vs. Zemm: The battle of the century!

Bring it!

[Liquid_Egg_Product]

@GNU Pepper said:

"ONLY DO ONE THING IN EACH FUNCTION, AND MAKE SURE THE NAME OF THE FUNCTION TELLS YOU WHAT THAT ONE THING IS CLEARLY".

[code]
function IncrementIntegerByAmount(int inputVariable, int amount) {

inputVariable = inputVariable + amount;

}
[/code]

But I have ANOTHER function where I have to do IncrementIntegerByAmount multiple times, so that function will have to do more than one thing...what to do, I'm so confused.