Bots be bottin'
-
I'm paging through Apache logs of a superseded webserver. DNS points to another server as of yesterday. Some long TTL on a few secondary domain names mean we still see some hits and I wanted to assess whether it's save to turn it off now. I wont hold back for this guy:
207.46.REDACTED - - [17/Mar/2016:06:41:09] "GET / HTTP/1.1" 200 29227 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:06] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:06] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:06] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:06] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:06] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:07] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:07] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:07] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:07] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:07] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 207.46.REDACTED - - [17/Mar/2016:06:42:07] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"Nothing much in
robots.txtin case you were wondering.
-
Oooh I just realized what it is. The bot loaded the homepage and got links to a few subdomains which point to the same server. Before querying these it dutifully requested their robots.txt. So, not really a WTF.
-
That's just the Bing Bot; I wouldn't worry about it
-
The Bing Bot runs Mozilla on an iPhone? I knew Microsoft has little love for their own platform, but that is adding insult to injury. Or is Bing Bot a 'garage' project?
-
The Bing Bot runs Mozilla on an iPhone?
you havent seen the user agents for IE lately?
the IE10 User Agent String is.......
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)DUN DUN DUNNNN!
-
It is strange that it's pretending to be an old iPhone though. Is it trying to crawl for a mobile page, too?
-
-
-
Every browser's user agent string starts with 'Mozilla', and usually mentions at least one other browser; Chrome's mentions Safari, I know that.
-
Now ... try to explain that to a group of 6th graders.
WTF is WRONG with our industry?! o_O
-
Every browser's user agent string starts with 'Mozilla', and usually mentions at least one other browser; Chrome's mentions Safari, I know that.
I refer to the link above. It's actually quite clever and explains the issue well :)
-
-
It PAAAAAINS us, precious!
We wants it to be better!
-
Sadly I'll take 95% backwards compatibility to:
"Why old-thing no understand new-thing protocol?" : "That is called forwards compatibility" : "Me no understand, if old and new thing talk must be backwards compatibility!" : "That's bizzaro-world compatibility but whatever..."
-
I ... I don't ... was what English?
-
WTF is WRONG with our industry?! o_O
Backwards compatibility and lazy people.
I saw this and ran-with-it.
-
-
I thought I was making fun of people that can't tell the difference between forwards and backwards compatibility? I must have swooshed.
away folks!
-
Wait... was IE *ever* better than Netscape?
-
Yes. Specifically, IE4-6 were better than Netscape 4.
-
-
Expected? When I get home I might try playing with user agent string to see if that does anything.
-
EXCEPT... 5 to 5.5 on Mac OSX.
Those really sucked.
-
-
-
I keep reading this topic title as bottles be bottling
-
I really just wanted to use my ponies, didn't actually care
-
-
Sadly I'll take 95% backwards compatibility
If it was just backwards compatibility, it would be fine. But this is ass-backwards idiocy-to-idiocy compatibility. Because from the start, there should have been a feature list and not a user agent. But user agent is all we've got, so we run with it.
-
Even that could be cocked up by an unscrupulous browser
-
That's not what I remember. Anyway, the timeline of web browsers indicates that when IE 6 was released, Netscape 6.2 was also being released, so it's hardly fair to compare IE6 to Netscape 4.
Anyway, if my memory is correct, Netscape 4 was a lot better than Internet Explorer 5... IE at the time only allowed you to use
document.writewhile the page was loading; after the page layout was completed, you couldn't change anything that would've required recalculating the layout.
-
Anyway, if my memory is correct, Netscape 4 was a lot better than Internet Explorer 5... IE at the time only allowed you to use document.write while the page was loading; after the page layout was completed, you couldn't change anything that would've required recalculating the layout.
Pretty sure you have that backwards.
document.write was the only way Netscape 4 could add text to a page. They even added a special
<layer>tag that you could populate usingdocument.write.Whereas IE could modify text in existing
divs (and possibly other elements) using theinnerHTMLproperty.
-
That's not what I remember. Anyway, the timeline of web browsers indicates that when IE 6 was released, Netscape 6.2 was also being released, so it's hardly fair to compare IE6 to Netscape 4.
Should have put this in my last post, but a lot of people avoided Netscape 6 because it was crashy as fuck.
-
Even that could be cocked up by an unscrupulous browser
There is a difference. Answering “yes” to “do you support X?” for any X is easy, but listing all Xs to a question “what do you support?” is not.
WebAIM: History of the browser user-agent string
The Old New Thing
