PHP, Perl, Python-- how three scripting languages starting with P fucked-up their respective releases



  • Interesting article. With an unhealthy focus on Perl.

    I should also mention that his naive "just start writing Perl 6 code" advice is the exact same advice Python 3 developers gave out-- it didn't really work.

    (Although, unlike Python, Perl apparently has a 'mode' to run legacy code in the same project as Perl 6 code, which is going to be pretty-impressive-- if it works. Then again, Perl also has like 6 users remaining and zero flagship apps.)



  • @blakeyrat said:

    Then again, Perl also has like 6 users remaining and zero flagship apps.)


  • FoxDev

    I'm not sure a Puyo-Puyo clone counts as a flagship app



  • I thought it was a Bust-A-Move ripoff. (Even for an open source project, it has a disastrously awful website. Attempt to download it. I dare you. Hint: the GIGANTIC "download" text at the top of the page is not a link.)

    Either way, yeah, a shitty clone of an existing game is not exactly what I meant.



  • @blakeyrat said:

    I should also mention that his naive "just start writing Perl 6 code" advice is the exact same advice Python 3 developers gave out-- it didn't really work.
    I think this is too pessimistic. I don't do heavy Python development, but I do do some, and from what I can tell, most of the libraries and such that I want are dual 2&3 and have been for a while. The scripts I write I think are all trivially 2to3-able, and I make heavy use of __future__ imports now.

    The article you link cites this one that says that only 9900 of 55000 packages (presumably on PyPi) are Python 3, but just looking at that number has a bunch of problems; probably the two biggest ones are:

    • It doesn't take into account the importance of packages. PIL has a Python 3 "version" in Pillow, lxml is 2/3, Pygame is 2/3. Twisted is the biggest Python 2-only library I can think of off the top of my head (and is, admittedly, a very notable omission).
    • It just looks at whether the package says it works with Python 3. Probably a large chunk of the "2-only" packages would work fine under Python 3 with no changes other than 2to3.

    At this point, the biggest thing I suspect is keeping people on 2 is "convenience" -- if you have a Python installed, it's probably Python 2, so you write to run on Python 2. When distros start shipping Python 3 as the default Python, people will start writing Python 3 more. There is some chicken-and-egg problem there, and it'll take a bit more time, but I think it's all but guaranteed to happen.

    I think you'll see adoption continue increase slowly until a big distro or two switches the default, and then I think the picture will quickly switch to the inverse of what it is currently: most big packages will continue to be 2/3 for a while but small packages and end programs will be mostly Python 3. (Or, if you're looking at PyPI counts, completely defunct with no updates in several years.)



  • @EvanED said:

    I don't do heavy Python development, but I do do some, and from what I can tell, most of the libraries and such that I want are dual 2&3 and have been for a while.

    When I was writing Python to do REST API imports and "data science-y" stuff, there was nothing that was 3.x compatible. That was only a bit over a year ago. (And I couldn't find a working SOAP library for Python 2.7 at all, but that's an old thread.)

    @EvanED said:

    I think you'll see adoption continue increase slowly until a big distro or two switches the default, and then I think the picture will quickly switch to the inverse of what it is currently: most big packages will continue to be 2/3 for a while but small packages and end programs will be mostly Python 3. (Or, if you're looking at PyPI counts, completely defunct with no updates in several years.)

    Right; but as the article says Python 3 was released in 2009. 2009. That's a long time to get to the point of "meh, maybe people will start using it I guess..."


  • BINNED

    Well, PHP 7 looks like it's slowly, very slowly moving in the right direction.

    I'm cautiously optimistic about PHP becoming decent one day. Maybe.



  • Good article.

    Is there an example of a language that made a successful code-breaking transition and came out fine?

    C# had a big step from 1 to 2, but in the best Microsoft tradition, everything remained backwards compatible. You could write C# 1 code today and it will compile into modern C#.

    Java has never changed much, it seems. Its changes were mostly a series of disappointing half-measures anyway.

    C has remained the same as it ever was.

    C++ is just adding more and more crap, but it's not breaking backwards compatibility, so it doesn't count.

    I would actually say fucking javascript is the closest thing we came to successfully moving a programming language forward. And even there, most breaking changes are hidden behind the "strict" switch and will only become mandatory with ES6.



  • @cartman82 said:

    Is there an example of a language that made a successful code-breaking transition and came out fine?

    There's no such thing.

    This is why, if you're a language or library developer, you need to spend a LOT of time thinking about shit BEFORE you ship. Because every line of code you write, every single one, you need to support. FOREVER.

    @cartman82 said:

    I would actually say fucking javascript is the closest thing we came to successfully moving a programming language forward. And even there, most breaking changes are hidden behind the "strict" switch and will only become mandatory with ES6.

    JS has never made any breaking changes I'm aware of.


  • FoxDev

    @cartman82 said:

    "strict"

    'use strict';

    yep. love that thing. it's one of the best examples i've seen of mode switching in a way that's backwards compatible. because nothing that's valid in strict mode is invalid in non strict mode and it all works the same you can develop in strict mode and know that your application will properly function in a environment that does not support strict mode.

    when ES6 finally hits that will be interesting, but since JS has a rich history of using polyfills to get around using new features in old environments i anticipate that it should be fairly smooth transition.



  • @blakeyrat said:

    JS has never made any breaking changes I'm aware of.

    With strict mode, a lot of the old garbage no longer work. Specifically, the with keyword and a bunch of arguments related stuff that I'm actually very fond of.

    And this will become the default with ES6.

    In fact, with the generally good vibes surrounding ES6, seems like they'll be the first ones to actually make the transition and not screw up.



  • @accalia said:

    yep. love that thing. it's one of the best examples i've seen of mode switching in a way that's backwards compatible.
    I'm sure this is based off of a pipe dream, but I wish they had done "use es5 scrict;" or something to allow paving the way for other breaking changes in the future (with a "use es7 strict;" or whatever).

    I am playing around with JS a bit for a personal project (and in the process learning a lot about it and new HTML stuff), and when I first came across strict mode it struck me as not even fixing the things I find most objectionable about JS. After watching some videos of Douglas Crockford talks I now get why they picked the JS parts that they did to fix, and I see the value and think it's well-designed, but I still have a pipe dream that someday we'll get a version where values don't get implicitly coerced to/from strings if you blink at them wrong.


  • Winner of the 2016 Presidential Election

    @cartman82 said:

    With strict mode, a lot of the old garbage no longer work. Specifically, the with keyword and a bunch of arguments related stuff that I'm actually very fond of.

    That's not a breaking change. Just don't add "use strict"; and your old code will continue to work.


  • 🚽 Regular

    @EvanED said:

    I'm sure this is based off of a pipe dream, but I wish they had done "use es5 scrict;" or something to allow paving the way for other breaking changes in the future (with a "use es7 strict;" or whatever)

    Nothing prevents using "use strict; require es7"; in the future. Heck, it even gives you more granular control.


  • Discourse touched me in a no-no place

    @cartman82 said:

    Is there an example of a language that made a successful code-breaking transition and came out fine?

    It's happened quite a few times. The usual reason for success is that there's a huge carrot — often performance or greater safety — to encourage people to move. Programmers will do a lot to squeeze more speed out of their systems, and they hate doing debugging…


  • Discourse touched me in a no-no place

    @blakeyrat said:

    This is why, if you're a language or library developer, you need to spend a LOT of time thinking about shit BEFORE you ship. Because every line of code you write, every single one, you need to support. FOREVER.

    Ha ha. No. That literally does not happen. But if “forever” is a definite length of time ( ;) ) like 10 years, then it does happen.



  • Speaking of Python 3 adoption rate being "abysmal", he really forgot that the adoption rate of Perl 6 is next to nonexistent.

    What the hell. People do write Python 3 code today, and we have real stuff that works in production environment. I treat Python 2 and 3 as if they were different languages, though awfully similar.

    Let me repeat that: we have Perl 6. It works, it will get a major release this year, and it is going to come with many more features than originally promised.

    And still zero production deployments. While PHP folks fucked up the PHP6 effort (because Unicode is hard), I believe the PHP7 is going to be real, with smooth upgrade path. Python 3 is going to get its sixth major release this year. And Perl 6 is getting a major release this year (a mantra they iterate over every year, I heard this in May 2013 at a Perl conference).



  • @wft said:

    And still zero production deployments.

    Because perl 6 isn't officially stable yet?

    If people are complaining it is slow then we won't be moving to it (from perl 5.10.1 if you don't read my other threads): we'd probably change to a completely different language if/when support is dropped for perl 5. We found even "use utf8;" slows things down measurably, which is probably where a lot of the slowness comes from in perl 6.



  • @dkf said:

    Ha ha. No. That literally does not happen. But if “forever” is a definite length of time ( ) like 10 years, then it does happen.

    Microsoft is still supporting stuff from 30 years ago.

    So is Lunix people, but with the caveat that:

    1. Their support is crappy shit ass dung
    2. They actually think the 30-year-old bullshit is good

    Apple just kind of deletes every HD in the place every 5-10 years or so, because it's not hip and trendy to work on code old enough to vote.



  • @wft said:

    Speaking of Python 3 adoption rate being "abysmal", he really forgot that the adoption rate of Perl 6 is next to nonexistent.

    How many people are using Perl at all? I thought the entire language was kind of dead other than legacy shit.



  • My company uses it. I know of at least one other company using it for new projects.

    Booking.com uses Perl almost exclusively, they have lots of legacy shit, but they've got lots of new shit too.



  • @EvanED said:

    I don't do heavy Python development, but I do do some, and from what I can tell, most of the libraries and such that I want are dual 2&3 and have been for a while.

    Yeah -- my (admittedly limited) experience with 2 vs 3 portability is that it's not a terribly huge deal to write code that works on 2.7 and 3.3+, although six is sometimes needed to help out in corner cases.

    @EvanED said:

    At this point, the biggest thing I suspect is keeping people on 2 is "convenience" -- if you have a Python installed, it's probably Python 2, so you write to run on Python 2. When distros start shipping Python 3 as the default Python, people will start writing Python 3 more. There is some chicken-and-egg problem there, and it'll take a bit more time, but I think it's all but guaranteed to happen.

    I think you'll see adoption continue increase slowly until a big distro or two switches the default, and then I think the picture will quickly switch to the inverse of what it is currently: most big packages will continue to be 2/3 for a while but small packages and end programs will be mostly Python 3. (Or, if you're looking at PyPI counts, completely defunct with no updates in several years.)


    Arch has already made the switch, and AIUI, Ubuntu will be doing so in the near future...

    so, the 2-3 transition is moving like a Greenland glacier: slow, but eventually grinding down and pushing aside any obstacle in its way.


  • BINNED

    I seem to have both 2.7.9 and 3.4.2 installed. Python 2 seems to still be the default though.

    onyx@jarvis ~> uname -a
    Linux jarvis 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) x86_64 GNU/Linux
    onyx@jarvis ~> readlink /usr/bin/python
    python2.7
    


  • @Onyx said:

    I seem to have both 2.7.9 and 3.4.2 installed. Python 2 seems to still be the default though.

    Yeah -- Ubuntu might have held off on Python 3 for a bit. I'll have to do some research sometime...


  • BINNED

    @tarunik said:

    Ubuntu

    Debian, baby. Ain't no *buntu in this here house!

    Meaning I'm likely to have it switched to Python 3 a bit later, at least on stable.



  • Who cares about Linux? OS X ships 2.7 and literally everybody I've ever known whose into Python uses OS X.


  • ♿ (Parody)

    @blakeyrat said:

    whose

    WHO'S


  • BINNED

    Python stuff runs on the servers as well. I'd say it's most useful there, for both automating stuff and running services.

    How many OS X servers have you seen kicking around recently?



  • MOBILE PHONE<fuckyou>



  • 😆

    (I have to say though, its Android version is OK. Although not ported by them, it's a third party project)



  • I don't know of any reason why Python couldn't just have added an extra "using version=3" keyword to specify that you're using Python 3 syntax and libraries, and maintained support and interoperability for Python 2 code.

    Heck, why hasn't a third party made a Python interpreter that does that yet?
    @EvanED said:

    I'm sure this is based off of a pipe dream, but I wish they had done "use es5 scrict;" or something to allow paving the way for other breaking changes in the future (with a "use es7 strict;" or whatever).

    Every language and file format should start with something like "javascript version=es5". Then it's much easier to make changes to newer versions.


  • FoxDev

    @anonymous234 said:

    I don't know of any reason why Python couldn't just have added an extra "using version=3" keyword to specify that you're using Python 3 syntax and libraries, and maintained support and interoperability for Python 2 code.

    mainly because there are breaking syntax changes between varsion 2 and 3. it's not just a case of libraries and keywords. the actual syntax of the language changed.

    yeah that could have been worked around, but that's a much bigger problem than javascript's "use strict"; statement which merly disallows certain things when in affect. the same parser can parse strict and nonstrict javascript so that's easier to implement.



  • @anonymous234 said:

    Every language and file format should start with something like "javascript version=es5".

    That's one thing HTML and XML got right. Shove the version number right there in the file.



  • So read the first line to determine the version and run the parser after that?

    I mean, I'm sure there's a lot of smaller problems to solve, but it definitely seems possible and worth it.


  • FoxDev

    okay, then how do you solve mixing versions in includes, because your project includes might not have upgraded yet.

    the transition from Python 2 to 3 was a huge shift, to the point where it was impossible to do automatically (the 2to3 conversion tool had several places where it threw up its hands and said "nope. can't do that" and python lets you write modules in C, that's where numpy gets most of its speed from, and the ABI for those modules changed drastically.

    i'm not saying it wasn't a problem that couldn't be solved, just that i think Guido and co. made the right choice. maintaining cross compatibility was probably more trouble than it was worth.



  • @accalia said:

    okay, then how do you solve mixing versions in includes, because your project includes might not have upgraded yet.

    You do the .net/Java thing and JIT compile both versions of the language to the same (or functionally-identical) bytecode. If they're smart.

    If they aren't, and there's no stable bytecode backing Python, then you're just kind of fucked.


  • BINNED

    On JavaScript front, it's easy in the browser at least, just stick it as an attribute in the <script> tag.

    Anything else that could mix library versions - any original code should just run the same version, version information could be added to the include / import / whatever statement.


  • 🚽 Regular

    @blakeyrat said:

    That's one thing HTML and XML got right. Shove the version number right there in the file.

    Don't forget encouraging the declaration of character encoding.

    Incidently, you can do the same in a comment in Python, like this:

    # -*- coding: utf-8 -*-
    

    They could have done the same for declaring language version.


  • FoxDev

    @blakeyrat said:

    You do the .net/Java thing and JIT compile both versions of the language to the same (or functionally-identical) bytecode.

    As i understand it this was impossible because:

    • there were multiple ABI changes that needed to happen to support the new features of Python3 and remove the obsolete features from Python2
    • Not all python code is precompiled to bytecode on disk, most of it stays as source code
    • In order to compile on the fly two interpreters would need to be maintained
    • Unlike Microsoft all developers on the Python project are volunteers and they didn't want to support the multiple parsers (that's really really really hard, just from the bug fix/regression perspective. the combinatorics between different combinations of versions in source files alone!)
    • BDFL Guido said they wouldn't support multiple parsers.

    @blakeyrat said:

    If they aren't, and there's no stable bytecode backing Python, then you're just kind of fucked.

    why do you think i now use Javascript for my personal projects? Moar Power! Moar stability! Moar Awesome! Less version conflicts!



  • @accalia said:

    there were multiple ABI changes that needed to happen to support the new features of Python3 and remove the obsolete features from Python2

    That's a valid reason, assuming those features are good enough to justify the change. .net has done that itself now twice.

    @accalia said:

    Not all python code is precompiled to bytecode on disk, most of it stays as source code

    I didn't say "pre-compiled", I said "JIT-compiled." Different. There's no reason they couldn't implement this; they have all the pieces in place.

    @accalia said:

    In order to compile on the fly two interpreters would need to be maintained

    So? Interpreters are small.

    @accalia said:

    Unlike Microsoft all developers on the Python project are volunteers and they didn't want to support the multiple parsers (that's really really really hard, just from the bug fix/regression perspective. the combinatorics between different combinations of versions in source files alone!)

    Ah, right, the classic open source reason: "We don't give a shit about actually creating a quality project."

    And BTW, being a volunteer isn't an excuse. This is plain ol' open source laziness. A lot of volunteers in other fields do work as well or better than professionals-- look at Habitat for Humanity, for an example. It's just open source where "volunteer" == "do a shitty job".



  • @blakeyrat said:

    That's a valid reason, assuming those features are good enough to justify the change. .net has done that itself now twice.

    If anything, the whole str/unicode/bytes dance had to happen at some point, and it's a breaking (even ABI-breaking) change then and there. I'm sure there were others of varying magnitude as well...



  • I would like to point-out for the record to people who bitch that Win32 API doesn't support UTF-8 when (most of) it was built after UTF-8 was invented that neither does/did any of the three scripting languages mentioned in this post. (Only Perl, IIRC, pre-dates UTF-8. The others have no excuse.)


  • Discourse touched me in a no-no place

    @blakeyrat said:

    You do the .net/Java thing and JIT compile both versions of the language to the same (or functionally-identical) bytecode.

    Assuming that nothing else has changed significantly. Standard library changes are major factors as well, and if someone's got code that depends on bugs in old systems being there, their stuff will be broken by newer code. (I actually do have some source code from 20 years ago that still works. Across a major version change too. I definitely wouldn't write it that way now though…)

    I've done supporting stuff for 10 years or more. I've done maintaining stable ABIs over that period of time. There's a fucking limit, especially if I'm not being paid for it. Unless you're making the assertion that only megacorporations should be allowed to build languages and libraries; if you're doing that, FUCK YOU.



  • @dkf said:

    Unless you're making the assertion that only megacorporations should be allowed to build languages and libraries; if you're doing that, FUCK YOU.

    I'm making the assertion that only corporations (or at least for-profit companies that pay developers money to do boring tasks) can do a good job of building languages and libraries.

    Volunteers can do whatever they want, they don't need my permission or anybody else's. If it's open source software volunteers, it just inevitably turns out shitty.


  • Discourse touched me in a no-no place

    @blakeyrat said:

    Only Perl, IIRC, pre-dates UTF-8. The others have no excuse.

    Python predates UTF-8 by around a year. (1991 vs 1992.) Perl is from 1987, and PHP from 1995. Which isn't to say that any of them have excuses.


  • FoxDev

    @blakeyrat said:

    That's a valid reason, assuming those features are good enough to justify the change.

    from what i've seen of those changes, i believe that they are.

    @blakeyrat said:

    Interpreters are small.

    the code to parse to a symbol tree is small, the code to actually run that symbol tree much less so, the code to switch between different intepreters is less so. the code to handle the cases where the callee and the caller disagree on the ABI to be used even less so.

    it's not that they couldn't have supported the multiple parsers and runtimes, but then they exponentially increase the number of odd edge cases that they now have to cover when interpreting projects that have different python versions. maintaining a language is already hard enough. one doesn't need to add the extra complications of supporting multiple incompatable versions of the language.

    remember that because of the ABI changes the suggestion to run one interpreter (or switch between interpreters) to execute both versions is roughly equivalent to saying that it's possible to run one interpreter for Java and JavaScript. it's technically possible, but why would you do that?

    @blakeyrat said:

    Ah, right, the classic open source reason: "We don't give a shit about actually creating a quality project."

    well if you want to represent it in an anti opensource way. I'd prefer to phrase it as "We're going to provide the best product we can with the time and resources available to us, understanding that 95% of our developers already have day jobs that they need to keep so they can support their families"

    @blakeyrat said:

    And BTW, being a volunteer isn't an excuse.

    No, it's not an excuse, but it does mean that as a volunteer you generally don't have a full 40+ hours a week you can devote to the organization you are volunteering for. This in turn does limit the amount of stuff you can do for the organization in a given amount of time.



  • @accalia said:

    the code to parse to a symbol tree is small, the code to actually run that symbol tree much less so, the code to switch between different intepreters is less so. the code to handle the cases where the callee and the caller disagree on the ABI to be used even less so.

    it's not that they couldn't have supported the multiple parsers and runtimes, but then they exponentially increase the number of odd edge cases that they now have to cover when interpreting projects that have different python versions. maintaining a language is already hard enough. one doesn't need to add the extra complications of supporting multiple incompatable versions of the language.

    remember that because of the ABI changes the suggestion to run one interpreter (or switch between interpreters) to execute both versions is roughly equivalent to saying that it's possible to run one interpreter for Java and JavaScript. it's technically possible, but why would you do that?

    You misunderstand me entirely.

    @accalia said:

    well if you want to represent it in an anti opensource way. I'd prefer to phrase it as "We're going to provide the best product we can with the time and resources available to us, understanding that 95% of our developers already have day jobs that they need to keep so they can support their families"

    95% of Habitat for Humanity volunteers already have day jobs, etc, and the houses they build are perfectly average in quality. Or better.

    @accalia said:

    No, it's not an excuse, but it does mean that as a volunteer you generally don't have a full 40+ hours a week you can devote to the organization you are volunteering for. This in turn does limit the amount of stuff you can do for the organization in a given amount of time.

    I'm just saying other volunteer organizations have figured it the fuck out.


  • FoxDev

    @blakeyrat said:

    You misunderstand me entirely.

    i disagree, i think i understand your point perfectly. but I welcome any additional information you wish to give me regarding your point or how you feel i'm miossing your point.

    @blakeyrat said:

    95% of Habitat for Humanity volunteers already have day jobs, etc, and the houses they build are perfectly average in quality. Or better.

    this is true, however those 95% of people constructing the house for habitat for humanity are simply following plans laid out by an architect, They also do not have any interoperability concerns with other houses that they built because houses are complete stand along things, not frameworks upon which other things are built.

    Your analogy is flawed there. The house once built is not maintained by the builders of the house, but by the occupants the house was built for/sold to. a language is maintained by the builders.

    @blakeyrat said:

    I'm just saying other volunteer organizations have figured it the fuck out.

    other volunteer organizations can manage with 90 to 95% unskilled or semiskilled labor, overseen and directed by 5 to 10% skilled persons who direct the work of the other laborers, and are generally employed full time by the nonprofit in question.

    Additionally most of those volunteers that show up to construct the house are on a one or two day charity drive, either sponsored by their place of employment or using their paid (or unpaid) time off. These are not volunteers that need to put in a full days work at their day job and then volunteer for their charity/nonprofit.

    I'd love to see a non profit organization that requires the level of technical skill on the part of all volunteers that software development needs be organized and successful using the same business model as habitat for humanity manages.



  • @blakeyrat said:

    95% of Habitat for Humanity volunteers already have day jobs, etc, and the houses they build are perfectly average in quality. Or better.

    Instead of LOC (debugged, tested, quality-ed) .... they're limited by how many houses that they can build...

    And the technology for building a single-family home hasn't been changing as fast...


    Filed under: "Dang... they upgraded my lumber from 3.1 to XP and now my old blueprints aren't compatible!!"

    (But, I'm with you, the OSS, "it works, you just have to.... install ten new things..." thing is awful)

    And... :hanzo:'d in a very complete way by @accalia


  • Discourse touched me in a no-no place

    @accalia said:

    it's not that they couldn't have supported the multiple parsers and runtimes, but then they exponentially increase the number of odd edge cases that they now have to cover when interpreting projects that have different python versions. maintaining a language is already hard enough. one doesn't need to add the extra complications of supporting multiple incompatable versions of the language.

    It also gets harder once you start integrating the standard library with the language. Libraries tend to change rather more often than languages, and they're not just a simple matter of “recompile the bytecode engine” since they've also got to do things like integrating with the OS. The expectations for such things change over time. Go back 20 years, and nobody really expected a lot of multi-threaded code (hardly anyone had the hardware to support it). Nowadays, everyone expects it. That's a fundamental difference in terms of implementations, with far-reaching consequences.


Log in to reply