Easing up on aggressive refactoring?

  • In the ClassicCityDevs Slack group's General channel, someone recently posted a link to a Why Is Computer podcast entitled 'Ways to Be Less Clever', which discusses ways even relatively sane practices such as the Red-Green-Refactor cycle can be mishandled or overdone. They specifically discussed the problem of premature refactoring; they argued that refactoring all apparent commonalities immediately after every green test cycle could obscure even better abstractions later on. This went together with an argument similar to the familiar one against premature optimization, namely that if you refactor before you have any user feedback, you could code yourself into a corner that would require you to roll back almost to the start of development.

    They argued that it is better to wait until you have both isolated several candidates for refactoring, and until you have more information about what the program is and isn't doing correctly.

    While I think that they are being wildly overly-optimistic in the assumption that a significant number of development shops are actually refactoring code, using TDD, waiting for minimum code coverage before releasing, not developing in production, or even reading their code once it is written unless forced to by customer complaints, for those who do actually know these principles at all, understand them, and more than lip service, this is a welcome anodyne to the aggressive 'refactor everything' mindset.

    I was only going to mention this in passing, to set up a discussion on a tool idea this gave me, but I realized this was a topic worthy of debate in itself. Any opinions?

  • Sometimes I can't separate the hipster from the genuinely good-intending person.

  • Honestly, it sounds like they're justifying a process of shitting horrible code without thinking, and then immediately cleaning up.

    I think it's time to slow down on the Agile buzz, and consider the value in being thoughtful when programming.

    It doesn't mean you have to be inflexible in your design, but it does mean you have to at least lay some groundwork before you start spitting syntax.

  • I dunno, while I can all too easily seeing that being what a lot of coders take from it, I don't think they intend that to be the message. Basically, what they are saying is that there are a lot of 'agile' coders who apply DRY but have either forgotten or are ignoring YAGNI, which almost the opposite of that (not quite, because as I've said before most coders who bother to learn and regurgitate the Agile cant at all seem to believe "YAGNI == don't think, code", which, as you say, is the bigger undefined).

    My bigger thing is that they seem assuming Agile and trying to fix it, when the real problem is that most coders and development shops aren't even at the point where they can understand the arguments put forth by Beck, et. al., never mind form any rational opinions on them and decide whether they fit their circumstances or not (which they often won't).

  • @ScholRLEA Simple rule.

    It if works don't fix it.

  • Winner of the 2016 Presidential Election

    @lucas1 said in Easing up on aggressive refactoring?:

    It if works don't fix it.

    Well, there are different definitions of "works"…

  • Useful and simple, yes, but not applicable. While refactoring for its own sake is pretty useless, the point is to apply it in the planning and coding stages of the development and maintenance cycle, which, by definition, means that the software either isn't working, isn't finished, or needs to be changed because the needs have changed.

    As for only changing the 'broken' or 'new' parts and leaving the rest alone, that sort goes to the purpose of refactoring itself, which is that when you are updating a program, if the code that you are trying to change runs but can't be understood well enough to make the changes is just as broken and code that doesn't run at all. The protocol is that you isolate the parts that need fixing, see if there's a way to make them simpler and easier to understand so that fixing them goes quicker (or maybe find a way to avoid having to fix them at all), then fix them.

    Yes, it is not uncommon for Agile zealots to refactor code regardless of whether it needs to be changed or not. That's the very same problem which the people in that podcast are talking about.

  • @lucas1 said in Easing up on aggressive refactoring?:

    @ScholRLEA Simple rule.

    It if works don't fix it.

    Until it stops working, and you realize that the code diarrhea that's been accumulating for the past year or so means no one has any idea how to fix it anymore.

    Especially in TDD, where the first pass is banging out code that meets the test cases and literally nothing else. Which is why I hate TDD - because how many developers do you know that won't just stop there?

  • @Maciejasjmj said in Easing up on aggressive refactoring?:

    Especially in TDD, where the first pass is banging out code that meets the test cases and literally nothing else. Which is why I hate TDD - because how many developers do you know that won't just stop there?

    This is a symptom of a bigger problem, common to pretty much all development processes but particularly severe in some branches of Agile: a sort of target fixation, where a developer, manager, or team focus on a single heuristic as if it were an absolute rule, while ignoring equally important heuristics, including the ones which are necessary for the target technique to be useful.

    On a wider scale, it can lead to 'golden hammer' behavior - the same processes, rules, and methods get applied mechanically, without thinking out which ones are going to be applicable to the given situation. Ironically, the whole purpose of 'agile', and one of the reasons it is called that, was to try and break away from that sort of dogmatic mindset - a clear instance of what Hegelians and Marxists call 'dialectic progression', in which a process intended to replace a non-functional dogma becomes dogma itself. In this case, it became dogma almost before it was fully worked out, which I expect must drive the originators up the wall.

    Part of the problem is that Agile (like OOP) came on the scene at a point where a set of problems it was very well suited happened to become a major focus of development work, leading to the belief that a) it was suited for all kinds of work, b) anyone could use it regardless of experience or skill, and c) you could make it work just by following a set of rules, without needing to understand the reasoning behind those rules.

    The problem was, none of these assumptions were true. Most of the times the process worked it was because the people using it were the ones who developed it, and so they knew it in detail, and knew it's pitfalls and traps. Since they were already conscientious, experienced developers with enough insight to come up with these and debate these principles in the first place, they had the skills needed to work well regardless of the processes they used; they were cherry-picking those techniques which worked for them before, and passing those along to others. They could make it work as a set of heuristics because they were only using it for things it worked well for in the first place, and knew which parts made sense to use in a given situation.

    As an aside, let's consider the things Agile is suited for, and see what it tells us. First, like with all craft procedures, it depends in the experience and knowledge of the craftsperson. This by itself says that it isn't going to work if your whole team is inexperienced, which should be obvious but apparently isn't. The whole view of developers as unskilled assembly-line workers or plug-compatible parts is high on the list of reasons why software projects fail, though as with many things the opposite view - the 'rock star developer' crap - is just as bad, and mixing those two contradictory approaches a la Alex St. Jackass is pretty much insane. Lack of nuance is a critical failing for any software methodology, but Agile is all about nuance.

    The second is that it is suited for projects that have most of the following characteristics:

    • Relatively fixed user base
    • A strong dialogue between the stakeholders, and especially, frequent user feedback to the developers
    • High granularity of the design, such that one part can be designed with minimal consideration of the other parts
    • Rapid turn-over of minor changes, but relatively slow turn-over of major ones
    • A limited subject domain that all the shareholders understand
    • Loose coupling between the components, such that adding or changing one part does not significantly change others
    • Both high stereotypy and high variability in the user interface (i.e., several forms with a shared general layout but many minor variations)
    • Few shared or centralized resources, if any

    Now, what sort of system does that sound like to you? If you said 'intranet', give yourself a gold star. Agile rocks at intranets, and it is no coincidence that it became a major design philosophy at the same time intranets were becoming common.

    What it doesn't do well at are things which are tightly coupled, need a lot of broad-scoped design up front, have limited direct user feedback or have to serve many different user needs, and change their structure or interfaces slowly - in other words, things like databases, which is why DBAs tend to be the ones who dislike Agile the most. Trying to force an Agile approach on database development, especially in a very rule-oriented manner, is pretty much a guarantee for disaster.

    But what do intranets usually need as support, and are often just a front end for? A database. Which means that a one-size-fits all process cannot be applied, even within a single project.

    My point is that the real thing lacking in most development processes is forethought - not just of the design, but of the process itself. All too often, there's no process in the process.

  • I've experienced several instances of what might be considered the opposite. After digesting the customer requirements, I found a way to reuse code for multiple purposes. Later, when the customer came up with additional requirements (or perhaps remembered their actual requirements) I had to break them apart because it turned out that they really wanted the two things to behave very differently.

    It was a little bit depressing to have to break up the elegant things I had made, but thems the breaks.

  • @ScholRLEA said in Easing up on aggressive refactoring?:

    while I can all too easily seeing that being what a lot of coders take from it, I don't think they intend that to be the message

    I think it helps to understand the process of coding better, if

    1. Stop thinking about coding like you're building a fixed structure, and refactoring means moving physical parts around.
    2. Start thinking about coding like each iteration is designing the next year's model of a make/model car.

    As you code, you're throwing parts in there, but when you go to design the next make/model, rearrange your modifications to fit better.

    Another way of thinking about it, is how Bethesda incorporates the most popular public mods into the base design of their next game, and it fits better than the previous mod did.

    If I take that frame of mind, refactoring after every green phase, is simply starting the design of the next model by rearranging everything you added to the previous model to fit better.

    And I still end up getting the impression that the article is advocating coding sloppy and fixing it later.

  • Discourse touched me in a no-no place

    @ScholRLEA said in Easing up on aggressive refactoring?:

    the 'rock star developer' crap

    Most “rock star developers” are really just a bunch of assholes using (the software-development equivalent of) autotune.

Log in to reply

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.