How to learn software design



  • Hi there,

    long time lurker here (silently suffered through the :disco: :horse: on mobile), now I finally created an account, so I can actively help derail some threads, instead of just waiting :trolleybus:

    Tl;dr version: I can code decently, but I am crappy at designing/planning/structuring a software project. How can I learn this?

    Some Background:
    Originally, I studied Physics, but I am now branching into scientific computing with my masters course, since I found that "pure" physics isn't really for me and I enjoy coding more.
    While my masters program teaches quite a lot about the mathematical concepts and optimal algorithms (I would probably have no problems coding a decently optimized and parallelized solver for PDEs), it teaches jack shit about actually writing full programs. Especially the planning/design-phase and structuring of your code is completely absent, not counting that one lecture about UML in the "intro to C++" class (another :wtf: on its own). Therefore I have quite a few problems, when I'm trying to create applications from scratch. I either end up with a horrible, messy pile of ravioli glued together to form sphaghetti, am refactoring large and basic parts of codebase almost constantly, or both.

    Since I don't want to end up as frontpage material, I feel I need to get better at planning and designing the application from the beginning. Is this just something that will improve on its own, when I gain experience, or is there something I can do to improve faster? Also, I'd be happy, if you can recommend good teaching material (books, websites, whatever)!


  • Winner of the 2016 Presidential Election

    @ScienceCat said in How to learn software design:

    Is this just something that will improve on its own, when I gain experience, or is there something I can do to improve faster?

    One thing I can recommend is to read code written by others. Try to contribute small bug fixes to your favorite open-source projects. In the process, you'll have to figure out the structure of the code. Take a look at the complex parts of the application and try to understand how they handled the complexity. Not everything you see will be a good example, but over time, you'll get a feeling how software design is supposed to work and which patterns are actually relevant and useful in practice.

    This is how I learned to structure my code. Unfortunately, that's the best suggestion I have.


  • Discourse touched me in a no-no place

    @ScienceCat said in How to learn software design:

    I can code decently, but I am crappy at designing/planning/structuring a software project. How can I learn this?

    First off, you're going to need to practice quite a bit to get good at this. Also, when something doesn't work, try to work out why and to think about how you might not make that mistake again. ;)

    Secondly, you start by trying to work out what things you must do, what things you would really like to do but aren't critical, what things are mere “it'd be nice to have”, and what things you shouldn't do. The last one is important! It stops you from throwing effort in stupid directions, and if you're lucky enough to have some success, it allows you to manage user expectations.

    Once you've worked out what things you must do and your pick from the “really like to” list, start looking for how to arrange things. Identify major areas of responsibility, how much data you're pushing around, etc. Those sorts of things will drive your architectural design, and it is difficult to recover from getting the architecture wrong (other than by burning the whole lot to the ground and starting over, which is horribly expensive). This is where you can use things like GRASP to help. Hint: if you're dealing with networked systems, you probably need to think about what messages you're passing around fairly early too; they interact with architecture.

    Once the architecture is there, you're getting close to the point where your existing skills will help.

    Overall, it's mostly common sense and experience. You can learn some of it from books, but you need to try things out for real too as that helps you remember that you sometimes should stop dicking around with architecture and just code something that's good enough for now. Because sometimes that's actually the right thing to do…



  • Learn to cook. Seriously. Get a recipe book/website, choose a soup, a main course and a dessert that you think will taste good together.
    You will soon realize you'll need to gather all the ingredients beforehand, to group the recipe steps in logical sections, do more than one thing at the same time... That's the same mental exercise you'll need to cook good software.

    For the first times you try this, invite your least favourite friend to dinner.. :laughing:



  • Another thing that can help is to just look at and play around in some actual systems. See what's safe and easy to change and why that is, and what's wrong with the other bits. Different systems sometimes have the same problems for very different reasons, or have a problem with something another does not have, despite doing certain things the same way. In this way, experience is immensely helpful.

    But the most important thing to remember is this: Software only exists to make hard things easy. That's why this field exists and will continue to exist for the foreseeable future. Some people don't take this very far though: by necessity, this means you have to make your software easy to use for you and your successors, and for every user. You have to put in research and serious effort to make everything as easy as possible for everyone. You;ll fail, as we all do, but that has to be your driving goal, or we'll end up with even more projects like git.



  • @ScienceCat I come from a similar background as you (I have a HND in Mechanical Engineering and a Degree in Software Engineering).

    I dunno about how to help you with specific code. But maybe I can help you with thinking.

    In programming you have Heuristics, you have generally good ideas e.g.

    • Single Responsibility Principle (for methods)
    • DRY (do not repeat yourself).

    You can ignore these when necessary. It kinda like ignoring relativity when working with speeds that are low percentage of the speed of light i.e. Newtonian physics is good enough for this.

    I would learn those programming principles and think about how you are going to apply them to a particular problem.


    Advice as a programmer .. .just try to reduce your code to the minimum about of number of lines possible. That is the very first thing, "how can I do his in less lines".



  • @lucas1 Just go with all of SOLID. There's a good wikipedia article on the concept.



  • @Magus errh ... whatever. The problem with a lot of these overbearing ways of making software is that you end up with Enterprise crap with is overly complicated. I am very KISS when approaching problems.


  • Winner of the 2016 Presidential Election

    @lucas1 said in How to learn software design:

    overbearing ways of making software

    Well, only I and D are slightly enterprisey. S, O and L are definitely good advice.



  • @asdf But I did that before I knew it had a name, which is kinda my point.

    If I am writing a simple crud app, so I need to follow it all? Maybe not.



  • @lucas1 It's more of a philosophy thing: making things while keeping SOLID in mind is a good idea; as always, adhering to it as law is probably just going to cause more pain.



  • @Magus Yes that is why I said I use those ideas, but I don't keep to an ideology. A lot of developers treat these things like religious texts and anything else is heresy.



  • @lucas1 But I'd still tell someone to read the Wikipedia articles on the subject, and read Clean Code and possibly Microsoft's Framework Design Guidelines. The information is enormously helpful.



  • @Magus That isn't wrong, but saying "follow SOLID" is wrong, even though most of the ideas work well in moderately to complicated software development projects.



  • @lucas1 I would still say it. But I also rarely mean anything definitively, so this is probably just a communication error.



  • I think you find that if you think about aspects of "good" code, they kind of coalesce into a single style.

    For example, code that's easy to debug is almost certainly also clearly-written. And vice versa, code that's clearly-written is also easy to debug. Similarly, code that's properly separated into logical modules is easy to unit test, and the reverse. And code with all those qualities is easy to maintain.

    So if you're writing code with these goals:

    • Clearly written
    • Debuggable
    • Logically compartmentalized
    • Testable
    • Easy to maintain

    You're really writing just one style of code that meets all of those requirements simultaneously.

    Your biggest nemesis is unnecessary abstractions. The only motto you really need is KISS: Keep It Simple, Stupid. (Actually YAGNI is a good one too: You Ain't Gonna Need It.)



  • When I was a junior developer, one of the seniors explained to me the three conditions under which your program is broken:

    • If it doesn't compile
    • If it doesn't produce the expected results
    • If you can't change it


  • @asdf said in How to learn software design:

    Well, only I and D are slightly enterprisey. S, O and L are definitely good advice.

    But you don't want to be SOL.


  • Discourse touched me in a no-no place

    @lucas1 said in How to learn software design:

    Advice as a programmer .. .just try to reduce your code to the minimum about of number of lines possible.

    A big caveat to that: it's not the newline that is expensive. It's the statement/expression, and bigger statements are much more expensive in terms of understanding than smaller ones. Also, using local variables to name intermediate values is potentially a great thing for understanding.

    But that's really all about the base parts of programming. The design and architecture levels really ignore that stuff. They're much more about identifying what units to break your program into in the first place so that you don't go crazy. Well, crazier. ;) Many programs can effectively reuse the architecture of others, which is nice, but you should at least spend some time working out if that's true this time too…


  • area_deu

    @lucas1 said in How to learn software design:

    just try to reduce your code to the minimum about of number of lines possible. That is the very first thing, "how can I do his in less lines".

    NO. That only leads to write-only code that even you won't understand anymore after three months.
    Reading and understanding (and changing) 20 lines of easy code is much much easier than the same function compressed to two lines.

    Space is cheap. Developer time wasted first on compressing the code and then on trying to understand what the code does, not so much.



  • @ScienceCat

    1. Contribute to other people's code
    It's much easier to work when someone else has already figured out all the architecture issues. You can then just plug your code into their system, use their style, tooling etc. Learn from the way they did things, but also learn from their mistakes.

    2. Use an opinionated framework or a project generator
    There are a lot of frameworks that pretty much tell you how you should structure your application. Eg., in the web programming, that's Rails and its clones. Even for non-opinionated frameworks, you can use something like yeoman to get you started.

    3. Produce shit load of crappy code
    In the end, there's no substitute for experience. You'll just have to make mistakes, start fresh, make new mistakes, over and over again, while your skills slowly improve. 10,000 hours of experience and all that.



  • @asdf said in How to learn software design:

    Try to contribute small bug fixes to your favorite open-source projects.

    Good Idea, I'll keep my eyes open for fitting projects!
    Tried to do it once already with my messenger app, but turned away pretty quickly due to hostile lead devs.

    @dkf said in How to learn software design:

    This is where you can use things like GRASP to help.

    Thanks, that's probably one of these basic knowledge things I am missing, I'll definitely look into it.
    Also, thanks for reminding me to spend more effort on gathering and organizing the requirements. It's one of these things they don't teach at university and I always tend to forget.

    @presidentsdaughter said in How to learn software design:

    Learn to cook.

    I actually do like to cook already. You're right, it's somewhat similar to coding, but that's the easy part (for me). My difficulties are more with recipe creation (to stay in your analogy). Which ingredients do I want? How should I combine them? etc.

    @Magus said in How to learn software design:

    Just go with all of SOLID.

    Thanks, I'll have a look into that.

    @blakeyrat said in How to learn software design:

    So if you're writing code with these goals:

    Clearly written
    Debuggable
    Logically compartmentalized
    Testable
    Easy to maintain

    You're really writing just one style of code that meets all of those requirements simultaneously.

    Yeah, my problem is basically "How do I structure my application so I can write this kind of code?"


  • Winner of the 2016 Presidential Election

    @ScienceCat said in How to learn software design:

    Yeah, my problem is basically "How do I structure my application so I can write this kind of code?"

    As others have said, there's no magic bullet. Reading up on patterns can help but only experience will tell you when a pattern is cromulent and when circumstances dictate deviating from them or using a different pattern.

    I'd recommend writing some toy projects, then extending them. Seeing which bits of your structure are maintainable and allow for adding new features vs the parts that are a tangled mess will give you a good idea.



  • @ScienceCat said in How to learn software design:

    I actually do like to cook already. You're right, it's somewhat similar to coding, but that's the easy part (for me). My difficulties are more with recipe creation (to stay in your analogy). Which ingredients do I want? How should I combine them? etc.

    There's only a few chefs in the world who create brand new recipes. Learning to cook is knowing recipes, and understand what are the key ingredients, which ones are replaceable and by what, etc.. You get that by cooking and eating new things.

    Example: A vegetable soup. If you ask me, I would make a Mediterranean-style one with mashed mixture of potatoes, onions and carrots as base, and headed cabbage. My mom, on the other hand, would replace the potatoes with zucchini (courgette) because of the calories. Are they the same soup? No. Do they taste the same? Not quite. But both are vegetable soup and both are acceptable if someone who doesn't care about calories ask for a vegetable soup.

    Breaking out of the analogy: you should get familiar with patterns (this can be a good 'index' of vocabulary and a door to start exploring https://en.wikipedia.org/wiki/Software_design_pattern) and anti-patterns (what you shouldn't do or you'll be on the front page).

    All that was said by everyone else can be summed up: read other's design and code and try to understand the 'why'. Write your own code and designs and tell to yourself 'why'. Rinse and repeat.

    Even if you can make an awesome Beef Wellington, that doesn't mean you're ready to make a complete dinner service for a restaurant.


  • Winner of the 2016 Presidential Election

    @presidentsdaughter said in How to learn software design:

    I would make a Mediterranean-style one with mashed mixture of potatoes, onions and carrots as base

    Side note: Don't forget the celery! The base of basically every Italian dish is fried onions, carrots and celery.

    @presidentsdaughter said in How to learn software design:

    My mom, on the other hand, would replace the potatoes with zucchini (courgette) because of the calories.

    I'd do the same, but not because of calories, but because it's more "Italian". ;)

    (Yeah, I know we have a cooking thread :arrows:)



  • @asdf There's other Mediterranean cuisines apart from Italian. And I don't like celery.


  • area_deu

    @cartman82 said in How to learn software design:

    3. Produce shit load of crappy code
    In the end, there's no substitute for experience. You'll just have to make mistakes, start fresh, make new mistakes, over and over again, while your skills slowly improve. 10,000 hours of experience and all that.

    This. Like when you want to become a writer, you should write a book with 100,000 words or so, and then throw it away and learn from all the mistakes you made and start over.
    The important thing is learning from your mistakes, and not just dump your crappy JS plugin on Github for the rest of the world to use because you think it's good enough.



  • @asdf said in How to learn software design:

    The base of basically every Italian dish is fried onions, carrots and celery.

    You just have to be careful to avoid obviously un-Italian things, like garlic, oregano, tomato, cheese, and olives!


  • Winner of the 2016 Presidential Election

    @Magus I think I may be :whoosh:ing here.



  • @asdf I was just naming things everyone thinks of when they think of Italian food, which are entirely unrelated to your list.



  • @asdf said in How to learn software design:

    The base of basically every Italian dish is fried onions, carrots and celery.

    ITYM France. There's an italian variant but it's parsley, lard, and onion. Carrots + celery optional.

    A lot of cuisines have a base holy trinity/quadrinity. Cajun cooking is onions, bell peppers, and celery. Polish is parsley root, celery root, carrots, and leeks. German is leek, carrot, and celery root. Spanish is theoretically garlic, onions, paprika and tomatoes, but literally every spanish or portuguese speaking nation has their own variation. West Africa is more or less the same, with the base of onions, tomatoes, and hot pepper and infinite regional variations.


  • Winner of the 2016 Presidential Election

    @AyGeePlus said in How to learn software design:

    ITYM France.

    Hm, is it too late to use the Mediterranean > Italian excuse? Just ignore that I used the word "Italian" above. :trolleybus:



  • One thing I think will go a long way to adopting the SOLID and DRY principles that have already been mentioned is to look into the MVC pattern (or any of its cousins, really). There are some different philosophies that differentiate some specific ideas surrounding it, but the idea is roughly the same: You want to separate the UI from the data and business logic.

    When you press a button, all the view layer should ever do is forward its click event to the logic layer. It's the logic layer which handles what should happen to pressing the button. If it needs to change the UI display, then it tells the UI display what to present (but only relays the minimal information needed for this; HOW this data is displayed is decided by the UI layer).

    That's the very brief explanation, anyway. Practicing this method may help you adopt some of the SOLID principles, especially single-responsibility and substitution principles. Note that the logic layer isn't a monolithic entity either, and should be broken down into their own parts, each with their responsibilities.



  • @Jaloopa said in How to learn software design:

    @ScienceCat said in How to learn software design:

    Yeah, my problem is basically "How do I structure my application so I can write this kind of code?"

    As others have said, there's no magic bullet. Reading up on patterns can help but only experience will tell you when a pattern is cromulent and when circumstances dictate deviating from them or using a different pattern.

    I'd recommend writing some toy projects, then extending them. Seeing which bits of your structure are maintainable and allow for adding new features vs the parts that are a tangled mess will give you a good idea.

    To explain a little further, write a bunch of toy projects that just do little things. Then take a simple program and apply a design pattern to it. Even if the design pattern would be unnecessary for that particular program, seeing how the pattern works in a simple, easy-to-understand environment often makes it easier to see how it can be useful for more complicated situations.



  • @cartman82 said in How to learn software design:

    In the end, there's no substitute for experience.

    And if your experience teaches you nothing else, it will be that six months ago you was an idiot. Your opinions on what's good and bad will change. Seriously, there's practically no objective metrics here. People think too differently. Stuff that you think is obviously shit might really not be. That link is a classic, and it's just a software version of Chesterton's fence.


  • Winner of the 2016 Presidential Election

    @djls45 said in How to learn software design:

    Then take a simple program and apply a design pattern to it.

    This is good advice. When I was learning about DI I made a ridiculously overengineered Hello World app that allowed injection of the UI layer (console, GUI, write to a file etc.), the message and anything else I could think of to abdicate responsibility. Obviously not necessary for something that simple but it gave me god insight into what the pattern could do.

    I still put too much logic into the UI of the real project and had to spend a day or two moving it all out into interfaces and dedicated classes, because it takes time to let it sink in, but it was valuable


  • Notification Spam Recipient

    Pick a well designed piece of software, write down all it's features, then reimplement the software without looking at the source of the original.

    You'll mostly likely implement many things differently, and run into a lot of edge case that behave differently. When you're sick of figuring how to make something work, go read the source code of the original. At that point, you're in a good position to understand why something was done in a particular way.



  • @Jaloopa said in How to learn software design:

    it gave me god insight into what the pattern could do.

    Probably a typo, but it still works.


  • mod

    Read a lot of design books by people you trust to be experts. Every yahoo with 6 months of javascript development under his belt thinks he's hot shit, but only a few experts manage to be trusted and have the weight of experience 10 years later. Beautiful Code. The Pragmatic Programmer. Clean Code. Code Complete. Anything by Martin Fowler. Not everything from any of those sources is perfect, but as you read more you'll start to be able to draw connections between them and get the gist of what you'll need.



  • Another idea is to explain what you are trying to do to at least two kinds of people: a programmer and a non-programmer (or at least think about how you would explain it to them). The programmer should understand your terminology and may be able to offer alternatives. Often just by explaining it, you can see how to make it better.

    Explaining to a non-programmer forces you to use non-technical terms or analogies that may help you understand it better yourself. As an example, my wife is definitely not a programmer, but she is always eager to learn about what I've been working on at work (or discussing here [yes, even this exact thread {she's sitting next to me}]), so by trying to teach and explain things to her, I often get a stronger grasp of whatever concept I am explaining.

    Often the most effective way to learn something is to have to teach it to someone else. Socratic method and all that jazz, or something.



  • @djls45 said in How to learn software design:

    @Jaloopa said in How to learn software design:

    @ScienceCat said in How to learn software design:

    Yeah, my problem is basically "How do I structure my application so I can write this kind of code?"

    As others have said, there's no magic bullet. Reading up on patterns can help but only experience will tell you when a pattern is cromulent and when circumstances dictate deviating from them or using a different pattern.

    I'd recommend writing some toy projects, then extending them. Seeing which bits of your structure are maintainable and allow for adding new features vs the parts that are a tangled mess will give you a good idea.

    To explain a little further, write a bunch of toy projects that just do little things. Then take a simple program and apply a design pattern to it. Even if the design pattern would be unnecessary for that particular program, seeing how the pattern works in a simple, easy-to-understand environment often makes it easier to see how it can be useful for more complicated situations.

    I'm currently thinking about writing a simple to-do-list-app which can also do some simple time-management (as in "you have x hours of activities planned today, so you have y hours left"). Would this work for these purposes, or is this already too complex?


  • Winner of the 2016 Presidential Election

    @ScienceCat Sounds like about the right amount of complexity. There's plenty you could add on to something like that, but it's not so trivial you won't be able to refactor it into a pattern



  • @cark said in How to learn software design:

    Pick a well designed piece of software, write down all it's features, then reimplement the software without looking at the source of the original.

    Did you think about something specific when writing that or could otherwise give a recommendation? I tend to choose poorly in these cases and would probably end up trying to reimplement something much too complex...



  • @Yamikuronue said in How to learn software design:

    Beautiful Code. The Pragmatic Programmer. Clean Code. Code Complete. Anything by Martin Fowler.

    Thanks for the suggestions! I'll have a look around if any of them are available in a nearby (university) library.



  • @ScienceCat Also, every person with any desire to touch the workings of software in any way needs to have read The Mythical Man-Month.



  • @ScienceCat In order to learn software design, you'd have to invent software design first. Because no one on Earth actually knows how to design software yet, and anyone who says otherwise is lying.

    We have some rules of thumb, some theories, some 'best practices' that amount to copying what worked for something else and crossing our fingers hoping it will work on something completely different, some cargo culting, and some flat-out witch doctor bullshit, but no software design, not so far.

    The good news is we're learning, at least some of us at any rate. It's slow and will probably take centuries to get to even the lowest level of real design, but it is improving.

    I recommend starting by keeping a critical eye about everything, and keep prodding at any designs you are working on. Be especially skeptical about any claims about the latest fad or silver bullet; nine times out of ten, it's just a rehash of something that was done better in 1965 or earlier.

    Second, get a good grounding in what people actually know, or think they know. The Royce paper is a seminal work, and a good place to start if you don't make the same mistake of reading only to halfway through the second page like 99.99999% of developers at the time did.

    The Mythical Man-Month is also a good starting place, but keep in mind that Brooks' main objective was to handwave the total failure of a software project he was the lead developer on (OS/360, the operating system on the System/360 mainframe). While he learned many valuable lessons from it, he's still trying to justify himself and it helps to keep that in mind.

    Programming Pearls by Jon Bentley is mostly about programming in the small, algorithms and creativity and so forth, but it does have some great things to say about design and engineering.

    I would recommend reading up on a little design patterns, Agile, Scrum, and so forth, but try to gargle the Kool-Aid rather than swallowing it; there are some great ideas in Agile, but also some terrible ones, and even the best parts only apply some of the time. Actually reading The Timeless Way of Building and/or A Pattern Language helps (confession: I never finished either one), as it shows where a lot of the ideas came from, but at the same time demonstrates why not all of them really fit software design as well as some people think.

    Learn to work in multiple paradigms, or at least read something about several different ones; one of the biggest problems for both coders and designers is a tendency towards myopia, a habit of trying to use the same tool for every job. Just knowing that a technique exists often helps, even if you don't know how to use it right then and there.

    When looking for help, try to explain what you are trying accomplish, not just the problem you've encountered. Anyone who is a regular at helping people on sites like SO or Daniweb has seen more shoe-bottle problems than they care to think about, and it makes it harder for everyone when this happens.

    Finally, don't take it too seriously. We aren't at the point where we can do that, yet, and it will lead you to a burnout (as it did with me).

    Programming is terrible—Lessons learned from a life wasted. EMF2012 – 1:06:44
    — Thomas Figg

    Coding Sucks: Why a Job in Programming Is Absolute Hell – 13:33
    — heroineworshipper





  • I forgot to say this, too: program development is only about 5% design, and 10% coding (and most of that is debugging and maintenance rather than writing new code). The really hard parts are all in interpersonal communication: getting design specs, maintaining a dialogue with and between the stakeholders, communicating design plans to others, getting documentation and requirements, writing comments and documentation, interpreting user feedback, and so forth. Given that programming itself tends to draw loners, this is a Bad Thing for anyone who wants a program done for them.

    That's about 50% of program development. Getting or having domain knowledge is another 25%, too, with the last 10% being various kinds of overhead (regulatory concerns, general paperwork, etc).



  • @ScholRLEA That video makes me feel sad and somewhat ashamed, honestly. Which I know was the goal. I really need to start messing around with servicefabric.



  • @ScholRLEA said in How to learn software design:

    the total failure of a software project he was the lead developer on (OS/360, the operating system on the System/360 mainframe)

    I'm not sure describing OS/360 as a "total failure" is even vaguely justifiable.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.