How much of what you do as a programmer requires domain-specific knowledge?
-
I took part in a discussion between teachers about the role of content mastery vs understanding (the whole "higher order thinking" issue). I'm very much in the camp that says you can't understand without already having facts mastered.
To help me ground my reasoning I thought I'd ask a question of you all--
What percent of what you do (roughly) is domain specific vs tool specific? As in, how much requires you to understand the business processes, the specifics of the thing you're writing the program for and how much just requires "general" programming skills (frameworks, languages, toolchains, etc)?
-
@Benjamin-Hall If you don't have enough domain knowledge, you cannot please a single user. You can learn it on the job, but your job, unless you write compilers and OS kernels, is generally going to be far more about specific domain knowledge than computer science theory.
-
@Benjamin-Hall 20% general domain knowledge, 30% general tool knowledge, 50% knowledge of our framework implementing the problem domain. Having just domain knowledge won't get you very far, but neither will tool knowledge.
-
There always needs to be someone with domain knowledge involved. If you have someone without it then you need to give them a really detailed design.
There are obviously some issues that I deal with that are fighting something technical (Oracle, some quirk of the widget library we use, internet explorer, etc), but that's got to be less than 5% of what I do.
We've had some weenies at our customer who wanted access to the database for their own analytical / data call purposes. But they really don't have much domain knowledge (I've been working with this customer for over 10 years now and have more institutional knowledge than a lot of people there, even if I'm always a little behind whatever new things show up over time). Writing intelligent queries to answer questions requires a lot of domain knowledge in order to get the correct information.
-
@Benjamin-Hall Hmm...taking some guesses at how much time I spent on things on different projects:
Device Testing: 10% domain knowledge, 40% general programming, 10% specific to the testing framework, 40% digital paperwork.
Making App Scriptable: 40% domain knowledge, 20% general programming, 40% specific to implementing scripting on Windows at the time (COM/MFC/ATL....)
Creating a simple PWA: 25% domain knowledge, 10% general programming, 25% web design, 40% web and PWA programming.
The more control you have over the design, the more domain knowledge you need. (Which is probably obvious?)
-
@Benjamin-Hall for me, probably 70% domain specific. The need for specific programming skills is very secondary because adequate knowledge of particular tools can be acquired as and when needed (and that skill-set is transient, because technology moves on).
My role may be atypical, because I spend as much time on data analysis, user-support and politicking as on software development. Most 'programming' jobs are probably more compartmentalised and focused than mine.
In general, I think domain specific knowledge is terribly undervalued and that's partly why so much useless and ill-designed software is churned out (by programmers who, from a position of ignorance or as victims of misdirected micromanagement, correctly solve the wrong problem).
-
@Benjamin-Hall said in How much of what you do as a programmer requires domain-specific knowledge?:
What percent of what you do (roughly) is domain specific vs tool specific? As in, how much requires you to understand the business processes, the specifics of the thing you're writing the program for and how much just requires "general" programming skills (frameworks, languages, toolchains, etc)?
70 percent domain, 40 percent tool, 20 percent general programming skill. Yes, that means things overlap.
-
@boomzilla said in How much of what you do as a programmer requires domain-specific knowledge?:
queries to answer questions
Reporting requires a lot of domain knowledge
-
@Benjamin-Hall In my daily work it's probably 50/50. Working in a small company with many clients means I have to wear a lot of different hats – senior dev, tech lead, debugging wizard, sysadmin, product owner (and I enjoy them all). In some of these roles, you only need tech skills, and in others, you can't get things done without understanding some of the problem domain.
In general, it depends on the role and experience level.
- Junior devs need very little domain knowledge because they get only small and well-specified (yeah, yeah, wishful thinking) tasks.
- Senior devs and team leads need a balance of both.
- Architects and SMEs need much more domain knowledge than tool knowledge.
- Tech leads need mostly tool knowledge.
-
@Benjamin-Hall said in How much of what you do as a programmer requires domain-specific knowledge?:
(frameworks, languages, toolchains, etc)
In many ways, those too are domain-specific knowledge. They're just the domain of computing and most programming is about bridging between that domain and another one.
For all but the simplest tasks, understanding the domain is very helpful. If nothing else, it lets you understand what the users are asking for and screaming at you about, and work out which bits can be ignored, which bits must be done, and which you should push back on because they'll just make everyone's life hell for no good reason.
-
@dkf said in How much of what you do as a programmer requires domain-specific knowledge?:
understand what the users are asking for
a lot of it boils down to terminology
-
For my job nature, it's 50/50.
However I delegate all the "business knowledge" part to my BAs. They tell me where I can find the data and how to find data dict. I list out what I can do and they tell me what they think will work, and what additional detail to pay attention for (say, the cutoff time of foreign factory machinery statistic is different than here). I implement them as well as other things like crunching multiple thousand pages documentation to figure out the correct XML or FIX protocol text to talk with the machines.
For technical knowledge, it's those SI related part like how to mix-and-match the libraries, applications written in multiple programming languages, running on multiple platform (Windows or Linux), with multiple authentication mechanisms with SSO like behavior. Need general knowledge to the protocols involved, the howto to work with different web servers, know to debug/troubleshoot on those platforms, and of course, some general networking knowledge.
The best thing is, the "writing code for project itself" part has option to delegated to software vendors, so I only need to worry about all the addon and so on later.
-
I'm probably a bit unusual in that I'm officially in an R&D position (not software development in any form, even if it's de facto part of my role) and most of my work is strongly related to internal consultants doing domain-specific work for a client. So almost all my interactions with my users require almost exclusively domain-specific knowledge, with the small exception of the obvious purely computer-related bugs (e.g. "when I click on the 'change colour' button the colour doesn't change").
For the development part (including specs, architecture, testing, docs etc. -- everything that doesn't fall under "directly helping users with their clients"), the domain-specific part is actually somewhat less important than knowledge of the framework we use. So for day-to-day work, probably something like 50% framework, 20% domain and 30% generic. Of course during the design phase of a tool it's going to be more like 80% domain and 20% framework, but not so much for the everyday incremental changes.
I guess one could argue whether knowledge of our framework is domain-specific or not, since that framework was developed over the years to answer our domain-specific problems...
-
@Luhmann said in How much of what you do as a programmer requires domain-specific knowledge?:
@dkf said in How much of what you do as a programmer requires domain-specific knowledge?:
understand what the users are asking for
a lot of it boils down to terminology
It's often broader than just terminology. There's usually a gap between what users say they want and what they actually want/need.
-
@remi said in How much of what you do as a programmer requires domain-specific knowledge?:
I guess one could argue whether knowledge of our framework is domain-specific or not, since that framework was developed over the years to answer our domain-specific problems...
It most certainly is domain-specific.
-
@dkf I would tend to agree, but some part of working with it is just understanding how the code is structured, so it's not any different than working with any 3rd-party library.
Imagine you're told to write some code to frobnicate all foobar objects that are stored in the flux capacitor of the system your company is building. Your company has a framework where you have a
FluxCapacitorobject, aFoobarobject and aFoobar::frobnicate()method. Does writing a function that doesforeach(foobar f, flux_capacitor->getFoobars()) f->frobnicate();require domain knowledge?OK, this example is a bit contorted to make it look obviously non-domain-specific and I'm certainly not saying that all uses of the framework are like this. But it's an illustration of why I decided to try and split between purely domain-specific stuff, and framework use that may or may not be domain-specific depending on the cases.
-
The part where the code executing also is something useful that makes money, is usually the part that requires non-programming-specific knowledge. So, to avoid domain-specific knowledge, just make tools for other programmers. If you happen to have any, consider encapsulating it into a library. Domain-specific knowledge is gross.
-
@Gribnit said in How much of what you do as a programmer requires domain-specific knowledge?:
Domain-specific knowledge is gross.
WHAT?
-
The basic problem is that I'm the only one in the world who knows how to write a ticket, (though many think they do, but don't) and no-one listens to me. The most important point in a ticket is listing the steps so that anyone off of the street could follow the steps, see the result as it is, and state what it is that the result should be. If the ticket includes these steps, then the amount of domain-specific-knowledge required is minimized.
Without those steps, then you have to figure out what you are doing, and domain-specific knowledge can be helpful.
-
I am a consultant, so I jump from place to place, and there is always someone that knows the domain around and it's usually not the other programmers. I ask them for the particulars of what I am to implement in code. Most of systems development is just figuring out who actually knows shit and who actually can make shit happen, sometimes you're in luck and it's the same person.. Writing code is fun and all, but unfortunately I rarely get to just hammer out code all day.
-
I think I'm the opposite of most programmers here since I'm a physicist first. All of the programming I do falls into three categories:
-
Data analysis: read raw data from file, do math, make pretty charts. General programming is secondary since the Matlab script only has to work once to generate the plots for the report/grant application/journal article.
-
Hardware interfaces for controlling experiments: move here, activate switch, read this data port. Very little abstract programming since most of this is C or LabView.
-
Simulations (where I spent most if my time): even the frameworks meant to make simulations easier to create require extensive domain knowledge to use--Geant4 being a prime example. There is no such thing as a particle simulation that includes all of particle physics, so you have to pick which details you care about. Just the photons released when high-energy electrons hit tungsten? Neutrons? Quark interactions? These decisions determine how long the simulations run and can affect the accuracy of other parts of the simulation.
So, about 95% is domain knowledge with some abstract programming knowledge where optimization is needed so it doesn't take hours to get results. The only exception is the creation of applications that will be used more than once, of which I wrote three in seven years, one in active use but nobody looks at the code because I wrote it while learning LabView (resulting in horrifically literal spaghetti code), another only used by me, and last used by literally nobody (see the Joel Test topic in the Lounge).
-
-
Surely this is a discussion around Conway's Law.
I've worked in some industries where it was absolutely essential. It really depends what you are doing.
I am currently working with ID scanning technology. It helps a lot to understand the legislation around it. I've also worked in dealing with hotel invoicing for travel agents and just trying to understand the process of invoices, charge-backs and expenses can be a headache.
I am sure if someone was building something more generic like a CMS or just a shopping site than the specifics of the business doesn't matter as much.
-
@MZH said in How much of what you do as a programmer requires domain-specific knowledge?:
I think I'm the opposite of most programmers here since I'm a physicist first. All of the programming I do falls into three categories:
-
Data analysis: read raw data from file, do math, make pretty charts. General programming is secondary since the Matlab script only has to work once to generate the plots for the report/grant application/journal article.
-
Hardware interfaces for controlling experiments: move here, activate switch, read this data port. Very little abstract programming since most of this is C or LabView.
-
Simulations (where I spent most if my time): even the frameworks meant to make simulations easier to create require extensive domain knowledge to use--Geant4 being a prime example. There is no such thing as a particle simulation that includes all of particle physics, so you have to pick which details you care about. Just the photons released when high-energy electrons hit tungsten? Neutrons? Quark interactions? These decisions determine how long the simulations run and can affect the accuracy of other parts of the simulation.
So, about 95% is domain knowledge with some abstract programming knowledge where optimization is needed so it doesn't take hours to get results. The only exception is the creation of applications that will be used more than once, of which I wrote three in seven years, one in active use but nobody looks at the code because I wrote it while learning LabView (resulting in horrifically literal spaghetti code), another only used by me, and last used by literally nobody (see the Joel Test topic in the Lounge).
I kinda feel like there's still a lot of locked up potential in programming as a tool in physics, because physicists are not as good of programmers as they are physicists.
-
-
@stillwater said in How much of what you do as a programmer requires domain-specific knowledge?:
@Gribnit said in How much of what you do as a programmer requires domain-specific knowledge?:
Domain-specific knowledge is gross.
WHAT?
Maybe he meant "total" or a "dozen dozens" instead of "disgusting"?
-
@djls45 said in How much of what you do as a programmer requires domain-specific knowledge?:
@stillwater said in How much of what you do as a programmer requires domain-specific knowledge?:
@Gribnit said in How much of what you do as a programmer requires domain-specific knowledge?:
Domain-specific knowledge is gross.
WHAT?
Maybe he meant "total" or a "dozen dozens" instead of "disgusting"?
Nah, it's kinda disgusting at its heart. It must be. Look at all the abstraction effort put into avoiding needing it.
-
@Gribnit said in How much of what you do as a programmer requires domain-specific knowledge?:
@MZH said in How much of what you do as a programmer requires domain-specific knowledge?:
I think I'm the opposite of most programmers here since I'm a physicist first. All of the programming I do falls into three categories:
-
Data analysis: read raw data from file, do math, make pretty charts. General programming is secondary since the Matlab script only has to work once to generate the plots for the report/grant application/journal article.
-
Hardware interfaces for controlling experiments: move here, activate switch, read this data port. Very little abstract programming since most of this is C or LabView.
-
Simulations (where I spent most if my time): even the frameworks meant to make simulations easier to create require extensive domain knowledge to use--Geant4 being a prime example. There is no such thing as a particle simulation that includes all of particle physics, so you have to pick which details you care about. Just the photons released when high-energy electrons hit tungsten? Neutrons? Quark interactions? These decisions determine how long the simulations run and can affect the accuracy of other parts of the simulation.
So, about 95% is domain knowledge with some abstract programming knowledge where optimization is needed so it doesn't take hours to get results. The only exception is the creation of applications that will be used more than once, of which I wrote three in seven years, one in active use but nobody looks at the code because I wrote it while learning LabView (resulting in horrifically literal spaghetti code), another only used by me, and last used by literally nobody (see the Joel Test topic in the Lounge).
I kinda feel like there's still a lot of locked up potential in programming as a tool in physics, because physicists are not as good of programmers as they are physicists.
There is a machine-learning initiative at the LHC to deal with the deluge of data coming out of the experiments there.
Machine learning proliferates in particle physics
A new review in Nature chronicles the many ways machine learning is popping up in particle physics research.
Deep Learning and its Application to LHC Physics
Machine learning has played an important role in the analysis of high-energy physics data for decades. The emergence of deep learning in 2012 allowed for machine learning tools which could adeptly...
-
-
...
-
I work for a very small company in a niche market and in a senior role. So, domain-specific knowledge is an absolute must in making design and architectural decisions. And there are a lot of pitfalls in that domain. A majority of the design is very typical for most general web-based software, but there's a significant part that needs serious consideration to be engineered the best, and you need to know much about the customer-base to come to those correct conclusions..
I would say the more junior you are, the less domain-specific stuff is needed, mainly because the design decisions are usually made by people above you. This doesn't mean you shouldn't learn that stuff, especially if you want to advance in your job or career, but you can kinda get by without knowing as much of that stuff, and your job should be coaching you into knowing more anyways.
-
@MZH said in How much of what you do as a programmer requires domain-specific knowledge?:
@Gribnit said in How much of what you do as a programmer requires domain-specific knowledge?:
@MZH said in How much of what you do as a programmer requires domain-specific knowledge?:
I think I'm the opposite of most programmers here since I'm a physicist first. All of the programming I do falls into three categories:
-
Data analysis: read raw data from file, do math, make pretty charts. General programming is secondary since the Matlab script only has to work once to generate the plots for the report/grant application/journal article.
-
Hardware interfaces for controlling experiments: move here, activate switch, read this data port. Very little abstract programming since most of this is C or LabView.
-
Simulations (where I spent most if my time): even the frameworks meant to make simulations easier to create require extensive domain knowledge to use--Geant4 being a prime example. There is no such thing as a particle simulation that includes all of particle physics, so you have to pick which details you care about. Just the photons released when high-energy electrons hit tungsten? Neutrons? Quark interactions? These decisions determine how long the simulations run and can affect the accuracy of other parts of the simulation.
So, about 95% is domain knowledge with some abstract programming knowledge where optimization is needed so it doesn't take hours to get results. The only exception is the creation of applications that will be used more than once, of which I wrote three in seven years, one in active use but nobody looks at the code because I wrote it while learning LabView (resulting in horrifically literal spaghetti code), another only used by me, and last used by literally nobody (see the Joel Test topic in the Lounge).
I kinda feel like there's still a lot of locked up potential in programming as a tool in physics, because physicists are not as good of programmers as they are physicists.
There is a machine-learning initiative at the LHC to deal with the deluge of data coming out of the experiments there.
Machine learning proliferates in particle physics
A new review in Nature chronicles the many ways machine learning is popping up in particle physics research.
Deep Learning and its Application to LHC Physics
Machine learning has played an important role in the analysis of high-energy physics data for decades. The emergence of deep learning in 2012 allowed for machine learning tools which could adeptly...
-
