Engineerering the ultimate deploy script

gleemonk

We're currently pushing to get all our projects into CI. One guy took it upon himself to develop standards for our deploy scripts. Because we have such a diverse host of systems and due to his desire to create the Ultimate Non-Repudiable Definite All-Encompassing Deploy Script, things got complicated quickly. Here's the part that triggers the script on the hosting after rsync did its thing:

# creates a file deploy/server.env with the environnement to be used on the server.
# pipes the env content directly through ssh. tee writes to server side file.
# if success, source it and run server.sh
SRV_DEPLOY="$PATH_APP/deploy/"
for var2export in PATH_APP PATH_WWW_ROOT
 do
  # ${!var} : indirection (variable variable)
  # ${var/x/y} : replace; here: single quote by escaped single quote
  # \' : single quote escaped from actual shell
  # $'val' : POSIX string (escaped single quote does not terminate single quote string: 'foo\'bar' )
  # Test: ( VAL="foo'bar" ; VAR="VAL" ; source <(echo "NEW=$'${!VAR/\'/\\\'}'" ) ; echo "_ $NEW _" )
  echo "export $var2export=$'${!var2export/\'/\\\'}'"
done | \
ssh ${REMOTE} "tee ${SRV_DEPLOY}server.env && source ${SRV_DEPLOY}server.env && ${SRV_DEPLOY}server.sh"

This is currently being copied to deploy scripts all over the place. I have proposed in vain to have this shortened to

ssh $REMOTE "PATH_APP=$PATH_APP PATH_WWW_ROOT=$PATH_WWW_ROOT ${PATH_APP}/deploy/server.sh"

Now you may protest: "But Gleemonk, properly escaping variables is a must! What if PATH_APP has a space in it?!" Well I have news for you:

PATH_APP="app"
PATH_WWW_ROOT="www"

And this is actually per policy. So the server.sh could just use app and www directly. Because they never change. The whole thing could be shortened to

ssh $REMOTE app/deploy/server.sh

But agreed: the imagined project that violates policy by not using standard app and www dirs would need a custom server.sh! And the deploy-script must work for all possible hostings. Which trumps all other reasons.

CHUDbert

@gleemonk said in Engineerering the ultimate deploy script:

Which trumps all other reasons

--> Obviously we need to have a Trump emoji.

Stolen from web:

or

@boomzilla @ben_lubar make it so you lazy moderators!

pie_flavor

@CHUDbert That would get out of hand very fast. Meanwhile, emojis are added by making a PR.

PJH

@gleemonk, it's like 'premature optimization', but somehow the 'optimization' bit got conflated with 'obfuscation.'

Is this guy trying to engineer his own 'job for life?'

zekka

i think i work with this guy

he has a strong opinion against storing credentials in code, which in practice means he stores them in code, but four templating layers deep in one of six directories called credentials

gleemonk

@PJH said in Engineerering the ultimate deploy script:

Is this guy trying to engineer his own 'job for life?'

It's a serious case of "it must be a general solution we can use everywhere". Where "everywhere" means "all the distorted projects I'm looking after". Rather than bringing the systems in line he's trying to write a script that works for all. The tragedy is that the other senior guys don't want to use those scripts so he's now sitting next to the junior guys "helping" them with the deploy scripts.

I've now had two code-reviews where I went " just delete this cruft" and the junior dev said "but it's documented that way". The confrontation is on as of today because I've proposed to change the documentation. Let's see how it plays out.

gleemonk

@zekka said in Engineerering the ultimate deploy script:

i think i work with this guy

he has a strong opinion against storing credentials in code, which in practice means he stores them in code, but four templating layers deep in one of six directories called credentials

Committing secrets?

In this area I have no disagreement with the complicator. He was actually the first to push for removing credentials from code. So it must be someone else you're working with.

zekka

@gleemonk i mean he just committed them to a different folder in the same repository, then wrote a bunch of ceremonial scripts to purify them

Zenith

@gleemonk said in Engineerering the ultimate deploy script:

It's a serious case of "it must be a general solution we can use everywhere". Where "everywhere" means "all the distorted projects I'm looking after". Rather than bringing the systems in line he's trying to write a script that works for all.

That's what happens when you aren't allowed to fix those projects. You don't really know that hell until you've lived through it.

Carnage

@Zenith said in Engineerering the ultimate deploy script:

@gleemonk said in Engineerering the ultimate deploy script:

It's a serious case of "it must be a general solution we can use everywhere". Where "everywhere" means "all the distorted projects I'm looking after". Rather than bringing the systems in line he's trying to write a script that works for all.

That's what happens when you aren't allowed to fix those projects. You don't really know that hell until you've lived through it.

I've lived through it enough to just change broken stuff. If managers have an opinion about that, I'll start being very opinionated about how they should do their jobs.

gleemonk

@Zenith said in Engineerering the ultimate deploy script:

That's what happens when you aren't allowed to fix those projects. You don't really know that hell until you've lived through it.

Oh but he is the one making technical decisions for his projects. He can absolutely decide to fix them. If in a code-review I'd seen the snippet above I'd brought it up and probably let it pass if he insisted. It's borderline usable after all. Right now he's trying to get other people to do it the way he's worked out. Now we see all the cruft he's been building up. His declared goal is to have this in every deploy-script as the standard way of doing things.

What I've posted above is only a snippet of similarily convoluted logic handling all parts of the deploy-process. A crufty standard like that would be disregarded widely and be ineffective and confusing. He's making the junior devs think that all this CI stuff is really complicated while the senior devs just shrug and use whatever worked before. I was really conflicted between just ignoring that proposal and ~~fighting~~trying to improve it. After posting here I realized how embarrassing it would be if we had this as our declared standard.

Zenith

@gleemonk I still don't see anything wrong with accounting for spaces in a path.