Building docker images within a buildroot image.


  • Discourse touched me in a no-no place

    My google-fu is failing, and would like some search suggestions. Because all I'm getting at the moment are largely suggestions for

    • docker containers for using buildroot in or
    • building the actual raw docker containers/images from scratch. using buildroot

    Neither of which are what I'm after.


    We use buildroot to produce the image of our OS which gets burnt onto CF cards which are largely viewed as read-only from that point on. (There are exceptions, but these are generally one-off personalisations to differentiate between each unit or project one-only's for something specific to that project. Imagine these as glorified INI files, for that is exactly what they are.)

    As such that image should contain almost everything that's needed for it to run, including configuration for any services it provides, such as ntp, https, iptables etc.

    We have this.

    However.

    One such new service is docker containers. Because we (or rather manglement) want 3rd parties to run services on our boxes, but in containers, not on the host.

    So, in pictorial form:

     +------------+
     | Our system |
     +---------------------------------------------------------------+
     |  +------------+    +---------------+    +-----------------+   |
     |  |            |    |               |    |                 |   |
     |  |  ntp       |    |  lighttpd     |    |  dns            |   |
     |  |            |    |               |    |                 |   |
     |  +------------+    +---------------+    +-----------------+   |
     |                                                               |
     | +------------+ +-------------+  +-------------+               |
     | |            | |             |  |             |               |
     | | docker1    | |  docker2    |  | docker3     |               |
     | |            | |             |  |             |               |
     | +------------+ +-------------+  +-------------+               |
     | +----------------------------------------------------------+  |
     | |                                                          |  |
     | |         docker engine                                    |  |
     | |                                                          |  |
     | +----------------------------------------------------------+  |
     +---------------------------------------------------------------+
    

    We have buildroot building 'our system' as an .img which can be dd'd to a CF card. It currently includes ntp, lighttpd, and dns from the diagram, including their special configs. It also includes the docker engine (or hopefully will by the time the current build has finished and is tested.)

    Is there any way of pre-populating the .img with all the configuration needed to produce those three docker containers before the build has finished, using buildroot or am I going to have to resort to some sort of 'if this docker container doesn't exist, create it from this Dockerfile on first run' sort of shenanigans?

    Basically, is there a way of getting buildroot to parse Dockerfiles to produce stuff like this:

    root@hpdesktop:~# ls /var/lib/docker/containers -l
    total 28
    drwx------ 4 root root 4096 Sep 15 14:04 174e7fd5aa249b17002665a54a1742684c085073236d6845b389e73082dec61b
    drwx------ 4 root root 4096 Sep 15 12:42 27f5e17b0b876c0e82ae5dc51474fd79b75a322985f21f6569619cdf380e7244
    drwx------ 4 root root 4096 Sep 15 12:42 4502e6e2aff2b221e60f1395ff0e086551ac9a19cf813f06a974a8918d25356d
    drwx------ 4 root root 4096 Sep 15 12:42 8bf8d5c0cc968d260936ed2d8fcd2577e50db04184ecc1395c18bc4cad9f142f
    drwx------ 4 root root 4096 Sep 15 14:04 8d2ca5da1ca0970b86f5240e522a086f7485f9f75fcd3b1581ca6639dbb8956f
    drwx------ 3 root root 4096 Sep 15 12:42 a0c7768e28965810313fadec93b3f81b0c226fbed954911ff35007a4eabbd860
    drwx------ 4 root root 4096 Sep 18 13:48 e3edb50ecae6e004f994637505b97a28ecb9dd52b0b2e46cdc632316600e8928
    root@hpdesktop:~# 
    

    as part of its output



  • Can you build the containers, docker save them, docker load them in the built image, and spin them up on start?


  • Discourse touched me in a no-no place

    @jazzyjosh said in Building docker images within a buildroot image.:

    Can you build the containers, docker save them, docker load them in the built image, and spin them up on start?

    The .img is simply built up from whatever filesystem Buildroot produces deep in the bowels of buildroot/buildroot/output/build/<name of package> or maybe buildroot/buildroot/output/staging/<sparse file system> - the image isn't 'run' on the system it builds on which, if I'm reading your suggestion correctly, it would need to do.



  • @pjh Right, I think we've crossed wires here.

    Build the docker image and then docker save it on the build system. Add the tar to the buildroot image.

    Whenever the system with the image starts up, have it docker load the saved images and start them.

    This won't actually do the loading or configuration of the containers for you, just keeps you from having to rebuild the image. Sorry if I wasted your time.


  • Discourse touched me in a no-no place

    @jazzyjosh said in Building docker images within a buildroot image.:

    Whenever the system with the image starts up, have it docker load the saved containers and start them.

    Gotcha.



  • @pjh Just realized I said containers instead of images. docker save doesn't save the running state, so make sure everything is configured via the Dockerfile or on docker run of the image.


  • Discourse touched me in a no-no place

    @jazzyjosh said in Building docker images within a buildroot image.:

    @pjh Just realized I said containers instead of images. docker save doesn't save the running state, so make sure everything is configured via the Dockerfile or on docker run of the image.

    Understood.

    I'm basically after what mechanisms are cromulent for doing what we're after rather than the minutæ of them at the moment...


  • :belt_onion:

    @jazzyjosh said in Building docker images within a buildroot image.:

    docker save doesn't save the running state, so make sure everything is configured via the Dockerfile

    I think even the Dockerfile metadata may be lost on docker save, unless that was fixed within the past year or so. So you might need a long-winded docker run command specifying the working directory, environment variables, command w/arguments, etc... but @JazzyJosh's approach will work and my under-caffeinated brain can't think of a much better one right now.


Log in to reply