Dagger for Docker’s CI/CD and Security Gaps
The idea I had was to share my Neo4j knowledge graph project on a Docker container for possible work and revisions by folks who could help with the project. This, again, is not a commercial project but a sandbox project involving oceanographic data analysis.
However, I spoke with at least two developers who were adamantly opposed to that, and they said I would need GitHub or git for any kind of work for several reasons — which I know of and are well known — such as its systematic approach, its effectiveness for pull requests and its ability to trace and audit past changes, etc.
But again, I wanted something simple, and I think it would involve just a maximum of two or three people to review the code in the application, and that’s about it because I’ll be doing everything else. When using code from other runtimes for my project, I could also just use a hardened Chainguard container from DockerHub for security, as opposed to having to bother with signatures and SBOMs and such. Because Chainguard consistently updates its container images, you can leave the security updates to it to manage.
While looking into the right option to share my knowledge graph application, I remembered a talk given by Chainguard developer relations advocate Adrian Mouat, at KubeCon + CloudNativeCon EU in Paris earlier this year. The talk was called “Building Container Images the Modern Way” (I was there for the talk, and it was packed).
Use @dagger_io to build pipelines input: @chainguard_dev’s Building Container Images the Modern Way – @chainguard_dev’s Adrian Mouat demo. It « real strength » is its modules, he said. @KubeCon_ @linuxfoundation @thenewstack pic.twitter.com/bTCpBcuV1C
— BC Gain (@bcamerongain) March 22, 2024
The key element I took away from it, not necessarily for my project, but in general, was that indeed, for deployments and in many other ways, GitHub is and has been the way to go. However, it could be supplemented by improvements, especially in terms of the CI/CD organization of pipelines.
Docker, for CI/CD specifically, falls short. Although going back to my original project, yes, Docker is perfect, I think, for what I want to do. But it’s emerged that, again, for CI/CD, Docker does have its shortcomings, as well as for certain security aspects.
One of the key points that came out of Mouat’s talk was how Dagger seems well-suited for CI/CD, and additionally, it can be integrated with GitHub for CI/CD projects, as Mouat explained.
I Want My CI/CD
Dagger offers programmable CI/CD through containerization. But as mentioned above, this is not an either-or situation, as is the case with my project. Again, my project is just about shipping a simple application in a container to a couple of people. It’s definitely not a full-blown CI/CD type of collaboration.
As Mouat defined during his talk, Dagger is a tool that leverages BuildKit‘s power to define CI/CD pipelines in code. It excels in creating complex build pipelines that can be reused across projects, providing strong caching and parallelism capabilities, he said.
These reusable build pipelines in a container are key.
“Defining reusable build pipelines as code is a critical precondition for reducing DevOps complexity and enhancing security and compliance. Allowing developers to run these pipelines locally using Docker BuildKit, a tool they are familiar with, automatically ensures consistency between development and production environments,” Torsten Volk, an analyst at TechTarget’s Enterprise Strategy Group, said. “This is critical for implementing policy-driven security to enable developer productivity while ensuring security at the same time.”
And again, Dagger is integrated with GitHub, GitLab or plain git, or can be. Therefore, you’re getting the best of both worlds in many ways, with the security aspects and simplification.
The programmability of the CI/CD pipeline and the different options that Dagger offers make it particularly apt for CI/CD. This is where a simple Docker container, while great for my project, would arguably fall short — at least according to one of Dagger’s creators, Sam Alba, co-founder and vice president of engineering at Dagger, and former vice president of engineering at Docker.
Before the Dagger project was created in 2018 and while at Docker,