These videos are extremely relevant to my job as a DevOps Associate at my current company. In our small team, your insights are invaluable. Thank you so very much. I'm checking out your other learning materials as well.
It sounds like you're talking Automaton theory by Alan Turing 😃. Good presentation of GitOps; However, I think only a few people truly understand and enjoy your talk. Keep up the good work though. Thanks for sharing!
8:14 As a one man team, I can not agree more. When you're spending the majority of your time working on application logic, not having infrastructure defined as code creates a lot of uncertainty when you need to modify something months later.
So from first principles, what we're trying to do with GitOps is: 1. Encode and compress instructions using domain specific languages to interface with machines. 2. Store those instructions digitally and make them accessible over a network to eliminate space and time constraints. 3. Keep the change history to make change auditable and reversible. I'm imagining a future in wich we interface with machines using natural language. No special syntax required. The spec emerges through conversation. Until we reach that point, GitOps will be the most efficient way. Software Engineers will be needed to translate from natural language to DSL and to create the machines interpreting that of course.
What an amazing 20 min I had watching this video, got what I needed to learn as a concept. Tools/Technologies to implement this concept will surely demand it's own learning curve... surely unlearning is another important philosophy which I'm fully agree as a passionate 'Software Engineer' Take away from your teaching on GitOps - "It's a matchmaking of Man-Git(Ops)-Machine", or in few more words "Must Code of what you desire, Git is single source of truth of your desire, prefer declarative approach and follow immutability ".
I think I agree with you that eventually all work is codified into code. But in the process of getting there, I sometimes want interactive processes that let me fiddle with ideas along the way in a lighter-weight fashion than having to check something into Git and waiting to see what pops out the end of the pipeline. I don't always know exactly what I'm doing. I don't always know the best way of solving a problem. I need interactive ways of fiddling with ideas. What I'm thinking about is something like Jupyter Notebooks for developers. Something that lets me interactively build up a set of information sources from different places and play around with different presentations. Once I've got it how I like it, then yeah, I guess I want to express that as a config file I put into Git. Same goes for interactive, manual testing. It would be very difficult for me to jump straight to a fully automated test suite without first having an interactive, manual test plan that lets me work on the testing kinks first.
Can we consider ArgoCD as a GitOps tool? If we assume that GitOps means that git is a single source of truth, doesn't UI offered by ArgoCD kind of breaks this principle, as it is possible there to manually synchronize to some different revisions?
Argo CD UI used to observe the states is ok. The part that allows you to create or edit apps directly in the cluster is contradictory to gitops principles.
17:55 I've seen people build UIs that work on YAML and I've seen editors that save directly back into Git. So it should be possibly, it would be good if the UI before it commits to Git asks why the change is made so it will be included in the commit message or even better: pull/merge request.
There are indeed UIs like that, some of them public and others built for internal use. Still, they are more of an exception than something common. As a side note, my team is working on a public project that will do something like that. We are in the early phase, so I cannot yet share any details. Soon...
@@DevOpsToolkit I have had some extensive ideas about what a generic webbased YAML editor could be like. My guess is 2 definition files are needed. One which is like a schema, which describes what the different options of the YAML format are (Kubernetes, Ansible, etc.). And the second is a description file which describes the UI which should be rendered with the labels that go with it and help texts. Maybe it's 1 file which contains both. My thinking the UI should map to different UI elements for example which fit something like Bootstrap/Vue Admin Templates (Kubernetes Dashboard uses Andular Material). I didn't have the time to build it, otherwise an open source webbased YAML editor might have already existed. :-)
Continuous deployment is about continuously deploying apps to production. That typically means the whole automation of the app lifecycle, from pushing a change to Git all the way until it is running in production. Between those two is typically a bunch of steps like building, testing, deploying to temp envs. etc. CD is the whole process. GitOps, on the other hand, is about defining everything as code and having a process that continuously reconciles the actual into the desired state (the one stored in Git). You can say that CD is an evolution of CI (a superset) and about the automation of the whole process with GitOps being a part of it. For example, in my CD pipelines, I would make some changes to manifests stored in Git and those changes would be picked up by GitOps tools like Argo CD or Flux. Before and after that, there would be many other steps in a CD pipeline.
Source code is not the source of truth. Every build will be different, in principle, because of external dependencies. Today's build is not guaranteed to be the same as tomorrow's, even when the code has not changed. Try it out!
Assuming that the exact dependencies are specified and stored in git, it is the source of the desired state. Now, I do agree with you that git is not the source of truth, but for a different reason. Live system is the source of truth. Git is rather the definition of the desired state. The actual state might differ from the desired state, so the actual state is the real source of truth while git is the source of our intention of what the actual state should be.
Good response. Thanks. Key points; it is not the source of 'truth' but is an expression of the desired state. Very important; we need to pin dependencies, otherwise the expression of the desired state is just a moving target. Finally, consider that the computed logic in the code might come up with a different result on each build. Simple case; DateTime.now is always different! i.e. the build output (the build artefact) is closer to the truth, rather than the code. The live system is the actual truth - very good point!
The example with date and time is a good one. That's why we have GitOps repos that define specific releases that should be running in an environment rather that the latest or source code. Eventually, the desired state is a collection of releases with env. specific variables and manifests rather than the source code.
@@DevOpsToolkit Works well with container images which can produce a static i.e. immutable build artefact. Does not play well with infrastructure, which does not have a deployable artefact. Great points, thanks.
That is partly true. Some part of infra are mutable, but others can be (and should be) immutable. Networking, for example is often immutable. VMs that constitute a cluster, on the other hand, can be based on images in a similar way as containers. But, the servers where those VMs are running are mutable. So yes, infra is not completely but only partly immutable.
Good video! Thank you for the content.
"If you are clicking buttons and filling forms, you are in the wrong field." Love it!
These videos are extremely relevant to my job as a DevOps Associate at my current company. In our small team, your insights are invaluable. Thank you so very much. I'm checking out your other learning materials as well.
It sounds like you're talking Automaton theory by Alan Turing 😃. Good presentation of GitOps; However, I think only a few people truly understand and enjoy your talk. Keep up the good work though. Thanks for sharing!
8:14 As a one man team, I can not agree more. When you're spending the majority of your time working on application logic, not having infrastructure defined as code creates a lot of uncertainty when you need to modify something months later.
So from first principles, what we're trying to do with GitOps is:
1. Encode and compress instructions using domain specific languages to interface with machines.
2. Store those instructions digitally and make them accessible over a network to eliminate space and time constraints.
3. Keep the change history to make change auditable and reversible.
I'm imagining a future in wich we interface with machines using natural language. No special syntax required. The spec emerges through conversation. Until we reach that point, GitOps will be the most efficient way. Software Engineers will be needed to translate from natural language to DSL and to create the machines interpreting that of course.
You summarised it much better than I did.
What an amazing 20 min I had watching this video, got what I needed to learn as a concept. Tools/Technologies to implement this concept will surely demand it's own learning curve... surely unlearning is another important philosophy which I'm fully agree as a passionate 'Software Engineer'
Take away from your teaching on GitOps - "It's a matchmaking of Man-Git(Ops)-Machine", or in few more words "Must Code of what you desire, Git is single source of truth of your desire, prefer declarative approach and follow immutability ".
That's a great summary!
lol I love the rant about needing to write code or find a different career
Nice flow
Very well put , especially on remining us (everyone) on basics. Thank you
My pleasure!
I think I agree with you that eventually all work is codified into code. But in the process of getting there, I sometimes want interactive processes that let me fiddle with ideas along the way in a lighter-weight fashion than having to check something into Git and waiting to see what pops out the end of the pipeline. I don't always know exactly what I'm doing. I don't always know the best way of solving a problem. I need interactive ways of fiddling with ideas. What I'm thinking about is something like Jupyter Notebooks for developers. Something that lets me interactively build up a set of information sources from different places and play around with different presentations. Once I've got it how I like it, then yeah, I guess I want to express that as a config file I put into Git. Same goes for interactive, manual testing. It would be very difficult for me to jump straight to a fully automated test suite without first having an interactive, manual test plan that lets me work on the testing kinks first.
Oh yeah. I also use different ways to learn and figure out what I want. Call that phase "learning". That's very different from going to production.
Git used to it! :)
Interesting. First time learning about Gitops and this was explained really well.
This is golden.
Excellent pitch! Many thanks! Do you have it a transcript somewhere? a blog perhaps? Would be good as a reference.
Unfortunately, I do not think a blog post based on that material. I can probably export the transcript from the video if that helps.
Can we consider ArgoCD as a GitOps tool? If we assume that GitOps means that git is a single source of truth, doesn't UI offered by ArgoCD kind of breaks this principle, as it is possible there to manually synchronize to some different revisions?
Argo CD UI used to observe the states is ok. The part that allows you to create or edit apps directly in the cluster is contradictory to gitops principles.
17:55 I've seen people build UIs that work on YAML and I've seen editors that save directly back into Git. So it should be possibly, it would be good if the UI before it commits to Git asks why the change is made so it will be included in the commit message or even better: pull/merge request.
There are indeed UIs like that, some of them public and others built for internal use. Still, they are more of an exception than something common.
As a side note, my team is working on a public project that will do something like that. We are in the early phase, so I cannot yet share any details. Soon...
@@DevOpsToolkit I have had some extensive ideas about what a generic webbased YAML editor could be like. My guess is 2 definition files are needed. One which is like a schema, which describes what the different options of the YAML format are (Kubernetes, Ansible, etc.). And the second is a description file which describes the UI which should be rendered with the labels that go with it and help texts. Maybe it's 1 file which contains both. My thinking the UI should map to different UI elements for example which fit something like Bootstrap/Vue Admin Templates (Kubernetes Dashboard uses Andular Material). I didn't have the time to build it, otherwise an open source webbased YAML editor might have already existed. :-)
Can you tell me what is the difference between GitOps and Continuous Deployment?
Continuous deployment is about continuously deploying apps to production. That typically means the whole automation of the app lifecycle, from pushing a change to Git all the way until it is running in production. Between those two is typically a bunch of steps like building, testing, deploying to temp envs. etc. CD is the whole process.
GitOps, on the other hand, is about defining everything as code and having a process that continuously reconciles the actual into the desired state (the one stored in Git).
You can say that CD is an evolution of CI (a superset) and about the automation of the whole process with GitOps being a part of it. For example, in my CD pipelines, I would make some changes to manifests stored in Git and those changes would be picked up by GitOps tools like Argo CD or Flux. Before and after that, there would be many other steps in a CD pipeline.
Thank you for the explanation.
Source code is not the source of truth. Every build will be different, in principle, because of external dependencies. Today's build is not guaranteed to be the same as tomorrow's, even when the code has not changed. Try it out!
Assuming that the exact dependencies are specified and stored in git, it is the source of the desired state. Now, I do agree with you that git is not the source of truth, but for a different reason. Live system is the source of truth. Git is rather the definition of the desired state. The actual state might differ from the desired state, so the actual state is the real source of truth while git is the source of our intention of what the actual state should be.
Good response. Thanks. Key points; it is not the source of 'truth' but is an expression of the desired state. Very important; we need to pin dependencies, otherwise the expression of the desired state is just a moving target. Finally, consider that the computed logic in the code might come up with a different result on each build. Simple case; DateTime.now is always different! i.e. the build output (the build artefact) is closer to the truth, rather than the code. The live system is the actual truth - very good point!
The example with date and time is a good one. That's why we have GitOps repos that define specific releases that should be running in an environment rather that the latest or source code. Eventually, the desired state is a collection of releases with env. specific variables and manifests rather than the source code.
@@DevOpsToolkit Works well with container images which can produce a static i.e. immutable build artefact. Does not play well with infrastructure, which does not have a deployable artefact. Great points, thanks.
That is partly true. Some part of infra are mutable, but others can be (and should be) immutable. Networking, for example is often immutable. VMs that constitute a cluster, on the other hand, can be based on images in a similar way as containers. But, the servers where those VMs are running are mutable. So yes, infra is not completely but only partly immutable.