(Simple?) Real World Azure Pipelines YAML Walkthrough
Vložit
- čas přidán 25. 07. 2024
- Real world pipelines for real world applications are more than just unit tests and deployments. They must also handle multiple environments, safeguard production, secure pull requests, version releases, promote Docker images and more. Let's walkthrough a relatively simple example that only has 2 environments and deploys to a managed PaaS. Although I use Azure Pipelines in this video, most of the concepts are vendor-agnostic.
Note: this is a very opinionated workflow, not a golden solution for everyone.
#DevOps
Image background by Jeremy Thomas on Unsplash
unsplash.com/@jeremythomaspho...
====== References ======
Containerized demo Node.js app in video
github.com/julie-ng/azure-nod...
YAML Pipelines and Documentation for the project
github.com/julie-ng/azure-nod...
====== Contents ======
00:00 Intro
01:50 Hello World Pipeline
02:57 Release Workflow, Automated Change logs
07:30 Variables
10:37 Dev Environment
14:00 Docker Image Promotion
19:00 Closing - Věda a technologie
This women is a national treasure, we must protect her at all cost !!! 😮 Thank you for this video Julie ! This is exactly the type of content and take I wanted to see on how to make azure pipelines !
That is soooo kind of you - thank you! ❤️
Came in to learn some azure pipelines best practices, and ended up learning about 50 new things in the first 10 mins. Every now and then you just come across a channel that makes you question whether you're even an IT professional, and I love that. keeps one going and eager to learn more. Great job Julie
Thanks Riadh! Keep learning. Never question if you're a professional if you're a constant learner :)
Started digging into azure devops for 24 hours, and scrolling through the pipeline at the beginning of the video made me aware a stage condition can be stored on a variable. LAND !
Your videos are hidden gems. Thank you for sharing all this knowledge 🙇♂️
Thank. you zlovcevic for the lovely comments! It means a lot to mean to hear that it's helping people and it motivates me to make more. Thank you!
I can't explain how much I've learned with this video. Thank you very much for sharing all this knowledge
wow! what a find you are Julie. Great content and very well presented. Thank you
Thank you for watching 🤗 let me know if you think I should make more pipeline YAML videos
Your channel should have more than 100k+ subscriber already. Great work!
Thank you ♥️♥️♥️♥️
Wow, absolutely wonderful. Im new to pipelines and i found your channel. Just amazing.
I would have loved to have a pipleline tutorial video for beginners by you. What to think of, how to structure it and so on...
Kepp up your good work and have a fantastic day =)
Love the video format.and cadence. Really helped
Glad it helped - thank you!
Great video.
I'm learning a lot from your material.
Thanks Carlos! Let me know if you have any particular questions or topics you'd like to see :)
Hey Julie, love your videos! I am learning a lot of nice tips as to best practices. If you're looking for specific topics and such I do have a suggestion. Could you do a "start to finish" video of building a pipeline? Most of your videos are shown from the point of "I have this thing already built and I'm tweaking it." But it would be great to see a breakdown of the whole thing start to finish and an explanation of each part. I know that's a little heavy in terms of a topic but I would find it incredibly useful. Great stuff!
Hi Matt would just explaining an existing pipeline but much slower help? Would including how to add it the CI UI (Azure Pipelines or GitHub Actions) help? Is that missing? What else may be missing for you?
I am concerned that building a pipeline from scratch on camera will take some time.
@@JulieNgTech I think explaining an existing pipeline would definitely help, no need to actually type the whole thing out on camera. I lot of times I look for videos like these in order to make "best practice templates" for my organization. When doing so it's super important to understand "why" it's best practice and how it all fits together. The one off tips are great don't get me wrong but the overall picture is extremely useful. The git repo you have that has the example yaml file has been invaluable. Part of me wishes I could peruse your devops instance to see if there is any configuration in there that you have done that I should think about replicating. That sort of stuff I can investigate but putting it all together in a video would really help.
@@mattkruczek2665 ok now I understand, the end to end perspective including UI is missing. It's on my list since it's something I regularly do with customers. I just need to record it for CZcams and let you know :)
@@JulieNgTech Awesome thank you SO much Juile, your videos are really helping me create our "standard" for dev ops.
@@JulieNgTech That would be helpful to start from scratch please. Probably good idea to break down the tutorial into smaller videos.
Love ur videos, Julie
Thanks Krisna!
I’d love if you go through real world use of yaml release retention policies. Would love to see something that works with other considerations like managing source tagging (which could change based on whether it completes the release cycle or fails at some stage), and also managing retention in light of multiple in-process releases for an organization that may not complete a full release cycle deploying to production before producing more builds to promote to various testing environments.
Thank you Julie, 吴小姐! I find your videos really helping to make real-world yaml pipelines!! I've been coming back and forth for different topics and trying to adopt your recommended best practices. Thank you so much. Please make more yaml pipeline videos!!
Do you provide training or consulting sessions to companies?
One last question, where do you get other opinionated approaches for organizing pipelines? 😃
I will definitely make more YAML pipeline videos. It's crazy cold and snowy this weekend so I may do a video walkthrough of some "real world" GitHub Actions pipelines.
Yes, training and consult is part of my job. I don't offer it outside of work. But if your company is deploying to Azure, then you may qualify for the FastTrack for Azure program: azure.microsoft.com/en-us/programs/azure-fasttrack/
Thank you so much for the feedback and I'm so happy that it's helpful. So much so that I asked my brother what you wrote in Chinese :) My last name is five btw, so "Wu" in Mandarin. Because my parents emigrated to the U.S. before names were standardized, we're "Ng"s
Truthfull laughter :) 4:13 - and laughtfull truth: 7:25(one week later....) - current media are missing with truth its good to have You here!
Hey Julie, love your videos! Thank you for sharing all this knowledge. Can you teach me how to create my own pipeline for my project? I'm learning DevOps.
Like a video on how to create a pipeline from scratch? Because there are so many beginning and overview videos out there already - what's missing in them that you think I can do that others have missed?
Shoutout's to the email notification at 2:26 that made me check through all my tabs
😂 I've since learned to keep my computer muted when recording. But it also means I might miss a meeting notification 😅
@@JulieNgTech small price to pay imo XD
great class! tks!
Thank you Rafael. Glad you found it useful. ❤️
How do I get the console prompt like yours with branch name and how long the last command took?
I use the Spaceship Prompt but have customized it to be more minimal. You can copy it if you want github.com/julie-ng/dotfiles
We need more pipeline videos Julie please do not stop filming 😢
hey, yeah, sorry. Been going through a rough few months and haven't found time or motivation to film. But thank you 🤗 for asking. That helps with the motivation bit.
Great video. I am just learning Azure Pipelines and trying to set this up for a monorepo. How would this translate to a monorepo? Say I have 15 "packages", Does each of them have a DEV/TEST/PROD pipeline? Thanks
If you're starting out on a new project, consider avoiding monorepos especially if you have 15 packages. That's going to be a nightmare. Even with 2 packages with potential dev/test/prod, you have at least 6 triggers to manage. Imagine the combination with 15 🤯
That being said there are uncommon use cases for a mono-repo usually because of legacy code that I've seen. And in those cases the teams are working towards splitting the monorepo so it's not a goal but an intermediary state. A long time ago I built an example here that may be helpful to you github.com/julie-ng/azure-pipelines-monorepo
@@JulieNgTech yes I was working from your monorepo example. I have both an existing front end (react/js) and backend (.net) monorepos for separate services and common libs. If I break up these into separate repos won't I still have to deal with these triggers in the individual repos?
I really like your videos, nice tone, lot of usefull information, thanks ! Keep going, please, even after the pandemic ;-)
Thanks Julien! Knowing it helps people, I will try to continue even as the sun starts to shine… thank you for the feedback, I think it will really help me try to include this in daily 9x5 work - which is the only way I can do this long term.
Very nice, lots of info!
Q: any advice, tips/tricks how to improve pipeline testing. I always create trillions of commits until it does what it should.
I sounds like you're already doing like real life engineers - lots of commits. I also need many "oops" and "try…" commits before I know it's properly working. That's why I miss having a single tenant Jenkins when you could just write groovy in the UI and run it and get result immediately without waiting…
If you're using GitHub Actions, you could try this github.com/nektos/act to run the pipelines locally for faster development.
@@JulieNgTech Thx! I am using AzureDevops Pipelines, so still have to look for something similar.
Nice one Julie! So many fish (es?)
Do you use this same folder structure to build infrastructure(using terraform)?
No, infra is different. IaC is risky. There is an example here though you can check out github.com/Azure/devops-governance be sure to browse the backend/ and environments/ folders, not just the azure-pipelines one. They should be documented relatively well. If not, open a GH issue and let me know and I can try to improve it :)
@@JulieNgTech thanks a lot. I am checking the repo.
Thank you
You're welcome, glad this was helpful for you :)
how much is it costing you to run all theses resources?
You have my shield
🛡️
Could you please show us Corporate workflow for AKS deployments using Azure DevOps :) and thanks for this vid
For workloads to AKS? I don't have a distinct video teaching. But the latest video is me trying to deploy to AKS using Copilot to write the GitHub workflow pipeline and k8s YAMLs. That might be interesting - or at least entertaining :)
For deploying the cluster? Does it have to be Azure DevOps? If it's GitHub Workflows, maybe :) Did you watch the GitHub workflow video? czcams.com/video/UG4o4lA2eak/video.htmlsi=V0AFugZT281ORKC9 Asking because I think the pipeline concepts are generally the same. The syntax will be different and GitHub has some extra constructs like the "environment" and federated identities (watch my most recent video about deploying with AI to see).
[4:31] That keyboard sounds like a repetitive injury building up. Be a keyboard ninja (make no sound) and you'll type faster too.
😂it's an ergonomic keyboard - not mechanical, but it still makes a lot of sound. I don't think sound is an indicator of RSI. Look at all the people who swear by mechanical keyboards ;-)
I have suffered from RSI in the past, but that was when I used the awfully flat Apple Keyboard. I now use a Surface Ergonomic keyboard and love it :)
Can you do a complete DevOps course?
Maybe… probably only if my employer sponsored it, i.e. let me do it in working hours - in which case it would/should be published on one of their channels. Have you searched CZcams for other DevOps courses? There must be many out there. What's missing that you're looking for?
@@JulieNgTech Securing the pipeline from SCM to Post Deployment.
@@logicstv I wrote a few articles in the MSFT Documentation. They should be generic enough that they're helpful regardless of whether you use Azure. Let me know if you they answer your questions
docs.microsoft.com/en-us/azure/architecture/example-scenario/governance/end-to-end-governance-in-azure#considerations
docs.microsoft.com/en-us/azure/cloud-adoption-framework/secure/best-practices/end-to-end-governance
I actually recorded versions of this video but the audio/video was so mangled, I need to re-record it.
PROTIP: [13:31] production.yml is very slim. It doesn't run any test or build docker images. It grabs from container registry and assumes it's ok.
Yep it only grabs the image and deploys because it's production. If the git workflow and permissions are configured properly then when this pipeline is running everything before it went fine.
Even Microsoft engineers are despite of windows. She is using linux shell in mac os. 😂
To be clear, I don't despise Windows. It's just not for me as someone who's used a Mac for almost 20 years and mostly works with non-Microsoft specific open source technologies 🙃
Julie's 'simple pipelines' are complicated...
For perhaps it was cool to see how you generated the change log and kept a variables file embedded with various conditions, overall good job.
Real world pipelines are rarely simple 🫠
I am a noob. This sounds like goobledygook to me
Sorry, that's why "simple" is in parentheses. Real world, esp. production code is always complex. If you do a CZcams search, you should find plenty of beginner videos.