Platform Engineering Is The New Kid On The Block
Vložit
- čas přidán 27. 06. 2023
- Platform Engineering is the new kid on the block in the world of DevOps and Continuous Delivery, at least that is the claim. To begin the process of adoption of CD it is useful to have people that know what they are doing and that can guide organisations toward success, but, at least in the messaging around Platform Engineering, it seems to be saying something different to that, and falling into a very old trap.
In this episode, author, speaker and software engineer Dave Farley explores what Platform Engineering is, what it should be, and where current proponents may be offering some bad advice.
-
⭐ PATREON:
Join the Continuous Delivery community and access extra perks & content!
JOIN HERE for as little as £2 ➡️ bit.ly/ContinuousDeliveryPatreon
-
👕 T-SHIRTS:
A fan of the T-shirts I wear in my videos? Grab your own, at reduced prices EXCLUSIVE TO CONTINUOUS DELIVERY FOLLOWERS! Get money off the already reasonably priced t-shirts!
🔗 Check out their collection HERE: ➡️ bit.ly/3vTkWy3
🚨 DON'T FORGET TO USE THIS DISCOUNT CODE: ContinuousDelivery
-
🔗 LINKS:
🔗 "What is Platform Engineering?" ➡️ platformengineering.org/blog/...
🔗 "How Platform Engineering Works" ➡️ humanitec.com/platform-engine...
🔗 "Setting the Stage for Platform Engineering" ➡️ softwareengineeringdaily.com/... platform-engineering/
🔗 "Platform Engineering VS DevOps" ➡️ www.puppet.com/p/free-trial-p...
-
BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is available as paperback, or kindle here ➡️ amzn.to/3DwdwT3
and NOW as an AUDIOBOOK available on iTunes, Amazon and Audible.
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 "Continuous Delivery Pipelines" by Dave Farley
Paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
Sleuth is the #1 most accurate and actionable DORA metrics tracker for improving engineering efficiency. Sleuth models your entire development cycle by integrating with the tools you already invest in. You get a full and accurate view of your deployments, see where true bottlenecks lie, and keep your team’s unique processes and workflows. With accurate data, Sleuth surfaces insights that your engineers can act on to improve - with real impact. ➡️ www.sleuth.io/
IcePanel is a collaborative diagramming tool to align software engineering and product teams on technical decisions across the business. Create an interactive map of your software systems and give your teams full context about how things work now and in the future. ➡️ u.icepanel.io/1f7b2db3
Tricentis is an AI-powered platform helping you to deliver digital innovation faster and with less risk by providing a fundamentally better approach to test automation. Discover the power of continuous testing with Tricentis. ➡️ bit.ly/TricentisCD
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com - Věda a technologie
How do you set up a development team to be the best it can be? I've put together my tips FOR FREE including; what size the team should be, and what skills do you will need, how to you divide up the work, and how to collaborate across teams. Get it direct to your inbox HERE ➡ www.subscribepage.com/organise-teams-guide
If you are around long enough eventually it's just engineering.
"just" :-)
"Platform engineering" is often a nice way of saying, _"We're moving back to specialist teams who don't understand the business context within which they operate - but we call them platforms now."_
Yes, exactly my concern - but much more concise 😉😎
And it's a necessary thing. See: cognitive load. "You build it, you run it" does NOT scale and results in excessive variation, sprawl, and unmanaged attack surface. Factor out the undifferentiated heavy lifting and accept that it's a commodity, shared service play.
@@alphas0ngand often removes any say from the folks who get the latest and great junk thrown over the fence to them to deal with leading to burnout or frustration quitting. Companies aren't dealing with problems, they're just throwing ideas at the wall and seeing what sticks. Well at least that's my experience, I am sure there are companies that do it well as well, I just haven't been at one.
Lol, this is so true 🤣
@@RU-qv3jl Ah... The mythical "surely someone, somewhere is doing it right".
I too have never encountered one of these unicorns 😂😭😭😭
I am a platform engineer and my work is basically to encapsulate the accidental complexity of the environments in which the code runs so the developers can focus on doing code. Why not let the devs do that too? cost and almost impossible to find people able to do so. Most devs we interview hardly know what TDD, DDD, TBD, and CI are now add on top knowing Docker and some infrastructure and you will look at a pool of candidates that is terrible expensive and small or non-existant.
In that scenario is where someone like me, 15 years doing software engineering and system administration, comes and encapsulates a lot of the complexity so the company can access a wider hiring pool and start adding value.
That's the hype. In practice, there really isn't a ton of accidental complexity and these snowflake abstractions are hinderances.
The opposite being job postings looking for a "full stack" developer. If such a person exists and they are good from soup to nuts, most likely a company can't afford them. There are only a handful of unicorns.
@@BryonLape And having to sift through all the resumes where they think sticking a json blob in mongo during the todo tutorial is 'full stack'. I see guys 2 years out of school applying for 'senior dev' positions. Yeah, no.
The problem is that you end up with developers that has no clue how their things actually works in the real world so they can't make informed decisions, or even know who to ask the right questions. How do you expect to grow seniors, innovate and be competitive when you have devs that has no clue about CI/CD, TDD/BDD, observability, operability or how their decisions affects the system in production and are stuck with the some kind of "I write code, then magic happen" culture?
While a good developer-friendly platform is a good thing, it should be there to help with the everyday BAU work and add constraints (governance and compliance). It shall not be an excuse to "dumb it down for developers".
An anti-pattern I see all the time is seniors "controlling" and doing "programming by remote control" with an attitude of "I must review everything because I'm the senior". So many organizations hire juniors to just fill a seat and expects them to be productive and because of all the new people they try to control the work with processes and the aforementioned programming by remote control, not (wanting to) understand that for every junior developers they add they are actually removing a senior developer for the next six months. Because that is the job of seniors - to mentor and coach so that the new people **can** be a trusted productive member of the team. An organization with senior devs that does not understand these "sociological" things is not a good place to be. I don't care if they have been doing it for 20 years and can write the most beautiful code ever written; if these things are alien to them they are **not** seniors (you'd be surprised how common this is).
The primary job of a senior engineer is to create more senior engineers.
@@ddanielsandberg That is the ideal but the truth is that a lot of people working as devs does not care at all about all of that and you cannot mentor someone who has no interest. Sure, some do and those are mentored and I explain to them why I do X or Z and how things work but those are like 1 out of every 100 and you cannot run a company with only the ones you like because then you will never have enough people.
I think the hype around platform engineering comes from the same managers that 10 years ago demanded big data solutions for dbs that fit in memory. A premature silo to pretend to be bigger than they are at the expense of the engineers
For me, as mobile engineer, overuse of platform team concept nowadays looks more like a result of dysfunctional processes.
“We need a platform team” is usually said when project is buried in technical debt due to inadequate power balance engineering vs product design. So the alleged solution is: let’s put a couple of engineers outside of product design so that they are not overloaded with features and can take care of overall quality.
This is a pretty good example of what I have seen. Another challenge I have seen also is that every team in the organization is taking a "you build it you own it" approach and has people sitting outside of the product cycle that ultimately are all building the same kinds of things, not talking to each other, and get bogged down in technical debt because of how many things they need. There is never enough people to always manage, upgrade, and maintain the underlying stuff so the pendulum shifted back to more a centralized model.
It makes sense to some extent that those people would come together as a team and collaborate to build shared services that team would ultimately use rather than building all the same services for their unique teams.
You're describing my current client that has a platform team for exactly this purpose. Suffice to say, it doesn't really help, because everything this team does is completely disconnected from the business domain, meaning they are throwing overkill solutions on really simple problems, and underdimensioned solutions on really hard problems.
It reminds me ESB. On my next to last job my company had specialized ESB teams (and even an architect responsible for all of that) to hide the complexity of communication between services. I was part of it for some time implementing several ESB-services. One job and couple of years later, my next company was decommissioning ESB-services (and teams) to bring control back to feature teams. They also introduced devops teams (they called theirs members SRE) to hide complexity of infrastructure and deployment. So devops created a pipeline, although it didn't deliver anything anywhere. I left that job too, so I don't know how it all ended.
There are no new ideas, just new ways of labeling them.
Yes, for instance: timeboxed waterfalls, also known as Scaled Agile Framework.
I wouldn't exactly call platform engineering new, I worked on one a decade ago. What's new is this insane hype around it.
Imagine expecting a software engineer to deeply understand networking and how a load balancer can be optimized or maybe how to somewhat standardize k8s clusters across hundreds of teams in order to reduce costs or maybe you want to use a data platform but need a high data governance ? Well, platform teams do it (with well-defined and automated APIs) so stream aligned software engineers can deliver what matters.
Team Topologies explains it really well.
Team topologies explains “platform teams”, I think this is different to “platform engineering”. Abstracting things is good, but is not about tools, it’s about design.
That’s true. I’ve seen some terrible approaches being done in the name of “Team Topologies”, transforming engineers in simple “code monkeys”.
That wasn't the job of a system admin 5 years ago? Before all the hype started?
@@RicardoSilvaTripcall Yes, and how badly that worked was one of the inspirations for Infrastructure as Code, DevOps and Continuous Delivery. I don't want to work on a project where I have to raise a ticket to change the config of my service again.
@@ContinuousDelivery That's where I think you are off base. Platform engineering is generally the same as TT platform teams. The marketing hype is Humanitec's fault. But they sponsored the latest Puppet survey which CLEARLY calls out Team Topologies as the canonical definition.
I think at least part of the issue here is we are talking about two very different things that have almost the same name. The new "platform engineering" role appears to focus on DevOps admin and common tooling. But the platform team as described in Team Topologies is a software engineering team that creates things like common code libraries and interfaces into security, connection pooling, auth, monitoring , logging, metrics, and other fundamental code requirements.
Platform engineering has co-opted the term platform and distorted its meaning. 0:02 0:02
I was on a project where we started using serverless and it was great. We used all the abstractions that AWS provided directly for infrastructure and we could write very clear code where we understood everything we were using, all of the the platform abstractions we were using were clearly documented by AWS and predictable and high-quality.
We were so successful the business started investing more into our serverless and bringing "serverless" enthusiast people on board. At that point, it all fell apart. The serverless enthusiast people started writing "construct libraries": more abstractions on top of the AWS abstractions to abstract away bits of infrastructure that might be useful in many places, so we could be "DRY". Now we had to contend with home-grown "platform infrastructure" abstractions that were not at the level of quality that AWS could deliver.
I work in the enterprise. I’m considered a platform engineer and previously called a DevOps engineer. I like the term because it puts me and my team into a product mindset instead of services mindset. Products scale much easier than services. When I build a platform, I’m building abstractions of hundreds of AWS services, services in AWS, on-prem and other SaaS services with APIs. I look at the services through the lens of the organization’s security and compliance policies. This saves developers so much time that they don’t have to worry about the security and compliance of a service because me and my team put those guardrails in the abstraction. We cannot expect developers to know all these things or be expected to learn them. It would be such a waste of money. Another benefit a platform is on boarding and off boarding developers, self-service and giving management the tools they need to hold development teams accountable.
Where can I get that T shirt? I am having a bad day with our "Platforms"
My understanding is that platform engineering helps with plumbing. For any decent sized project you want devs to be working on business problems, not battling arcane platform/infra stuff.
In last 3 orgs we had foundational libraries that help with things like message passing, auth in web services, caching, working with relational DB (both data and schema), logging, metrics, healthchecks, object storage, notifications (sms/email).
Previous employer had me and one other colleague who built this stuff for several dev teams to use (we worked on business tasks as well), and eventually they introduced the dedicated platform team to improve the libraries and support other tech stacks that are in use.
In my opinion those libraries are essential even if you use one tech stack across the company. They make it easy for easy onboarding, switching developers between projects
yea that's a good platform team, but in most current platform teams, they don't work on core libraries!... they work on abstractions over cloud services/platform!!!
Thank you Dave, As Covey says you have to start with the end in mind. And in Business 'the end goal' is not use cloud or containers it is to make money
I am on the platform team. It works well, and I enjoy it. Nothing wrong with Platform teams.
On the platform team 🙂Must be the global one, where everything is abstracted to so many levels that you do not even realized that you are inside the Matrix
Loving the Tshirt Dave!
I worked at a fortune 100 company with a forced 'platform' and it was terrible. Forced upgrades that could just blow up an entire sprint with breaking changes, yet you can't get things like a modern version of java on the platform. Some terrible leaky yaml mess over the real cloud apis, only you can't google anything and slower, undocumented bug fixes. A new cloud service that solves your problem? Not for at least a year for the 'platform team' to add another bad abstraction over the latest cloud services, assuming they think you need it and they feel like supporting it.
All while touting innovation and autonomy.
this is an example of a platform team without proper product management, which is a disaster.
yep, that sums up most platform teams... when it should be a team that work on core libraries and apis cross-cutting concerns instead! ...why build an abstraction on top of a cloud api is beyond me!..
@@alphas0ngwhat's proper product management though? A magic bullet? A dream? A promise? An elusive target? A mythical place? The way it's managed in reality is known: negate stream aligned team needs. Comfy.
It starts with folks like Marty Cagan, Melissa Perri, Theresa Torres, Steve Blank. There's a rich set of insights and literature. @@Fred-yq3fs
Today one of my colleagues told me he worked with the LMAX Disruptor in one of his previous roles (at a different employer). It just so happens that the dependency is available in our artifact repo. :) I made a slight adjustment to my version of the EventLoop pattern so that it uses an ArrayBlockingQueue instead of a LinkedBlockingQueue, because I hoped the focus on contiguity would show an improvement. I didn't get much of a response when I was asking a general audience of technologists if we have a way to evaluate cache hits and misses in our infrastructure. :( How are we even going to do Cloud Native if we can't evaluate cache hits/misses :(.
Great video - and some great thoughts shared in the comments.
I agree, the same problem that happened to "DevOps"
From being the culture that helps in deploying code as seamlessly (without friction) as possible.
To
Any team of sysadmin that knows jenkins...
Exactly!
I agree with the statement "this is what platform engineering teams do, and this cannot work in cases other than the most simplistic".
But
I think 95% of these organizations have over-engineered themselves into frustration, and in reality they ARE operating in simplistic situations. The cost then is self-inflicted.
Don't forget: elegance comes from simplicity.
Don't forget: vertically sliced, highly available, modular, hexagonal monoliths go a very long way.
Don't forget: each microservice in isolation is exactly a modulith as described above, and independently deployable.
Ironically, as a consultant, those self-inflicted costs you mention ends up as revenue in my pocket. And when I tell literally every single one of my clients (except one) that the cost is indeed self-inflicted and it only ends up enriching us consultants, they:
1. get mad at me for telling them
2. don't let me help them correct the problems
3. force me to dig the hole even deeper
4. pay me good money to make the problem worse
¯\_(ツ)_/¯
Thank you so much!🙏
Spot on analysis Dave !
Here we go again, another new term to label old stuff. 🙄Here's the deal. What we're all doing is developing software for users; we provide environments for work/entertainment/play. Platform engineers are just software engineers developing the SOS (same old sh!t) over and over again, but combining them in different ways. We just keep creating layers of abstraction to define niches and to differentiate ourselves, but the user-facing result is always the same. Always.
Having said that, the platform I've been developing over the past several years actually does something truly different for users: allowing non-programmers to define those work/entertainment/play environments, and it does so in a novel way (I'll shame more when we're closer to release). It's a tough job (
I guess I've been "platform engineering" this whole time.. CV updated!
Centralised platform engineering teams are a variation on Competency Centres or Centres of Excellence. Platform Eng as a decentralised role isn't so bad considering roles in IT are not defined in any standard way.
I think it was a PR mistake to focus this on cloud engineering. But there is a gap where secondary systems that support the product would be better served by a dedicated team to enhance and optimize, while another team focuses on the product innovation.
In my experience, there have always been developers who stepped up to do this work. But this can hurt them if their review metrics were skewed towards measuring product/sales-driven impact. To me, this is just a formal way to recognize the work that's already being done. Or, a label to give to management when you're asking for more headcount because the capacity for this work is currently lacking!
Edit: Also, I'm assuming this is all in-house, not "someone else" optimizing those secondary systems
Everywhere I've been where they've had a platform team it's just a re-introduction of the operations silo.
I didn't say that there was, Platform Team != Platform Engineering
This can be done very well and very poorly. Unfortunately, I mostly see the latter. My first exposure was the former because we were focused on making things easier for our customers, the other development teams.
The hype about platform engineering comes from people selling tools. Saying that platform engineering replaces DevOps is like saying transmissions replace cars.
platform engineering is actually a reaction to the failure of DevOps - not the tooling of DevOps, but rather the belief that we can have these "rock star" developers who can do all of their fantastic daily dev-stuff on-top of procuring, designing, maintaining and consistently improving cloud infrastructure, CI/CD processes and other stuff that takes a huge amount of cognitive load.
@@billsmoke4919 That's not a failure of DevOps. That's a misunderstanding of DevOps. DevOps is about improving the delivery flow, not "the development team owns everything required to deliver."
I've worked as a platform engineer at a geospatial research company for a couple of years now. The whole point of my position being created was that the company tried the whole DevOps thing (like many other companies) which led to the Devs and data engineers being completely burnt out from having to maintain infrastructure, monitoring, observability, CI/CD processes on top of their regular roles. I've got over 15 years experience working as a network engineer, sys administrator, linux admin and have quite a solid basis in AWS, so I do a lot of the infrastructure stuff that Devs don't give a fuck about lol. I don't think Platform engineering is hype, to second another persons comment, platform engineering is a reaction to failure that was DevOps, or trying to entirely amalgamate developers with the enormous task of maintaining networks, CI/CD processes, observability and cloud infrastructure.
But this type of work already existed 5 years ago, until someone preached that devs should know and take care of everything, so now we're going back to the place we shouldn't have left in the first place.
This Hype-driven development world we are living in today has caused more harm than good ...
@@RicardoSilvaTripcall "so now we're going back to the place we shouldn't have left in the first place." 100% agree with you Ricardo. Ops and infrastructure (especially with networking, linux, security, compliance, CI/CD and cloud architecture) are not things to take lightly.
I am certainly not suggesting that we should take these things lightly, nor that they skills are unnecessary, the problem is how to organise those skills.
What is the difference between you optimising generic tools and technologies, and Google, Amazon or Microsoft doing the same?
Surely it is *context* if you are doing this generically, then forgive me, but my working assumption until proved wrong, is that the cloud providers are in a better place to generalise those things than you. However, what you have, or at least should have, that they don't is that *context* you are doing this in the context of the code that your teams are building, and at that point that is design and architecture and is problem-specific and not technology-specific.
In the teams I worked on, very technical teams, we'd want your kind of expertise, but working essentially as part of the team, or at least in the context of, the specific work of those 'stream-aligned teams'.
I have no problem with the problem of 'Platforms' in fact I think that they, and the abstractions that they represent are an inevitable consequence of decent design. But that is NOT how 'Platform Engineering' is being positioned or described.
@@ContinuousDelivery Hey Dave, thanks for replying, a big fan of your work.
To answer your first question, I think that although the FANG companies you mention do have way more resources, it doesn't necessarily mean that they are better at generalising technologies. I know this from experience when myself and the data engineers I work with were using AWS EMR and Apache sedona in a project that was pretty unique (unique in that we couldn't find any examples of anyone else doing the same). While AWS support was useful to a certain degree, the vast majority of the discovery and advancements of the project came from us optimising and adapting the services by ourself.
This is similar to other projects involving ECS, Elastic Beanstalk and EKS - if you are doing generic, bog standard projects using these tools then you are right, but from my own experience the minute you start blending these with other tools and combine them with more unique forms of architecture, then you'll find their support (and documentation) is pretty damn thin.
Ultimately I completely agree that the platform engineer should be both aligned with the dev team and that platforms create abstractions that make it easier for them to focus on their bread and butter. I think the way in which Platform engineering is described can simply be put down to tech recruiters and recruitment agency pushing forward the 'next big thing', and in that form it is definitely a hype train that is quite different in reality.
@@RicardoSilvaTripcall That "someone" being Werner Vogels saying "you build it, you run it."
I like your way of thinking. These are indeed the universal concepts of software engineering, but I struggle to see the contradiction.
Hiding complexity from stream-aligned teams requires both a technical expertise (i.e. a case of separation by tech) as well as a good abstraction. The fact that platform engineering doesn't define how to get there (the case of the last couple of pictures) does not mean there is something wrong with what it does say. Yes, people tend to focus on a more simplistic view because it is so much easier to promote to the masses, and I think this is what happens in the sources you cite, but again, it does not rules out the fact that a good abstraction has to be there.
I feel like the "power" of platform engineering is in reducing the scope. If a platform engineering team focuses on a single product, there might be just enough natural domain constraints for a good abstraction (still built on top of complex tech developers not necessarily understand) to emerge. Otherwise, yes, it is a case of general cloud which is already there.
I think you can abstract a lot of it away with things like serverless functions, dapr, opentelemetry etc.
Even docker works as an abstraction layer and isn't difficult at all, you can generally generate a dockerfile from your IDE.
K8s can get a bit more complicated but there are managed k8s offerings in cloud that are usually good enough.
The reputation of RDBMS to not scale well is over exaggerated too. You can get a lot of performance out of a good schema, and you can eventually scale horizontally with sharding.
How can this be a full time job of many developers? It 95% of the time some one time thing you apply and make reusable within your usual feature team.
I don't even understand how is the performance observed of such teams. Also I don't see any business value that these teams add, how would you hire these people who are out of touch with the actual business. I would never let such team evolve from within my business. There is no need for separate teams who deal with purely technical concerns.
I'm part of such a team, and I can tell you that all your observations are 100% correct. It's madness.
I think this is preparing the area for new category of software for witch companies will be able to sell solutions for. "Are you developing your own internal platform? Here is a one-click solution for that!..."
I think I have a name for that... It's called "The Cloud" 🤔 Amazon could call theirs "AWS", Google perhaps "Google Cloud", Microsoft could be a bit quirky and call theirs "Azure" 😉
Is UML part of platform engineering?
No, UML is just a tool.
"We can solve any problem by introducing an extra level of ... abstraction." or... When your only tool is abstraction... ;)
Tell me if you agree with this litmus test:
If your stream aligned team needs to create a ticket and toss it over to another team to get some infrastructure built out for them, you're doing neither "DevOps" nor "Platform Teams" the way Mr Farley would recommend.
If your stream aligned team has the expertise and empowerment to get that infrastructure built out when they need it, then they are.
It doesn't mean any one person necessarily has all the skills, it just means the team does.
And the skills should (deliberately, ideally) interrelate, overlap and complement each other.
Frankly. I'm just lost. I don't think this video helped. *shrugs*
the important thing is the platform surface must slope away from the tracks
bob the builder shirt is 10/10
Agree with you that Platform Engineering is a total crappy trends... I will get back to you later during one or two week about what I call the KissOps Principle,,, because I need to do a total revisit to my first *Boring* and *too-long* KissOps presentation
I can't find any one to show how "Platform Engineering" is done, as if they created this new thing for shareholders. It honestly sounds like cloud engineering, but for developers instead of IT Operations, so the dev half of "DevOps".
Thing is, businesses are trying to save money by layoffs and downsizing, not throwing more money in IaC.
a lot of the platform engineering hype comes from some well know facts... if one opens all cloud provider services whiteout curation to a large enterprise, we going to have many team investing huge amount of effort to end up with a platform for their own tiny application that looks nearly the same as everyone else... k8+some gitops+ maybe some crossplane this would be the most common pattern with some lambda or ecs niches... most of this stuff is often completly irrelevant for the aplication design... if you get your rds db with crossplane or cdk cloudformation is irrelevant for the aplication itself.... you also mention complexity... I think you underestimate the complexity say, the CNCF ecosystem.. should one use flux or argocd? developing your own gitops framework is a huge effort and likely you would do it worse than the existing frameworks that already implement many of the best practices... and it goes on and on... service mesh, canary, test automation, observability, bla bla... countless tools and frameworks, many equivalent that barely change the application architecture or design at all. so yes, this is a lot about tech, curating choices so that every developer in the organization does not have to. this by itself, ofc will never solve the problem of bad application design... maybe some people early in the hype cycle some silly things like PE being a silver buller... I guess this not the problem of PE itself but of marketing and sales people being over enthusiatic as it happens with any other IT hype.
i agree completely. The most important thing about platform engineering is really to provide some good abstractions that lets one easily deploy multiple different cloud resource configurations (e.g. lambas, fargates, load balancers). These abstractions should be generic in a way that the developer can declare which configuration they want and enable them to set up a configuration that actually solves the complexities mentioned in the video (eg scaling) comfortably in an easy and reusable way for the common patterns.
Exactly. How many different ways are there to persist data on AWS? Without a platform team, your "build/run" product teams will use an unmanageable assortment, with bad outcomes in terms of staffing, security, resiliency, and other quantifiable business outcomes.
Platform engineering like this sounds like a fkn nightmare.
If you've just started out, you can even become a prompt engineer.
In general, being too specialized in one particular area and knowing nothing else about anything else is stupid. Managers think that this is a good approach because it works in pencil factories.
I subscribe to your vision.
PE applied to CI/CD pipelines is falsely viewed as a way to move away the design and responsibility of CI/CD to a central/magic team who somehow knows it all.
It ends up lengthening time to delivery by creating more friction, and needless constraints.
The fallacy here is to pretend that one size fits all. The industry hard sells PE with savings (hire cheaper non devops devs) that will never eventuate.
Root of the problem is devs have to play too many roles. Fire the testers, we have unit tests and devs are now running the tests. Fire the architects we have platform as a code now. Fire the managers we have scrum masters. Fire the infra people, we have devops, fire the security people we have secops.
I don't think that has really changed, at least for the kinds of systems that I have spent most of my career involved in, probably at the more complicated end of the spectrum, it has always been like that, but that is the job really IMO.
If you are doing something "vanilla", then it was then, and is now, a bit easier. One of the BIG traps now that I see lots of teams fall into is making simple (vanilla) problems complicated by assuming that they must use the techniques of orgs that have complex problems. Microservice in tiny teams, K8s for simple apps that will never need to be internet-scale and ignoring old stuff that works like the RDBMS programming model.
@@ContinuousDelivery I'm working for big financial companies here, and the business politics is all that matters. They aren't interested in solutions, just how much cost cutting they can get away with, fetch bonuses, and anything that fails falls under calculated risk, hire 200 external experts and the public forgets the outages in a few weeks time.
Rinse and repeat. It may have never been different but as time progresses it gets worse and worse. It's not even about how teams are put together or piling roles onto people that haven't got a clue as to what they're supposed to do. IT people try to do a good job, failing that just make do, failing that damage control etc. It's must godawful stupid and destructive to their own business.
@@imqqmi Sure, and there is certainly a degree to which we probably can't change the world, but if we don't try, it is certain that the world will never change 😉
I see this as, an extremely common, failure on 2 fronts! We techies did some dumb things when we didn't know how to build SW, we still often do. That is the cause of one of the failures, in that it encouraged managers to "micro-manage" us, and they have even less of an idea of how to build software than we do.
Since then, we have learned what really works. We have experience, evidence and scientifically justifiable studies of what works. But we still abdicate responsibility for our work to the non-technical people who don't know what they are talking about. So the failures are - we ask permission to do a good job from people who don't know how to do a good job and we don't try to do a good job, because we are more familiar with practices that don't work very well. This is false economy in every respect.
I worked on a team that built one of the world's highest performance financial exchanges, we built the first production ready version, including going through the hurdles to get approved by our financial regulator with a team of about 15 people in 8 months. Meanwhile one of our market-makers, a very large, very famous, financial institution, had a team of 120 people and a plan for 6 months to write the adaptor between their trading system and our exchange, they were late by 2 months!
So the non-tech folk's assumption of how to do better is completely wrong, they go slower and spend more money, and write worse SW.
So if you want to cut costs, you need to think in engineering terms, and for that we techies need to first believe that we have something valid to say when it comes to how we work, and then engage with the people that don't know how SW works and teach them. - Sorry feeling a bit "ranty" today 😉
@@ContinuousDelivery That's quite an optimistic view, but certainly a valid one! I've done a project a few years back with a piece of software that was badly maintained and logs not monitored in any way. The team was originally from a different company which was bought by the current one. The team was quite dejected at that point and completely ignored by the rest of the company. General thought was the software was eol. Still year on year turnaround was the highest in the company.
Since nothing could be won by managing it they were mostly left to their own devices. I was able to overhaul many of the shortcomings, reinstate log monitoring, reduced number of outages, reduced the risk of failures to virtually zero (system had issues every 2 weeks or so, having to work during nighttime or real time contract sales of our clients would just stop). After a couple of years I asked for more business centric changes, new products, sales went up and we got noticed by management and they fired 25% of the team, and announced rebuilding the software despite the current one was totally viable! They would replace it in 9 months, after 3 months they didn't even manage to put out job descriptions for new devs to do the job! In the face of so much stupidity I just left, what else can be done? It was no lack of communication, the conservative leadership still had the defunct sw in mind despite multiple success stories and presentations and plans that we presented. There was a lot of spite and old wounds apparently. Some people will never change even if they would look good if they'd make the right decision. My life is too short to try to move mountains. Well I've tried but I'm not going to bother anymore.
A few days back while met with Alexis C.E.O of Weaveworks, we had a similar conversation, this is how we came up with defining, Platform Engineering = PE is how we simplify computing for every developer in our Organization by creating the environment for them to run their apps those environments are the platform and the job of the platform team is to choose how they are built and where they run and make sure they're always available in an easy way and separating those concerns b/w the automated platform and app devs who have an easy time that is PE, but remember no one size fit all think twice when defining platform in general: czcams.com/video/p6D-NYkVp9E/video.html
The new blue pop-ups are vey distracting, and actually detract from the presentation instead of enhancing it. Really gives me that "death by PowerPoint" feeling. And whatever you do stop pasting them in front of Dave. The whole video seemed a bit rushed, and not as well thought out as most of the content here. Would like to have some of these distinctions fleshed out better, e.g. SRE vs Platform Engineer, what is meant by platform, how to "vendor platforms" come into play (we have lots of those where I work). Microsoft Dynamics 365, PEGA, and Oracle Cloud, are all "platforms" but to varying degrees. Please consider some deeper videos on these topics.
I know you're talking about something important but I'm just laughing about your shirt.
😉
oh lord another buzz-word - why doos the digital domain get so bored with titles that they feel the need to create yet another hat for developers to wear with their other ten hats
This new kid on the block will soon become a disruption being a centraliser that enable self-service capabilities, problem is that too many companies just jumped into this space too early making many mistakes without understanding the life cycle and sustainability of the tools and workflows and you are just too quick to conclude that
Focusing too much on reducing cognitive load inherently encourages programmers to specialize until they only have a single responsibility. A bunch of people trying to dogmatically follow Team Topologies are spawning the human equivalent of distributed monoliths..
The term "platform" came about from failure of DevOps. The idea that everyone on the team understands and do ops (both the dev side and prod side) turned out to be deeply misguided. And the reason was mentioned in the video many times - cognitive load. Nobody is able to simultaneously develop the useful application for the customer and juggle the dev & prod infrastructure (cloud or on prem or both in most cases) at the same time. And most people don't even want to attempt which is pretty understandable. Perhaps not the best analogy but as a pilot I certainly would not want to simultaneously handle building the runway, route air traffic and take care of passengers, their luggage and operate the airports while of course being expected to pilot the plane from A to B. Which is what DevOps was essentially asking of all members of SW teams. Hence Platform Engineering, for better or worse to distribute some of this stuff around. It is not a trend, it is a reaction to a failure of the previous idea (everyone does everything). And yes, there is a danger as mentioned in the video. But that is not danger specific to the "platform" stuff, knowledge silos and disconnect between teams have always been there and likely always will. Not much to do with this new "trend".
Exactly this
Was devops ever about every team member having to know both? I always thought it was about team as a whole handling both…
@@qj0n That right there is the problem though. It wasn't supposed to be about every team member having to know both but then so many roles were "rebranded" to devops engineer and ops teams became devops teams.
We're just now making the same mistakes about PEng we made about DevOps 😬
I see no evidence that DevOps "failed." Nor am I aware that any of the definitions included "the idea that everyone on the team understands and do ops." It was a much needed corrective to broken waterfall processes.
I tend to find it is the idea that teams should have people with overlapping responsibilities and skill sets, to avoid creating "gaps" where problems occur.
"Software Engineering" and all other related disciplines masquerading as "engineering" are in actual fact psuedo-engineering (with the exception perhaps of some embedded and non-web based distributed systems, e.g. avionics and military control systems). "Platform engineering" is just another psuedo engineering term that will end up as another silly discipline, creating more bad standards and extremely messy and inefficient "bolt-on if it fits" technologies, such as god-awful Javascript. Please spare us the MESS!! Sigh.
no it is not uncle. platform engineering is dead. you are too old.