"Building Observability for 99% Developers" by Jean Yang (Strange Loop 2022)
Vložit
- čas přidán 5. 10. 2022
- The rise of SaaS and APIs means that it's easier than ever before to build applications quickly. A small, otherwise under-resourced engineering team can quickly deploy more functionality than they were ever able to before--but what about making sure the system keeps running, or deploying changes with confidence? As it turns out, we'll also want easier visibility, easier monitoring, and easier observability. This is especially true for teams that don't necessarily have large, expert DevOps teams.
In this talk, I'll discuss the visibility and observability needs of “99% developer” teams--and how we can be applying techniques traditionally applied to compilers and program analysis towards building "one-click" observability for these teams.
Jean Yang
Founder and CEO of Akita Software.
@jeanqasaur
Jean Yang is the founder and CEO of Akita Software, a company working to bring visibility and control to modern web applications at the API layer. Jean has a background in programming tools and was previously an Assistant Professor of Computer Science at Carnegie Mellon University. She has a PhD from MIT and a bachelor's degree from Harvard University. The MIT Tech Review named her one of the Top 35 Innovators Under 35 in 2016.
------- Sponsored by: -------
Stream is the # 1 Chat API for custom messaging apps. Activate your free 30-day trial to explore Stream Chat. gstrm.io/tsl - Věda a technologie
Marvelous presentation.
Great talk and specifically the Q&A section
Great talk. Many systems already use a reverse proxy like NGINX. Any thoughts on integrating at that level? Maybe that's what you mean when you answered about TLS
With a proxy, packets are copied from kernel memory into user memory (modified by NGINX) and back into kernel memory before they can be sent to their final destination. A passive eBPF module observes packets from inside the kernel with no additional copies or latency.
A proxy is a second address that all your mail get sent to. There, a guy opens it up, reads it, seals it back up, posts it, and you wait again for it to be delivered to you. A passive eBPF module is a guy with x-ray vision sitting in the mailman's van
@11:30 We have that anti-pattern all over industry. But it *is* an anti-pattern. When the only record of requirements is the emergent behavior of the existing code, this is unprofessional. Industry hires handymen when they need craftspeople; designers when they need engineers.
If you want to make tools that anyone is actually going to use, you gotta meet people where they are. A screwdriver shouldn't break if someone tries to use it for hammering, prying, or scraping. 99% of users will at some point use a flathead for something other than screws, and if it can't survive that they're not going to buy it again.
So cool
Types don't exist across APIs. But they do. They're called SDKs. If you use something like GraphQL or Protobuff, typesafe SDKs can be generated for both the server and the client, that's how I made sure the frontend and backend in my company are integrated properly without dumb gluing errors even though they speak through HTTP.
But that's when you are gluing the APIs that you have control over... you've missed the whole point of the talk. The whole outside world does not, and will not, standardize to a single type-safe spec for their APIs.
I agree but you should be assuming the api you’re writing against has incorrectly implemented the contract and you should be able to observe that.
I tried to use this approach against a PagerDuty integration and found that the state of OpenAPI code generators are not on par with OpenAPI spec descriptions. Especially in the case where user defined inputs have extensibility baked into them by the API, which is becoming more common as time goes on.
There are still only bytes going over the connection. You assume that both sides are interpreting those bytes the same way. But that only works if both sides use the same version of the same schema. Which is easy in a frontend/backend situation but gets harder if server and consumer are more decoupled.
> in my company
Very cool! You have different problems than this solution seeks to address! Your solution is probably very effective in your case! However it's not useful to the bottom 51% of developers that have never used code generation! That's the majority!
Why not have logging implemented in the system?
TL;DR: Developers don't know their own system.
Watch @17:02 for some of the reasons why.
There's too many systems, not enough time, not enough control, and not knowing what needs to be logged
Logging is slow and adds complexity. Imagine trying to log something that's a property of a nullable thing. Now your log, which isn't you primary function of your system can introduce a run time error.
These cross cutting concerns should be implemented orthogonal with the system.
“I don’t know how many of you get to program Haskell for smart contracts, and how many here are living in the real world” at 1:58. The thing is : this talk is much about the complementary relation between execution and observation, which is also what smart contracts in Haskell are about.
99% of developers aren't writing smart contracts & they aren't starting to. I believe that's the statement that was made, nothing less, nothing more.
E somehow got stuck on a surface level similarity that's also deep beneath the surface. Must be a Haskell programmer