Video nenà dostupné.
OmlouvĂĄme se.
Not My Best Week As A Staff Software Engineer
VloĆŸit
- Äas pĆidĂĄn 5. 11. 2022
- This was not my best week as a software engineer. Essentially I wrote some bad code which resulted in needing to issue a hotfix, that would make for a bad day but to top it off my hotfix resulted in yet another hotfix. This was one of the worst weeks of my career and in this video I'll walk you through what happened, what I learned, and how you can avoid making the same mistakes.
đ¶ââïž FOLLOW ME đ¶ââïž
Join Our Discord - / discord
Twitter - / codyengeltweets
TikTok - / codyengeltalks
Medium - / membership
Subscribe To My Newsletter - www.engel.dev/
đĄGet 20% Off Of Brilliant - brilliant.sjv....
đ„ My CZcams Gear - kit.co/CodyEng...
đ„ My Desk Gear - kit.co/CodyEng...
đ” My Background Audio - www.epidemicso...
*The above links are affiliate links.
#softwareengineer #lesson #computerscience
Really appreciate you opening up about your experience with a prod breaking bug. Not alot of developers would be able to self-reflect, hold themselves accountable, and learn how to implement solutions to mitigates mistakes in the future.
One thing I've learned is that mistakes will happen and it is alright. Mistakes is the only way we as developers can probably grow. I know when I was just starting out in my career, I had the complete opposite mentality. I didn't think mistakes was an opportunity to learn and wanted to be the perfect developer. That mentality almost got me fired and put on a PIP.
Thanks again for sharing this Cody and keep up the good work man!
You're welcome and thanks for watching T-mo âșïž The pursuit of perfection is a surprisingly good way to get fired, thanks for sharing your own personal story about that. Mistakes are perfectly fine and happen all the time, the most important thing is to learn from them and ideally not repeat them again.
@@CodyEngelCodes Couldn't have said it better myself! Keep up the good work and content Cody! Love seeing you videos appear in my recommendations!
This is a fantastic video! I've found myself debugging using the logging/monitoring tactic as well! I want to be able to identify possibility for downstream and cascading issues right away by the naked eye. I try to keep up with historical errors and repetition in errors through some way of logging. I think it's really important to be able to recognize and error you've seen before and be familiar with the potential cause and solution. I encountered a few situations where I asked another engineer about an error and they said "I've seen this before but I don't remember the fix". It's never good when errors, solutions, and documentation live in the engineer's head only.
Exceptions always disliked them for this reason! You get an unpredictable program follow. When I went to C++ one of the argument was: âwe have exceptions!â I never used them, always did an immediate check and either return -x for errors and 0 for okay. In Java projects Iâve worked on it was alway a hell!
I like rust where you get Result object back with either an Ok or Err that you can unpack. Itâs the cleanest most predictable way imo.
And logs are golden! Totally agree, you want to known what was the last thing your program did before it was caught in an unhandlable exception.
Itâs the only question on interview that o ask that o find valid. âWhat was your biggest mistake, and how did you deal with it?â
People who say they have never made a blunder are either lying or donât do Jack shit. And I donât want either.
At this customer I have the running gag when people mess: âwell done! It took you longer than me. Because last year I was preparing for a database merge script and it ran nicely but I needed to do a few more fixes so I removed the tables and I wondered: âwhy is my disk not running hot as I migrated from dev on to my laptop it should be⊠oh fuck me!â Yeah I had removed 3 tables from the actual dev environment. It was the second week in. I tried to restore but no rights, and I just owned up. A lot of laughs and we restored it.
Recently (as in 2 days ago) that colleague that laughed (out of sympathy) made that new environment for which is merged database inconsistent, by having missed a crucial input validation logic đShit happens and as a team you come together tease a little and solve it.
Mistakes will always happen and thereâs no reason being angry about it because people realize as to what to do to prevent them next time. I for example decided not to use sql manager studio because the connect connection names were so similar and I clicked the wrong connection of the two selected the âmigrated NOTâ tables and delete the wrong ones - speed kills đ
I made 3 big mistakes in my career ones every decadeâŠđ
I think you skipped over the first mistake, which you mentioned earlier:
The change took weeks to develop.
Small changes over gigantic ones, if possible.
In almost 30 years in the business Iâve seen some monumental fuck ups. But never anybody being fired over them. The biggest and funniest was a startup that I worked for who procured a energy provider (in my town) with 80k clients. We had to literally risk the whole business putting that organization as is to our backend processing - basically two systems with interoperability so they could move clients slowly into the native billing and trading engine but they would already trade on our platform and be billed. When we are done suddenly the authorities come to visit with detectivesâŠ
Turned out that the two directors of that energy provider were suspect of tax envision, trading with prior knowledge and trade fraud. And since we were now basically owning their company they came for all the bookkeeping etc because the two had fled the country đđđ
And indeed they found evidence to indict them on all 3 charges and an international APB is still out for them đ
But that meant that this acquisition was unfair so the customer base was to be distributed in equal share to all energy producers/providers. So we ended up with 200/300 users extra for what we paid 4 million and spend another 500k rebuilding a system that hardly was needed. đ
Iâm glad I didnât have to pay that.
But then again both startups that this CEO and CTO ran, backed by one of the Netherlands biggest billionaire actually made a very nice profit when sold.
Bravo for using your bad experience as a "teaching moment"! Hard to admit when things like this happen. Definitely a take away is that some of the tests you are doing need to cover the race conditions (or whatever it was) that was cropping up, but, more generally, add that sort of situation to your checklist of types of tests that need to be created when you are working on creating tests. Simulating race conditions can be tricky, depending on the language and runtime system you are using, but if you have something like NodeJS where testing frameworks like Sinon which interposes on function and method invocation and lets you specifically schedule executions to simulate difference race conditions, it is pretty easy once you get the hang of it.
Thanks for your comment and support! You're absolutely right - learning from our mistakes and turning them into "teaching moments" can provide valuable insights for ourselves and others. We should indeed consider the importance of testing for race conditions and other concurrent scenarios while building our test suite.
As you mentioned, simulating race conditions can be challenging depending on the language and runtime system being used. However, utilizing tools like Sinon for NodeJS can make this process more manageable. We appreciate your suggestion and will definitely consider incorporating these testing strategies in our future projects. It's always great to share knowledge and help one another grow! Keep those insights coming! đ
Was an intern at this smaller company. Ran an update SQL query without a where clause on a production table on a service I had recently made⊠was not fun. Rebuilt the tables manually, and since then always check my update queries
Oof, that's never fun, happy to hear you were able to resolve the issue though, messing up SQL data in production is no joke đ
Simply love this video.
Glad you enjoyed it!
Love all your videos
Mistakes are part of our job, the best way to deal with it is learn from that experience, just as you are doing. I am wondering if you have a qa team on that project?
Nope, but between working on projects with QA and without QA I've found the same number of bugs are caught and end up in production.
i can't find the link to the recommended video, can you post the link here or @ the description
Then please, how do you create logs? can you make a video on it?
Secondly, what are hot fixes? thank u!
Hmmm should have been at the end of the video but here's the link: czcams.com/video/uJiDK2uZL3c/video.html
A video on logging is a great idea, I'll add it to the list. And hot fixes are essentially unplanned releases that go out to resolve some kind of bug.
@@CodyEngelCodes thank you so much cody!
You're welcome Stanley!
I'm promoting LDD - Log Driven Development đ
Yes!!
sup with meeting with HR?
Itâs an app. No one died. Itâll be okay.
How can you be so sure?
@@CodyEngelCodes lmao
Click bait!
What happened with HR?
Nothing happened with HR. More of an illustrations that you can have a really bad week without a 4:30 meeting with HR on Friday đ
We were trying to reduce latency for a legacy api and found an external service REST call to be the bottleneck. Based on the parameters of request, we decided to cache it. And next day, we started receiving escalations for customers being shown wrong information! The response of the API we cached had user information coming although the parameters didn't imply that. Our mistake- we ignored the attributes being passed in headers. đ„Č
That's a big oof but also makes total sense. Software is complex and when common sense practices aren't followed (like making it obvious what information is returned by an API) then it's easy to make mistakes like that.