How Should I Implement This Backend?
Vložit
- čas přidán 16. 08. 2023
- Here I need your input on my project! 😃
⚡️Checkout Miro here:
miro.com/yt/backend-creation/
Follow me on instagram: @withmarko
Hey, I made a discord server: / discord
=== Links ===
☕️ My Notion Template:
links.withmarko.com/notion-te...
☕️ Wallpapers:
withmarko.com/wallpapers
☕️ Mac Bluetooth Toggle:
gist.github.com/Markonis/fb15...
☕️ My e-Ink Tablet:
links.withmarko.com/boox
☕️ My phone:
links.withmarko.com/phone
☕️ My Free Window Manager:
links.withmarko.com/one-menu
☕️ My Productivity Mouse:
links.withmarko.com/mx-master-3s
☕️ My Productivity Keyboard:
links.withmarko.com/productiv...
☕️ My Headphones:
links.withmarko.com/headphones
☕️ My Microphone Setup
links.withmarko.com/microphone
links.withmarko.com/audio-int...
links.withmarko.com/cloud-lifter
☕️ My Camera
links.withmarko.com/main-camera
links.withmarko.com/main-lens
=== Business Inquiries ===
Email: business@withmarko.com
Instagram: @withmarko
=== Affiliate Disclaimer ===
I might earn a small commission at no cost to you, if you click on the product links above.
My only advise would be: Don’t try to make these difficult decisions all at once, go with whatever you think is the easiest and replace stuff when necessary.
I can't count how many projects I started exactly like this.
I spent hours trying to learn which language, framework, database, architecture, hosting service etc. Just to end up giving up before even finishing an mvp.
@@elan2199 you may think there could've been a better way but trust me, all options will give you headache at some point, some of the more so than others, that's true...
Yeah, I thought the same thing. It sounds over-engineered for a new project (Unless you are a youtuber wanting to make content) But if you want to seriously build something, just build.
I thought the same thing, yeah. It is like, put to much effort to bring up technologies that may be aren't aligned with the company, for example: Who will even maintain a rust application if it is not used widely in the company? I thought if it is started simpler things could be replaced if needed. Find specialized developers is hard if you thing about business.
As a intern seeing a senior dev opening and talking about the perks of a personal project give so much insight about how to view things in software development.
Amazing content, you ernead a subscriber
I was about to say the same thing!
you cant go from intern to senior dev
@@myonlylovejesus887 maybe there are some steps between but off course you can. Nobody is born a senior in anything my friend
My personal advice: go with the thing you know the most and let you to develop faster to get an MVP as soon(and cheap) as possible. Then if the idea works, you can improve it as the business grows. In my personal opinion, if the project is intended to become a business it is the best to think in how you could test your idea and then the engineering stuff we all loved. I am working on a personal digital business too and I started to think in the same stuff(language, platform, etc) then I realized I wanna to have a customer testable version as soon as possible and I went to node/express and heroku and a well-organized monolith (think I can develop faster) instead of my initial idea of microservices, aws and golang
Underrated comment. One of the best I've read, far.
Exactly build the basic and optimise it later
Ha I literally just wrote that actually that’s hilarious
That's right. Though on my day work we have our super big 11 years old monolith project with a lot of legacy, we still are able to move some parts of it to microservices all the time, update codebase etc. For the start - the main purpose is make testable product for you potential clients. Your project won't likely be that enormous and difficult through first couple of years, so that you would not be able to change anything in architecture. As you hire bigger and more experienced team in the scope you are willing to implement the architecture in, you will be able to scale and navigate product development in any direction you want
Hello Mark, these chill videos are the best (watching at 3:00am)😂You never dissappoint
Hey thank you guys, this puts a smile on my face! 😊
1:46 🇲🇽
Lol ! I'm pretty close to 3:00am . These video just popped up.
3:33 x)
Absolutely love the vibe of your vlogs!
Your videos are so clean. Love it!
I was also considering between Go and Rust for my current startup and I decided to use Go. I absolutely agree about all the reasons that you mentioned and my experience has been great with Go so far. It's extremely easy to start with and it gives you ability to write production ready code on week two. Rust is great when you need some of it's low level functionality but I feel it's an overhead in 90% of all cases, especially if we're talking about backend development.
That being said, Go has a great ecosystem and an amazing standard library that allows you to focus on your product instead of fighting with the compiler itself. I know, the Rust borrow checked is supposed to help in a long run but the question is if you're ready to sacrifice 50% of your development performance for 10% of execution speed/memory.
Hey thanks for sharing your experience. Yeah, I'm leaning towards go myself. Am excited to dive deeper in one of the next episodes! Is your startup live, btw? 😃
@@withmarko i think go is good but things become messy once the project is bigger
i built a project with it before and then things become pain
so i actually switched to node / express and it was an easier time for me tbh
We are using go in our Startup as well and so far we are pretty happy. With the right structure the code is maintainable and easy to understand. We are also using the microservice architecture, because we have a large user base. And that is working for us pretty well.
I decided TS (nestjs)
Yo I was waiting for this, keep it up bro.
having the rain noise in the background while walking through the backend language options was a great touch Marko, loved it!
also, for a backend language recommendation I'm going to go ahead and give you a very unorthodox option: side-server Swift, using a framework like Vapor. It has all the performance that you liked from Rust, but with a language that is vastly superior and more intuitive in just about every sense.
the rain behind during the video voice recording is just 🥰
sooooo relaxing and really good video
Hello Marko!! Just a quick life update, I’ve been following you from the start and you inspired me to get into coding. Tomorrow, I graduate from college with a Computer Science degree! I also just got MacBook (my first one! I wanted it for coding) for my birthday on August 4th! I am starting my personal coding journey now since I am finally done with school and you are a BIG reason why I got involved in this community. So for that, thank you sir! You’re the man! Keep up the awesome work 😊
Congrats!
Congratulations !! I just graduated 3 days ago and got my fist Mac!! Let's go🙌 you got this! We got this!
@@vloginnwithekom8326 yeah we do!!
I am very happy to read this message and I congratulate you
Congratulations! I also hope to graduate from college in January 2024, insha'Allah.
oh my goosh.. i TRULY dont know ANYONE else here in YT with such a good chilly Vlog Vibe.. KEEP GOOING !!!
This video was very helpful to me. Thanks Marco!
I love those personal projects. Its stressfull but also super fun to deal with all these decision on technology, to make sure you comply with gdpr, to do the system design etc. Its so much fun
Your videos are outstanding, Marko!
Cheers from Brazil!
Hey Eduardo, nice to see a familiar face in the comments! 😃 Thank you my friend 🙌
Hello Markο
Your chill video are amazing!
Really like watching!
Mark I just enjoy your coding videos you're an awesome content creator as well as a passionate software developer
Definitely a huge fan of the frequent vlogs!
haha love your videos so fun and chill to watch. Just why you always show your watch lool
Norway! Awesome! Holland was similar but has been getting more relaxed with some stores now staying open later. Most things close at 5, some now 6. There’s a “sales-evening” once a week, Thursday in big cities and Friday in smaller cities, where shops stay open until 9. Grocery stores close anywhere between 6 and 10. And some other shops also stay open until 10. I don’t understand some of the logic here, but I guess it works.
amazing stuff mark :)
Awesome chill video as usual
Nice video as always Marko :)! Hope you can get around the hosting problem with your app
I like your videos, the sound, audio, shots, editing, everything is best, 😊
For deployment, you could start with docker-compose which is great for local development as well as self-hosting. Then later can use the Kompose tool to quickly start using K8s for larger prod deployments.
K8s is very complicated lol. I’d suggest he uses Cloud Run because the containers scale up and down based on traffic.
Kubernitis is a disease that is slowly strangling our industry. Just last week we had a terrible experience in production - i would not recommend this to anyone i like.
@@tobyzieglerrr Yes, it is like a nightmare. I always encourage people to avoid using this garbage.
Dude your videos are epic so is the city .
This video is so well made,how was i not subscribed
Hi Marko! Thanks for your videos, they are really nice and relaxing. About the macros in Rust, you never need to use if doesn't feel comfortable with it. It's just a feature, nice to have, for the once wanted to use. Like in C, you had macros too, not as powerful as Rust (but that's why being developed Rust), and write code for decades without using them. Regards and keep going!
How beautiful is Norway. I hope some day I'll visit the country.
I would love more Notion based setup videos. It's crazy good!
hey @Marko whatever place you are at, seems to be pretty cool to live and work. Also i like the way you detailed out the thing in this video.
very nice the cafe, awesome!
Hi Marko, nice video as always
Hey Marko! About the infra part of the product I think that the easiest way here is to go serverless. In AWS you have to the define the subnet where your lambda will be executed, and this way you can restrict to a specific availability zone to avoid GDPR issues and still be very scalable.
The DNS can be solved with Route53 regional rules that depending from where the address of your service is requested it will be solved to a different API gateway (either in Europe or outside Europe).
Regarding HTTPS you can even use ACM to sign the certificate and attach it to your API Gateway, and use cloudfront for caching the results.
I would also go here with DynamoDB for scalability and ease of management. Also the price is super low when compared to other DBs and if the product grows a lot you can even make the tables global to decrease your time to info!
Really nice format of comparison
Just as a couple others have said, rust is designed to be memory safe. The unwrap function isn't the intended way to get the value from the result - instead use the match statement to handle what happens that unwrap fails or not.
Also regarding gradlie, have you seen the new kotlin dsl for gradle? Worth having a look at and brings gradle more up to modern standards.
Great video as always marko!
I've been writing go for about three years myself and I might be biased but I think given your options and the cons weighed against the other, it's the obvious option. It's easy to pick up, fadt and has a great community.
bro, i always watch your videos, and i know that i will learn something from you
Thank you for your video.
Hi Marko! I recommend you go with AWS. I am currently using it on a project I work on and it's phenomenal. You can have all of your diagram components as AWS services. In terms of the GDPR, you can have different storage instances per region to keep the data in the same continent as the user. Unfortunately it cannot be self hosted though.
Hey thanks for the input. Yeah AWS is extremely powerful, I also used it in the past, I'll have to look into the regional instances, and what the legal language is around that 😃
Every language has pros and cons, maybe the best thing to do is considering all the libraries which will make your project easier. If your project is Kafka-heavy then go with Java, or Scala. Almost every language supports Kafka as a message broker, with consumers and producers, but if you want to do many stateful operations, the kafka-streams library could save your day, and it's only Java (there are some ports around, but the java library is developed by the same team around Kafka).
Considering the first two pieces of your architecture, the catch-all kafka writer and the ingress, I think that kafka-streams could be a good fit. For example, if you plan to implement rate-limiting based on messages, you could use kafka-streams time aggregations with just a couple lines of code. Moreover, kafka-streams offers you local storage which works like an embedded database on the services, and for persistent backups you could use the Kafka Connect API from a kafka topic into almost any persistence layer.
watching these videos give me hope on coding
Finaly new video🎉
Hey Marko, I really liked your background raining sound(I don't know it was real...yes it was 🤣). Its makes feels very confort , but I not saing that I'll almost sleeping 😁Excelent chapter!!!
This is the perfect video! I was thinking about writing my own in-memory database like Redis, but I got stuck at choosing a language. I eventually went with Golang. I guess it's a safe bet to go with Golang - best in both worlds
Hey thank you so much, I'm glad you like it! 😃 Yeah I am leaning towards golang for sure these days, it's got a lot of great things going for it, and I love how simple and minimal the language is! Do you have a link to share with your project, it would be exciting to check it out! 😃
Did you have contact with Java in new versions and spring boot? I think that they are much more fun to use than before
Wow… that is impressive research!
The negative earnings with the laugh took me out. Such great content to start the day with :)
I enjoyed your run through of each of the languages from your perspective.
How about using C#? Similar syntax and feature set compared to Java but you get nullable referencing in newer versions (to help with catching NREs at compile time).
You may be possible to implement your entire back end using PAAS or managed cloud services (such as Azure with Event Hub or AWS managed Kafka).
Hey that's a great point that C# has seen a bunch of improvements recently. I will have to check it out. As far as using PAAS, I'm considering a self-hostable alternative 😃
@@withmarkoC# also allows for the same kind of query syntax that you pointed out as nice in Rust. Switching from Java to C# is easy peasy.
And C# is faster than Go...
can u explain?
@@Parzival-ht6ey
@@TheFeljoy LINQ yeah
Hey Marko! First of all as always awesome content! Your production quality through the roof 🚀
Second I've got a quick question for you and my fellow coders: how do you stay pumped and interested in coding non-stop?
Lately, I've hit a bit of a motivation slump. Coding isn't grabbing me like it used to, and when the college day comes to an end I'm spending more time playing games and watching shows/animes than on coding. How do you keep your motivation up? And do you maybe have some tips on how you can improve yourself as a programmer in general?
Maybe this would make for an interesting video topic if that's something you are willing to do?
Hey dude, we all go through these periods. Just thinking of a project and trying to implement it is what keeps you motivated all the time. That's what works for me at least. If I stop somewhere, I need to find a solution and it's all I'm thinking of.
@mermigkis I agree with you. Having one or more project(s) to work on definitely keep me motivated.
But relying solely on motivation is tough. You have to be consistent (not only work on something when you have motivation).
Personally, I found having accountability partner(s) helps with consistency. Having someone to discuss your project(s) with, share ideas, and ask for advise fuels the passion for coding.
Hey! Love your videos. As a web dev myself I am faced with issues relating to gdpr a lot. I had discussions and talks at length with lawyers and people who should know what they are doing but unfortunately because there are barely if any legal decisions/cases and the actual legal texts are very confusing in the ways they can be interpreted I would argue it's wrong to say google analytics is flat out "illegal" in the EU.
Almost every big tech company nowadays is based in the US so even using things like Microsoft Office which is perpetually online now, any browser, etc would be "illegal" in the EU because they transmit data to the US. There are a few regulations like the "standard clauses" which handle these cases.
I've been immersed in this topic a lot but unfortunately the tldr of the situation is, that it's just a big gray area with a lot of unkowns as of today.
Unless we have actual cases that have decisions made by judges, there is no real "right" way atm. just a "probably correct, but it might also be wrong"-way...
At my company we have similar requirements to your project. It's self hostable, it's GDPR compliant and privacy focused. What we do is, we have a docker file for each component such that we can host the entire infrastructure on kuberenetes for our SaaS solution but we also provide a "all in one" image for selfhosters if they just want to get something up and running.
Could be an idea for you as well? You could just start out with one image that encapsulates all of your components and once you need to scale you can figure out kuberenets.
Nice video Marko. Please what font are you using for your vscode?
Hi Marko, this recommendation I'm about to give may not be perfect but I believe it is still pretty good none the less (dunno how to spell it right :P). Anyways, since I see from the video that you really like and need good request paths handling and paths defining I highly recommend Nestjs (backend TypeScript framework) which I know "It's TypeScript which means it's slow and yada yada yada" but lemme tell you it's actually quite fast for a TypeScript framework (especially recently since they made a new feature which makes Nest 20x faster than before, still experimental). It is pretty easy to pick up and maintain the codebase. Also, Nest is made for big projects with a big amount of data flow. It is heavily reliant on dependency injection which is a double edged sword but in my experience I had no problems with it at all in Nest. When it comes to hosting honestly it really depends on you situation. If you need a server on demand with a specific amount speed and reliability then serverless is the way go to but don't take me seriously with this last one cause i still need to learn a thing or two about it.
I hope I helped with your decision and good luck with the project :)
struct tags was also one thing i didn't like about go, but you'll get used to it
Great vlog, super authentic ... where did you get the beats used ?
Hello mark love watching the videos I know you have probably answered this before but I really like your bag that I see in your day in the life videos and was wondering what bag it is keep up the good work!
i feel like you should use go but that is just because im bias. keep up these videos marko!
Sounds like a great use case for CQRS when speaking of backups but you already have that part down.
Language wise I would use Java with Quarkus which will make everything really really easy and maybe deploy to OpenShift if that is something you are comfortable with.
Honestly Mark, you should rather use typescript with some sql modules. Kafka is more for global and complex data analytics. I think that you should try scala as an option but try using some data frame in TS (like pandas in python).
We need more vlogs!😀
Have you checked faasd? For the backend I would use Bun + TS or Golang.
Oslo is a great city btw. (dobar izbor) I visited it 2 weeks ago.
I feel your pain with the build system, but I'd still seriously consider Kotlin for your use case. It's the best of JVM.
Points to consider 1) Its a personal project so I think not more than 2-3 people will work on this hence code maintainability will not be a bigger issue hence complexity of the language will not have that much of a impact 2) Since it will be a data analytics software therefore runtime minimization should be a major priority but I think that also depends upon the dataset, So you should have an idea about the size of the data so that can be a tradeoff with the language complexity
Rust's .unwrap may feel clunky, but only because it is an anti pattern, .unwrap is supposed to be used only when you are 100% sure that it will not fail.
Otherwise you should be using pattern matching, which is similar with go in the sense that you check for every possible error
Hey! Love your videos. Where I can get this wallpaper?
I suggest going with Rust and Actix. Regarding the macros, Actix's macro is very clean. One more thing, Rust Result is not meant to always be unwrapped everywhere. It is meant to handled.
for API in Rust, try `axum` rather than `warp` if you want to see "lesser" macro in Rust
hey man, I really appreciate your affords, can you please make a video about frontend as you explain backend stuff?
Give the bun ts/js runtime a try. You can stick to ts while getting nice performance.
marko u are best
Bun is out! It might be the perfect balance for using typescript and getting performance improvements
You can talk about technical stuff and not make it boring. Please tell more about how it will be possible to selfhost, but also provide cloud. Because it seems like good idea for my situation. Thanks for video.
I’d suggest to dockerize your go binaries and deploy as AWS Fargate tasks.
C# also seems to be a good another good choice.
where do you get this background music, suits very well. Can u please share the link?
6:31 Have similar syntax with C# and LINQ with EF CORE and have good performance. Best of two worlds.
Hey I’m still learning about the whole architecture aspect and I was wondering what is the name of that diagram? Like what kind of diagram it is, thanks!
One thing about Kafka: You have to be sure that with Kafka you have to put in a lot of work manually also still in production if you want to scale up. There is other choices like Azure Service Bus that take away all that workload but you have to pay for that then. Worth looking into that though, I think.
depends...If you use Kafka through a project like Spring Cloud Stream (using Java) then it's a breeze to use. The setup is very minimal
@@mario_luis_dev Thanks a lot for that! Sounds interesting. I will definitely have a look at that!
I would not underestimate Scala. It is that type of language "Once you master it, you will never want anything else". BTW Just visiting Oslo and falling in love with the city Norway overall :-)
This project would be great with Elixir as the backend and pheonix live view for your dashboard
😂😂😂😂
how do you like c# for the backend
Hi Marko,i am planning to buy a macbook for study and do you think macbook air m2 or macbook pro 16inch 2019?
No bs I’m literally learning about how to implement node js and blender right now in my bootcamp.
Marko Cooodessss 😂
Great breakdown of the different choices Marko! Question for you - how do you prevent burnout with programming? As someone who has tried to build projects with similar complexity as this outside of my day job, I find myself fizzling out a few months in. What’s your secret?
If you rely on motivation, at some point that will run out. You must set a goal and be disciplined to achieve it.
Do you use the iPad to help with your work, or are you satisfied with the Mac only?
Also regarding Kubernetes, try Rancher. Quite a lot of YAML pain goes away.
Elixir gen servers and supervisors could be a good fit for this
You can run serverless assuring Data Sovereignty in europe or germany leveraging Azure for example.
For example Azure Germany was specifically built to handle the data residency, security, and compliance needs of German businesses. It ensures that data is stored within German borders.
Hi! What kind of watch do you have?
I think Go or C# make the most sense given your background if you want to learn something new.
i have a question , which language to learn for backend development, most of them say is Node js but on internet it shows node js is not very secure, i want to decide between node js, pyhton django or any other ?
How about the php marko and its framework
Marko, I want to thank you for these helpful videos. I needed your advice . I am currently studying data science. I wanted to know your opinion about this field. I learned the basics of Python. Is it possible to study data science after that ? Or is it better to learn data science after learning some programming language ?
as far as my shallow understanding goes, Data Science is about Statistics (gathering data, evaluating / processing data, presenting data), programming is how you apply, all the math, in production.
You will sorta need both, because there is a reason people say, programming is just math, but practically applied.
can you please share your career journey in detail. which language you learn first and where you learn from??
how do you learn all this? I don't understand any of it. all of this is so overwhelming. do people like work in jobs and learn this stuff by working on projects, or are they expected to know all these stuff before applying as a fresher? or they taught in college?
Love u❤
What wallpaper do you use?
please I want to start in the field of back end and what is better in javascript or c# at the moment
c#