Wow! Thank you very much for this video Code to the Moon! Just the two of us at the moment, but my brother and I have huge plans for SurrealDB. We are working on lots of performance improvements, GraphQL integration, and our own key-value storage engine coming in due course. Thank you once again for this video!
_Came for Rust, stayed because the unique abstractions in the design actually managed to make the idea of database structure actually appealing to me..._ 😹
These awesome projects like SurrealDB are a result of developers using a language in which they don't have to care about memory and bang their heads when their application segfaults. Awesome work by SurrealDB devs!!
This looks really interesting. Having worked for many years with traditional SQL, and having migrated a few years back completely to NoSQL databases for speed and scalability reasons, I'm really curious to see how SurrealDB scales. If it combines the scalability of NoSQL databases with the ease and familiarity of SQL then could be a very exciting!
Thanks for posting this! I've surveyed the DB landscape wayyy too much and still, this is my first time seeing this database! VERY impressed and intrigued. Also, great job on the overview! Really concise and thorough introduction.
I'm sure this will become very popular quickly. It seems to be very well thought out and a great fit for a ton of use cases, it's written in Rust which is always a plus and it has super consistent and slick looking branding and all that. The logo looks so cool, I'm tempted to use surreal just so I have a reason for putting that sticker on my laptop :-P
I agree Flying Squirrel! To your point - In addition to being feature packed, it has incredible branding which I feel like matters more than most technical people would like to admit
The devs fpr this project should feel proud. Its not easy to create a whole new database architecture, let alone one that leverages the benefits of it predecessors. This will catch on
Those multi-cluster stuffs that are in "future" on surrealdb's roadmap are even more ambitious. If they can deliver on those and survive Jepsen, I'll tip off every hat to them!
Nice video and nice product! For the curious, this db uses TIKV under the hood which uses RockdDB, an LSM tree based database. Cassandra uses the same technology.
@@SurrealDB Do you think it’s more effective to build your own storage engine right now or have a pluggable architecture? I ask because there’s several features that I think would be great if users had a choice and they seem complex to implement in that way out of the box, so if users can choose the right storage engine for their use-case, it would probably be better. Some of the feature choices I’d like to see are LSM vs. b-trees, enabling/disabling record versioning for time-series data, enabling/disabling zero-copy for websockets (I really want something to dethrone Kafka that will integrate natively with a primary data store (Sorry, Redis. Keep dreaming)), and compression algorithm choices. Some of those aren’t even really in current storage engines, so I’d be really interested to see how they can be implemented. I’ll certainly be keeping an eye on this project. Wish you the best of luck!
@@CharlesChacon We always intend to support lots of pluggable storage engines to give developers the choice. Our own key-value store will enable us to offer additional functionality down the line that we can't currently do with the current architectures out there.
There’s a JVM based multi-storage mode database called OrientDb. They have the same concepts/similar concepts. Distributed fault tolerant. Document store, relational, and graph, schema is optional, integration with kafka, and more. Uses SQL variant for query. The documentation is a bit lacking though.
@@codetothemoon I haven't done any performance testing. The backend storage engine is very robust and a lot of integration with other systems like Kafka, MQTT, etc. It's a pretty "featureful" database.
The content of your channel is real: great video + audio quality + thumbnails + transitions + rhythm and so much more. Really inspiring! Thanks for that super fast and amazing tutorial! 🙏🏻
I'm starting to see more Obsidian in rust dev videos... I'm inclined to think you guys have super meaty brains... 🧠 side note let me know if someone is making Obsidian in Rust... so I can meme about it.
@@leeroyjenkins0 Obsidian MD is not open source, so I don't know what it's written in and note-taking being called a fad is funny can't tell if you're trolling or legit.
Funny story about why I used Obsidian for this - there isn't any readily available SurrealDB SQL syntax highlighting, so I had to hack Obsidian to use the same custom JavaScript syntax highlighting that the SurrealDB website uses 👍
I like the idea behind this. I'm just not sure if it'll fit any of my projects. I have things split into three distinct layers SvelteKit (Cloud Edge Deploy) -> FastAPI (Local Python Microservices) -> PostgreSQL (Local DB). Mostly I prefer the DB to be as plain as possible which is just storing the data and the Local Microservice layer handles Auth, Data validation, some caching, etc, etc. This means what I'd really be looking for if I was switching to a new GraphDB is a Rust or Python equivalent of FastAPI to connect to SurrealDB. It'll be interesting to watch this project grow and see what other utilities pop up from this.
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
It's probably not a good fit for every use case, especially when one db paradigm can handle all of your access patterns and you don't need the built-in auth and REST APIs that Surreal has. But yeah I agree it'll be cool to see how the project develops!
I'd really like a nice way to represent rust style enums in my database. I've never seen a database do that before. I would use Surreal for everything if I could do that
Absolutely revolutionary. Got a question about the performance of the edge queries - while most were in the 100 microsecond range, the edge queries were in the 500 microsecond range. Does that give anyone pause? I have no idea how this stacks up with other graph database tech so it could be a nitpick.
Great question - I hadn't delved too deeply into performance benchmarks. In the docs they openly state that the performance currently doesn't yet quite stack up to the more established databases, so I'm wondering if we should wait for a more stable release before reading too deeply into them. Definitely something to keep tabs on though, and it's not unreasonable to be cautious about types of queries that seem to be slow in the beta.
I've been working with standard MySQL for several years now and just started implementing PostgreSQL for my own projects outside of work. This may have just disrupted everything I thought I was going to be doing. Much more research to do now.
Good idea, I've added it to the video ideas list! There are definitely some gaps currently in the documentation. But the SurrealDB team is really responsive on their Discord server, you might try getting help there!
@@codetothemoon yes, especially for small projects it's like having a firebase in a box, at least for me having something like this, if the security layers and event emission works properly, completely removes the need for a server between the client and the DB. And the fact that you can embed js functions into fields makes it even possible to have some business logic too (with limitations I guess) Definitively not the way I would use for a well structured long-term project, but for everything else I don't know... on paper it kinda fits perfectly
This is really cool and all, but when evaluating new things to work with, "version control" is the first thing that comes to mind... if this can be easily integrated into Git-like repos, it'd be a gamechanger for me. Also... localization... and transaction support across tables.
Agree, version control is huge, and I'm not sure if Surreal plans to support it. At one company we had implemented our own homegrown version control inside Postgres, and the number of bugs and general pain it caused over many years was hard to fully quantify. It'd be nice to have something like that built in to the database.
Summary: Surreal DB is a new database that aims to take the best aspects of each major database Paradigm and combine them all into one database. It is schema-less by default, has a sql-like query language, and supports record links to establish relationships between records in different tables. It also has a graphql-like query language for fetching data from multiple tables at once, and supports websockets for real-time updates.
The current scalability story seems to be pretty good. They've stated in their docs that performance isn't up to par with the more established databases yet (I believe they specifically use Postgres as an example). I'm not sure how much security auditing they've done, but I'd imagine they'd wait to do that until a more stable release (as opposed to beta).
Thanks so much! I'm actually just beginning to get my feet wet in the backend development world, and I have to say it's a very exciting time to be doing so. I've got just enough experience with a couple other relational and graph databases to see the massive advantage of some of these features, but I've yet to really dive into Rust. I'm getting the impression that its time is coming soon, but according to the forum murmurings that's an impression that Rust has been pretty good at creating for quite some time without a lot of real progress. However, looking at the hard facts with this framework, it seems like they might finally have a real game changer on their hands. I hope it works as smoothly as it appears to.
Thanks for watching! Fwiw you don't need to have any Rust knowledge to use SurrealDB! I agree it does seem like a game changer - the current version does have some of the quirks that you'd expect in any beta software, but overall it seems to work really well in its current state!
@@codetothemoon I managed to get an instance served up locally and it worked great, but I'm wondering if it's possible to create a cloud-hosted instance through something like Google Cloud Functions or if that is something I will have to wait for until they release their cloud hosting features?
I'm always suspicious of db tech that is "all of the things". Why is this a good idea? How will it help me ship my saas app faster? Is it going to be around for 50 more years and how do you know?
I think caution is healthy when approaching ambitious projects like this. Multi-paradigm is a great idea because you don't have to spend time deciding which paradigm is the best fit for your app because you have all of them at your disposal. Also you don' t have to use multiple DBs when multiple paradigms are required to support your access patterns. Of course, multi-paradigm is only part of the SurrealDB story.
I have a soft spot for databases and SurrealDB sounds very interesting. I must say that I have often problems with joins, so not needing it sound great. I am not proficient with rustlang yet, so I wonder if I cound use some other language with SurrealDB?
I love it, except for one thing: is there a simplified schema definition syntax? I support defining strong schemas, but the code for doing it looked really verbose, too wordy vs normal SQL
yeah afaik this is currently the only way to do it, at least the only way that exists in the documentation. The team seems really receptive to feedback, maybe hop on their discord and shoot them a message!
It’s pretty cool for a multi-model approach. Right now I’ve switched to graphs exclusively because moving away from SQL and tables has been easier for me. I haven’t seen the need to keep them.
@@jaimemorganhitchcock6195 I understand the desire to combine different DB models. A better metaphor for how I see this is sticking with a hybrid vehicle instead of going full electric because it’s assumed that having both gas and electric together is more convenient. Convenient sure but is it a better design? I don’t believe so. Currently the database I mainly use is Dgraph. I’ll see how Surreal compares but I haven’t found anything that’s close that simplifies my architecture as much as Dgraph.
Dgraph is definitely a cool database @Khari99. With regards to multi-model, we wanted to ensure that SurrealDB was easy-to-use coming from all different database backgrounds. In addition, the design is such that the performance isn’t affected regardless of what data model you use in SurrealDB - obviously there are no JOINS, so SurrealDB is definitely closer to MongoDB/Dgraph as a concept. We have some really great things planned on our roadmap, so hopefully we’ll sway your mind in the not-too-distant future 😊!
@@jaimemorganhitchcock6195 While I like Dgraph a lot, I don't think it's perfect. Outside of the GraphQL API, I feel like there are a lot of ideas that you can pick up on to make SurrealDB better. 1. Automatically generated CRUD queries from a schema definition. I've only seen this implemented with GraphQL so I don't know if this is possible for your architecture. But this significantly reduces how much code needs to be written and maintained in the server. 2. Implement lambda functionality. Dgraph has a philosophy of moving the DB to the middle layer and allowing queries to trigger lambdas that either run custom computations or can reach out to a server externally for some request if needed. I really like this approach as it helps further reduce architectural complexity. But right now I'm limited to JavaScript lambdas. Rust lambdas would be amazing. 3. Auth & Access Control patterns. I looked through the documentation for Surreal and haven't seen anything around this topic. I understand the philosophy of leaving this up to server logic but if these features are baked into the database, it's one less thing for me to worry about. Even though I moved away from Fauna, they have this integrated into their DB, which is why I tried them out in the first place. 4. Logical Inference Rules. I've only seen these implemented in TypeDB. one of the things that Dgraph is missing that almost makes me want to develop a DB in Rust myself is the ability to create a knowledge graph intuitively. If you could look into how TypeDB accomplishes this and think of a similar implementation, this would be awesome. 5. Indexes. I also didn't see this in the documentation. Dgraph has indexes that enable full-text search (along with many other index types) out of the box. I would rather not have to implement elasitcsearch separately.
I'd like to know more about row-based permissions and end-user authentication features. They are both mentioned on their website as features of the DB, but I can't find any documentation about these features.
There are definitely some gaps in the documentation. But the SurrealDB team is really responsive and helpful on their Discord server, maybe try getting help there!
@@codetothemoon that’s a good idea! I’ll definitely check it out. I really hope they transition to a documentation first approach, good documentation is fundamental for stuff like frameworks and databases. Thank you for the tip though!
Hi I! We’ve already been building production applications on it for a few years now. However we are looking to release the stable Version 1.0.0 really soon!
My understanding is that it aims to support all of the complex types of queries you'd expect to be able to perform on a traditional relational database. JOINS specifically are eliminated as you are expected to use record links, graph edges and FETCH instead, but you can achieve basically the same thing. I believe subqueries are currently supported (but maybe ask on the SurrealDB discord server to be sure). Not sure to what level CTE-like patterns are currently supported or what is planned there, again maybe check with Tiobe and Jaime.
very nice modern all-in-one remote and local db. very promesing. hope the new investors do not take away to many features to a payed enterprise version.
thank you! actually currently working on a new video that will feature SurrealDB (but won't focus specifically on it). I'd love to do another full video including all of the new stuff since this one, we'll see if it bubbles to the top of the video idea list 😎
This is very interesting to me. The features are indeed ambitious but quite compelling. Would require a lot of testing before using it in a production product I suspect but I want to try it in my normal research and experiments code. As usual, well explained and informative.
I think SurrealDB is really interesting too, but yeah I wouldn't use it in production quite yet. I'm definitely looking forward to watching how the project develops!
Personally I think I'm waiting tor the interface to be implemented. Currently using PocketBase and it's great to use - but I'm always keeping an eye out for alternative to broaden my horizons. SurrealDB looks great - but the lack of a UI right now is keeping me away still. Hopefully soon!
Wanted to try it out, followed the docker steps, wanted to access the web UI and got a 403 error. I don't understand why that web UI is even hosted externally.
Looks like a great product. Thanks. How to we do a full text search on a field? Is it efficient (unlike oracle search where % wildcard or any text partial search will kill the efficiency while search?)
The docs mention support for full-text indexing, but I'm not sure exactly the steps required to enable that for a field (maybe just create a specific type of index on it?). The docs do appear to have some holes in them at the moment, but the team is extremely responsive and helpful on their discord server, maybe drop in and ask them!
great tutorial. at 10:36 maybe we should add player= $after.id field so we can fetch the player who levelled up. just an idea that I learned from previous 10 minutes. thank you very much.
I've just logged an issue because the keyword "schemafull" is incorrect, the suffix should be "ful" with one "L". I feel a bit like a pedantic ass, but it seems like a jarring oversight to have a misspelled keyword.
@@maninalift No, the GitHub log is great! SCHEMAFULL is a concatenation of 'schema' and 'full'. I agree with you that as one word it should probably be more inline with what you suggested, but we wanted to make sure it made sense to those who already know about 'schema-less' or 'schema-full' databases.
@@jaimemorganhitchcock6195 I think this refers more so to schema migrations/versioning. Aside from the under-the-hood capabilities, I think this would be one of the top feature requests
I'm curious about the what kind of backup solution they have. Over the years many databases and come and gone, all promising to be easy turn key solution to all problems. But when you start to look at them more in detail, as you do when you use them for real project, you always find that the old databases have these "complexities" for reasons not apparent initially.
I think this healthy skepticism is a great way to approach new databases like this, and one that I had going in as well. During the time I immersed myself in it, I tried to uncover any gaps between its promises and what it delivers, and I couldn't find any. That's not to say there aren't any, just not any that I could find. The biggest issue I ran into was missing / incomplete documentation, which I'm confident will be addressed. YMMV
you can teach us crud and aggregations with math like add or sum because data is raw you can process it in query so the data can be presented to clients
Jaime I think he is referring to automatic schema migrations like what FlywayDB does in the traditional SQL world. Sean - correct me if I'm wrong about this 😎
@@nobytes2 "potentially" because it's not automatically faster, it depends on what you write, but for the same code in c and in rust you will either have rust and C roughly equal (same llvm backend) or rust faster because the compiler has more information to prove more things about the program and use more agressive optimisation. If you want data you can look for it yourself you have Google too.
What tooling has been made thus far because typing all that out like you've sucks and is error prone. Any intellisense like in a db editor? Any integration with a rust client or typescript?
Hi Sean. We are working on our Rust and TypeScript libraries as we speak. These should be available very soon! We realise the tooling is our weak point at the moment, but we have some big releases coming soon!
You mention Events being used for real time updates to users via websockets, is that use case one with an example in the SurrealDB docs? Is it something we can build out in Rust? Is it better to directly interface with SurrealDB's API?
Hi Sean. As mentioned on my other comment, we are working on the Rust client to connect Rust with a SurrealDB server. However you can already use the database library itself embedded in Rust. Not sure if that answered your question or not, but let me know!
At the beginning there is a standard SQL on the left and some cryptyc text on the right, which is claimed to be equivalent to SQL on the left. I would expect some explanation about what is written on the right and what it does. I tried to understand that almost half time of the video and still not sure I understand.
Yeah the features are incredible. I know back when it was first released there were some concerns about performance. Not sure if there are still such concerns.
@@codetothemoon yes you really should. The most interesting thing I find at EdgeDB is that queries and schema are much simpler and shorter than with a normal orm or raw sql. That is partly because you don’t need stuff like pivot tables to represent many to many or one to many relationships. It is 100% OO.
Thanks - unfortunately I don't have the queries posted anywhere, but you can check out README.md on the SurrealDB GitHub repository for some good examples!
Wow! Thank you very much for this video Code to the Moon! Just the two of us at the moment, but my brother and I have huge plans for SurrealDB. We are working on lots of performance improvements, GraphQL integration, and our own key-value storage engine coming in due course. Thank you once again for this video!
Hey just wanted to say thank you for this amazing DB keep up the good work.
@@alandto3996 Thank you very much for the kind words. We certainly will!
Thank you very much! ❤️
Looks like a brilliant project. Would love to see key-values and websocket auth pubsub soon! 💛
@@fire17102 WebSocket auth already here Tami. Live queries and pub-sub coming very soon!
_Came for Rust, stayed because the unique abstractions in the design actually managed to make the idea of database structure actually appealing to me..._ 😹
Is it unique? Looks like OrientDB, which has been around for a decade.
Thanks Wolveric, glad it piqued your interest!
OrientDB is written in Java - I'm sure it's fast, but there's kind of an upper bound to how fast it can be...
Me too
A directional edge connection is the first time I've seen an arrow operator feel appropriate
❤
In F# you use this operator
It's similar to the syntax of Cypher, the query language of Neo4j
Arrows for marking function output types and for dereferencing a pointer to a compound object both seem appropriate to me. Just my 0.002¢
I agree! Really the perfect operator for working with graphs
This looks extremely interesting. I suspect this will grow very quickly.
Thank you very much @theherk! We have a lot planned on the roadmap!
Agree!
DEFINE, CREATE, RELATE, SELECT ... all six letter commands. I *love* it.
Wow, I hadn't noticed that!
keen eye
Ok, this is insane. I was dreaming of a DB that would do graph connections in trust to move away from neo4j , but this is amazing. An absolute killer.
I agree! It definitely has massive potential.
at 9:15 you could of simplified the statement to be *SELECT armor:dragon
Thanks Dabbo will give this a try!
These awesome projects like SurrealDB are a result of developers using a language in which they don't have to care about memory and bang their heads when their application segfaults. Awesome work by SurrealDB devs!!
Time to start playing with the new boy in the block. Thanks for sharing.
nice, thanks for watching!
This looks really interesting. Having worked for many years with traditional SQL, and having migrated a few years back completely to NoSQL databases for speed and scalability reasons, I'm really curious to see how SurrealDB scales. If it combines the scalability of NoSQL databases with the ease and familiarity of SQL then could be a very exciting!
Agree, it seems like the team is aiming to do just that!
Thanks for posting this! I've surveyed the DB landscape wayyy too much and still, this is my first time seeing this database! VERY impressed and intrigued. Also, great job on the overview! Really concise and thorough introduction.
Thanks for the kind words Mike, glad you found it valuable!!
Holy crap. If this pans out to what it’s teaching for this could single handedly solve a lot of the database headaches.
I agree! Choosing a database paradigm is really challenging, the prospect of not having to make that choice upfront will be so liberating...
You guys are great and SurrealDB genuinely feels interesting. Can't wait to play with it.
Thank you very much for the kind words Onizuka ❤️
It's worth giving a try for sure!
I'm sure this will become very popular quickly. It seems to be very well thought out and a great fit for a ton of use cases, it's written in Rust which is always a plus and it has super consistent and slick looking branding and all that. The logo looks so cool, I'm tempted to use surreal just so I have a reason for putting that sticker on my laptop :-P
I agree Flying Squirrel! To your point - In addition to being feature packed, it has incredible branding which I feel like matters more than most technical people would like to admit
The devs fpr this project should feel proud. Its not easy to create a whole new database architecture, let alone one that leverages the benefits of it predecessors. This will catch on
I agree that what they've done is really impressive! Seems like they have a ton of momentum already, hopefully it continues
Those multi-cluster stuffs that are in "future" on surrealdb's roadmap are even more ambitious. If they can deliver on those and survive Jepsen, I'll tip off every hat to them!
100% agree, can't wait to see how things pan out. I actually hadn't heard of Jepsen, I'm glad you put this on my radar!
Nice video and nice product!
For the curious, this db uses TIKV under the hood which uses RockdDB, an LSM tree based database. Cassandra uses the same technology.
Good to know, thanks
Thank you Ismaj. We are working on our own key-value store which will bring some additional benefits to SurrealDB in due course.
@@SurrealDB Do you think it’s more effective to build your own storage engine right now or have a pluggable architecture?
I ask because there’s several features that I think would be great if users had a choice and they seem complex to implement in that way out of the box, so if users can choose the right storage engine for their use-case, it would probably be better.
Some of the feature choices I’d like to see are LSM vs. b-trees, enabling/disabling record versioning for time-series data, enabling/disabling zero-copy for websockets (I really want something to dethrone Kafka that will integrate natively with a primary data store (Sorry, Redis. Keep dreaming)), and compression algorithm choices. Some of those aren’t even really in current storage engines, so I’d be really interested to see how they can be implemented.
I’ll certainly be keeping an eye on this project. Wish you the best of luck!
@@CharlesChacon We always intend to support lots of pluggable storage engines to give developers the choice. Our own key-value store will enable us to offer additional functionality down the line that we can't currently do with the current architectures out there.
Thanks, and thanks for the info - I wasn't aware of TiKV's relationship with RockDB!
There’s a JVM based multi-storage mode database called OrientDb. They have the same concepts/similar concepts. Distributed fault tolerant. Document store, relational, and graph, schema is optional, integration with kafka, and more. Uses SQL variant for query. The documentation is a bit lacking though.
Nice, this actually wasn't on my radar. How does it stack up performance-wise?
@@codetothemoon I haven't done any performance testing. The backend storage engine is very robust and a lot of integration with other systems like Kafka, MQTT, etc. It's a pretty "featureful" database.
The content of your channel is real: great video + audio quality + thumbnails + transitions + rhythm and so much more.
Really inspiring!
Thanks for that super fast and amazing tutorial! 🙏🏻
I'm starting to see more Obsidian in rust dev videos... I'm inclined to think you guys have super meaty brains... 🧠
side note let me know if someone is making Obsidian in Rust... so I can meme about it.
@@leeroyjenkins0 Obsidian MD is not open source, so I don't know what it's written in and note-taking being called a fad is funny can't tell if you're trolling or legit.
Funny story about why I used Obsidian for this - there isn't any readily available SurrealDB SQL syntax highlighting, so I had to hack Obsidian to use the same custom JavaScript syntax highlighting that the SurrealDB website uses 👍
I couldn’t believe you got that event to work just like that 👏
Thanks, I practiced a bit before recording the video 🙃
I am brand new to DAW and soft soft - these tutorials are excellent an very helpful to get soone like up and running. Appreciate
Glad you found it valuable, thanks for watching!
Looks like I found a new go-to databse for prototype projects (and non-prototype ones once it matures a bit)
same here!
What a cool demo! It really showcases SurrealDB's simplicity and usability. Thanks!
Thanks and thanks for watching!
If they manage to do the websocket part, it will immediately become much more useful.
Agree, it will be a game changer!
I really like what I'm seeing, I just hope it won't end up like rethink db
Thank you very much indeed @gorudonu ! We have lots planned for SurrealDB!
Agree it seems promising!
tbh I can't wait what sorts of projects the community will come up with now that we have rust
agree, there are so many interesting Rust-powered projects popping up, even CLI tools to replace ones that have been around for decades...
This looks very promising!
Thank you very much @Elf!
I agree!
I love it. Gotta test this
It's worth giving a try!
It's pretty SURREAL that a database engine like this exists... Too bad I work as a front end dev and never have to deal with databases
Perhaps this tech could have some side-project potential for you?
It's ok, given that there seems to be a new JS framework every day, there's bound to be a SurrealJS at some point... 🙃
You can start creating your side projects with Nodejs just for fun :D and connecting with SurrealDB
I like the idea behind this. I'm just not sure if it'll fit any of my projects. I have things split into three distinct layers SvelteKit (Cloud Edge Deploy) -> FastAPI (Local Python Microservices) -> PostgreSQL (Local DB). Mostly I prefer the DB to be as plain as possible which is just storing the data and the Local Microservice layer handles Auth, Data validation, some caching, etc, etc. This means what I'd really be looking for if I was switching to a new GraphDB is a Rust or Python equivalent of FastAPI to connect to SurrealDB. It'll be interesting to watch this project grow and see what other utilities pop up from this.
Hey Matt! I understand your tech stack position. We have some awesome things planned for SurrealDB so hopefully we can sway your mind in due course 😀
@@jaimemorganhitchcock6195 If you get full-text search in there, I’ll bite!! 😄👍🏽
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
It's probably not a good fit for every use case, especially when one db paradigm can handle all of your access patterns and you don't need the built-in auth and REST APIs that Surreal has. But yeah I agree it'll be cool to see how the project develops!
I'd really like a nice way to represent rust style enums in my database. I've never seen a database do that before. I would use Surreal for everything if I could do that
That would be cool!
Absolutely revolutionary. Got a question about the performance of the edge queries - while most were in the 100 microsecond range, the edge queries were in the 500 microsecond range. Does that give anyone pause? I have no idea how this stacks up with other graph database tech so it could be a nitpick.
Great question - I hadn't delved too deeply into performance benchmarks. In the docs they openly state that the performance currently doesn't yet quite stack up to the more established databases, so I'm wondering if we should wait for a more stable release before reading too deeply into them. Definitely something to keep tabs on though, and it's not unreasonable to be cautious about types of queries that seem to be slow in the beta.
As soon as LIVE queries are implemented, I am switching for sure :)
they do sound pretty handy!
I'll definitely be trying this out on my personal app later!
Nice, let us know how it goes!
I've been working with standard MySQL for several years now and just started implementing PostgreSQL for my own projects outside of work. This may have just disrupted everything I thought I was going to be doing. Much more research to do now.
Nice! Saw you found your way onto the SurrealDB discord server, seems like a great community is growing there. Let us know how the journey goes!
Super cool. I just need to figure out how to use this at work.
Hi Nathan. Thank you very much. Join the Discord or GitHub if you have any questions - surrealdb.com/community
Convincing others to buy in is the hardest part! Even for game changing technologies.
It’s totally worth learning new skills cause then you get excited and confident to create
It's true! 🔥
After web framework galore we now have database galore
For reals!! This is getting ridiculous!! 😂
Hah yeah, I'm skeptical of all new projects - JS, Database, or otherwise - but this one really seemed to stand out from the crowd.
Wow that looks bonkers good!
Thank you very much Ratstail91 ❤️🙌
I think "bonkers good" is a fitting description!
I'm ready to marry this db
💍
😂
hah me too!
This is super cool, would love to check it out
Thank you Zaha! Join the Discord or GitHub if you have any questions - surrealdb.com/community
Yeah I think it's worth a look!
I'd love to see a video showing how to use it with Rust. The documentation doesn't seem to exist yet.
Good idea, I've added it to the video ideas list! There are definitely some gaps currently in the documentation. But the SurrealDB team is really responsive on their Discord server, you might try getting help there!
This is the most interesting thing I have seen in a while in the tech world
Same, it seems pretty disruptive if the vision fully materializes!
@@codetothemoon yes, especially for small projects it's like having a firebase in a box, at least for me having something like this, if the security layers and event emission works properly, completely removes the need for a server between the client and the DB. And the fact that you can embed js functions into fields makes it even possible to have some business logic too (with limitations I guess)
Definitively not the way I would use for a well structured long-term project, but for everything else I don't know... on paper it kinda fits perfectly
This is really cool and all, but when evaluating new things to work with, "version control" is the first thing that comes to mind... if this can be easily integrated into Git-like repos, it'd be a gamechanger for me. Also... localization... and transaction support across tables.
Agree, version control is huge, and I'm not sure if Surreal plans to support it. At one company we had implemented our own homegrown version control inside Postgres, and the number of bugs and general pain it caused over many years was hard to fully quantify. It'd be nice to have something like that built in to the database.
Summary:
Surreal DB is a new database that aims to take the best aspects of each major database Paradigm and combine them all into one database. It is schema-less by default, has a sql-like query language, and supports record links to establish relationships between records in different tables. It also has a graphql-like query language for fetching data from multiple tables at once, and supports websockets for real-time updates.
Sounds about right to me!
How scalable is this? What read/write performance does it have comparing to postgres? Did they pay for a security audit?
The current scalability story seems to be pretty good. They've stated in their docs that performance isn't up to par with the more established databases yet (I believe they specifically use Postgres as an example). I'm not sure how much security auditing they've done, but I'd imagine they'd wait to do that until a more stable release (as opposed to beta).
Give me a typesafe TS client and I’m sold!
This is coming very soon!
Sounds like it's in the works!
Thanks so much! I'm actually just beginning to get my feet wet in the backend development world, and I have to say it's a very exciting time to be doing so. I've got just enough experience with a couple other relational and graph databases to see the massive advantage of some of these features, but I've yet to really dive into Rust. I'm getting the impression that its time is coming soon, but according to the forum murmurings that's an impression that Rust has been pretty good at creating for quite some time without a lot of real progress. However, looking at the hard facts with this framework, it seems like they might finally have a real game changer on their hands. I hope it works as smoothly as it appears to.
Thanks for watching! Fwiw you don't need to have any Rust knowledge to use SurrealDB! I agree it does seem like a game changer - the current version does have some of the quirks that you'd expect in any beta software, but overall it seems to work really well in its current state!
@@codetothemoon I managed to get an instance served up locally and it worked great, but I'm wondering if it's possible to create a cloud-hosted instance through something like Google Cloud Functions or if that is something I will have to wait for until they release their cloud hosting features?
I'm always suspicious of db tech that is "all of the things".
Why is this a good idea? How will it help me ship my saas app faster? Is it going to be around for 50 more years and how do you know?
I think caution is healthy when approaching ambitious projects like this. Multi-paradigm is a great idea because you don't have to spend time deciding which paradigm is the best fit for your app because you have all of them at your disposal. Also you don' t have to use multiple DBs when multiple paradigms are required to support your access patterns.
Of course, multi-paradigm is only part of the SurrealDB story.
This is cool and interesting. I might try it out, but I’ll be keeping work with single databases for their specified purpose.
Nice, yeah definitely something to explore but probably not ready for production yet. Also, I see you are a fellow Babylon 5 fan!
I have a soft spot for databases and SurrealDB sounds very interesting. I must say that I have often problems with joins, so not needing it sound great. I am not proficient with rustlang yet, so I wonder if I cound use some other language with SurrealDB?
Kimmo - absolutely, I believe they have client libraries in several other mainstream languages. No Rust knowledge necessary.
I love it, except for one thing: is there a simplified schema definition syntax? I support defining strong schemas, but the code for doing it looked really verbose, too wordy vs normal SQL
yeah afaik this is currently the only way to do it, at least the only way that exists in the documentation. The team seems really receptive to feedback, maybe hop on their discord and shoot them a message!
Could we say that this is SURREAL
❤❤
Indeed!
It’s pretty cool for a multi-model approach. Right now I’ve switched to graphs exclusively because moving away from SQL and tables has been easier for me. I haven’t seen the need to keep them.
Graph is definitely the future @Khari99! That is exactly why we wanted to merge Graph with traditional database concepts.
@@jaimemorganhitchcock6195 I understand the desire to combine different DB models. A better metaphor for how I see this is sticking with a hybrid vehicle instead of going full electric because it’s assumed that having both gas and electric together is more convenient. Convenient sure but is it a better design? I don’t believe so. Currently the database I mainly use is Dgraph. I’ll see how Surreal compares but I haven’t found anything that’s close that simplifies my architecture as much as Dgraph.
Dgraph is definitely a cool database @Khari99. With regards to multi-model, we wanted to ensure that SurrealDB was easy-to-use coming from all different database backgrounds. In addition, the design is such that the performance isn’t affected regardless of what data model you use in SurrealDB - obviously there are no JOINS, so SurrealDB is definitely closer to MongoDB/Dgraph as a concept. We have some really great things planned on our roadmap, so hopefully we’ll sway your mind in the not-too-distant future 😊!
@@jaimemorganhitchcock6195 While I like Dgraph a lot, I don't think it's perfect. Outside of the GraphQL API, I feel like there are a lot of ideas that you can pick up on to make SurrealDB better.
1. Automatically generated CRUD queries from a schema definition. I've only seen this implemented with GraphQL so I don't know if this is possible for your architecture. But this significantly reduces how much code needs to be written and maintained in the server.
2. Implement lambda functionality. Dgraph has a philosophy of moving the DB to the middle layer and allowing queries to trigger lambdas that either run custom computations or can reach out to a server externally for some request if needed. I really like this approach as it helps further reduce architectural complexity. But right now I'm limited to JavaScript lambdas. Rust lambdas would be amazing.
3. Auth & Access Control patterns. I looked through the documentation for Surreal and haven't seen anything around this topic. I understand the philosophy of leaving this up to server logic but if these features are baked into the database, it's one less thing for me to worry about. Even though I moved away from Fauna, they have this integrated into their DB, which is why I tried them out in the first place.
4. Logical Inference Rules. I've only seen these implemented in TypeDB. one of the things that Dgraph is missing that almost makes me want to develop a DB in Rust myself is the ability to create a knowledge graph intuitively. If you could look into how TypeDB accomplishes this and think of a similar implementation, this would be awesome.
5. Indexes. I also didn't see this in the documentation. Dgraph has indexes that enable full-text search (along with many other index types) out of the box. I would rather not have to implement elasitcsearch separately.
This is really cool, I haven't heard of many folks switching from pure relational to pure graph!
The event thing was crazy, amazing
agree!
I'd like to know more about row-based permissions and end-user authentication features. They are both mentioned on their website as features of the DB, but I can't find any documentation about these features.
There are definitely some gaps in the documentation. But the SurrealDB team is really responsive and helpful on their Discord server, maybe try getting help there!
@@codetothemoon that’s a good idea! I’ll definitely check it out. I really hope they transition to a documentation first approach, good documentation is fundamental for stuff like frameworks and databases. Thank you for the tip though!
Really strong stuff you serving today!
Authors really taked their code to the moon 😄
lol thanks! 🚀
Reminds me a bit of datomic but in rust...
How long until it’s production ready?
Hi I! We’ve already been building production applications on it for a few years now. However we are looking to release the stable Version 1.0.0 really soon!
hadn't heard of datomic, thanks for putting it on my radar!
Good to know, looks very interesting, I will still use the rock solid postgres but this kind of project can have they use cases
hard to go wrong with Postgres!
Looks amazing!
As a data analyst/engineer the one thing I really want to know is: Does this support complex joins, subqueries & cte?
My understanding is that it aims to support all of the complex types of queries you'd expect to be able to perform on a traditional relational database. JOINS specifically are eliminated as you are expected to use record links, graph edges and FETCH instead, but you can achieve basically the same thing. I believe subqueries are currently supported (but maybe ask on the SurrealDB discord server to be sure). Not sure to what level CTE-like patterns are currently supported or what is planned there, again maybe check with Tiobe and Jaime.
This looks pretty interesting. I’ll have to play with it some
it's worth checking out!
SurrealDB seems very interesting. If I ever am granted the luxury of time I'll gladly experiment a bit.
Nice, it's worth checking out!
Thank you for this video,
You rock!
Thanks for watching Scott!
First vid I watched *exploring* SurrealDB.
Looks like something I gotta know, being a big PostgreSQL fan.
Thanks :)
Thanks for watching Tobias!
This is such an informative video. Thanks for creating. I am going to try myself.
Thanks Singh! Fantastic that you are going to start creating! Please let me know if there's anything I can do to help.
@@codetothemoon Thanks for replying. Will definitely do 😊.
This feels surreal. The web UI seems not ready yet but the project is interesting.
Surreal indeed! Yeah my understanding is that the UI will be available relatively soon.
Fantastic video, loved it. I will be playing with it this afternoon.
Thanks Thomas, let us know how it goes!
very nice modern all-in-one remote and local db. very promesing. hope the new investors do not take away to many features to a payed enterprise version.
Yeah I'm curious as well to see what the cloud version looks like!
Can we get update on it since whole year passed. Maybe to cover updates and upgrades? Very nice video.
thank you! actually currently working on a new video that will feature SurrealDB (but won't focus specifically on it). I'd love to do another full video including all of the new stuff since this one, we'll see if it bubbles to the top of the video idea list 😎
That's super cool; thanks for bringing this!
Thank you very much David
Thank you David!
Thanks for watching!
looking forward to learning from the rest of your videos. Subscribed!
Thanks Justine, very happy to have you onboard!
RetroSun thanks to other comnts i gotchu; click on gms when you’re in the channel rack to open up the gms nu and in the light blue
?
This is very interesting to me. The features are indeed ambitious but quite compelling. Would require a lot of testing before using it in a production product I suspect but I want to try it in my normal research and experiments code. As usual, well explained and informative.
I think SurrealDB is really interesting too, but yeah I wouldn't use it in production quite yet. I'm definitely looking forward to watching how the project develops!
looks amazing!
Glad it piqued your interest as well!
Mylosp just a legend really aren’t ya
I try to be! 🦄
Personally I think I'm waiting tor the interface to be implemented. Currently using PocketBase and it's great to use - but I'm always keeping an eye out for alternative to broaden my horizons. SurrealDB looks great - but the lack of a UI right now is keeping me away still. Hopefully soon!
I haven't yet tried any of the "*bases", keep hearing good things about them. I agree the UI will be a game changer - the screenshots look really nice
@5:47 It works for me with SELECT math::sum(strength) FROM player GROUP ALL; (I'm using version 1.2) - so use GROUP ALL, rather than GROUP BY ALL
Wanted to try it out, followed the docker steps, wanted to access the web UI and got a 403 error. I don't understand why that web UI is even hosted externally.
Hi Salzian. Our web UI isn’t released just yet. When we launch it it will be embedded into the binary itself.
Looks like you got an official response, just wanted to say the WebUI is going to be a game changer when it is released...
This is absolutely the level up!
I agree, the SurrealDB team is moving mountains!
Looks like a great product. Thanks. How to we do a full text search on a field? Is it efficient (unlike oracle search where % wildcard or any text partial search will kill the efficiency while search?)
The docs mention support for full-text indexing, but I'm not sure exactly the steps required to enable that for a field (maybe just create a specific type of index on it?). The docs do appear to have some holes in them at the moment, but the team is extremely responsive and helpful on their discord server, maybe drop in and ask them!
@@codetothemoon Thank you!!
i generally hate sql and most sql-like languages.
This is really awesome though.
Thank you very much Mo!
Glad it piqued your interest as well!
great tutorial. at 10:36 maybe we should add player= $after.id field so we can fetch the player who levelled up. just an idea that I learned from previous 10 minutes. thank you very much.
I've just logged an issue because the keyword "schemafull" is incorrect, the suffix should be "ful" with one "L". I feel a bit like a pedantic ass, but it seems like a jarring oversight to have a misspelled keyword.
Thank you for pointing this out @The Laughing Philosopher. Did you also message on Discord?
@@SurrealDB no. Should i do so?
@@maninalift No, the GitHub log is great! SCHEMAFULL is a concatenation of 'schema' and 'full'. I agree with you that as one word it should probably be more inline with what you suggested, but we wanted to make sure it made sense to those who already know about 'schema-less' or 'schema-full' databases.
@@jaimemorganhitchcock6195
The pattern holds for words that use the "less" suffix.
eg
Tuneless / tuneful
Faithless / faithful
Tasteless / tasteful
Joyless / joyful
Doubtless / doubtful
Harmless / harmful
Shameless / shameful
Schemaless / ...
@@maninalift We have just pushed the change for the use of SCHEMAFUL with a sinlge 'L'. This will be in the next release!
Interesting new DB, love Rust too.
I agree!
Very helpful, and surprisingly therapeutic
Nice, glad you liked it! "Therapeutic" is something I'm very happy to have achieved!
Does it have some kind of migration tool to help migrations versioning?
Not just yet, but we are working on migration and sync connectors from Postgres, MySQL, MongoDB and RethinkDB.
@@jaimemorganhitchcock6195 I think this refers more so to schema migrations/versioning. Aside from the under-the-hood capabilities, I think this would be one of the top feature requests
Good question, for many people using Surreal will surely be dependent on this!
I'm curious about the what kind of backup solution they have. Over the years many databases and come and gone, all promising to be easy turn key solution to all problems. But when you start to look at them more in detail, as you do when you use them for real project, you always find that the old databases have these "complexities" for reasons not apparent initially.
I think this healthy skepticism is a great way to approach new databases like this, and one that I had going in as well. During the time I immersed myself in it, I tried to uncover any gaps between its promises and what it delivers, and I couldn't find any. That's not to say there aren't any, just not any that I could find. The biggest issue I ran into was missing / incomplete documentation, which I'm confident will be addressed. YMMV
it's still full of bugs
you can teach us crud and aggregations with math like add or sum
because data is raw you can process it in query so the data can be presented to clients
I would love to do another SurrealDB video that goes more in depth!
When doing an UPDATE what happens? Are migrations with an up/down still advised? Any gotchyas or considerations about what are 'safe' migrations?
Any defining a schemafull table locks us in!? Oh yikes.
@@seannewell397 Hi Sean. Please could you clarify what you mean by 'migrations with an up/down'.
Jaime I think he is referring to automatic schema migrations like what FlywayDB does in the traditional SQL world. Sean - correct me if I'm wrong about this 😎
Looks incredible but would love to see a perf / scaling follow up to really get a better feel of how SurrealDB stands up in real life scenarios
I'd love to do such a follow up, but I might wait until it goes 1.0 to do so!
Most production dbs are written in c, and tested throughout decades. It will be extremely difficult to beat performance but you never know.
@@nobytes2 well-written rust is potentially faster than C due to the rust compiler having more information to make agressive optimisations
@@eliottveyrier6253 show data or else stop making guesses, "potentially" lmao 😂
@@nobytes2 "potentially" because it's not automatically faster, it depends on what you write, but for the same code in c and in rust you will either have rust and C roughly equal (same llvm backend) or rust faster because the compiler has more information to prove more things about the program and use more agressive optimisation. If you want data you can look for it yourself you have Google too.
What tooling has been made thus far because typing all that out like you've sucks and is error prone. Any intellisense like in a db editor? Any integration with a rust client or typescript?
Hi Sean. We are working on our Rust and TypeScript libraries as we speak. These should be available very soon! We realise the tooling is our weak point at the moment, but we have some big releases coming soon!
Looks like you've got an answer directly from Jaime!
You mention Events being used for real time updates to users via websockets, is that use case one with an example in the SurrealDB docs? Is it something we can build out in Rust? Is it better to directly interface with SurrealDB's API?
Hi Sean. As mentioned on my other comment, we are working on the Rust client to connect Rust with a SurrealDB server. However you can already use the database library itself embedded in Rust. Not sure if that answered your question or not, but let me know!
Might be making a Rust integration tut for SurrealDB at some point...
At the beginning there is a standard SQL on the left and some cryptyc text on the right, which is claimed to be equivalent to SQL on the left. I would expect some explanation about what is written on the right and what it does. I tried to understand that almost half time of the video and still not sure I understand.
onto the software. I'd like to make soft from my ukulele, keyboard, etc., but don't know how to transfer. Do I need a microphone or
SurrealDB thankfully doesn't require a ukelele, keyboard or microphone. Unless you're using it to store your band's music
Keep up the good work!
thank you!
i'm in love, but i would like very much to know the benchmark and performance of it
Yeah the features are incredible. I know back when it was first released there were some concerns about performance. Not sure if there are still such concerns.
Edgedb is also very cool!
Thanks for putting it on my radar, maybe I will check it out!
@@codetothemoon yes you really should. The most interesting thing I find at EdgeDB is that queries and schema are much simpler and shorter than with a normal orm or raw sql. That is partly because you don’t need stuff like pivot tables to represent many to many or one to many relationships. It is 100% OO.
Great video, thanks for sharing. SQL novice here, would you mind sharing the SQL/SurrealDB queries? Would love to look them over to better understand.
Thanks - unfortunately I don't have the queries posted anywhere, but you can check out README.md on the SurrealDB GitHub repository for some good examples!
this is amazing!! thanks
thanks, yeah I think SurrealDB is definitely worth keeping an eye on!