As a much younger software engineer, I still remember this phrase I was taught regarding relational database development. The key, the whole key, and nothing but the key. So help me Codd.
I once was interviewed for a senior development position with a major multinational bank with their database guru. He was the one man who knew how the bank managed all of its data. He revealed that they had just one database. It had 600 tables. There were no joins, no relationships, no structure; It was just a massive dumping ground. Trades, transactions, accounts, … everything. The goal was to take this unholy mess and redo something better, while keeping all the existing business units running without interruption. It was an Oracle database.
I recall working with such a DB dumping ground as a new hire at one of my previous jobs. Source data from a global company in the banking and payments space. It came with hundreds of mostly useless stored procedures and functions that were supposedly written around 2015, but contained zero proper joins in the code. My new colleagues were waiting for me to make snarky comments about it!
As a pre-SQL programmer, SQL was a game changer. Using SQL gave me a logical, abstract view of the data structure, significantly reduced my design and coding time, and allowed changes to the database structure without having to break code or migrate data. I used several systems on PCs like DBase and Firefox which had SQL database at their core. Also need to remember that systems back then were so constrained by CPU performance, memory storage, and disk capacity that SQL would have been too large to run on some of the early computers. Great video as always!
@@zzbeasley Agreed. I first met dBase on an Osborne luggable with 5 1/4" floppy drives and no hard disk! Later used dbase on PC and I liked Superbase under DR DOS/Gem. I think Superbase used dbase for data storage with Superbase giving a GUI front end.
I'd argue that SQL is one of the most successful inventions of computer science, on the level of C or even more so. Nothing better has come up despite decades of attempts, and pretty much all complex datastores eventually support SQL as they grow.
Humankind is generating vast amounts of data. Making sense of it all is beyond what a relational database can do well. The next step is "big data" non-relational databases. There are no keys, there is no structure to the data. The goal is to find the relations, not order them in that way. It is where we are headed. A relational database is great for what we use it for and will be used that way for a long time to come, but big data will look information a new way for a new purpose.
"Humankind is generating vast amounts of data. Making sense of it all is beyond what a relational database can do well"... did you ask ChatGPT to write this bunch of vague and disconnected catch phrases just to oppose the OP ? What would be the practical purpose of "making sense" of all data we are producing? Make a real life Matrix ?
@@chpsilva A comment section is not the place to have a deep discussion about big data. I don't see that I opposed the OP, I presented what the next generation of databases are about and where the research is going. I have spent my professional life programming SQL relational databases, I do it for a living. If you are interested in the big data concepts I am talking about do a google search. Big data is useful and yes, despite your reservations, it is something that matters.
Database researcher and university professor here: Very good presentation as usual, though it of course ends with the state of things in the mid 1980s and a lot has happened since, not just in academia but also in industry. As early relational DMBS go, IBM's System R (which has been alluded to in the video but not named and whose DNA lives on to this day in IBM DB2) has had significantly higher impact and relevance than Ingres. Since you mentioned Boyce and Stonebraker, a number of others would have deserved mention at least as much, such as Jim Gray or Pat Selinger. Also, you added to my pain as an academic trying to attract young researchers into the field by making databases look real bland and boring (though important to business). Unfortunately, so many undergrad courses do the same. There is beautiful systems and theory research to be done, and there are interesting fundamental questions that arise here that are much cooler than anything mentioned in this video (though ultimately enabled by Codd).
For sure, a database is way more than an Excell spreadsheet. With column based storage, data compression, and parallel processing. We get things like Google, a keyword database of the entire internet. And Twitter a database that adds records at a rate of 100's of gigabytes a second, all instantly searchable.
Not the original poster but curious; a bachelor's degree in mathematics with no theoretical exposure to databases, and 6 months experience programming with SQL.
I had the privilege of digitising some Betamax tapes from the Australian Computer Society many years ago from the 10th Australian Computer Conference held in 1983. Chris Date, a relational database expert, gave a presentation on one of these tapes and made many claims about what the future would be for databases. "Like it or not, SQL is going to become a very important language. It might become an actual standard, and it almost certainly will become a de facto standard." The full speech is here czcams.com/video/VnNbddUMZQI/video.html
@@MePeterNicholls I used one, a Key West Big Voodoo. It doesn't handle dropouts in the vertical blanking very well. The tapes were very degraded unfortunately.
@@MePeterNicholls The biggest problem were dropouts. A digital timebase corrector used with a domestic video recorder is not going to help with that. Sure they can on studio machines like Umatic, Betacam, MII etc., but domestic machines lack the required RF output signal. However it might be that the particular player could have usefully had the DOC (dropout compensator) sensitivity adjusted, I recently showed that with a Beta machine on CZcams.
As a newish developer, of all videos, this was the most exciting to see when I saw the thumbnail. SQL has become one the foundational blocks of software development and even today, when all the competing NoSQL paradigms (MongoDB, Firebase, Redis, etc) have claimed a space in database management, we are coming back to SQL with new ideas thanks to PostgreSQL and SQLite related projects like Turso and libSQL. And I really like the image in 14:50 while distant in time is so relatable to my knowledge. A testament on how SQL stood the test of time and won. Really thanks Jon for this video.
The problem is that most NoSQL databases (Mongo is fully ACID but records aren't usually normalizaed each document has all data so this doesn't mean much and so multi document transaction were only introduced in Mongo 4 mere 4 years ago) aren't fully ACID (Atomicity, Consistensy, Integrity and Durability). For things like banking transactions are a must: the money that gets out of one account must get in on another or the transaction must rollback. Unstructured or partially unstructured data are a case for NoSQL, many NoSQL databases are very simple and good in replication, but at a cost of using GUIDs as the primary key (as a surrogate key) this is not always desirable. When you get billions of records that need integrity and complex queries (Mongo performance with complex queries is abysmal) that must be done fast SQL is a must (good part of my life I passed tuning badly designed and maintained databases, and if I got a Dollar for each time I heard this can't be done faster or this can't be done in SQL I would have double what I earned).
@@agranero6 I agree with you, but my point really was that we are ending the era when NoSQL was trendy and "you have to use it to be cool" like many other bad ideas mostly in webdev (microservices for all, complex SPAs for everything, rewriting everything in the latest language, etc). Nowadays everybody is rediscovering that SQL in its many forms is a solid language with solid ideas. Even SQLite is becoming popular! 😜 For me, I'm in both sides: For small projects I'm happy with tools like Cloud Firestore, but for more complex projects nothing that good ol' SQL.
Do professors tell you that physical structure of the RDBMS is as important as logical structure? As a contractor DBA I make much more money from physical structure issues than from logical ones.
@@kondybasAgreed to a point. Usually, the logical design isn’t that bad. My big problem is middleware… stuck threads, garbage collection, multi-threaded architecture and lack of commits.
I've been using SQL since September 1999. Database programmer has never been my formal title, but I have been one for 25 years this fall, in addition to my main programming job that's normally identified by the other programming language. It's been SQL the whole time, but the other language has gone from C to Perl to PHP to Java over the decades. What makes SQL different from other programming languages is that it's used by far more than just programmers. I've seen programmers write horrible SQL and non-programmers write terrific SQL. If you can wrap your mind around set calculus you'll become very good at SQL, and you don't need to be a programmer at all. Either way, practice helps. I was only what I would consider good at SQL after 15 years and I'm now fairly proficient at SQL such that I can jump right into any database get going.
_Lots_ of people know where and how SQL got started. They just had to graduate college before the rise of Java. Heck, I've still got a Relational Databases textbook in a box somewhere.
I have to profess my love of this channel and your content. I don't know how I was lured into this channel at first, I'm not Asian, and I'm just a mere MSP technician who grew up in the 80s and 90s. But you always seem to hit the right note, and Sunday nights haven't been the same since. You're doing it right, sir. 🤩 I eat these videos up, and wanted to say "thanks".
its only called Asianometry because there's a focus on Chinese, Taiwanese, Japanese, Indian technology sector! the channel is not actually about asian culture lol
@@Old_Jack_Ketchi agree. It is probably because the topics in the video's are not ancient history and still have relevance to this day. But even if you started studying or working in ICT you might have never heard of many contextual details in these vids but when you hear or see them you can often instantly relate them to personal experiences (at least i can) and that's entertaining. Greetings from ASML-city!
@@thinkingcitizen I get that. I meant in terms of the initial draw for me, what made me pick this content, based solely on the name of the channel - It speaks to how genuinely interesting the selection of topics are. 😉
Really enjoyed that, but then I do have a 35 year SQL career behind me. Furthermore, almost all of it was new to me despite my extended immersion. I have to be honest, when I hear Codd's name, I usually switch off: it's usually in relation to Codd's Rules, academic rules of thumb that in practice are nothing more than common sense and second nature in this line of work. I watched a video on them that I happened to stumble across a few months ago, and yes, it is all common sense codified into a set of rules. So I congratulate you in managing to do a 20 minute video on SQL without ever referring to Codd's Rules!
for our management information systems program in university we barely do the SQL behind these academic concepts and it's really dumb, I am currently building a .NET interface for a access database just so I can practically learn this stuff.
Information exists outside of youtube, ya know. Pretending youtube is some super-special, locked down knowledge store that will never go away and is vital to maintaining a record of history... come on now. 😏
@@mercster Asianometry helps spread this information to a wider audience without them actively searching for it. Almost every person watching this video had it recommended to them via the CZcams home page or through another recommendation method on the site. There are other sources for this information, but this video is a good way of pushing out that knowledge to a large number of people who would have not previously considered or cared about the topics discussed on the channel. Best regards.
@@broadestsmiler I'm not taking anything at all away from Asionometry, whose videos I very much enjoy. What I'm getting at is, when you lavishly heap praise in an overly-enthusiastic, hyperbolic manner that doesn't really accurately reflect reality, you diminish the genuine value.
@@mercsterNot everybody has the time to delve into historical documents. I see this channel as a low-treshhold gateway into the early history of the computer industry. Not overly theoretical yet very informative.
I work in a social work/SUD treatment agency, and we somewhat recently began working with a company that helped us get backend data out of the cloud database. I haven't worked with SQL since college... The last few months have been a blast, and believe you me, I've been able to flex comp sci skills in a very not comp sci industry haha.
I like how You dig out filthy details: "Deliverability, Redundancy and Consistency of Relations stored in Large Data Banks" with the exact date on it +++
Hello Jon, In my opinion, this is one of your best video. As a side hustle to my job, I've been writing in SQL for more than a decade. I'd read about how it come to be, but the story you tell is at another level. Thank you Anthony
Gotta say I wish my dad was still around to appreciate this channel. As a historian and library scientist with most of his career transpiring before the internet explosion, he didn't really get to witness much refinement of video content. The variety and scope of the stories shared through Asianometry are just plain great... (Although I'm sure he would've griped about lack of formal reference summaries to back up details.)
I knew Charlie Bachman and can confirm he was a stand up guy. Kind, patient, a good teacher and a good boss. I worked for him for a few years when he was promoting his entity-relationship model which was more abstract than the Hierarchical (IMS), CODASYL (IDMS) and Relational (DB2) models and was useful for porting and reverse engineering.
I remember learning SQL in the early 2000's and dutifully memorizing that it was the unification of relational algebra and relational calculus. The fact that almost nobody would even bother to learn that today attests to how useful and intuitive SQL really is. The key was, I think, that they modeled it after regular English and the implicit set-theoretic operations in natural language - "select birthday from people where ...""
Great video as always! I would love future videos on more modern SQL since the 1980s -> e.g. the breakout of OLAP vs. OLTP databases, SQL in the cloud, Presto, etc.
Phenomenal video. The inoculation to being overwhelmed by a technology is to follow the simple ideas and history that built its complexity. You have a remarkable talent for leading audiences along those journeys. I hope that you'll take us on similar tours of these omnipresent, yet terrifying, tools that power our world. In the meantime, avoid dying of a brain aneurysm. Because evidently that's something I get to worry about now.
Later on in my life I am finding a lack of great videos that bridge the human gap between the great ideas we need to learn and how and there they came from. It's so easy to give up because everything comes down some arbitrary decisions than are then later generalized and that's what wins out. I am one of those people who needs that back story to calm down the "Why this way??" questions in my head with out mastering something enough to know why. Not much point to this for others. Just engaging the algorithm with something more than "Thanks for another good video. I enjoyed it."
Interesting, informative and well done, on a subject that I worked with for a number of years, but had not idea of 'the rest of the story'. Thank you for all the work and then sharing.
Big thumbs up and thank you , databases often overlooked are the backbone of most system design. One of the first high school jobs was digitizing the music collection the national radio station into a DBase IV system. I then used my database programming superpowers to make a movie rental database back in the days when stores rented VHS tapes using Novell Netware and Btrieve. A lot of money to be made from database design in the 1980/1990s! It was the HTML of the era.
I worked on databases on IBM mainframes. IMS was described as hierarchical, Total was a network database (you had two levels). I then came across early DB2 using Structured Query Language and of cause Codd’s relational database principles. I did a bit if work on Oracle and quite a lot using UDB (the djstributed version of DB2 for middle tier rather than mainframe systems. Interesting to see how other software sellers went about implementing Codd’s rules.
S-Q-L (originally SEQUEL) had to change its name to S-Q-L due to an existing language called SEQUEL and subsequent trademark dispute. I try my best to use S-Q-L on Monday, Wednesday, Friday, SEQUEL other days, and "Squirrel" when I have had too much to drink.
squirrel 😂 - I can laugh about that joke without an IQ of 10000. - I didn’t have to come up with it, I only needed to understand. Edit: on the other hand: it seems a bit unfair, as squirrels (allegedly) sometimes forget where they stored all of their seeds for the winter, thus enabling unintended growth of trees… (- „trees“ ?😂) Edit 2: maybe the 10000 wasn’t in the decimal system, but binary 😂
Writing SQL is an incredibly therapeutic, zen experience. I cannot properly express the joys of having an end goal of what data you want to have rolled up and the fun, challenging path of writing SQL to get you there. I changed fields, in a way, and no longer get to use SQL. I miss working with it all the time!
I interviewed with Oracle and three other Bay Area companies before moving there in 1981. Oracle was my second choice. It was a prototypical startup - but at the time, of course, it looked like a young and untested outfit. What might have been… (I had plenty of time and read E. F. Codd's papers at the not-so-busy consulting firm I signed with instead, and got heavily into databases later.)
My second cat, the last offspring of my first dilute Calico whose litter came while I was learning relational databases, was named SQL. A damned good cat, indeed.
I appreciate the label ‘vicious’ applied to Oracle as that is how they were described by Michael Stonebraker and colleagues, developers of Ingres, Postgres, etc. In the 1990s. Oracle would do whatever it took to get the business. (I worked as a programmer in the CS Division at UC Berkeley in the 1990s)
I've been needing a good overview on databases! You always put out videos addressing things I'm interested in :) Wait a minute... show us the database you use to predict our viewing desires!!!
We usually prefer the term tree instead of network to avoid confusion with a computer network and because network is a wider concept (equivalent to a graph): networks don't have to be in tree form, they can for instance be semi-lattices for instance. Trees are good for seaching as we can traverse with simple algorithms and find a key in logarithmic time, if the tree is well balanced, and we can balance them for efficiency. In general graphs this can't be done. In your video you confuse related tables with trees: the trees in databases are assembled inside the indexes to the keys in the database. The relationship between tables doesn't need to be in the form of a tree and it usually is not.
I worked as a SW developer for 30 years. I started with IMS, IDMS then DB2. DB2 relational DB is the best for design, development and serving the business.
Excellent, as always! The second word in Hawker Siddeley is usually pronounced as two syllables, sid-ley. Source: I was an avid 10-year-old aicraft spotter in London in 1972 when RDMS was getting going, and the HS Trident was my favourite airliner. (Though it started life as the de Havilland Trident.)
i had a professor in the early 80's who was a disciple of Ted Codd. he was always spending class time relating personal anecdotes about Codd instead of the class material. according to my Prof, IBM was not happy with Codd's research since they thought what he was doing was going to conflict with IMS and could cut into the profits IBM was making from it. my Prof respected that Codd was able to persist in spite of the corporate discouragement, and eventually created the System R project, which turned into DB2.
This was great. I hope you will follow up with the saga of the database wars of the late 1980s and early 1990s between Oracle, Ingres, Sybase, Postgres, and MS SQL Server.
Okay this was the first time for me there wasn't a fascinating video hidden behind a boring title. That is just me tho. Still watched it. Keep up the great work ❤
i am about to graduate with my buisness degree in Management Information Systems and im so excited! Im currently staring at a 3nf normalized ERD for a travel company. Its kind of like micromanagement games like openttd, you gotta make sure all your little guys are transporting what you need to each other.
While working on the Allen Brain Atlas I got to work a bit with Jim Gray ... I best remember talking over beer about both technology, sailing, and comparing Seattle to the Bay Area.
You forgot to mention the predecessor from SQL (that I learned), it was dbase IV, and it had already SQL implemented in it. I programmed a lot in dbase III and IV mostly for bookkeeping purposes. At that time, this was very powerful, but I quit programming when I was conscripted for the army. After my 3 years duty, I lost interest, unfortunately. But SQL isn't much different from the other products. And later in my Carrère working in BPCS I used my knowledge to create my own query's and forms to extract data out of the mass database. My boss didn't know much about databases and programming or making forms on that level. Enjoyed my work very much. Making things a lot more efficient. I remember saying to him, everything is there, you just have to know how to extract things that are relevant for you and the team. But I left the job because it became boring and other opportunity's that were better paid.
wow hearing about Codd! jeeeze that takes me back to the 80s when i was a simple college student & my computer sciences teacher was telling me all about this interesting language. he was of course an ALGOL 68 man & me a humble assembler / dbase III guy.
When I entered college in the fall of 1979 I was hired by its Educational Computing Service department which did contract programming for the university and government agencies. At the time CODASYL network databases were the most likely type a programmer would encounter and an absolute pain in the ass to deal with. Not because they were hard to navigate but because it was exceedingly difficult (read: expensive) to change the schema. I still have a copy of "An Introduction to Database Systems" by C. J. Date on my bookshelf from 1981 when relational databases were still controversial but it was clear to me would quickly become the standard.
I would love a video on Visual Basic. I have a belief that VB is awesomely important both for applications running today especially in business but also for being one of the first examples of a minimal learning curve way to create applications really cheaply, especially in contrast to next with objective c
It may be "elitist", but... While conquering C, decades ago, I couldn't get past the literal meaning of the name of my first language: BASIC The name stands for "Beginner's All-purpose Symbolic Instruction Code". Can't remain only a "Beginner" forever. Can't imagine life without a pointer... Pointless existence... 🙂
@@rustycherkas8229 I know personally of at least three (maybe four but hazy memory) companies selling applications built using VB in active development to this day which started life as one-man bands. People may scoff at that language and look down on their work because of it, and those one-man band folk will be crying into their champagne tonight in their private south of france villas.
I started my professional career in GIS in the early 2000's I realise how critically essential relational db's are for applications like GIS. (table joins , primary key, etc for doing thematical maps and spatial analysis) Thnx for this great video! Because this kind of historical context was never part of the curriculum we had to study.
E F Codd's ideas area a bit more subtle than just applying to databases. They also apply to any type of software, where you should just update the software in one location.
Excellent video. I didn't know any of this and I've been writing SQL code since the late 90s- nothing too complex. Most of it involved pulling data from process control historians, doing some calculations and writing it to html files so operators and accountants could have real time access to inventories without having expensive process control client software. I find myself wishing the windows file system had SQL abilities on an almost daily basis.
Jon, have you ever considered packaging all your videos as a structured "history of computing" course? Or multiple courses (history of semiconductor manufacturing, history of hardware, history of software, etc)? I bet home-schoolers and others would be happy to pay for access to an experience that is more ordered and structured, like a miniature college-level history course. I think I'd pay for that. In any case, thanks for the videos. They're fantastic.
Bachman was a visionary, Cood a bookkeeper. As I'm currently learning neo4j, it seems obvious that graph databases as Bachman envisioned is the way for modern software and modern data representations.
I've left databases and network/web related things until last in my CS journey, still not really had cause to dive into them yet, file system hierarchies, or just one long json/bson/hashmap have been fine for my purposes so far, but I have seen bits of SQL here and there, and have been diving into bevy/entity component systems recently and have just realized how similar they are... in some ways the same philosophical hierarchical vs relational debate seems to somewhat be re-emerging in game design, in terms of how to structure dynamic interacting entities in graphics/simulation, scene trees vs entity component tables.
Thanks for the metaphorically to plant new trees. I am stacked with MS Access and I thought I need to plant more trees: unfortunately the garden is already full.
A fantastic video as always, but I can’t help but worry about the company represented at @11:21. With no IT Support staff, what are they going to do when the database goes offline? 😬
When I started in '97, Rapid Application Development was hot. Since those tools often used key-value pair "databases", I avoided SQL altogether. I dabbled in Clipper (dBase III) and VBA for office automation/workflows but eventually used Lotus Notes/Domino for 10+ years (distributed storage model for LAN and/or offline use, replication for minimizing network traffic, support for rich text and attachments, shared-nothing servers so you could cluster a VAX with a mainframe with a Solaris, PKI easily integrated in custom apps, ...)
This is a great video. I recently wrote a satirical critique of genealogy software's GEDCOM tool which was created in 1984 and is still being used to transfer data among different genealogy software applications. My thesis was that if GEDCOM had been created a few years later, it would have been a SQL database instead of the hierarchical nesting of asymmetrical inconsistencies that it is. Unfortunately, hierarchical structure is what was tried first and GEDCOM still hobbles genealogy data transfer in 2024 while the rest of the world has adopted RDBMS decades ago.
As a much younger software engineer, I still remember this phrase I was taught regarding relational database development. The key, the whole key, and nothing but the key. So help me Codd.
Also, in Codd we trust.
I Love That!
lame
@Bebtelovimab looking forward to the noML hype of the 2060s
I learned exactly the same phrase in my SQL systems SLIS course @ Univ of Western Ontario in 1989!
I once was interviewed for a senior development position with a major multinational bank with their database guru. He was the one man who knew how the bank managed all of its data. He revealed that they had just one database. It had 600 tables. There were no joins, no relationships, no structure; It was just a massive dumping ground. Trades, transactions, accounts, … everything. The goal was to take this unholy mess and redo something better, while keeping all the existing business units running without interruption. It was an Oracle database.
Nice
BlackRock 👀?
I don't feel so bad anymore, the one I work with is approaching 1900 tables and it still keeps referential integrity, mostly.
Lol, ever seen just one table with 3 columns: name, value, type?. Barf city.
I recall working with such a DB dumping ground as a new hire at one of my previous jobs. Source data from a global company in the banking and payments space. It came with hundreds of mostly useless stored procedures and functions that were supposedly written around 2015, but contained zero proper joins in the code. My new colleagues were waiting for me to make snarky comments about it!
“Codd, despite having a fishy name “ 😂 😂
I came here for this comment 😂😂😂
His worst enemies were two guys called Fisher and Hook.
Made me smile, too. The video is well done.
I came here to groan at that 😅
@@cjsveningsson I love groanworthy bad puns.
As a pre-SQL programmer, SQL was a game changer. Using SQL gave me a logical, abstract view of the data structure, significantly reduced my design and coding time, and allowed changes to the database structure without having to break code or migrate data.
I used several systems on PCs like DBase and Firefox which had SQL database at their core.
Also need to remember that systems back then were so constrained by CPU performance, memory storage, and disk capacity that SQL would have been too large to run on some of the early computers.
Great video as always!
I think you meant to say "FoxBase" or "FoxPro."
@@vulpo FoxPro was derived from dBase which became popular on IBM pc's in the early 80s
@@atheistbushman dBase was an awesome tool on the PC.
@@zzbeasley Agreed. I first met dBase on an Osborne luggable with 5 1/4" floppy drives and no hard disk! Later used dbase on PC and I liked Superbase under DR DOS/Gem. I think Superbase used dbase for data storage with Superbase giving a GUI front end.
Good
I'd argue that SQL is one of the most successful inventions of computer science, on the level of C or even more so.
Nothing better has come up despite decades of attempts, and pretty much all complex datastores eventually support SQL as they grow.
Humankind is generating vast amounts of data. Making sense of it all is beyond what a relational database can do well. The next step is "big data" non-relational databases. There are no keys, there is no structure to the data. The goal is to find the relations, not order them in that way. It is where we are headed. A relational database is great for what we use it for and will be used that way for a long time to come, but big data will look information a new way for a new purpose.
"Humankind is generating vast amounts of data. Making sense of it all is beyond what a relational database can do well"... did you ask ChatGPT to write this bunch of vague and disconnected catch phrases just to oppose the OP ? What would be the practical purpose of "making sense" of all data we are producing? Make a real life Matrix ?
@@chpsilva Cook his ass
@@chpsilva A comment section is not the place to have a deep discussion about big data. I don't see that I opposed the OP, I presented what the next generation of databases are about and where the research is going. I have spent my professional life programming SQL relational databases, I do it for a living. If you are interested in the big data concepts I am talking about do a google search. Big data is useful and yes, despite your reservations, it is something that matters.
I find it remarkable that it is the only widely used non-imperative language.
Study math, fly bombers, invent the dominant database structure. What a life!
+Program differential equations for missles
Yes, to fly murder machines, how amazing
SQL is so useful. And you can learn 90% of what you need in two weeks. There's a reason it's so ubiquitous.
Database researcher and university professor here: Very good presentation as usual, though it of course ends with the state of things in the mid 1980s and a lot has happened since, not just in academia but also in industry. As early relational DMBS go, IBM's System R (which has been alluded to in the video but not named and whose DNA lives on to this day in IBM DB2) has had significantly higher impact and relevance than Ingres. Since you mentioned Boyce and Stonebraker, a number of others would have deserved mention at least as much, such as Jim Gray or Pat Selinger. Also, you added to my pain as an academic trying to attract young researchers into the field by making databases look real bland and boring (though important to business). Unfortunately, so many undergrad courses do the same. There is beautiful systems and theory research to be done, and there are interesting fundamental questions that arise here that are much cooler than anything mentioned in this video (though ultimately enabled by Codd).
Maybe he will cover MUMPS.
For sure, a database is way more than an Excell spreadsheet. With column based storage, data compression, and parallel processing. We get things like Google, a keyword database of the entire internet. And Twitter a database that adds records at a rate of 100's of gigabytes a second, all instantly searchable.
Please post interesting fundamental question(s).
@@Martinit0 What is your background so I can gauge my wording?
Not the original poster but curious; a bachelor's degree in mathematics with no theoretical exposure to databases, and 6 months experience programming with SQL.
"Codd, despite having a fishy name, did not want this." 😂😂😂😂
I had the privilege of digitising some Betamax tapes from the Australian Computer Society many years ago from the 10th Australian Computer Conference held in 1983. Chris Date, a relational database expert, gave a presentation on one of these tapes and made many claims about what the future would be for databases. "Like it or not, SQL is going to become a very important language. It might become an actual standard, and it almost certainly will become a de facto standard."
The full speech is here czcams.com/video/VnNbddUMZQI/video.html
Awesome find! Thanks! 😎✌️
Thanks. (You needed a time base corrector inline with the signal)
@@MePeterNicholls I used one, a Key West Big Voodoo. It doesn't handle dropouts in the vertical blanking very well. The tapes were very degraded unfortunately.
@@256byteram ah 👍🏼 find one these days is hard enough too tbh
@@MePeterNicholls The biggest problem were dropouts. A digital timebase corrector used with a domestic video recorder is not going to help with that. Sure they can on studio machines like Umatic, Betacam, MII etc., but domestic machines lack the required RF output signal. However it might be that the particular player could have usefully had the DOC (dropout compensator) sensitivity adjusted, I recently showed that with a Beta machine on CZcams.
As a newish developer, of all videos, this was the most exciting to see when I saw the thumbnail. SQL has become one the foundational blocks of software development and even today, when all the competing NoSQL paradigms (MongoDB, Firebase, Redis, etc) have claimed a space in database management, we are coming back to SQL with new ideas thanks to PostgreSQL and SQLite related projects like Turso and libSQL.
And I really like the image in 14:50 while distant in time is so relatable to my knowledge. A testament on how SQL stood the test of time and won.
Really thanks Jon for this video.
Part of Data-oriented programming.
The problem is that most NoSQL databases (Mongo is fully ACID but records aren't usually normalizaed each document has all data so this doesn't mean much and so multi document transaction were only introduced in Mongo 4 mere 4 years ago) aren't fully ACID (Atomicity, Consistensy, Integrity and Durability). For things like banking transactions are a must: the money that gets out of one account must get in on another or the transaction must rollback.
Unstructured or partially unstructured data are a case for NoSQL, many NoSQL databases are very simple and good in replication, but at a cost of using GUIDs as the primary key (as a surrogate key) this is not always desirable.
When you get billions of records that need integrity and complex queries (Mongo performance with complex queries is abysmal) that must be done fast SQL is a must (good part of my life I passed tuning badly designed and maintained databases, and if I got a Dollar for each time I heard this can't be done faster or this can't be done in SQL I would have double what I earned).
@@agranero6 I agree with you, but my point really was that we are ending the era when NoSQL was trendy and "you have to use it to be cool" like many other bad ideas mostly in webdev (microservices for all, complex SPAs for everything, rewriting everything in the latest language, etc). Nowadays everybody is rediscovering that SQL in its many forms is a solid language with solid ideas. Even SQLite is becoming popular! 😜
For me, I'm in both sides: For small projects I'm happy with tools like Cloud Firestore, but for more complex projects nothing that good ol' SQL.
@@agranero6 this, so much
Still can’t believe they named it “mongo” lmao
Being a DBA student, it's interesting to see where all modern concepts began.
Do professors tell you that physical structure of the RDBMS is as important as logical structure? As a contractor DBA I make much more money from physical structure issues than from logical ones.
@@kondybasAgreed to a point. Usually, the logical design isn’t that bad. My big problem is middleware… stuck threads, garbage collection, multi-threaded architecture and lack of commits.
I've been using SQL since September 1999. Database programmer has never been my formal title, but I have been one for 25 years this fall, in addition to my main programming job that's normally identified by the other programming language. It's been SQL the whole time, but the other language has gone from C to Perl to PHP to Java over the decades. What makes SQL different from other programming languages is that it's used by far more than just programmers. I've seen programmers write horrible SQL and non-programmers write terrific SQL. If you can wrap your mind around set calculus you'll become very good at SQL, and you don't need to be a programmer at all. Either way, practice helps. I was only what I would consider good at SQL after 15 years and I'm now fairly proficient at SQL such that I can jump right into any database get going.
There are literally billions of SQL databases in this world, yet very few people knew where or how it got started until now.
Thanks again, Jon.
There are billions of ignorant people too.
I'd use the word 'care'.
At least now I know how SQL got its pronunciation ! Can't wait for the sequel ! 😊
... and how Larry Ellison got his dirty mitts on things
_Lots_ of people know where and how SQL got started. They just had to graduate college before the rise of Java. Heck, I've still got a Relational Databases textbook in a box somewhere.
Yeah if only knowledge were stored somewhere other than CZcams... WAIT A MINUTE! 🤦♂
Moral of the story: Concepts are great for awards but actual working code/products can make you rich.
it's the moral of the story if your ideology revolves around the idea of making money.
@@Bestmann3nlet me guess, you're a socialist and think " money bad! Me no like! >:("
@@Bestmann3n The moral still works if you just want awards.
I have to profess my love of this channel and your content. I don't know how I was lured into this channel at first, I'm not Asian, and I'm just a mere MSP technician who grew up in the 80s and 90s. But you always seem to hit the right note, and Sunday nights haven't been the same since. You're doing it right, sir. 🤩 I eat these videos up, and wanted to say "thanks".
its only called Asianometry because there's a focus on Chinese, Taiwanese, Japanese, Indian technology sector! the channel is not actually about asian culture lol
It’s oddly addictive for reasons I can’t figure out. I usually set the videos to my ‘watch later’ list for when I’m driving my daily commute.
@@Old_Jack_Ketchi agree. It is probably because the topics in the video's are not ancient history and still have relevance to this day.
But even if you started studying or working in ICT you might have never heard of many contextual details in these vids but when you hear or see them you can often instantly relate them to personal experiences (at least i can) and that's entertaining.
Greetings from ASML-city!
@@thinkingcitizen I get that. I meant in terms of the initial draw for me, what made me pick this content, based solely on the name of the channel - It speaks to how genuinely interesting the selection of topics are. 😉
Really enjoyed that, but then I do have a 35 year SQL career behind me. Furthermore, almost all of it was new to me despite my extended immersion.
I have to be honest, when I hear Codd's name, I usually switch off: it's usually in relation to Codd's Rules, academic rules of thumb that in practice are nothing more than common sense and second nature in this line of work. I watched a video on them that I happened to stumble across a few months ago, and yes, it is all common sense codified into a set of rules. So I congratulate you in managing to do a 20 minute video on SQL without ever referring to Codd's Rules!
for our management information systems program in university we barely do the SQL behind these academic concepts and it's really dumb, I am currently building a .NET interface for a access database just so I can practically learn this stuff.
Your videos are so important. Documenting and popularising the foundational building blocks of our modern world.
Agreed, this channel is a gold mine of technology history
Information exists outside of youtube, ya know. Pretending youtube is some super-special, locked down knowledge store that will never go away and is vital to maintaining a record of history... come on now. 😏
@@mercster Asianometry helps spread this information to a wider audience without them actively searching for it. Almost every person watching this video had it recommended to them via the CZcams home page or through another recommendation method on the site.
There are other sources for this information, but this video is a good way of pushing out that knowledge to a large number of people who would have not previously considered or cared about the topics discussed on the channel. Best regards.
@@broadestsmiler I'm not taking anything at all away from Asionometry, whose videos I very much enjoy. What I'm getting at is, when you lavishly heap praise in an overly-enthusiastic, hyperbolic manner that doesn't really accurately reflect reality, you diminish the genuine value.
@@mercsterNot everybody has the time to delve into historical documents.
I see this channel as a low-treshhold gateway into the early history of the computer industry. Not overly theoretical yet very informative.
I work in a social work/SUD treatment agency, and we somewhat recently began working with a company that helped us get backend data out of the cloud database. I haven't worked with SQL since college... The last few months have been a blast, and believe you me, I've been able to flex comp sci skills in a very not comp sci industry haha.
I like how You dig out filthy details: "Deliverability, Redundancy and Consistency of Relations stored in Large Data Banks" with the exact date on it +++
Hello Jon,
In my opinion, this is one of your best video. As a side hustle to my job, I've been writing in SQL for more than a decade. I'd read about how it come to be, but the story you tell is at another level.
Thank you
Anthony
Gotta say I wish my dad was still around to appreciate this channel. As a historian and library scientist with most of his career transpiring before the internet explosion, he didn't really get to witness much refinement of video content. The variety and scope of the stories shared through Asianometry are just plain great... (Although I'm sure he would've griped about lack of formal reference summaries to back up details.)
5:40 in CS, the “top” of the tree is its root, not the canopy as you depict it.
I knew Charlie Bachman and can confirm he was a stand up guy. Kind, patient, a good teacher and a good boss. I worked for him for a few years when he was promoting his entity-relationship model which was more abstract than the Hierarchical (IMS), CODASYL (IDMS) and Relational (DB2) models and was useful for porting and reverse engineering.
I remember learning SQL in the early 2000's and dutifully memorizing that it was the unification of relational algebra and relational calculus. The fact that almost nobody would even bother to learn that today attests to how useful and intuitive SQL really is. The key was, I think, that they modeled it after regular English and the implicit set-theoretic operations in natural language - "select birthday from people where ...""
I'm in the "Esss Que El" camp. Good to know Asianometry is the "See quell" camp.
Jon missed another opportunity to cement his brand by mispronouncing industry terms.
Great video as always! I would love future videos on more modern SQL since the 1980s -> e.g. the breakout of OLAP vs. OLTP databases, SQL in the cloud, Presto, etc.
Phenomenal video. The inoculation to being overwhelmed by a technology is to follow the simple ideas and history that built its complexity. You have a remarkable talent for leading audiences along those journeys. I hope that you'll take us on similar tours of these omnipresent, yet terrifying, tools that power our world. In the meantime, avoid dying of a brain aneurysm. Because evidently that's something I get to worry about now.
No mention of C.J. Date, Codd's most fervent acolyte?
15:06 Nice shot of the Air North Hawker Siddeley from Whitehorse, Yukon. My home town.
Later on in my life I am finding a lack of great videos that bridge the human gap between the great ideas we need to learn and how and there they came from. It's so easy to give up because everything comes down some arbitrary decisions than are then later generalized and that's what wins out. I am one of those people who needs that back story to calm down the "Why this way??" questions in my head with out mastering something enough to know why.
Not much point to this for others. Just engaging the algorithm with something more than "Thanks for another good video. I enjoyed it."
I look forward to your productions. You’ve found your stride! You are on an excellent video production roll!
Interesting, informative and well done, on a subject that I worked with for a number of years, but had not idea of 'the rest of the story'. Thank you for all the work and then sharing.
Big thumbs up and thank you , databases often overlooked are the backbone of most system design. One of the first high school jobs was digitizing the music collection the national radio station into a DBase IV system. I then used my database programming superpowers to make a movie rental database back in the days when stores rented VHS tapes using Novell Netware and Btrieve. A lot of money to be made from database design in the 1980/1990s! It was the HTML of the era.
I worked on databases on IBM mainframes. IMS was described as hierarchical, Total was a network database (you had two levels). I then came across early DB2 using Structured Query Language and of cause Codd’s relational database principles. I did a bit if work on Oracle and quite a lot using UDB (the djstributed version of DB2 for middle tier rather than mainframe systems. Interesting to see how other software sellers went about implementing Codd’s rules.
10 IQ: "S-Q-L"
100 IQ: "Sequel"
1,000 IQ: "Squeal"
10,000 IQ: "Squirrel"
underrated
S-Q-L (originally SEQUEL) had to change its name to S-Q-L due to an existing language called SEQUEL and subsequent trademark dispute. I try my best to use S-Q-L on Monday, Wednesday, Friday, SEQUEL other days, and "Squirrel" when I have had too much to drink.
squirrel 😂 - I can laugh about that joke without an IQ of 10000. - I didn’t have to come up with it, I only needed to understand.
Edit: on the other hand: it seems a bit unfair, as squirrels (allegedly) sometimes forget where they stored all of their seeds for the winter, thus enabling unintended growth of trees… (- „trees“ ?😂)
Edit 2: maybe the 10000 wasn’t in the decimal system, but binary 😂
Wonder what DB model squirrels use to find their hidden hazelnuts again 😉
One could say that SQL was an _injection_ the industry needed?
Hahaha. I'll get my coat.
I can RELATE to that. WHERE did you learn such corny jokes?
When you get your coat just remember to don't DROP it on the TABLE.
It definitely made a _statement_ no one was _prepared_ for.
@@raylopez99 🤣🤣
I'd rather SELECT someone else to make jokes...
Fantastic video. Subscribed! Your content has a wonderful niche that needs filling!
My manager met Charlie Bachman in the 90s. I used to use his tool also called Bachman to write DDL and create DB2 tables. Video brought back memories.
Writing SQL is an incredibly therapeutic, zen experience. I cannot properly express the joys of having an end goal of what data you want to have rolled up and the fun, challenging path of writing SQL to get you there.
I changed fields, in a way, and no longer get to use SQL. I miss working with it all the time!
I interviewed with Oracle and three other Bay Area companies before moving there in 1981. Oracle was my second choice. It was a prototypical startup - but at the time, of course, it looked like a young and untested outfit. What might have been… (I had plenty of time and read E. F. Codd's papers at the not-so-busy consulting firm I signed with instead, and got heavily into databases later.)
My second cat, the last offspring of my first dilute Calico whose litter came while I was learning relational databases, was named SQL. A damned good cat, indeed.
May he rest in peace
I appreciate the label ‘vicious’ applied to Oracle as that is how they were described by Michael Stonebraker and colleagues, developers of Ingres, Postgres, etc. In the 1990s. Oracle would do whatever it took to get the business. (I worked as a programmer in the CS Division at UC Berkeley in the 1990s)
Let's give a little credit also to Chris Date who somehow managed to document Codd's work in a form that mere mortals could understand.
I've been needing a good overview on databases! You always put out videos addressing things I'm interested in :) Wait a minute... show us the database you use to predict our viewing desires!!!
That was a great one. Thanks and well done.
Great video. I love Asianometry's style of presentation.
What a great video, thanks a lot for making ❤
We usually prefer the term tree instead of network to avoid confusion with a computer network and because network is a wider concept (equivalent to a graph): networks don't have to be in tree form, they can for instance be semi-lattices for instance. Trees are good for seaching as we can traverse with simple algorithms and find a key in logarithmic time, if the tree is well balanced, and we can balance them for efficiency. In general graphs this can't be done.
In your video you confuse related tables with trees: the trees in databases are assembled inside the indexes to the keys in the database. The relationship between tables doesn't need to be in the form of a tree and it usually is not.
this was very much needed ... thanks
Great video! Thanks for producing it.
I worked as a SW developer for 30 years. I started with IMS, IDMS then DB2. DB2 relational DB is the best for design, development and serving the business.
Absolutely amazing story! Another very informative and well researched video!
One hopes you do a video on IBM’s Query by Example and System R and how a salesman and programmer from Amdahl managed to start Oracle. Or not.
Oh this is my absolutely favorite topic, tech history. This was such a good video.
Wow! Glad I subscribed your channel!!
I've finally found the video to share how I'm feeling. I've loved using Prefect. Your videos are the vvibes!!
That was a quick one Jon!
Great job with the "despite having a fishy name" joke. I laughed 😂. Great video!
Codd looks like Dirac. And the controversy between Bachman and Codd reminds me of Einstein and Bohr vis a vis quantum mechanics
Please never stop making videos!!!
Codd looks kinda like Paul Dirac in that photo
Nice vid. BTW Trees are traversed from the root(trunk) to the leaves, not from the leaves to the trunk (usually)
SQL just goes on and on and shows no sign of being superceded. I'm convinced that Data is still using it on the Enterprise...
Excellent, as always! The second word in Hawker Siddeley is usually pronounced as two syllables, sid-ley. Source: I was an avid 10-year-old aicraft spotter in London in 1972 when RDMS was getting going, and the HS Trident was my favourite airliner. (Though it started life as the de Havilland Trident.)
I love all your videos. But the your computer history videos are above all else. Please, do more computer history videos!
i had a professor in the early 80's who was a disciple of Ted Codd. he was always spending class time relating personal anecdotes about Codd instead of the class material. according to my Prof, IBM was not happy with Codd's research since they thought what he was doing was going to conflict with IMS and could cut into the profits IBM was making from it. my Prof respected that Codd was able to persist in spite of the corporate discouragement, and eventually created the System R project, which turned into DB2.
This was great. I hope you will follow up with the saga of the database wars of the late 1980s and early 1990s between Oracle, Ingres, Sybase, Postgres, and MS SQL Server.
Okay this was the first time for me there wasn't a fascinating video hidden behind a boring title.
That is just me tho. Still watched it. Keep up the great work ❤
i am about to graduate with my buisness degree in Management Information Systems and im so excited! Im currently staring at a 3nf normalized ERD for a travel company. Its kind of like micromanagement games like openttd, you gotta make sure all your little guys are transporting what you need to each other.
While working on the Allen Brain Atlas I got to work a bit with Jim Gray ... I best remember talking over beer about both technology, sailing, and comparing Seattle to the Bay Area.
Larry Ellison has entered the chat
One
Rich
A**hole
Called
Larry
Ellison
One Rich As.... Called Larry Ellison
You forgot to mention the predecessor from SQL (that I learned), it was dbase IV, and it had already SQL implemented in it. I programmed a lot in dbase III and IV mostly for bookkeeping purposes. At that time, this was very powerful, but I quit programming when I was conscripted for the army. After my 3 years duty, I lost interest, unfortunately. But SQL isn't much different from the other products. And later in my Carrère working in BPCS I used my knowledge to create my own query's and forms to extract data out of the mass database. My boss didn't know much about databases and programming or making forms on that level. Enjoyed my work very much. Making things a lot more efficient. I remember saying to him, everything is there, you just have to know how to extract things that are relevant for you and the team. But I left the job because it became boring and other opportunity's that were better paid.
Many thanks again. I look forward to a sequel on the easel SQL products and how it later became so widespread. Fingers crossed.
Nice movie!
Excited to see the SQL.
great video man, shoutout to those legends
wow hearing about Codd! jeeeze that takes me back to the 80s when i was a simple college student & my computer sciences teacher was telling me all about this interesting language. he was of course an ALGOL 68 man & me a humble assembler / dbase III guy.
When I entered college in the fall of 1979 I was hired by its Educational Computing Service department which did contract programming for the university and government agencies. At the time CODASYL network databases were the most likely type a programmer would encounter and an absolute pain in the ass to deal with. Not because they were hard to navigate but because it was exceedingly difficult (read: expensive) to change the schema. I still have a copy of "An Introduction to Database Systems" by C. J. Date on my bookshelf from 1981 when relational databases were still controversial but it was clear to me would quickly become the standard.
I would love a video on Visual Basic. I have a belief that VB is awesomely important both for applications running today especially in business but also for being one of the first examples of a minimal learning curve way to create applications really cheaply, especially in contrast to next with objective c
Basic: "We let dumb folks program."
I miss Visual Basic 😭
It may be "elitist", but... While conquering C, decades ago, I couldn't get past the literal meaning of the name of my first language: BASIC
The name stands for "Beginner's All-purpose Symbolic Instruction Code".
Can't remain only a "Beginner" forever. Can't imagine life without a pointer... Pointless existence... 🙂
@@kyriosity-at-github Yep, there's a reason no killer apps were ever written in BASIC.
@@rustycherkas8229 I know personally of at least three (maybe four but hazy memory) companies selling applications built using VB in active development to this day which started life as one-man bands.
People may scoff at that language and look down on their work because of it, and those one-man band folk will be crying into their champagne tonight in their private south of france villas.
Awesome video topic!
Maybe WinFS with the relational model applied to file systems.
Great Video! You're ability to redraw the history of software and it's engineers is extraordinary! More of software history!
Great video about SQL! Enjoyed it a lot.
Wake up babe Asianometry just dropped a new video
great video! you should try to interview King Larry!
I started my professional career in GIS in the early 2000's
I realise how critically essential relational db's are for applications like GIS. (table joins , primary key, etc for doing thematical maps and spatial analysis)
Thnx for this great video!
Because this kind of historical context was never part of the curriculum we had to study.
E F Codd's ideas area a bit more subtle than just applying to databases. They also apply to any type of software, where you should just update the software in one location.
ROTFL - a vicious new competitor! Can't wait for the next instalment Jon.
Excellent video. I didn't know any of this and I've been writing SQL code since the late 90s- nothing too complex. Most of it involved pulling data from process control historians, doing some calculations and writing it to html files so operators and accountants could have real time access to inventories without having expensive process control client software.
I find myself wishing the windows file system had SQL abilities on an almost daily basis.
Jon, have you ever considered packaging all your videos as a structured "history of computing" course? Or multiple courses (history of semiconductor manufacturing, history of hardware, history of software, etc)? I bet home-schoolers and others would be happy to pay for access to an experience that is more ordered and structured, like a miniature college-level history course. I think I'd pay for that. In any case, thanks for the videos. They're fantastic.
Bachman was a visionary, Cood a bookkeeper.
As I'm currently learning neo4j, it seems obvious that graph databases as Bachman envisioned is the way for modern software and modern data representations.
Im a BTA major and have spent a lot of time doing SQL labs. This video was amazing.
12:06 - I spilled my tea!
Excellent and enyoyable report on a dry but important issue. Thank you
I've left databases and network/web related things until last in my CS journey, still not really had cause to dive into them yet, file system hierarchies, or just one long json/bson/hashmap have been fine for my purposes so far, but I have seen bits of SQL here and there, and have been diving into bevy/entity component systems recently and have just realized how similar they are... in some ways the same philosophical hierarchical vs relational debate seems to somewhat be re-emerging in game design, in terms of how to structure dynamic interacting entities in graphics/simulation, scene trees vs entity component tables.
Thanks for the metaphorically to plant new trees. I am stacked with MS Access and I thought I need to plant more trees: unfortunately the garden is already full.
For the IBM Series/1, before SQL, there was QBE (Query By Example), I believe on both the EDX and RPS platforms.
A fantastic video as always, but I can’t help but worry about the company represented at @11:21. With no IT Support staff, what are they going to do when the database goes offline? 😬
When I started in '97, Rapid Application Development was hot. Since those tools often used key-value pair "databases", I avoided SQL altogether. I dabbled in Clipper (dBase III) and VBA for office automation/workflows but eventually used Lotus Notes/Domino for 10+ years (distributed storage model for LAN and/or offline use, replication for minimizing network traffic, support for rich text and attachments, shared-nothing servers so you could cluster a VAX with a mainframe with a Solaris, PKI easily integrated in custom apps, ...)
Great history! Thanks for putting this together.
Btw is a decade "fast"? Certainly in geology, but in computers...?
This is a great video. I recently wrote a satirical critique of genealogy software's GEDCOM tool which was created in 1984 and is still being used to transfer data among different genealogy software applications. My thesis was that if GEDCOM had been created a few years later, it would have been a SQL database instead of the hierarchical nesting of asymmetrical inconsistencies that it is. Unfortunately, hierarchical structure is what was tried first and GEDCOM still hobbles genealogy data transfer in 2024 while the rest of the world has adopted RDBMS decades ago.