Calling All Wizards: Let’s Build the Ultimate Data Org Guide
Vložit
- čas přidán 20. 01. 2022
- Check out the forum thread, it splits into several other threads: forum.level1techs.com/t/the-u...
We would love to hear your ideas!
**********************************
Check us out online at the following places:
+ Website: level1techs.com/
+ Forums: forum.level1techs.com/
+ Store: store.level1techs.com/
+ Patreon: / level1
+ L1 Twitter: / level1techs
+ L1/PGP Streaming: / teampgp
+ Business Inquiries/Brand Integrations: Queries@level1techs.com
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.
-------------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons.org/licenses/b... - Věda a technologie
Books on screen:
FreeBSD Mastery Advanced ZFS -- Jude & Lucas
Assembly Language For Intel-Based Computers -- Kip R. Irvine (3rd ed.)
(Indistinguishable red book, seems to have "Computer" in the title)
Fingerprints of the Gods -- Graham Hancock
A New Kind Of Science -- Stephen Wolfram
In The Plex -- Steven Levy
HP-100LX (a palmtop PDA) User's Guide
Robotics 2000 -- ASCE (a scientific conference)
QED: The strange theory of light and matter - Richard P. Feynman
The Art of War -- Sun Tzu
The New Hacker's Dictionary -- Eric S. Raymond (3rd ed.)
Star Trek Technical Manual -- Rick Sternbach and Michael Okuda
Boiling the IT Frog -- Harwell Thrasher
What if? -- Randall Munroe
The Adventures of Sherlock Holmes and Other Stories -- Sir Arthur Conan Doyle
Surely You're Joking, Mr. Feynman -- Richard P. Feynman
More Joel On Software -- Joel Spolsky
Antifragile -- Nassim Nicholas Taleb
Psychopath Free -- Jackson MacKenzie
The Adapted Mind -- Jerome H. Barkow, Leda Cosmides, John Tooby
The Art of Computer Programming - Donald Knuth
haha, nice! searchability! :D
Lucas has excellent books on FreeBSD.
I'm guessing this was accomplished with good old human biological OCR technology ;-)
@@jjdawg9918 I'm guessing it was a simple AI system that most people think of 'facial recognition' but is used to identify damn near everything... which is amazing, but could could be used in scary ways - but like this, BRILLIANT! mainly because we don't see it Identify Wendell and automatically list his credit score, bank info, life history, and criminal record LOL :)
I need that assembly book for exploits lol
Technology extends the boundaries of what we are, it is the scaffolding that extends our reach and vision. As someone who is a long time believer in owning my own media I am really looking forward to this series!
ive thought for years of technology as extenion man not unlike a hand or an arm. even chimps use tools,
+1 to this!
Now imagine that scaffolding collapsing. How can we prepare for that?
@@svenkarlsen2702 I would go back to living on a farm.
@@LAWRENCESYSTEMS agriculture is not easy enough for people to succeed at just by intuition and necessity.
I remember when I was in university finding an old ratty book in the library where decades of students had annotated the text and commented on and disputed the annotations of the previous students. It was something far beyond what the original published book had ever been.
I get what you mean, however more often than not, I have found annotations like that are often wrong. My experience is limited to physics/maths though, so the idea of what is right is probably more definitive unlike something like software engineer/architecture, or literature
Reddit on paper.
Would absolutely love to learn more about your workflow, reading habits and organization strategies. This is the content I've been waiting for.
I have - until recently, preserved information by keeping the respective browser tab open. It started years ago when i realizes that bookmarks kinda don't work when you have too many. Now, with 900 tabs open, I realize that again. I have no solution.
Ha, I did the exact same thing, although I've kept it down to only about 100-150 tabs and more recently reorganized a good portion of my bookmarks into categories and even sub-categories, for example I occasionally like to cook, so I have a cooking folder with sub-folders for breakfast/desserts, so yeah I've sorta half way fixed my problem, but that doesn't mean it couldn't still be better.
My advice is to simply do the same thing you would do with your files, create a system to organize stuff by, make sure to have a miscellaneous/catch-all folder to toss most things you aren't sure what to do with into and most important is stick to it, if you get lazy then you'll just fall back into having a mess of 100s of things everywhere.
I do the same. I use firefox with tree style tabs and fold trees that are not actively used. They will survive restarts, but sometimes not crashes. I use firefox data mining tools to extract tabs from session files (lz4 compressed json) when that happens. The problem with bookmarks is that it does not save pictures, contents, and state of the webpage, nor does it allow you to use the back button. Sometimes I read pages or threads and only a particular section is important, keeping the tab will ensure when reloading it goes back to that position. Also, going through trees of active tabs to find something that I saw on some page is quick since you don't have to open each page to look at it, and it allows you to see which tab is opened from which and the order of opening, allowing you to skip trees or sub trees.
Generally the software limit for firefox is somewhere between 2500 to 4000 tabs before encountering strange behavior or just crashing, this assumes enough RAM and no GPU acceleration (because GPU vram will runout and cause a crash, unless you have 16GB+ vram). What you can do to have more tabs is run firefox, firefox developer edition, and waterfox together in one system with sufficient resources, that will get you around 9K~12K tabs.
Around 8000 tabs is sufficient for me until the information is no longer useful, or I have found spare time to save important topics/searches into folders (along with all the text, assets, pictures, links, and screenshot of important parts).
@@evolution1565 That does sound amazing.
I have to say though, I watched the video above another time now and Wendell planted a seed in my head when he said "Wiki". An organise place where I could put the links, the most valuable content of those links, in case I am afraid they could disappear, and my own thoughts and maybe pictures on the matter, always ready to be expanded...
That might become a thing for me and it would also solve that other question of "where do I put my own thoughts?"
I struggle to keep tabs under 1k😭
@@evolution1565 hmm, nice catch, last time Firefox crashed I lost around 4000 tabs and only got some of it back through session backups as it corrupted the session files and well since I don't exactly reboot my PC every night the default session backup really was painful, well at least I had to spend the time to reevaluate through history and previous session backup and dropped a lot of tabs that had been there for way too long, though gotta say, would be impossible to use tabs with chrome in this manner
I've spent much of my adult life looking for a self hosted evernote/Onenote alternative. I'm excited for what is to come with this series.
Check out awesome self-hosted, have a look at the Note-taking category, a couple suggestions are Paperwork or Laverna
Joplin is pretty good I hear.
I am very satisfied with dokuwiki.
I didn't care for a self wiki or focument management system, but I understand they have their place.
This is a remarkable new project. The concept of using technology as "my brain outside my brain" is something I have been exploring for awhile but have not yet been able to "nail it". I'm not the most regular visitor to the L1 forum, but will definitely be headed over for this. Thanks L1 community & Wendell!
My long-term dream project has always been to digitize every book that I've read, every document that I've found interesting, and every video on CZcams to archive; and organize all of that into a database that's hosted at my home for my descendants to add their knowledge to and to get information from. I feel like the sum of all of the information that I've taken in would be a kind of digital imprint of my mind not exactly an AI duplicate of myself but close enough to one that I could think of.
neat idea....a record rather than a duplicate (and something that later gens could argue about and dispute, too!)
Would really hope that CZcams allows MP4 downloads officially with a tick box that the uploader can set, because you're not supposed to download CZcams videos except using methods officially made available by CZcams.
What a wonderful and timely project. As my parents are aging, I'm becoming the custodian of our family knowledge and history. From more recent digital archives of photos and videos, to reels of home movies, boxes of cassette tapes, shelves of photo albums, and bundles of handwritten letters. Everything is accumulating in my house. I'm digitizing what I can, but I've quickly discovered that this is just the first step. How to keep everything organized and then make it available for the next generation is truly challenging. Looking forward to this series and developing solutions with the community.
1:30 "There's mess nobody wants to clean this up" Exactly what I do at my job and people praise me for it. And I am like: "I just don't want it to be stupid, there is no secret formula to what I do".
as a teacher i feel the same way about using excel spreadsheets constantly....other people are like oooooo aaaaaaaa how do you DO that and im like your way just sucks, do it better...
As a bit of a data hoarder, I find this to be an important topic. Preservation of information is key to educating the next generations. Otherwise the kids of the future are only gonna learn from platforms like Tik Tok, and that's a scary future.
As an educator unfortunately they already are
Tik Tok is what happened when we allowed Instagram go full metastases, it's crazy, I swear modern kids have even lost some of their peripheral vision because of it, it's the only explanation on why they drive like absolute idiots, I've been crashed from behind twice while riding on my bike by kids that are filming themselves, we live in an absolute idiocracy.
That's not even a future, that's a dead-end. I don't see anyone learning GPU design from a 30-second video.
It reminds me of the essence of the gentoomen library. I had a former boss who built his own wikipedia with his own infrastructure that contained every IT solution he’s had to conjure up. That was a profound idea to have access to very niche pieces of information aggregated from hours of sifting through google, forums, manuals, and videos. But the web browser interface is limited as is trying to document complex diagrams and solutions using a wiki markup. I think you are onto something that will be the necessary Open Sourced library of backend technologies that young folks like me will need decades later as a way to avoid the malevolent information technology convergence that profit driven companies make us increasingly dependent upon today. It would be great to have an archive of mostly digital stored multimedia (in a variety of file formats) and a suite of AI and OCR software to sift through any information stored in any odd format via voice then display it via AR.
I’ve gotten into the habit of writing notes to my future self. Things that I’ll wish I’d remembered, like when working on the house, or things I build. At the moment there in notes on the phone, and Bitwarden for sensitive stuff. It is reassuring that I’m not the only one working to
Capture my brain knowledge
Org-mode
i just email myself.....its not great but after i get them i put them into their folders and come back to them when needed.....google notes works too but im in email so often it almost makes more sense
There’s a raspberry pi project to turn the pi into an offline Wikipedia. And you can download various archives to the hard drive connected to the pi. Throw a battery bank, keyboard, mouse, screen and solar panel into a pelican case and you have a laptop with the knowledge to aid anyone in a post apocalyptic world.
I was in love with the LiveScribe Echo - because it was a standalone device. The cloud and smartphone reliance of the replacement model is such a disappointment.
I loved the hell out of my Palm TX and Lifedrive, heck, even the z22 and the much better Tungsten E2 were amazing devices. I like my iPhone and my iPad, but they're attention seekers, Palms were straight to the point. And my Android phone is just a freaking trojan, I avoid using it because how insanely using it affects the ads I get everywhere.
I absolutely love this (especially the book server prospects). This series could potentially be my favourite thing on CZcams!
All about that book server. Even without the annotations, easy annotations stored on a book server just sounds like a dream.
HomeAssistant running in a VM has been huge for me and actually got me to go back to college and go back to looking at IT as my career after giving it up. I gave it up because I hated the CS program at the local community college, didn't learn much and was bored constantly. Learning YAML and getting away from the nabu casa subscription, thus learning a lot of networking stuff, I remembered how fun technology can be. Literally a lightbulb moment!
You are a wonderfully kind, genuine, and thoughtful person. Thank you for doing this.
When I worked on help desk, I learned how to automate my notes as well as create flowcharts.
It brought me to a point of working with interesting cases within a year because of my knack of weeding out rudimentary stuff.
Paperless-ng radically changed how our family lives - receipts being available for warranty and returns, manuals and paperwork for appliances and technology you've acquired, all your Bill's, insurance, medical, and legal paperwork... manually addressing that sucks more time than I ever realized...
Now any time we have anything of that nature come in, we just chuck it in the scanner feed tray, and then shred it - automatically organized by type, fully searchable, and you never have to worry about things like "those led bulbs had a 5 year warranty... did we keep the receipt, and if so, where is it?" ... type a one word search in your browser, and you're set.
This is fantastic. I've been thinking very similarly recently regarding information and offline archival
This adds clarity to something I've been mulling over for years. Challenge accepted!
Great concept! I'm looking forward to more of this series!
this is definitely gonna be my favorite video series and a motivation for me to contribute to the forums
You guys manage to be very inspirational to me and I appreciate your existence. Thank you. Just finished watching and man a book server that sounds nice. Like a Kindle but better.
Exiting! Great idea, looking forward to see what is coming up!
Really looking forward to this series!
This really sounds interesting. I'm looking forward to the new series! Thank you Wendell!
I'm excited for this series as well. Im currently building my first homelab server. I always watch you Wendell as well as Tom Lawrence.
Thank you, Wendell !!
really looking forward to this project, nice work!
Totally looking forward to this content! It’s something I’m trying to do for myself so this will probably be very helpful for me
MORE of such videos! THIS is actually quality content with an uplifting spirit!
The concept of this series is exciting! I am looking forward to more.
I love these long form video essays, you may not worry about speaking, but you are better at it than you think. More content like this is inspiring, I tend to save these video to my personal server in case a youtube apocalypse were to ever occur along with any other videos I find educational
Great theme, goal, and purpose for the series. Can't wait. The frustration of not having a clear path, tools, and schema to capture, organize, and retrieve personal inquiries, slice, dice and reference materials across all media types and channels, in really high. Even though I place the most blame on self focus, bandwidth, and persistence to learn the underlying tools. But there are so many, its tough to know where best to dig in.
I recently rambled through an ex sysadmins office, there were dozens of books, lik literally dozens, on network, tcp/ip etc.
I didn't understand why he kept all of them around back then, maybe there was much more written into these books other than what the author had put there.
Loved this video, Can't wait for the follow ups to come
Damn Wendell, love what you had to say in this video. Hope I can be apart of this. I think digital annotations are huge and missing from so much of what I consume
It wasn't rambling. We get it! It's true! Looking forward to the series :)
I love this project. I've been seeking ultimate personal knowledgebase + other content organisation for decades. I'm still searching and changing things, because, as you suggest, nothing works well forever. I have found immense value in projects and publications like you're beginning.
I'll be following and hopefully contributing. 👍
Fantastic insights on this one. A pleasant surprise.
This video is so inspiring and I want to contribute this project in life, even if I’m too late to help out your part of the project of improving the way people can use computers. I want computers to be like on Star Trek more designed to help you understand unknowns and reference lots of data and like you said make annotations but hopefully keep them in an open format. I really really love this whole videos sentiment.
Love the reading desk lamp!
I've had an idea for a few years that I never explored, recording your desktop at all times and using machine learning to make the videos searchable for images and text on the screen. Then you can literally search your desktop memory and jump directly to the exact frame in a video where you had or worked on an idea
Wendell, is preservation through a Younger Dryas event on the agenda? 😀 I'm in. Instead of throwing books away to make space on the bookshelf, I de-spined and duplex scanned them into pdfs.
Would be neat to make some automated book scanning, probably without ripping them apart, too
Without a doubt the best thumbnail of all time.
I forgot how much I love this channel.
I just joined the forum some days ago (actually it says it was exactly one week now) and this is the first video about a topic I read there first.
And yeah it's an interesting topic... The thread led me to to a concept of ordering things and thoughts I hadn't heard before. Luhmanns Zettelkasten - this guy (Niklas Luhmann) seems to have had a very special kind of system to store all his thoughts in a box and was therefore able to write that many complex and interesting books and essays...
I've been struggling with that problem for three decades. See you on the forum, Wendell.
I love the idea for this series
looking forward to this, normally i never watch regular Lvl1, only the news.
I'm so looking forward to this.
I’m extremely excited about this!
Knuth being in the book pile definitely sparks joy. Looking forward to this series.
Great Wizard, here I am. Thanks for reading my mind and parsing my worries with solutions and great thoughts on the matter, I guess we wizards are in sync indeed. If we act well, we can preserve some part of the culture in case unspoken things happen, acts of nature, random actions that may render these so-called web services provided by the "cloud" vendors inaccessible or have its services corrupted for unknown amounts of time if not forever. Ridulian crystal wasn't invented yet and the time immemorial ways of storing data are but prototypes and ideas at this point. Thank you Wendel the "Great Wizard which acts as a Wizard Janitor for computers and data", for making this video! Have a nice weekend!
Awesome concept!
I think this is a very worthwhile set of projects. I have been around tech since the '60s and watched many, many formats come and go. Having a personally curated library is definitely the way to go. I still cannot bear to part with the many paper books I have.
BTW, the only truly durable format I have found for images is a black and white photograph with annotation on the back in #2 pencil. That has held up for over a century on some of our family history. As for digital, I have found ascii text has endured 50+ years. I have hopes for flac audio and jpeg being fairly long lasting.
Copying to new formats every 5-10 years has been important for me. Thankfully storage costs (dollars/byte) have continued to drop and reliability has gone up.
I am looking forward to contributing to this effort.
This is exactly what I've need for a long time.
This is a fantastic idea. Your final point about the rise and fall of software is well made. I personally like OneNote because of it's ease of capture in the wild, but will I be able to open a OneNote notebook in 30 years?
I doubt it, I used to love using Outlook on my old ass iPad 4, then one day they said "hey, this device stopped receiving new updates two years ago so you won't be able to even use the last compatible version of Outlook on your device". Somehow Apple seems to have the most open garden now.
same, i have some peace of mind thinking that at the very least i would be able to export everything as pdf, but yeah, one note def has an expiry date.
Love that you have "what if?" on that stack of books, really recommend it if you want something science and funny.
That thumbnail is incredible!
"No one should limit us in the realm of possibilities.
We can all play a part in innovation pressing forward."
Yup. I won't go into it, but myself and at least three others would be greatly helped by an improved ability and ease of archiving and organizing our information as well as our thoughts and notes.
Good lord Wendell you hit the jackpot subject here. So many of us are trying to organize our lives and keep from drowning in world of digital clutter. Eat your heart out Marie Kondo.
I am so happy you exist. Let’s go !!!
Seeing Wendell hold the Star Trek book now explains the Red Shirt/ Blue Shirt look on his show...
Wendell, looking forward to this series! Checkout Eben Moglen’s freedom box project, if you are not already familiar.
I would like to add for your consideration the software Obsidian (note taking software). For myself with a tangential thinking pattern, it is a note taking system that allows me to impose my own order and structure to information I need but cannot remember mostly due to the infrequency of use. I think it works with some of your goals as it is self hosted, personalized and somewhat standardized though the use of markdown. Look forward to seeing what comes of the series.
Only issue is the syncing requires subscription on mobile in a frictionless manner.
This is the most poignant and important video that I've seen in a long while. Much love Wendell 👏
Thank You Mighty Friends!
Genuinely, this is so fucking cool, Wendell. I cannot wait for more of these videos.
edit: _engagement_
I don't quite follow this ramble. It's not clear what the ultimate goal is. But I've been using a wiki for noting down a lot of different things and this has helped me see three things I want to have.
* A way to much more quickly insert photos. Especially for my hardware projects that'd be very helpful.
* A way to consistently reference books I'm reading. Currently I just write down the name of the book and page. I would want something more rigid.
* A good way to reference the notes I've made from my book. Will not always be reasonable because of DRM but
having the possibility of linking to my wiki seems great.
This is something I would love to chip in on, one Janitor to some others.
Love this idea.
Wendel that's a huge order but it's achieveable. It gets my mind kinda racing when I was listening to you. I also had a livescribe pen until my cisco instructor told me I could not use it. A few professors said no as well because it would also record the lecture as I was struggling to take notes.
I like where your head space is thinking about this. I'd like to help, I don't know how much help I could be but whatever I can do if sounds like fun. See you on the forums.
This excites me. I've been archiving for years and I'll say the upkeep and ingestion routine is becoming a bit much for me. I've been out of the coding realm for way to long personally, but if we can conjure up a great idea for platform or software that isn't out there, I'll knock the rust off.
If you do compile a lot of documentation info into one place, maybe look into something like Notion? You can even copy public pages into your own personal Notion so you can annotate and alter it yourself.
A good example is the Pokédex someone made with it, but there are a few competitors in that space as well if you want to shop around.
I have mad respect for you Wendell.
Looking forward to this series, although I got distracted when I saw my own face appear at 3:14. Does this count as an official appearance on L1? (Thanks algorithm, LOL)
Love these videos. Wendell will be the light that leads us out of the darkness.
Big fan of using the org mode format. It's like markdown on steroids but everything is still fundamentally just plain text. When partnered with things like org-roam and org-roam-ui it becomes an even more powerful network of information for me to capture everything.
I hear this and think that we need to build some good OSS software to help with some of these issues. I'd really love my phone to automatically upload all the photos I take to a NAS backup for example.
Thank you!
I'm very interested in the subject, heading to the forum.
Greatest mind expanding book trilogy: The Reality Dysfunction - Peter F. Hamilton
There ought to be plenty of projects for image recognition for text. It would be cool to get somewhat usable solution for digitalizing paperback books that is FOSS (notes being separate for a start).
Appreciate the subtle plug for xkcd
yes! awesome project Wendell!!! I have been trying to write a tool for organizing things/information from various sources (books, audio, video, data, ..) and write documentation. It's not usable, really it's horrible, but I learned a lot from my previous tries. I still want to write that software, although with a more narrow scope
(feature creep is a thing ^^). In the meantime I'm curious what everyone else comes up with and which tools allready exist. Sometimes it is hard to search when you don't know the right words to look for.
On another note: At Uni and at work I got really frustrated with bait and switch software, tools that disappear, changes to subscription models, changes to software features that I didn't want, ... All in all I started realizing how little controle I have over huge parts of my life. I understand the rational from companies, but these changes for the most part are not in my interest. I started changing software I use to open source alternatives and started contributing (money and time) and I started to gather what I call my digital toolbox. Wendell, your videos and the Level1 forums where a huge part of me becoming convinced that we all can have that control back and of me starting to learn and contribute in this space. Thank you!
Long time watcher first time engager… this is super interesting please follow up with some more of this… in my professional career in IT at an MSP… the constant battles with good documentation of clients was an ongoing issue and I feel this is a bit of a personal extension of it, I have moved out of IT but I still am looking for a good way to brain dump and quickly reference material on the run
Tech and knowledge meets philosophy, next episode: tech religions (AKA linux distros) :D
loving your work as always but this is a special topic, looking forward to getting stuck into this
I love this!
Thanks!
Used to sell em at Tiger Direct back in the day!
Brilliant words.
Thank you
Love this
I also have that Steve Wolfram book.
This is a topic which needs a great software solution. For me the big target is scientific papers. Reading a paper you want to take notes, link code and data, categorise but also find references and similar works. That information and analysis is critical to producing new knowledge at the bleeding edge and should be open, free and available for decades or hundreds of years. It's a huge UX problem and solving it would directly improve science.