Markov Chains and Text Generation
Vložit
- čas přidán 4. 08. 2016
- Markov chains are used for keyboard suggestions, search engines, and a boatload of other cool things. In this video, I discuss the basic ideas behind Markov chains and show how to use them to generate random text.
My code to generate text: github.com/unixpickle/markovc...
My code to generate line drawings: github.com/unixpickle/markovdraw - Věda a technologie
Really nice explanation, great video
what I would like to add to 10:18 is, that using longer word sequences also has disadvantages. the longer the sequences, the less likely it is for them to appear in the training text several times followed by different words. word sequences that occur rarely would have a probability of 100% for a specific following word, and so in the generated text it is totally deterministic by which word this sequence is followed. the longer the word sequences are, the more sense the generated text makes, but as well it loses its variability and in the extreme it will just be exactly like the training text.
Amazing video and very well explained. I've been interested in Markov chains for a while now, and this was really good!
This guy is genius.. combining theory with application. Worth subscribing to.
Loving the direction your videos are taking mate. Good job!
Very smart! Keep the videos coming I enjoy them a lot, very interesting stuff!
You are so cool mate. Great Work. The world needs more and more creative people like you.
Woow, Nice explanation. Thank you i really learn about markov chains from this video.
Really amazing and very well explained!! Cheers from India
Nice little ending
great video as always !
I am wondering if you teach an online course related to this topic on UDEMY or Cousera? Truly I like your way of explaining things!
Thats pretty cool! Is there a way this could work for musical patterns as well? How would you be able to integrate the music into the program?
This is actually amazing! Take my like & subscription!
Cool explanation! Really appreciated the graphics that went along with it. An interesting addition to markov chains comes with including 'actions' to get markov decision processes. Where performing an action changes the probability of going from one state to the next. Throw in some rewards for state transitions and you've nearly hit reinforcement learning territory!
great video !
for the last example with pen strokes. if you had made it non-markov with an LSTM you can maybe really generate convincing characters. very fun idea!!
Alex Graves did this in a really interesting paper. The results are cool! arxiv.org/abs/1308.0850 (see page 25).
How do you generate the first word? Isn't there an initial distribution ?!
wow, incredible, one question, how do you get all this wikipedia stuff for the computer learning it?? thank you
I suppose downloading the html and using a script to extract the tags. or using a bot to datamine the sites. I guess there's a tool for it somewhere online.
Hold on, let me parse that phrase...
"Pitchers can also refer to translation to machine code" => Pitchers (the vessels) or pitchers (the plants) are another way of saying that we're compiling source (assembly language, probably) into the machine code that goes into an executable file. In short, pitcher plants can also be used for compiling source code.
"For correctly rounded when converted to Islam in the material" => But if you want proper rounding (like with floating-point numbers), the source code needs to be converted to Islam first, not MIDI or PDF. (Refer to the meme where searching "how do i convert to" leads to Christianity, Islam, and PDF as the next word in the sequence; I hear predictive text is based on some sort of Markov chain, so that's interesting.)
If you want to pitcher your code so that it works best with floating point numbers, you need to convert your source files to Islam first.
Will you consider to do some GO lang tutorials? I think Go is rather new and still not many learning materials out there. It will also be great for people who want to understand your code.
This is a great idea. Since I have been doing all my code in Go, I think it would be fitting for me to make some tutorials on it. :)
what's ur reason for choosing Go? Especially considering using it for machine learning.
For the past few years I have used Go and JavaScript for virtually everything. Thus, I wrote this ML stuff in Go so that, if I made some useful machine learning APIs, I could easily use them natively from other Go programs. I can also use my Go stuff on the web using GopherJS, which is a fairly reliable and fast way to compile Go to JavaScript.
For the time being, it may still be a problem of personal taste. (Especially when talking about building programs with rather small scale.)
But that's also why I hope macheads101 could do it: still very few programmers would code these things in GO but Alex did. It should be interesting to see why he likes GO, and how he coded everything in GO.
+1 subscriber
You still active?
11:25 You say you "trained" a markov chain, I'm not being critical, just asking for clarification. Do you mean you just built a markov chain? When I hear the word train, I think of backprop and gradients
training is used in a very loose sense. Anytime a model with parameters gets its parameters set to a set of values based on some dataset it is training loosely speaking. Making a n-gram model is also "training".
I put 500th like
converted to islam