Lecture 23: Computational Complexity
Vložit
- čas přidán 28. 07. 2024
- MIT 6.006 Introduction to Algorithms, Fall 2011
View the complete course: ocw.mit.edu/6-006F11
Instructor: Erik Demaine
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu
this classroom has a nondeterministic amount of chalkboards
good one!
Really!?? :p
NP-eed!
The classroom has an uncountably infinite number of chalkboards
funny enough, but in USSR some universities used exactly same board setup. Is it ancient Greeks or Dutch, came up with this moving boards?
01:10 three complexity classes P, EXP, R
11:02 most decision problems are uncomputable
19:12 NP
19:43 NP as a problem solvable in P time via lucky algorithm
26:07 NP as a problem which positive result can be checked in P time
31:00 P = NP?
37:50 NP-complete, EXP-complete
40:35 reductions
"you cant engineer luck". Absolutely loved it. Thanks a lot :)
con hod.|||||||||||||||||||||||||||||||||||
This lecturer is the man!
Professor Srinivas Devadas, who thaught other lectures of this same course is great too.
I love that they still use the chalk board. All of my lectures are with power point presentations and very little use of the chalk board.
I'd prefer chalkboard over boring power point slides.
The best is to use the chalkboard for teaching and explanation, and PowerPoint presentation for just visual information.
I just love MIT video lectures
Very nice lecture! Clear explanations of everything and very easy to understand
Wow look at that hand held self powered calcium deposition printer, it's like the future.
+Seán O'Nilbud this made me lol XD
All the cool kids have them.
most people use whiteboard nowadays..
Finally I'm able to understand NP complete. Awesome lecture, thank you so much 🙂
i dunno why but the chalk thumping the board is actually kinda soothing....still thanks for the lecture
Way more in depth than I needed for my exam but couldnt switch off.
helped me get an A+ in tst.. so my verdict is that it's actually awesome
Thank You so much sir for such a great explanation .
You are a great teacher Erik!!!
Interesting to see how a MIT guy has got time to write stuff on blackboard. My uni lecturers do not even know what a chalk is.
*clicks through powerpoint reading out the slides*
This was so brilliant! Thank you so much 😊
Fun fact: He is the youngest professor ever hired by MIT.
Age?
@@Gulag00 20
Sakeeb Rahman jeez
he literally enrolled at college at the age of 12
@@srn306x I think he might be smart
Just one thing I’ve noticed, the halting problem is undecidable, running the program for which you’re trying to determine if it’ll halt or not doesn’t really solve the halting problem itself. Great intro regardless of that hick up.
When he put the decimal in front of the decision problem table to turn it into a real number, my mind was blown. When he said that one can view programs as natural numbers, I expected him to somehow represent decision problems as real numbers, but I didn't now how. Goddamn.
Algorithms are usually functions. Functions have domain and range. Some decision problems can be such that they have finite domain, others don't.
So, The Meaning of Life, the Universe and Everything is in which set?
"You can't engineer luck" - wow this is the best explanation I have heard regarding NP
25:38 I’m pretty sure that you can force a death every time by just … doing nothing and let all the pieces stack up in the middle. No way to clear lines, because no piece spans the entire width. Therefore, it will halt in linear time w.r.t. the height of the board.
Unless you dont have enough pieces
I never understood this topic in school....will watch later to gain some insight. Hope it's good! :)
what was the exception @2:00? Had they done a sudoku solver?
This was a great lecture !
love it
I knew, I would find you here. When does the next episode of computer science comes?
+Fatih Erdem Kızılkaya funny you ask, i'm just about to render the next video - will post tomorrow!
+Art of the Problem Great, looking forward to it. By the way I love how you make complicated things simple and beautiful. Please keep doing what you are doing.
which lecture is the implementation of this lecture
this episode is the most important for understanding the basics of complexity theory :O seriously.. why is this not 1.
Great lecture! Thank you
is the numbering of computer programs related to Gödel numbers?
You cannot solve longest path problem with reduction and Bellman-Ford Algorithm because there may be negative cycles that can't be handled by Bellman-Ford Algorithm when you negate the weights. The longest path problem is NP-hard.
19:19 "Out here Nothing happens"? Isn't it where everything happens and we have no knowledge of how they happen?
You know the amount of different sudokus out there would it not be the number of ways you can lay out 1 to 9 without repeating multiplied by the number of permutations possible when the first columb is filled 1 to 9 from smallest to largest.
enjoyed the lecture very much
thanks for posting
I do know a problem that's worse than EXP, quanitifier elimination on the theory or real numbers is the classical example. this is EXP_2, double exponential time.
Eric you beauty... Superb teaching... :)
pudiera medirse con matemáticas vectoriales como la de los poliedros en otra di mención multiplicado por el numero de posibles cara frac-tales dividido por el posible tiempo elevado a la masa del objeto. ¿?
I could watch Erik Demaine talking about math all day long. I don't understand anything but it's like music to my ears.
Wow man I think I love this guy
Very well explained, 100 times better than my prof
Thank you! Great lecture!
I have a question (at 18:25)
Given infinite space, you can write a program for every problem by hardcoding the solutions
Just going
if input = 0, output = 0
if input = 1, output = 0
if input = 2, output = 1
if input = 3, output = 0
if input = 4, output = 1
etc.
Just do this for every problem.
Wouldn't that mean every problem in R is computable?
(I'm obviously wrong, but what am I missing here?)
Great question dude, i also thought the same, but when i look it down deeper, i think than R is a range where we'd probably say that this problem is impossible to solve so maybe thats what it means, but this is yet a very deeper topic to dive in, and i'm obviously wrong here too.
Can anyone explain to me please why since there is a 1 to 1 relation between problems and decisions he said that there are way more problems ? I just did not get that... :(
I LOVE YOUR VOICE
Is it possible to get a better quality of the videos, I really want to watch it in like 720p or 480p
why not?
39:48 - he says we know EXP != P and then he proceeds to say that proving NP = EXP isn't as famous as P = NP problem and it won't get you a million dollars. My question is: Doesn't proving NP = EXP prove P != NP as well? On the other hand I know proving NP != EXP does not prove P = NP, however I still find his wording a bit weird.
He doesn't say we have proved NP = EXP, in fact explicitly says we haven't. And there are problems that were once NP that have been discovered are, in fact, P (see primality problem), so saying *NP = EXP != P, so therefore NP != P* is just giving up.
yes, would answer both questions and get you the prize
why is Go not exp complete? GO have way more possible moves than chess no?
When you're dealing with the infinite, every single finite thing is the same size: nothing.
Also, might I add that words can also be represented by integers, therefore most decision problems cannot be written.
Amazing lecture
Hmm. One question. why can we assume that each program is only capable of solving 1 problem?
"I am NP-Complete" :D
While theoretically true that most (almost all, in some sense) problems are not computable, all of the problems that people encounter are in some way relatively small, in the sense that they can be understood or contemplated, i.e., they are dependent somewhat on the intelligence of people for determining the knowability of the problem, i.e., even recognizing that the problem exists, whereas most (again, almost all) of the noncomputable problems are larger or more difficult than what people can understand or conceive. So saying that most problems are noncomputable is a red herring. How many and which problems that people can recognize or understand are computable?
Excellent video.
Wouldnt the shortest point in 3d between 2 points be super easy. All you would have to do is put those 2 points on the same plane. Or is there some magical path that you can follow that is outside any 2d plane which is shorter?
Exactly. You could have your destination be completely inaccessible on that plane.
Great lecture.
Oh great Erik has become a confident lecturer now!
+Arya Pourtabatabaie His name is spelled Erik. Just being a grammar troll. :)
+Dawn Lassen Fixed :D
can someone explain to me what studying is going towards?
what is an example of a problem that is in (EXP - NP)
Wow, 17:15 blew my mind :D
Hello Prof. Erik Demaine,
is the RSA algorithm is in P or in NP ?
Thank you in advanced,
(I know this is very late) Well from my understanding if it would be in P, almost all our ways of communication online would be unsecure. RSA relies on modular arithmetic with uncertainty to be NP. This way guessing the right key would take EXP time as the mod N gets bigger and bigger.
The act of encrypting/decrypting a number using RSA can be done in polynomial time if you have the private key.
The security of RSA encryption rests on the belief that integer factorization is a computationally hard problem (integer factorization is inefficient if time complexity is measured in number of bits, e.g. it can be inefficient to find the factors of a 1000-bit number).
However, integer factorization is actually believed to be in NP-Intermediate (a problem that's in NP \ P, but not NP-Complete). In the context of the diagram, it's believed this problem comes strictly after the notch where P ends but strictly before the notch where NP ends.
Why does one reduce non-deterministic to guesses or it's just an explanatory trick?
b/c if an algorithm is non-deterministic then there is no way to know the answer before you run the algorithm. that's kind of like a guess. there is no way to know what the answer to a guess will be before a person makes a guess. with a deterministic algorithm you always know the answer before you run the algorithm. for a given input, you get the same answer every time you run the algorithm
I don't know why I'm here but I'm only in pre-calculus and this seems interesting
Very GOOD! Thanks a LOT!!...
give an example ?
What are they trying to find?
hmm is the idea that lucky algorithms aren't realistic still valid given quantum computing algorithms?
Great talk.
so solving for pi is not in R.
I interpret that at 33:29 you just said QC is easier than development.
at 25:40 he say "can I die" should not be in np, but that's solvable in O(n) (worst case), which makes it a P problem, shouldn't that make it an np problem as well?
Andreas Kristoffersen Yes you are right. the professor did not state the answer correctly. "Can i survive [this series of tetris pieces]?" is NP. Since we can guess a series of moves, and if we survive, we stop execution and return true. "is there no way to survive?" would be the complement of "Can i survive?" and this would NOT be in NP. Since you would have to check all possible combinations of pieces, and prove that there is no solution that would make you survive this is called co-np.
"is it not possible to survive" is not the same as "can i die". "can i die" is in NP, since you can find a solution that you die from in polynomial time. "is it not possible to survive", you cannot guess a solution, and must check all possible solutions so this class of problem is in "Co-NP".
hope this helps.
why is @Jeb_ in video xD
What about other classes, I want to know about #-P Complete, can you please provide something on this???
Maybe check out the course he mentioned. I believe it was 6045? Somewhere in the video he mentioned that this is just a 1 hour taste of what these people do. Or you could also youtube it with your keyword
Check out complexity classes on wikipedia for all of them
The proof around 16:00 is ridden with flaws. For example It’s true that 0.00110010111… is just one number in R but to then say the whole of R is much bigger than N is irrelevant.
this is really interesting, but does anyone know where i can find the paper to the proof that most decision problems are not solvable. Have some queries in my head that perhaps reading the complete proof will help!
Check out the notes from the mitocw website they write it down
@@sriyansh1729 thank you for the reply! but it has been 11 months since i asked the question so I'm not even sure what I was asking about haha but will have a look at the notes
LOL
if a problem takes N*log(N) time to resolve, is it in the EXP or in the P category?
+TheSymetrie77 It's in EXP but it's also in P. N*log(N) is O(poly(N)).
N*log(n)
so is this really a question sort of about proving the transcendence from one state to another if the first is not a subset/lacks properties for existing in the second like if there is a state between finite and infinite existence where functions of time can cross between? or like a "tunnel" to travel through both?
Why is p fixed in polynomial time? Why not linear time?
Linear time is contained within polynomial time. Like the expression 2x +1 is technically a polynomial and also a linear expression.
Compared to French course.
It's very different.
It's less abstract : no formal proofs, no maths...
I find it really interesting and really cool.
But I wonder if it's enough accurate to have a full meaning of the topic.
Can you tell me the name of the French Course? How to get it?
4:20 Wow, someone really did clean that chalk board.
Good teacher!
hi! it has been 9 years, are u still there?
BOINC distributed computing science projects are P or NP problems right? #BOINC
+James D Correct. Some of them are P problems (prime search, SETI, etc...) and others are NP (protein folding)
Thanks for answering my question. :)
On the subject of solving the P versus NP problem, instead of trying to solve an NP-complete problem in polynomial time, why does no one take the approach of trying to prove that a problem that we already know is in P to in fact be NP-complete? Of course, I'm assuming that P = NP.
I couldn't agree with proof for amount of decisions. On first step it composes a real number out of all decisions (fine), on the next step it start talking about cardinality of real numbers. For uncountable amount of decisions you need to show that there is more than one such 'real number' string of decisions. And I see that he depleted all his infinities on writing down all possible decisions.
How can you solve the problem "will I survive playing tetris with a lucky algorithm?"
To me this is one of the problem in not in R because to understand if you survive you have to play for an infinite time.
When Tetris is introduced, Demaine stated that the list of blocks you are going to play with are given. So with that (finite) list you have to check somehow (lucky guess, for instance) if there is a surviving strategy.
If you prove that NP = EXP, and we know that EXP != P then we also prove that NP != P. So you would get the money! :)
+Rodrigo Camacho True, but perhaps he's betting that if a proof regarding NP and EXP is discovered, it will only prove that NP ≠ EXP, which wouldn't prove or disprove that NP ≠ P.
Proving NP = EXP is at least as hard as P != NP, well, by reduction.
Lol u tried simple logic
Okay, but what does "polynomial time" mean - and why does everywhere I look to try to learn the answer to that question assume that I already know the answer?
Polynomial time means the time complexity (time it takes to solve the problem) can be written as a polynomial function.
f(x) = x^10 + x^5 + 3
where parameters to f are the sizes of the inputs.
Here are some example of the time complexities of NP problems:
f(x) = x!
f(x) = e^x
I appreciate the effort; without contextualizing x, though, your examples don't mean much. I have since come across a source that explained it in a way I could understand, specifically because it gives the single most important piece of information in understanding this model, that I could not find anywhere else.
I knew what a polynomial is. What I didn't understand was what was meant by time as it relates to complexity: specifically, that complexity measures time in calculations.
notoriouswhitemoth
Martin already contextualized the "x" variable: it's the size of the inputs. I'll give it a quick try and then point you to the link at the end which should be helpful.
Since you already know what a polynomial and a function are, when we say that the an algorithm has a "time complexity" of some polynomial function, say n^2 + 5n + 7, where "n" is the "size" of the problem instance, this describes the "order of growth", or how quickly the problem and computational effort to solve it grow in relation to the size of the input.
In plain English, if the function above represented the time necessary to get a list sorted in alphabetical order, then "n" would represent the number of items in the list you want to sort, and the result of evaluating the function would be the number of computational steps that would be necessary to sort the list using said algorithm.
Clearly, you can see, even without running the algorithm, that a list with 100 items will require less computational steps than it would if you wanted to sort a list of 1,000 items. But with a polynomial function that tells you how an algorithm will 'behave' based on the size of the input, you can get a more specific idea of what this difference actually looks like. And this is what would allow you to compare this algorithm to another algorithm that is also meant to sort lists and decide which one should require less time to complete as the size of the input keeps growing.
Lastly, in Big-O notation, the number of terms is reduced to only the most significant/dominant term for simplicity, because this is meant to be an approximation for when the size of the input, N, becomes extremely large (i.e. tends to positive infinity). This means the function above would be expressed as O(n^2) in Big-O notation.
In any case, hope that helped. I still recommend you read this post:
stackoverflow.com/questions/487258/what-is-a-plain-english-explanation-of-big-o-notation
@@notoriouswhitemoth The complexity of an algorithm is how fast it grows depending on the size of the input (n). If you like, you can think of this input as a list of numbers. Exponential time algorithms double (for example) every time you increase the input size (n) by 1. Polynomial time algorithms grow much, much slower than exponential time algorithms relative to the input size, and this is particularly noticeable (and relevant) when n is very large.
You can think of exponential time problems as problems that take too long to solve with a large n value to be of any use, and polynomial time problems as problems that are generally solvable in a reasonable amount of time, even with a large value of n.
That's the simplest explanation I can give without going a lot deeper.
If we could consider relativistic effects time during algorithms execution (E.g. observer is very very close to a black hole's event horizon, but computer is far from it), might we say that NP has its limit to P? In other words is it possible to find such reference of frame in space-time?
Whoa that's a crazy idea. I don't have a clue
Game solvable in exponential time is best!
I am NP complete but with result no.
Is there such a thing as R-completeness? If so, what are some examples of problems that are R-complete? And what would that mean, exactly?
Qubrof I don't think there is such thing as R-completeness. Put simply, as shown in the graph, you have some point to which you define a class of complexity, for example for P class: You have a point greater than which you define next class of complexity called NP, so anything on the point is an intersection of P class and P-hard class. But since the class R extends to infinity, i.e., there are not any class beyond the R class, there must not exist R-completeness.
Put it in the other way, we define X - hard (X being any one of the class) as the set of problems that are at least as hard as every problems in X class.
Now, let's assume we have R completeness, Now R complete = R (intersection) R-hard
Now, R-hard is the set of problems that are at least as hard as every problems in R class, i.e., at least as hard as the hardest problem in R class. Now, the hardest problem in R class in not defined as infinity. That's why R completeness must not exist.
PS : That was just the way i thought! I don't know if there are some R completeness but from the current logic and intiution that i have got, it seems that it is not posible!
Nootan Ghimire But there is actually a class beyond R, it is the class of R-hard problems, and this class is not empty since it contains the halting problem, which is not in R.
So it seems quite reasonable to define a class of R-complete problems. Such a problem B belongs to R and has the property that for each problem A, if there exists any algorithm to solve A, then A reduces to B in the appropriate sense. Vaguely now, this may be related to the concept of a universal Turing machine.
Edit: actually I suspect tetris is an example of an R-complete problem. It certainly belongs to R. And it is complete for R for much the same reason it is NP-complete: if you have any problem in R, then it reduces to tetris. It does not always P-reduce. But it reduces in the appropriate sense.
39:30: Proving exp is at the same spot on the line as NP would prove P=NP, right? If we know that P is not equal to EXP that is. Then you would get $1M
Just awesome:).
This lecture states theorem "most decision problems are not solvable by program". The funny guy Erik forgot to clarify the condition of prove: "solve with with infinite precision". That is what makes this statement "depressing". In my view most practical problem have bargain about precision of solution, that is how the program (from N) solves any real decision problem (from R) => life is full of practical optimism! It is all about point of view on practical need in "precision". But that is a topic for philosophy class. Chin Up Canada! ;)
((Great lecture! I was just wondering is reducing 3-sat really proof that Tetris is NP-complete? If it proves Tetris is atleast as hard as 3-sat and hence NP-hard wouldn't you still need to show that Tetris can also be reduced to some NP-complete problem for it to be proven NP-complete? I guess this might be obvious to someone who's not a layman. Still I'm curious.))
I realise I need to clarify my question.
It's my understanding that:
1. 3-sat is NP-complete.
2. 3-sat can be reduced to Tetris which shows Tetris is harder than
3-sat.
3. Since Tetris is harder than 3-sat, Tetris is NP-Hard.
4. NP-complete problems are NP-hard problems "contained" in NP.
5. All NP-complete problems are hard but all NP-hard problems are not
NP-complete.
Based on these assumptions what I meant to ask was.
"If Tetris is harder than an NP-complete problem Tetris is NP-hard, but
wouldn't Tetris then need to be reduced to an NP-complete problem to
establish that it is NP-complete and not just NP-hard?"
I appreciate any answers I may get. I wouldn't keep on pursuing these
questions if the answers weren't valuable to me and I appreciate the
patience shown.
+Gustaf Smith 3-sat is in fact np complete.
+TheFabus95 Thanks for the answer. My question was a bit unclear since I don't fully understand the concept. It didn't really clear up what I was interested in knowing but my question may have been redundant in the first place. I'll try to rephrase my question.
It's my understanding that:
1. 3-sat is NP-complete.
2. 3-sat can be reduced to Tetris which shows Tetris is harder than 3-sat.
3. Since Tetris is harder than 3-sat, Tetris is NP-Hard.
4. NP-complete problems are NP-hard problems "contained" in NP.
5. All NP-complete problems are hard but all NP-hard problems are not NP-complete.
Based on these assumptions what I meant to ask was.
"If Tetris is harder than an NP-complete problem Tetris is NP-hard, but wouldn't Tetris then need to be reduced to an NP-complete problem to establish that it is NP-complete and not just NP-hard?
I appreciate any answers I may get. I wouldn't keep on pursuing these questions if the answers weren't valuable to me and I appreciate the patience shown.
You are correct: If we reduce 3-sat to Tetris and we know that 3-sat is NP-hard, it follows that Tetris is NP-hard, but not necessarily NP-complete. Think of reducing a problem to the halting problem, which is also NP-hard.
But we also know that Tetris is in NP. Thus, Tetris is NP-complete, since the NP-complete problems are those who are (1) NP-hard and (2) contained in NP.
However, note that reducing Tetris to a NP-complete problem L does not prove anything about the NP-hardness of Tetris. The fact that L is NP-complete means that every problem in NP (like Tetris) is reducible to L.
I hope this answers your question.
+TheFabus95 Thanks again!
This really cleared things up for me. Especially you pointing out Tetris being in NP. Somehow I had missed the whole proof of Tetris being in NP as a part of the NP-completeness proof.
I don't know how I managed this since he even says "therefore Tetris is in NP", and it can't get much clearer than that.
Much appreciated!
brilliant lecture
+vishnu karthik hi dude :D
Haha hi man. It's a small world
16:00 makes no sense. But then again, maybe the proof is left as an exercise for the reader
i agree, if he arbitrarily puts a dot at the start of the numbers in the set of inputs to make it a real number between zero and one, he could just as easily do that to the numbers in the set of programs and make it a real number between zero and one.
How is shortest path in 3D different to 2D? o.o
I CLAIM THAT A PATH IN 3D IS THE SAME AS 2D WITH MORE VERTICES
very interesting