Hey folks, small bit of advice - stop using seasons to talk about timeframes. It only applies to half the world at best. Q1, Q2, Q3, Q4 is a super easy substitute and is not ambiguous. Thanks very much for your work; mad respect.
When I started ML, I really wanted to learn PyTorch because I thought it would be more intuitive. But I stuck with TF because good things happen when you have 10x the number of engineers working on one thing ...
The major problem is yet to be touched in at least tf.keras. u can't retrain a slightly different model with ease. Easy reloading of model for slightly different graph is the need of the hour in research.
"No questions asked": An expression indicating that one will not be questioned or hassled about something, typically as an incentive for sharing some information or doing something that otherwise may be the subject of suspicion, further inquiry, or punishment.
I'm not a computer science person but I love watching videos where I understand the language spoken and still have no idea what is being said. A bit like listening to politicians talk I guess.
I really hope TF2.0 get better. The previous version is very difficult for beginners to enter. I think it was really good to have developed it with the previous version. TF2.0 wants to include many features such as user error alerts.
I heard that the reason why they don't use numpy is because numpy doesn't run on a gpu. I haven't confirmed this but it wouldn't surprise me if it was true.
i'm new to tensorflow, i have never touched machine learning code, but I have a question, why the activation functions are defined as strings? would it be easier if they where strongly typed?
They are. You can write them as strings or as something like tf.python.keras.activations.sigmoid The strings happen to be a bit better for documentation especially when you repeat that stuff over and over again. There are some times. Where From -- import -- Just doesnt cut it because of the overlaps to different functions
The argument you are talking about is strongly typed, because Python is strongly typed. The type is `Union[str, Callable[[Tensor, Tensor], Tensor]]`. The string dispatch refers to built-in loss functions, otherwise you have to pass a propper Callable.
It's amazing, but these updates are somewhat panic for those who are already working in such a big project/research. As today I faced many problems in Keras 2, I installed Keras 1.1.0 then it worked 😁
I may not be getting something, but saying that the goal is user-friendly interface and then using pip instead of (or not providing) conda version isn't that making half-steps?
How do I replace the function tf.contrib.seq2seq.prepare_attention method in TensorFlow 2.0? I want to make a Recurrent Neural Network and LSTM, with encoders and decoders, but this: "tf.contrib.seq2seq.prepare_attention" is deprecated
I’m a die hard PyTorch coder. Tensorflow 2.0 is practically Keras (user friendly and not practical for research) and PyTorch can use C++ inference more quickly compared to tensorflow. I recommend using wandb for hardware usage comparison. Tensorflow released 2.0 because they know they lost against PyTorch 1.0 to develop complex algorithms. Tensorflow 2.0 is recommend for people who wish to make production faster, and possibly unstable for lack in hardware control, which is okay but lacks intuition in machine learning. It’s like comparing angular 1.0 --> 2.0 to react js...we all know it this ends. The only thing Tensorflow won is marketing and the framework is already open sourced.
Very interested in getting TF 2 ported to the new architecture of Lucata(Semi-stealth) The new architecture should change the cost performance paradigm. Need folks that are interested in helping. Love this as it seems like all the attention is on DNN.
Great for prototyping but after that you need to get it in production which is really a pain in the a**, even with these new improvements, you should really start focusing a bit in that direction too.
We used it in a production app to do image recognition in Unity. Was super hard and we had to invent processes to get it deployed. Really glad they're focusing on usability now.
On one hand my codebase becomes much cleaner now and I highly appreciate the debugging capabilities; On the other hand: It feels like Google's Angular Framework (for web dev'): Easy things are easy to setup but if you go slightly apart from toy use cases you have to do hours of API research before you really understood the concepts that are not well explained in the docs. I started right now with TF 2 (did tf1 and pytorch + some spiking simulators before) and it feels exactly like this again. Remark on the presentation: I personally dont like this extreme branding (tf-shirts, tf-colors on the wall, a text that sounds like some advertising guys wrote it). Its better to let the devs speak in a way like the c++ gods on the c++-cons
If you use Visual Studio Code, you can download themes for its GUI / Code highlighting. I personally use "One Dark Pro". If you copy and paste your code into Microsoft's Word/PowerPoint or Mac's Key Note, it reserves the styling and also looks amazing.
I didn't get it, he says Distributed computation, Optimization on the graph aren't going anywhere with Eager execution. But afaik there's a tradeoff with those things if you don't use static graphs, the performance usually isn't as good on huge, complex models. Has Google managed to find the best of both worlds ?!
It's a wild guess, but my understanding is that they do construct a graph under the hood, however also carry around a function to eagerly execute everything. So it's a bit of a hack that makes it work eagerly when you want to see it, and not eagerly when you're not trying to see it. Effectively there are two ways of evaluating everything implemented, however they do return the same results. One is fit for debugging and the other is well fit for optimization.
I was wondering about that too. A couple of points come to mind. First, some people might want a normal static graph, but the ability to step through it with a debugger. Then, after debugging, you could run it more optimized and in non-eager mode. Second, in certain RL setups for example, you might have multiple actors, each running on a single CPU core, where each actor has a (simple) batch size of one. Now, if the graph is dynamic in this case, it may do, for example, ABBA(x) or BBAA(x) or something else, where A and B are "blocks" of layers, and each of them is separately optimized on the CPU core.
Haha It's no any problem, in these days everyone is dependent on others. On CZcams Microsoft official channel, in Play Store Microsoft apps. Collaboration is key thing in the companies
Does anyone actually have a real life example of a useful regression model or classifier? Most hard problems today need super high accuracy for long time periods of time, and other problems want to know "Why" a certain outcome was predicted, which usually is pretty difficult without domain expertise. Amazon store doesnt count, they can afford false positives.
Look, when i try this: print(tf.add(35, 2.5)) python returns an error: TypeError: Cannot convert provided value to EagerTensor. Provided value: 2.5 Requested dtype: int32 but if print(tf.add(2.5, 35)) then all ok: tf.Tensor(37.5, shape=(), dtype=float32) what the heck?
it's because tf.add takes two arguments, "x" and "y", such as: tf.math.add( x, y, name=None ) and as exposed in the documentation "y" must have the same type as "x", so you can use a int in the "y" position if "x" is a float, but you can't do the opposite.
Agreed pytorch is now running this race on one leg... But are we sure this is where we want ML to go? It didn't exactly pan out for three decades of MS ruling the OS market now did it? Stifling innovation is never a good thing. We are passengers again folks...just passengers ...
Won't lie. Been using PyTorch but now super tempted to go back to TF and Keras.
Hey folks, small bit of advice - stop using seasons to talk about timeframes. It only applies to half the world at best. Q1, Q2, Q3, Q4 is a super easy substitute and is not ambiguous. Thanks very much for your work; mad respect.
Google frameworks in general are always most complicated in market. Happy that keras was integrated.
No sess.run() anymore?
Guyz, this is huge
finally
Yeah and they just mentioned it offhand too...like as if its just a 'meh'...
rofl hahahaha
I genuinely don't know whether this is a sarcastic remark or a genuine one aha
Well, TF's eager mode has been around since 2017. It's just no one seemed to care for it much.
Congrads. TensorFlow can finally work like PyTorch!
And TF getting better n better !
*indeed*
Really interesting! But I have a question. Do you guys have a date for when TensorFlowi will be compatible with Python 3.7? Thanks in advance!
This TF 2.0 seems amazing! Making things hard for Pytorch.
But, what happens now to Keras library (not tf.keras, just keras)?
Keras developer François Chollet got hired by Google to work on tensor flow and Keras. So nothing bad happens.
Alberto Blanco Garcés separately maintained and merged to tensorflow
Keras just got integrated into tf.keras and that tf.keras is pretty powerful than normal keras.
Sounds good and all, but I don't really see a reason to change back from PyTorch
Where can we find the video for "2.0 and Porting Models" workshop mentioned at 7:55 ?
I like to watch this stuff because I figure someday I'll understand it.
When I started ML, I really wanted to learn PyTorch because I thought it would be more intuitive. But I stuck with TF because good things happen when you have 10x the number of engineers working on one thing ...
lol
Absolutely!! They never stop making it work, even when its working...
In the past couple of years, I couldn't find any researcher who hasn't switched to pytorch. Not sure whether this will make a difference.
Yeah, pytorch seems to do everything that's mentioned here already.
The major problem is yet to be touched in at least tf.keras. u can't retrain a slightly different model with ease. Easy reloading of model for slightly different graph is the need of the hour in research.
"No questions asked": An expression indicating that one will not be questioned or hassled about something, typically as an incentive for sharing some information or doing something that otherwise may be the subject of suspicion, further inquiry, or punishment.
3:13 this is amazing
Finally! TF 1.x was not easy to use, this seems like fun, can't wait to build with this.
where is the colab file for this presentation ?
Perfect timing to start my journey =)
Same here, good luck!
I'm not a computer science person but I love watching videos where I understand the language spoken and still have no idea what is being said. A bit like listening to politicians talk I guess.
Wtf ahaha👍
Nice for when you want to fall asleep.
hahahaha
Actually kids learn this way. When we are born we know nothing - a brain is just a clean slate.
Nice thought
Does it support python 3.7 now?
I tried it and it worked with 3.7
And ATI grafic cards
This has been needed since... well, TF 1.x. Super excited.
Thank u for this awesome video
I really hope TF2.0 get better. The previous version is very difficult for beginners to enter. I think it was really good to have developed it with the previous version. TF2.0 wants to include many features such as user error alerts.
this is great. this is amazing. great work guys. can't wait to get my hands dirty with tensorflow 2.0
why does the script rename `tf.enable_eager_execution` to `tf.compat.v1.enable_eager..`? Doesn't v2 just enable eager by default? @7:30
Great presentation .Pytorch's getting all tensed up
I wish they use NumPy rather than their own as their data container
Seriously! OpenCV already uses NumPy and this would make the experience so much better.
I heard that the reason why they don't use numpy is because numpy doesn't run on a gpu. I haven't confirmed this but it wouldn't surprise me if it was true.
@@timharris72 you are right, numpy doesn't work with GPUs. PyCUDA is the closest option for this.
Pytorch tensors are pretty close to gpu numpys
Numpy is slow so they go for C++ backend
Any chance TF 2.0 will be able not to raise "Not enough memory" error, but calculate maximum number of samples in batch for us?
i'm new to tensorflow, i have never touched machine learning code, but I have a question, why the activation functions are defined as strings? would it be easier if they where strongly typed?
You can do either
They are. You can write them as strings or as something like tf.python.keras.activations.sigmoid
The strings happen to be a bit better for documentation especially when you repeat that stuff over and over again. There are some times. Where
From -- import --
Just doesnt cut it because of the overlaps to different functions
The argument you are talking about is strongly typed, because Python is strongly typed. The type is `Union[str, Callable[[Tensor, Tensor], Tensor]]`. The string dispatch refers to built-in loss functions, otherwise you have to pass a propper Callable.
Does someone has a tutorial of how to set in profuction the models that you train ?
It's amazing, but these updates are somewhat panic for those who are already working in such a big project/research. As today I faced many problems in Keras 2, I installed Keras 1.1.0 then it worked 😁
Video ends at 10:10
I may not be getting something, but saying that the goal is user-friendly interface and then using pip instead of (or not providing) conda version isn't that making half-steps?
I'm thankful that the syntax didn't change too much.
Is it compatible with all the python 3 version ? because that was a big disadvantage of the previous version
1.13 is compatible with 3.7
Simply Amazing !! I am in love with 2.0
Why is the guy not in the teaser pic?
so, will tf2.0 support python3.7?
Yes, as of 1.13
github.com/tensorflow/tensorflow/issues/17022
2.0 looks amazing! And the presentation is excellent.
I need to learn so much more
This is the peak.
TensorFlow [ Keras + Pytorch ]
Wow fantastic that's the right way for developing
Meanwhile Stanford CS224N switched to PyTorch this year
How do I replace the function tf.contrib.seq2seq.prepare_attention method in TensorFlow 2.0?
I want to make a Recurrent Neural Network and LSTM, with encoders and decoders, but this: "tf.contrib.seq2seq.prepare_attention" is deprecated
Is google promoting keras and shelve estimator at some point in future? There was no session on enhancements in estimator(or I have missed?).
Fantastic!
without static graph, gpu memory managing will be a problem.
Tensorflows new flexibility looks very pytorch. Am I wrong?
as a low level api user not a fan of keras, but happy to see the low level api change though.
It's good for user?
TY!!
I’m a die hard PyTorch coder. Tensorflow 2.0 is practically Keras (user friendly and not practical for research) and PyTorch can use C++ inference more quickly compared to tensorflow. I recommend using wandb for hardware usage comparison. Tensorflow released 2.0 because they know they lost against PyTorch 1.0 to develop complex algorithms. Tensorflow 2.0 is recommend for people who wish to make production faster, and possibly unstable for lack in hardware control, which is okay but lacks intuition in machine learning. It’s like comparing angular 1.0 --> 2.0 to react js...we all know it this ends. The only thing Tensorflow won is marketing and the framework is already open sourced.
Very interested in getting TF 2 ported to the new architecture of Lucata(Semi-stealth) The new architecture should change the cost performance paradigm. Need folks that are interested in helping. Love this as it seems like all the attention is on DNN.
Do still have to recompile, to run on older Intel CPU's?
That keras integration though
Great for prototyping but after that you need to get it in production which is really a pain in the a**, even with these new improvements, you should really start focusing a bit in that direction too.
How do I install pip 😒
You need pip to install pip
This is cool
hope datasets are not still broken
Brilliant
Tensorflow 2.0 = Tensorflow 1.0 + Keras?
I love 60 fps
Just to prove how good pytorch is. All has been done by pytorch years ago
Ouch! that hurts. ( so true ). ps: We should use the 2, no hype is good.
you mean chainer?
would love to see it working on pypy interpreter also
Can documents be shared?
Can you name a concrete result, a
product, which is created with TF
and Keras and which can be viewed
on your website?
We used it in a production app to do image recognition in Unity. Was super hard and we had to invent processes to get it deployed. Really glad they're focusing on usability now.
that ending tho
What if we want to use its low level features for research purposes? I did not like tf2. I dont like keras. I will migrate to pytorch.
“If we can do it at Google, you can probably do it too, heh” lol
On one hand my codebase becomes much cleaner now and I highly appreciate the debugging capabilities; On the other hand: It feels like Google's Angular Framework (for web dev'): Easy things are easy to setup but if you go slightly apart from toy use cases you have to do hours of API research before you really understood the concepts that are not well explained in the docs. I started right now with TF 2 (did tf1 and pytorch + some spiking simulators before) and it feels exactly like this again. Remark on the presentation: I personally dont like this extreme branding (tf-shirts, tf-colors on the wall, a text that sounds like some advertising guys wrote it). Its better to let the devs speak in a way like the c++ gods on the c++-cons
whoop whoop
What _TF_
How, Great Presentation. But I wonder how to present the code like yours, such as in 15:03, the code is looks pretty elegant.
Using python classes
If you use Visual Studio Code, you can download themes for its GUI / Code highlighting. I personally use "One Dark Pro". If you copy and paste your code into Microsoft's Word/PowerPoint or Mac's Key Note, it reserves the styling and also looks amazing.
Where can I get a tensor shirt
I didn't get it, he says Distributed computation, Optimization on the graph aren't going anywhere with Eager execution. But afaik there's a tradeoff with those things if you don't use static graphs, the performance usually isn't as good on huge, complex models. Has Google managed to find the best of both worlds ?!
It's a wild guess, but my understanding is that they do construct a graph under the hood, however also carry around a function to eagerly execute everything. So it's a bit of a hack that makes it work eagerly when you want to see it, and not eagerly when you're not trying to see it. Effectively there are two ways of evaluating everything implemented, however they do return the same results. One is fit for debugging and the other is well fit for optimization.
I was wondering about that too. A couple of points come to mind. First, some people might want a normal static graph, but the ability to step through it with a debugger. Then, after debugging, you could run it more optimized and in non-eager mode. Second, in certain RL setups for example, you might have multiple actors, each running on a single CPU core, where each actor has a (simple) batch size of one. Now, if the graph is dynamic in this case, it may do, for example, ABBA(x) or BBAA(x) or something else, where A and B are "blocks" of layers, and each of them is separately optimized on the CPU core.
If you want to use Keras, then pip install keras. I came here for tensorflow.
I'm surprised Google's using GitHub since its owned by Microsoft now
Haha It's no any problem, in these days everyone is dependent on others. On CZcams Microsoft official channel, in Play Store Microsoft apps. Collaboration is key thing in the companies
1:46 For installing TF 2.0.
Does anyone actually have a real life example of a useful regression model or classifier? Most hard problems today need super high accuracy for long time periods of time, and other problems want to know "Why" a certain outcome was predicted, which usually is pretty difficult without domain expertise. Amazon store doesnt count, they can afford false positives.
XG Boost tells you what features are the most correlated
2:49 Using Keras in TF 2.0. tf.keras
Eager by default!!!
And break whatever is called Tensor flow 1
cool
Look, when i try this:
print(tf.add(35, 2.5))
python returns an error:
TypeError: Cannot convert provided value to EagerTensor. Provided value: 2.5 Requested dtype: int32
but if
print(tf.add(2.5, 35))
then all ok:
tf.Tensor(37.5, shape=(), dtype=float32)
what the heck?
it's because tf.add takes two arguments, "x" and "y", such as:
tf.math.add(
x,
y,
name=None
)
and as exposed in the documentation "y" must have the same type as "x", so you can use a int in the "y" position if "x" is a float, but you can't do the opposite.
@@EduardoWurch Yeah, thanx.
Lol, apparently Google engineers seem to have no problem to violate the addition's commutative property.
Loads and loads of apis
When will the anaconda version be released ??
I would guess not until after full release?
import keras as kerasflow
Tensorflow copied Pytorch.
So what can Pytorch copy from Tensorflow now...
Pytorch copied from Chainer
"If we can do it at Google, then you can probably too."
i found this line irritating
구글 사랑해요
Agreed pytorch is now running this race on one leg... But are we sure this is where we want ML to go? It didn't exactly pan out for three decades of MS ruling the OS market now did it? Stifling innovation is never a good thing. We are passengers again folks...just passengers ...
If it's free, open source and without (ad) revenue, where's the market?
tf.keras==keras
TF2 :O
ok
*with tf.Session() as sess:* arggg I hated that
Did an AI design the new TF logo? It's horrible.
16:32 we need more women in the industry
Joseph T , why?
JavaScript devs in the house??