Vložit
- čas přidán 12. 10. 2018
- Most of our IOT devices are insecure and vulnerable. It’s high time to learn how to make them more secure, also because unsecured devices will no more be able to use valuable services without using the HTTPS protocol. Already now, Google services, for example, do no more accept unsecured connections. But is it complicated? Let’s find out!
Our ESP8266 and ESP32s support such secure connections. In this video, I will show you, how to change your unsecured sketches it in a few simple steps. And you will learn some basics about encryption and certificates. Which you can use during the next discussion with your boss or your colleagues.
We will cover:
1. How does SSL work? We just need the most basic knowledge
2. How can we access cloud services using HTTPS with our ESP8266 and ESP32?
3. How can we create trust?
4. How much memory is needed on our devices?
Links:
Sketches: github.com/Sen...
Supporting Material and Blog Page: www.sensorsiot.org
Github: www.github.com...
My Patreon Page: / andreasspiess
My Bitcoin address: 19FSmqbBzb5zsYB1d8Bq4KbxVmezToDNTV
If you want to support the channel, please use the links below to start your shopping. No additional charges for you, but I get a commission (of your purchases the next 24 hours) to buy new stuff for the channel
For Banggood bit.ly/2jAQEf4
For AliExpress: bit.ly/2B0yTLL
For ebay.com: ebay.to/2DuYXBp
www.facebook.c...
/ spiessa
www.instructab...
Please do not try to Email me or invite me on LinkedIn. These communication channels are reserved for my primary job
Equipment in my lab: www.sensorsiot.... - Věda a technologie
Comparing the CAs to the Mafia was absolutely great! :)
Yes, especially considering they get a lot of money as well.
Thank you!
Absolutely nailed it.
Totally agreed!
One or two years ago I saw a series of your videos that I really liked. These days I spend hours learning from your videos. The explanations are among the best I have heard in my entire life, the format and presentation are excellent. You are a very good communicator, the speech is clear, precise and summarized. I also like your humor and comments, it makes the content lighter. Sincerely grateful for sharing your knowledge in this way and working so hard to make these super lessons.
Thank you for your nice words! Glad my videos are helpful.
Even though I already know most of the SSL details and almost skipped forward, I'm glad I didn't because you explain things very nicely. So well that even after reading about HTTPS from 3-5 different sources, it finally clicks when you explain it. You should teach professionally :-)
Thank you. I teach sometimes at Universities. But here I have a bigger audience ;-)
This is a great intro into SSL for anyone new to the subject, I do alot of conversions myself as I use ( mostly dreaded ) KeyStore/TrustStore in Java and have to provide the chains as you describe in the browser. I would just add that anyone venturing into this territory ( even under windows ) to familiarize yourself with the openssl command and its syntax, just be aware that I have found some quirks with windows that the only work around I found was to move all the required files to a linux system ( VM ) and finish up there.
I to use letsencrypt using the certbot engine to roll my keys when they expire automatically ( In my home ), super slick and easy to get up and going once you understand the basic principles.
Great video as always.
Thank you! I think your comment is more for the server side. In this video, I tried to focus on the client/IOT side to be able to access HTTPS servers. So far I never built a server myself (other than my Raspberries behind my firewall).
Fantastic!
I had used public/private key encryption for years, both as a user and a programmer, without really understanding how it works; I only thought I did. You explained it simply, and in a way I will never forget, before minute five in this video.
Bravo, sir!
Glad it helped!
This is something at everyone should be thinking about before any communications take place!
I hope I am able to remove some of the fear many people have to start with this technology.
yes Andreas -your down to earth tutorials really do help!
I agree we should think about data security. Having done so I use two alternative approaches to securing traffic with esp8266. Firstly put http traffic from esp8266 through a VPN tunnel if endpoints are controlled, secondly use SSL reverse proxy server. A raspberry pi or similar can perform both functions.
@@andybarnard4575 would you explain to us how you do that please?
At a high level, yes, for detail I always use Google... I use esp8266s mostly as servers, not clients, and I use orange pi on armbian for the SSL part. For reverse proxy 1. install apache on a suitable server (apt-get install apache2 or similar...) 2. get a dynamic dns name (eg. from afraid.org, use their updating script) 3. get a lets encrypt cert from certbot.org use full automatic method 3. configure local router to always give ESP8266 server same LAN IP address 4. install apache_mod proxy and configure using the 'digitial ocean' guide (google reverse proxy and digital ocean). 5. Access esp8266 securely over internet. 6. In VPN scenario you have two sites both with dynamic DNS e.g. as step 2 and a box at each end 7. Install VPN server on one site, VPN client on another. 8. Many solutions for this e.g. open vpn again use digital ocean config guide, but for other reasons I use L2TP with client from a Mikrotik RouterOS running on a HAP lite and configure a server on the main site using Softether VPN. Both have good config guides. On server side need to make sure firewalls and portforwading is configured. Thats how I do it, and just as a for instance. Result is ESP8266 sketches communicating over internet in secure manner but without having to deal with SSL themselves. Hope the concepts at least are of some use to you.
When I first started reading about the ESP8266 when it came out, one of the first things I thought was "ok, but what about encryption?" and was surprised at how hard it was, and how uncommon. Today's IoT infrastructure is pretty unsafe.
I think the best model right now is to use SBCs as central hubs to microcontrollers, since even the cheapest $10 SBCs can do HTTPS just fine. Sometimes, even the work of microcontrollers can be done by the SBC, especially when it isn't timing-critical. Plus you can then code logic as scripts rather than C.
Microcontrollers shine when low-power and real-time processing is required, but the difficulty to make them secure must not be ignored. It's always good to assume that these devices aren't safe and consider the implications. "What could a hacker do with this?" For things like lamp colors, air conditioner automation, motorized blinds, homemade weather stations, etc - then even HTTP is good enough.
I agree. My problem was more that many companies do no more accept HTTP connections. And I find the combination of ESPs and cloud services a good thing.
I had just about given up on this, but this got me on the right track! Thank you!
You are welcome!
Excellent. Thank you for making a complex subject accessible. Wouldn't it be nice if we didn't have to do this but unfortunately there are always some people who will try to cheat.
:-)
I think cheating was already in the first plans of whoever designed humans...
In the esp8266 versions you have "connect(); verify(); connect(); send()", you can leave the second connect() out. Good to see the esp getting better at tls encryption!(when i researched the cert method wasn't available yet)
Thanks for the tip. I used the example files and did not bother too much...
Thanks Andreas, perfect, all boiled down to what we need to know.... Feel confident to get it done in my own sketches now....
Glad to read that. Thank you!
That "guy with a Swiss accent" sure is smart! Thanks Andreas another great lesson :)
You are welcome!
Excellent video, thank you. I loved the Mafia parallel, "is he a friend of yours, or is he a friend of *ours*?" as a colleague used to say at work a few years ago. This is something I have been meaning to do for quite some time and your explanation made it even simpler. :-)
This was the purpose of the video!
Great video and an excellent tutorial on security which usually doens't get too much attention.
It is not only about security. If our cloud services change, we will no more be able to use them :-(
Some notes:
- If you are running the server (for example when communicating between an esp32 and your PC) you can also create your own certificate authority and make your client esp32 trust that CA. Usually referred to as "self signed" certificates. Same security, less Mafia :)
- For validation via fingerprints you can also use the fingerprint of the certificate authority (or any other point in the chain). Not sure if that's easily available with WifiClientSecure (my esp32 is still in the mail)
- Supporting more cipher suites may actually be bad, as an attacker can remove secure ciphers from that list via a downgrade attack. Therefore you might end up using an insecure cipher.
If you control the server, other options may actually be better for performance. For example you could avoid the asymmetric crypto part by supplying your device with a static symmetric key once. If you only care about integrity (no one else may edit the data) and not about confidentiality (no one else may see the data) just signing the data is enough.
Not sure how much of those is exposed in easy to use libraries for the esp32, but since it can do https, both ideas should be fairly easy to achieve.
Thank you for your comment. This video was focussed on accessing available services outside the firewall. So far, I do not encrypt the traffic behind my firewall. I am sure your comment helps if somebody wants to do that.
This doesn't have anything to do with firewalls. It's about running your own server, something you can do either locally, available only to your LAN unless you configure the NAT and have a static IP address or use a dynamic DNS service such as No-IP (free with limitations); or remotely, with a hosting provider.
This is extremely informative, thank you very much.
You are welcome!
Thank you Andreas, great explanation. Loved the Mafia comparison.
You are welcome!
thanks alot. i can now explain with confidence if someone asks me instead of blabbering on. Couldnt find any easy explanation/comparison elsewhere.
I also had to search and combine different sources...
thanks lot.... my smart home system with telegram bot stopped working few days ago due to this issue.... i didn't found any documents or tutorial to understand this...... but now i think i can make it work again... thanks a lot 🙂
Glad I could help
Great practical tutorial, thanks a lot!
You are welcome!
Thank you for this tutoriel, it was just the right thing to get me started on my project. Really easy to understand ! Your channel is really a gold mine for IOT users !
I'm working with an ESP32 and an MQTT Server. I found that the way to make the connection secure with the server is close to what you show in this video, with some nice certificate. It's working quite nicely in local network, and it's in part thanks to you !
But if my ESP32 is outside the network, then I manage to reach the server (with it's public ip and some port forwarding on my box), but I can't connect to it. Did you ever had a similar problem ? It's quite mysterious : I know my certificate is ok, as is my server. But suddenly the server told me my certificate is corrupt. Almost mystifying, really.
A project I was involved (IOTappstory) had to solve this problem. But I do not know the details. I only know it was not easy :-(
@@AndreasSpiess Ah, I can believe it, network problem is never easy. I don't believe you've got an hint on how it was solved ?
excellent video! thanks fro pulling all the information together to help explain certs, its always been something i only quasi knew how it functioned but this sure clears it up and i want to update my DIY sketches now to secure them :)
The same on this end. So I invested the time to learn it and thought it might be of value for others...
Thanks again for another useful video and the sketches too.
You are welcome!
I wish I was swiss you have so much freedom there
We also have our wives who restrict it considerably ;-)
Excellent - but I'm going to need to watch it a few times to grasp the methods for implementing HTTPS...
I also had to watch several videos to understand it. You are not alone ;-)
As always great info and great explanation, however, it would be great to know how much power and data bandwidth SSL consumes over non SSL.
Another viewer shared his experience. I pinned the comment. Maybe you read it. The bandwidth usually is no big issue.
Great tutorial as always! However, I think one important step is missing here - how to get a certificate from trusted CA. As I see from your screenshots you're using LetsEncryptIt certificate, do you plan to create another video on this? Thanks!
This video did not cover the web server part. Because of that, I did not cover the installation of certificates on the ESP. In this scenario, there was no need for creating certificates as this is done by the service providers.
If I find a scenario where we need a certificate on an ESP I will cover also this aspect. So far I did not find one.
@@AndreasSpiess Accessing ESP device web interface isn't this scenario?
No, only connecting a web server from the ESP. Of course, the SSL theory applies to both scenarios.
Wow, this is a pretty complex topic. It’s gonna take some practice to get my head around it for sure.
I also only understand the basics ;-)
Fantastic video.
Thanks for sharing 👍😀
You are welcome!
Great vid !
Thank you!
FYI
Line 31 of Python code is different between what is shown in the video and what the actual code. I found the video version worked.
On video: hexList = list(''.join(map(chr,hexData)))
In code: hexList = list(''.join(hexData))
Thanks for the correction. Maybe it will help somebody in the future...
OH MY GOD THIS COMMENT JUST SAVED MY ASS THANK YOU SO MUCH
What about power usage? encryption/decryption doesn't happen for free, more power and more data usage...
Ahhhh, I didn't think about that while I was watching. Not such an issue if the device plugged into a wall socket. But if it's powered by a LiPo (or similar), that could make a huge difference
There is a post of another viewer (Frank Hessel) which covers that aspect. I did not investigate into the connection times necessary using HTTPS vs. HTTP. HTTPS times definitively are longer. But, if our service providers change, we have no choice:-(
Would the ESP be abloe to connect to the authority certificate to download the new one (and convert it) when it's expiring in 10 years?
No. You have to do it yourself.
@@AndreasSpiess Thank you. OTA update it is, then
Hi Andreas,
I know that this video is 3 months old but is really difficult to follow you, and sometimes expensive, but ever fun.
I have tried the example with the ESP32 and everything works fine, but if I'm not wrong, you said that the use of client.setCACert(root_ca); is mandatory.
Because I'm really old but still a little rebel, I tried the same sketch without setting the root_ca, by simply comment the line.
And it works and the resulting JSON is exactly the same, point that simplifies a lot the https connection.
If someone want to try it, not only in the example but in real life, any comment will welcome.
Thanks for all your effort and enthusiasm!
Rom
Maybe they changed the behavior of the library. As mentioned in the ESP8266 example, the certificate is not for the site, it is for you to check if you are connected to the right site. The ESP8266 always returned the string, also w/o a certificate.
Thanks for the tutorial. I wished i had something similar as i was handling HTTPS connections with Google to get my emails.
I hope you were successful in the end...
Andreas Spiess, I was.
If you don't have the ability to use https (like with my with my cheap attiny) I setup up a ssl proxy (look for tls offloading or (I think) tls termination proxy) on a PI (which is my cheap server) to handle the encryption part.
Two "possibles issues" :
1) One more potential failure in the chain
2) I must trust my own network
We used these proxies in the past and they work well. I wanted to show that we can do the same only with an ESP. Your method, of course, is still possible and can solve some issues, especially issues with resources on the ESPs
Thank you so much. In my case, I used the root_ca to secure MQTT, specifically HiveMQ. However, I don't understand if there will be an exchange of symmetric keys or if the esp8266 will simply use this certificate to encrypt payloads to the broker.
I am no specialist. So I do not know the details :-(
Very interesting Andreas, if we could integrate this with wifi manager that could store certificates, the link between esp8266 iot devices with sensors would be more secure.
I think you could do that. But if the certificate is valid for a few years, you probably do not want to change it through WiFimanager.
Ah, yes, I think I see the point - being able to change the key without programming. Perhaps that is something that could be added to IotAppStory.com for example?
Could you not update the keys using OTA updates or a form of external memory?
I was thinking the external memory, using an EEPROM. OTA of the flash might be a bit much if you needed to do it every other week just for a key. imho
You can update the keys from the outside. But you should not forget it. Otherwise, you cannot do it anymore ;-)
I like the idea of longlasting certificates more appealing, though.
Very nice video. I shall do my best to share the mafia-centric description of PKI trust! I think that in my Home Automation use-case, the ESP8266 and ESP32 devices will have long-lived MQTT-over-TLS sessions established, so the impact of doing the TLS session establishment and public key cryptography won't really be that noticeable. Thanks!
Thank you! I do not know how MQTT is implemented and I agree if they can keep the connection open you do not have a lot of overhead (other than heap memory).
Have you programmed MQTT-over-TLS on an ESP8266 via the Arduino IDE?
If so, could you please share the code for that?
Thanks!
I m gonna watch this multiple times..😴😴
Enjoy!
Hi Andreas, have you dealt with Matter on ESP32 yet? It might be worth making a video about it. 😊 Sorry for being a little bit off-topic.
It is too new for me and currently has no advantage over Zigbee (or even a disadvantage). I will cover it when it has more value, I think.
Just today joined your patreon! - It seems like for a deployment of a device for several years or more, you must create a scheme to replaced expired certs or otherwise the old fingerprints. I guess if this is the plan, you probably have some way to update the whole "sketch" anyway.
Thank you for your Patreon support! You are right, the certificates have to be replaced. Usually after 2020 or 2022. Maybe we will have better possibilities then and can change our sketch accordingly...
I will make now requests to the server it can't refuse... :D
:-))
As usual, a great video. Hoever, I did miss the part that talks about performance of the asymmetrical (handshake) part of the connection. Depending on the cypher i may take upto 3 seconds to het the connection up and running. When using mqtt this can be a mayor pain in the bottom. I would have loved to have some more info in that since it's hard to find...
I do not use encryption in my LAN.
@@AndreasSpiess I don't understand. The video is about using SSL on (for example) an ESP32. An SSL connection starts with a handshake that uses asymmetrical encryption. I can be very slow especially if you reconnect every few seconds (MQTT for example). It made using MQTT over SSL nearly impossible for me. Hence my question...
@@Bigman74066 sorry that my answer was so short. what I wanted to say is that I use SSL to contact internet services like google. So I do not need frequent repetition.
MQTT is only used for my sensors on my Wi-Fi. So the 3 seconds are not a. If Problem for me.
@@AndreasSpiess thanks for clearifying. Maybe someday a video will pop up about performance of MQTT over SSL on an ESP32. You never know!
Useful video 👍 Excellent 👍
Thank you!
You got that wrong.
Encryption != Security
Encryption = secrecy if the encryption is strong.
I think encryption can be part of a security concept.
@@AndreasSpiess a solid authentication is much more worth than encryption. Also a secure handshake and strong random number generators are necessary before you can think of encrypting anything.
IoT device usually have trouble generating solid random numbers and usually have backdoors or major implementation flaws of the APIs for logins. Just adding encryption is usually just a patch which do not cover the holes below.
I do not understand a lot about the underlying technology. I just want to access some services which do no more accept HTTP requests. I do not transport real secrets with my devices :-)
Currently, everybody uses HTTP and I thought even a bad implementation of encryption is better than that.
BTW: Why do you write, that authentication is more valuable than encryption?
@@AndreasSpiess authentication is signing your messages cryptographically. Without it your encryption is easily breakable if you're in the middle - which is the assumption for "why we need encryption". For strong authentication and strong encryption you need a good handshake method, and for this to be secure, you need a good source of true random numbers. If an attacker can guess the random numbers you generated for your handshake it's likely that he can decrypt the handshake. With a really badly designed handshake it would also be possible to calculate the private key, after guessing the random numbers.
So in short random number generators are the first step to implement a secure communication.
If I understand right, you always need a "man-in-the-middle to "hijack" the connection. So the effort is considerable to inject data to a web request from an ESP using a poor encryption. BTW: A board with a receiver has no problem to get random numbers. Just listen to the receiver on an empty channel (white noise). But I do not know if the ESP library uses this process.
I am trying to get the certificate like above but, its getting any idea to get the certificate?
This is an old video and a lot changed since then. Maybe you look at the newer example files of the Arduino IDE?
Is there any disadvantage to putting the certificate.cer into the SPIFFS storage?
I do not like SPIFFS for such a small amount of data. You have to upload it separately. The library probably takes more space than the certificate. Otherwise, you can do it.
thank you sir
You are welcome!
I am using he esp32 WiFiClientSecure library.
But it does not have a cilent.verify function. Just to check I put a wrong root CA cert and it still connected to the server and gave html data.
So, am i doing something wrong or if not then how can I verify for the esp32 that I am connected to the server I wish to be connect.
Thanks in advance.
This is strange. I thought in my examples this did not work
How can I connect to https when using an Arduino Nano or a Teensy for example? Or do you have to use an ESP8266 development board/MCU?
I have an ESP8266 WIFI Module (ESP-01) connected to a Teensy 3.2 board, but when I compile it says "ESP8266WiFi.h cannot be found". If I change the board to "Generic ESP8266" I get an error saying "Multiple libraries were found for ESP8266WiFi.h". So confusing.
I only work with ESP boards if I need WiFi, so I do not know,.
Nice explanation, and good info. Thanks. Have committed some $$ via Patreon
I found an error message trying to convert a root cert file to .cer format using Cert to ESP8266.py. Fixed by removing the attempt to map the chr function across the hexData.
ie:
replaced this
#hexList = list(''.join(map(chr,hexData)))
with this.
hexList = list(''.join(hexData))
Thank you for your support! I am not a Python specialist and I found the script on the internet. When I used it I had no errors, if I remember right. Now your code is in the file.
Hi Bernie, how did you do this?
Already tested. This should works:
hexList = list(' '.join(map(chr,hexData)))
The 2nd not working. Thanks!
Thank you
You're welcome!
If you do not want to bother with manually writing a fingerprint or certificate in your sketch, there is also a framework that automatically includes all root certificates in your ESP8266 sketch. With this you can do HTTPS requests to any URL, and it will always be secure:
maakbaas.com/esp8266-iot-framework/logs/https-requests/
Thank you for the link. Seems to be a good approach. Unfortunately not for the ESP32...
Great video as always Andreas. Detailed but not boring. Very good presentation indeed.
Maybe this goes outside of this video's content since we are talking about the security... how do you handle securing your keys or certificates? After all your 8266 sketch can be read by anyone therefore an attacker can also read the certificate information. Is there a way to secure the certificate or public key info written into the sketch?
As the name implies: The public key does not need to be hidden. That was the invention. And I think, the inventors got the Nobel prize for that.
Many webservers are hosted on cloud where single physical server host many web servers and uses SNI (Server name indication) to resolve the server name. Many small IoT controllers do not support SNI feature. Do you know whether Esp32 libraries support SNI?
No, I never had to solve this issue.
Couldn't get you example sketches to work, sketch always stops with "connection failed", so the fingerprint was not even checked yet.
Eventually i added the next line to the sketch (directly after "WiFiClientSecure client;"):
client.setInsecure();
Don't know what was wrong with my setup, but it works flawlessly now :-)
Thanks for the update. Strange...
What about client certificates? Quite often I want to verify also client in server side with client certificates
I do not know. I only wanted to access https websites. Maybe you find some other sources for that topic.
@@AndreasSpiess some HTTPS sites that wants to authenticate client requires client certificates. But yes, it's not so common
Hello Andreas, is adding ATECC608 chip to the circuit add any advantage ?
I do not know :-(
IMHO TLS on platforms without even basic stuff like memory protection is maybe not totally pointless but only slightly better than it not being there at all. Especially when you have so little memory to start with and the memory pressure created by the TLS library will make it easier to create overflows etc. I say this after having implemented TLS on a Cortex M3 based product with more memory that's out in the wild in a few hundred thousand units... when I think about how many issues the industry-standard OpenSSL had/has I really don't have much hope that any of the embedded TLS libraries, which AFAIK are all based on some old BSD licensed code that was kicking around, being all that good.
In most applications you'd use a microcontroller for you don't even want the privacy (encryption) bits. You mostly want to validate the source of messages. I think there are cheap i2c secure elements that can do HMAC signing and validation. That seems like a better solution IMHO.
If you have an upstream service that requires HTTPS it seems to me like a gateway to that service running in the cloud would be better than trying to do it on the ESP itself.
I am no security specialist and I just wanted to show how we can make sure that we still can use our cloud services if they only accept HTTPS connections. Maybe it is not secure, but I cannot change anything because I have no knowledge in this area.
If you have to build a sellable product this is another story, AFAIK the new Arduinos use such an I2C chip. I also do not understand what the encryption blöocks in the ESP32 do and if they are used by the libraries.
I wasn't being negative about your video. I was being negative about the state of this stuff in IoT in general.
>we still can use our cloud services if they only accept HTTPS connections.
Many cloud services seem to assume you have a platform that can do TLS properly. AWS IoT for example requires TLS, device certs etc but didn't take into account that a little microcontroller based IoT thingy might not have the right time. Without the right time TLS is useless and many IoT platforms might only have a time that is within the right month and not within the right day that would be required for the very short expiry certificates AWS, Google etc want to use. There is a disconnect between the service providers and people making stuff that uses the services.
>AFAIK the new Arduinos use such an I2C chip.
There are a few examples of it. One of the earliest IoT platforms (electricimp) used an i2c secure element.
At least the ESP32 example connects to NTP to get the time. Now I know, why :-) Thanks!
I made a OTA, one time, from a private GitHub repo. On internet I found that was is not possible, but I made it.
Cool!
My esp32 comes hot when it is connected to your youtubechannel
Try watercooling.
Funny, thats what my wife says when she comes into my office on sunday mornings, "ah, you're watching your hardware porn again.." - just revenge for me saying the same when I catch her browsing running shoe websites.
:-))
Great overview, security is a vital topic for IoT and advanced tech is nothing without such good pedagogical presentations. I wonder if it is that easy to have the esp32 as an https server ?
I think it is possible. However, I do not use encryption behind my firewall.
I also would not venture opening up a port for an esp32 through my double routers walls, I use a VPN for that. But the IoT is pushing with things like Thread that standardizes bridging ipv6 sensors to the internet, we'll see how IoT security will evolve.
You do not need to write a programm to read that file. They are usually read with HEX Editors.
If you have a close look they need a few characters more at the end of each line...
Thank you for the perfect explanation. It is very useful an focuses on the important facts.
Is there any possiblilty to download the SHA1 fingerprints from a server of website?
It would be easy to update the fingerprints by stating the URL and getting back the SHA1 fingerprint like in the browser. So the sketch could get it once the fingerprint is expired and I would not need to update it manually.
I do not know.
This would not be secure, because if you would have a man in the middle, it would just provide you with the wrong fingerprint and you would not know.
Call me paranoid, but I struggle to call 'secure' anything less then SoC with OpenVPN for wireless or Internet-facing applications.
Good video. What does "esp8266/Arduino CI has failed") mean? I get 'fd1' as a reply. The certificate verification was successful
You may use HxD hex-editor to convert binary data to C o other programming languages. Simply open the file, select all and copy as => C (the editor is here mh-nexus.de/en/hxd/)
True. But if you closely look at the Arduino sketch you see, that you would have to add some stuff by hand. I was too lazy for that.
Dear Andreas, since this video is already older and I am fighting with micropython on an ESP32, didn't find much in the internets: It would be fantastic if you could try HTTPS requests on micropython :) maybe even with proper certificate checking - or would you suggest CircuitPython? So far, I like Thonny a lot..
After my Toit "excursion" I will not cover higher languages for quite some time. The time is just not ripe for mainstream. At least not in this community...
@@AndreasSpiess Ah OK thanks you for your answer!
Thank you!
Consider also https for esp8266/32 in server role. It's pretty easy with reverse proxy and raspberry pi and nginx for example. Some additional config is necessary for separation iot and normal sides of home network.
So far I do not use encryption behind my firewall (for IOT devices). This video was mainly to ensure we still can use useful cloud services. But if you want to access your ESP from the outside it might be necessary...
Usually, I use MQTT instead of a web server on the ESP. I find it more appropriate for the small resources of our devices.
Nginx is on my video list for a long time..
I know that this is old but I was strugglin with this as I want to do my IoT devices as secure as possible, keeping in mind good practises.
Since I have some others (In fact, a lot) ESP servers at home which I want to reach from the outside (All of them are HTTP with basic authentication), I think that my best shot is to build a reverse proxy with a SBC (Probably a raspi), isn't it?
Whad do you think @Andreas Spiess ? Do you have any vids on this topic?
Thanks in advance!
Another very useful video. Thank you!
But I also think it's important to learn how to provide a secure https connection on the little websites *hosted* on an ESP*.
All those important little web interfaces, often with username and password fields to access them, etc. - so many 192.168.*.* admin interfaces need to be secure.
I see that recent updates to the ESP8266 libraries appear to contain a lot more examples for this, such as 'WiFiHTTPSServer', which also contains a script to generate a 'Self-Signed Certificate' to enable your ESP*-hosted website to run via https.
You are right, you can also encrypt these connections. I usually do not encrypt the traffic behind my firewall and use MQTT for the connection to my ESPs . So I had no need for this scenario so far.
Not too many real projekts recently. I know, they cause much more work but I prefer them compared to all the pure educational videos. Nothing beats the reminder device. :-)
Thanks for the reminder ;-) I try to mix the different things...
Hi, great work as always. So correct me if i am wrong, these rules out self signed certificates?
Certificate generation is on the server side. This video focuses on the client side and assumes, the server thing is up-and-running. I also did not cover the certification of the ESP device itself as so far, I had no need for that...
Hi Andreas, what 8266 core do you recommend? I use 2.4.0 because I had problems with newer ones which consume much more memory that the 2.4.0.
I did not care too much recently and usually upgrde to the newest version. I only care about memory consumption if I do not have enough ;-)
great!
Thank you!
Andreas and what if people are using Arduino + Ethernet shield? :-)
They are unable to use HTTPS.
Hi sir, can we know if we are doing encryption for local IP communication with a smartphone or a web browser with letsencrypt SSL, does it need internet on browser-side : like my local IP ,eg 192.168.4.1. , will be safe ? how we can securely transfer data, should be use cryptography algo etc
Unfortunately, I am no security specialist :-(
Wow, interesting subject
Thanks for sharing 👍😀
You are welcome!
Can you please make a video on how to use encryption libraries such as wolfssl with esp32?!
I am no security specialist :-(
Great explanation!! Just one dubt: Who create the symmetric key? The iot device? One of its libraries? Thanks!!!
AFAIK public key methods do not symmetric keys
@@AndreasSpiess no, of course. Not public key. I'm talking about the symmetric key that iot device and server share encrypted (around 4.10 min in your video). Who generates its?
Anyway, now, there is a new library/method in arduino so called BearSSL. Can you give us some explanation about?? Thanks!!!
I assume the key is generated by the device. I am not sure I will cover BearSSL as I am no specialist here.
Better and faster is built in esp encryption that can be used with udp/tcp and it is much safer and free)
You are right. But the purpose of that video was to enable our devices to use services on the internet even if they change to HTTPS.
So far, I do not encrypt behind my firewall.
The code used in HTTPSRequest.ino for the esp8266 is it not outdated? Because I see the BearSSL library is more often used these days and AxTlS library is
deprecated. Does this matter as to the safety of the connection? Please clarify, thank you!
This is an old video. BearSSL does not change the basic concept. So there should be no difference in security.
Andrea : in esp8266 python error:
CertToESP8266.py", line 31, in
hexList = list(''.join(hexData))
TypeError: sequence item 0: expected str instance, int found
:(
thanks for help
marc.
I do not use MicroPython on the ESP8266. Maybe you google?
@@AndreasSpiess but you did write the script to convert certificate to string.
Hello. i found the error : replace hexList = list(. ..... by:
hexList = "".join(map(str, hexData))
:)
This is the first video I am watching from this channel. I want to stream my ESP32 Cam remotely. When he says "the rest of the code stays the same", what code does he mean? It looks a lot different than the example camera code.
I do not know the ESP32 cam and if it uses the http protocol :-(
I've tried converting espressif arduino-esp32 camerawebserver project from http to https with little luck. Do you know of examples that might help?
No, I never tried it. And the https stuff recently changed in the ESP32 framework. So you have to use the newest examples, I think.
hey is there any easy way we can make https requests without any fingerprint cause we will need to update the fingerprint everytime it changes and it won't be a good idea
I am no internet security specialist, so I do not know.
Why not store the X.509 cert on flash or a SD card so the cert is not hard coded in the sketch so using the card so the cert is read from it and if they are changed because they they have expired.
This is possible, of course. But if they expire in 4 years I do not care too much...
Hello sir, u make some great videos.
I cant wrap my head around the following concept. Maby you or anyone else out here could point me in the right direction.
Wat I am working on is the following,
I am building a vendingmachine thats controlled by an ardiuno. I want it to work as followed, a customer goes to my website on their phone, this opens a webstore so they select the products into the basket and pay online witj ewallet or online bank. Just like any other webstore, than the webstore needs to send.the data after payment verification to the andrino that gives out the product. I understand I can connect it with utp to the internet, but what protocol or software I use on the website to instruct the arduino? Or would it be easier to use a phi that hosts the webstore, and connects via lan to the.andrinos?
I would divide your project in parts and build one part after the other.
Hello Andreas thank you very much for the video. Although i have a problem, when i put in the certificate and make a const char for it i get this error: no matching function for call to 'BearSSL::WiFiClientSecure::setCACert(const char*&)' please help me!
Your function is different to mine I did not use BearSSL. And I do not know how it works because I never tried it.
great tutorial is there any way to do this in AP mode, do you know any tutorial, link or information about it ?, I feel lost
Maybe here: github.com/fhessel/esp32_https_server
Thanks for the explanation, but I tried the same way to call my https service, which is showing error code -1 with https.
Any guidance would be greatly be helpful
Quite a lot changed since I made this video. So it might no more be up-to-date
Vpn might only be helpful in cases where you own the endpoint of the connection as well, if i'm not wrong. But i'm asking myself if its possible to access a local proxy server via http and let it do the heavy https stuff with the outside world?
It is but it leaves your IOT devices vulnerable to local attack, personally I don't want anyone else able to access my cameras, heating controls etc. nor do I want to leave any easy entry point into my home network. I use ssl on all my gadgets even though everything external is handled by a proxy.
please how can i make my sim800 communicate with SSL/TLS1.3
I do no more use the SIM800 because it does not support 4G
Hello Andreas, is there a possibility to nurzen this SSL encryption via LAN, for example. with ethernet.h?
Many greetings
I do not know, I never needed it.
@@AndreasSpiess Thank you for your reponse.
It's a shame, I currently have a project in which I want to communicate https post via a server. Unfortunately, I can't find any suitable examples. Http alone wouldn't be a problem.
Hello Andreas,
I am having trouble getting your python script to work properly. I loaded it into my Raspberry and ran it in the same folder as my ca.crt file (also tried changing the name to ca.cer). I ge this error:
python CertToESP32.py
// ca.crt
const char* test_root_ca = \
Traceback (most recent call last):
File "CertToESP32.py", line 39, in
first = (chr(content[i]))
TypeError: an integer is required
The certificate is in the right format with -----BEGIN CERTIFICATE-----, but it doesn't have those nice /n at the end and hasn't been adjusted. I guess that is the purpose of your script! By the way, I created my certificates using OpenSSL
This is an old project. So it is possible the code does not work anymore.
@@AndreasSpiess Okay, thanks for letting me know! I suppose I can manually count the characters and hit ENTER a few times for the two certificates I need.
What about secure SSL connections with a esp only, not the www? I mean you can't create a certificate for the random esp ip etc?
The scenario covered in this video was purely connecting an ESP to an HTTPS address. There are many more scenarios thinkable, but so far I never encountered one.
How do you learn all that man ,, Mashallah you are so amazing at this,, I really want to visit Swiss land to meet you ! thank you for the great content.
It is very easy to learn stuff: Curiosity also in my age and hard work ;-)