Broadcasting Your Voice with ESP32-S3 & INMP441
Vložit
- čas přidán 15. 05. 2023
- The goal is to capture audio samples from the microphone and transmit them to a Node server, allowing all connected clients to play the audio.
The ESP32-S3's I2S interface is set up to handle the audio data using Direct Memory Access (DMA) buffers. DMA allows for efficient data transfer without involving the main processor, offloading the task to a dedicated DMA controller. By configuring the DMA buffer in I2S, the captured audio samples can be stored and transmitted seamlessly.
By combining the ESP32-S3's I2S interface, the INMP441 microphone, and the Node server including WebSocket, the project enables the capture and distribution of audio samples in a networked environment. This setup opens up possibilities for applications such as audio streaming, voice communication, and more.
Overall, the project focuses on configuring the hardware components, setting up the DMA buffer in the ESP32-S3's I2S interface, and establishing the necessary communication between the microcontroller and the Node server.
[ESP32-S3-DevKitC-1-N8R8 Development Board]
amzn.to/3FZmfAM
[INMP441 Omnidirectional Microphone]
amzn.to/3Ma8C47
[Project Code]
github.com/0015/ThatProject/t...
#I2s #DMA #INMP441 #WebSocket #TCP #ESP32_S3 #visualization - Věda a technologie
Check More Projects - youtube.com/@ThatProject
Github Repository - github.com/0015/
Join FB Group - facebook.com/groups/138965931539175
nice but your capture microphone seemed croppy, your broadcast inmp441 mems microphone seemed good as it was being heard in the background with an 500ms delay, correct me if I am wrong. By the way, why didn't you record that one? the output seemed to better represent the quality of the microphone, or you could capture the raw data straightaway and output to the audio channel on the video editor that you use. it would be a better representation of the audio as it doesn't go through a second media (the speakers) to then get captured by a third media (the capturing microphone)
I particularly think the galaxy s20+ mems microphone has a astonishingly good audio quality. I'm gonna try it and share the results as well as soon as I do it. (if someone wants to try as well shall google for "s20+ mems microphone aliexpress").
@@WillianMai I just wanted to show it as it is.
Trying the galaxy s20+ mems microphone is a good idea. I will try it too when I get the chance.
Can you please help me with the code.
Can you please point out the repo for webrtc based esp32 microphone
@@testdev1804 I don't have a project for WebRTC using ESP32 microphone.
i love your videos, keep up the good work man. Thank you for getting my passion back for such projects with micro contolers
Great project, thanks for sharing with us.
Perfect timing. I'm doing a project to use the INMP441 as a vibration sensor. I wanted to send the signal over WiFi. Now I have a basis to start on.
Thanks!
hi there, did you get it working?
@@kibandesteven5117 Yes, it works perfectly. I made an application in VB that brings in the signal and displays it compared to 5 other sensors. I put that on a windows tablet. The sensors are battery powered and very small. I used an aluminum piece to mount the INMP441 and suspended it with a rubber diaphragm so that the aluminum is pressed against the surface to get good sound transmission. I can see which sensor is near the source of the sound by the SPL displayed in bar graph, one for each sensor, on the tablet.
Great video thank you for sharing ,useful for predictive maintenance
Thanks for sharing, very interesting project
Amazing video sir!!
Best of the best 🥰
Interesting concept Eric! I understand this video was mostly about learning how DMA buffers work, always thought you needed a dedicated audio codec chip?! ESP-ADF would totally be worth looking into should you want to improve audio quality :)
I can't find many references to ESP-ADF. Anyway, I'll try to do it.
Will this work with the esp32s3 non devboard, the esp32-s3with the coupled camera attachment.
Great content. thnx
Can you please give us a tutorial to get the INMP441 audio sound from a speaker realtime using MAX98357 using ESP32
waiting for your webrtc variant of esp32 and this microphone real-time audio streaming with good quality
very cool !
How to expand this project to send data to not only local server?
At which Bitrate and samplerate is a 2 way communication to/from an esp32 s2 possible If used for nothing else than audio processing?
Bitrate and sample rate are not easy to calculate. At least I don't think there will be any problem receiving data through i2s. It seems that the bitrate/sample rate can be adjusted depending on how the data in the i2s buffer is handled. If you can empty the buffer as quickly as possible, you may be able to have a higher bitrate/sample rate.
HI, maybe stupid question, because i am amateur. Could it be possible to create two ways flow between ESP32 and web, where both side can send voice and recieve voice...basically like a call with WebRTC between two web apps.
As far as I know, it is still difficult to operate WebRTC in an MCU environment. It can send and receive voice data in half-duplex like a walkie-talkie. czcams.com/video/kw30vLdrGE8/video.html
Hi Eric, glad you continue I2S streaming projects. Could you make a demo that shows how to ask questions to ChatGPT with the ESP32 and I2S mic and show the answer on OLED or even better read the answer out loud by ESP32 I2S speaker? Thx.
Oh, that's an interesting idea.
@@ThatProject Yes sir, it will be very nice
thats what im working on
@@ThatProject please make a video on it please
hi great work!!
i have couple of questions please answer them.....my sincere request
1.what is the use of worker.js file?
2. In which part of the code in audi0_client.html is receiving the audio chunks?
3.Instead of immp441 as audio input i want to give it by microphone and then to client page is there any way?
1. You can think that worker.js is a background task. We need to convert data from ESP32 Mic to Int16Array to play. After converting, it returns to the main task.
2. Where the receiving the audio chunk is ws.on("message", (data) => in server.js
3. It depends. Maybe you need to get the PCM data or other data type from your MIC.
Sir, please tell me how to send real-time audio messages from mobile phones to ESP32 and make two-way voice calls with ESP32?
Thank you for a really informative video. Can you offer guidance on how to use ESP-NOW to transmit voice quality to a second ESP32?
Oh, that's an interesting idea. So far, in my project, I have only transmitted simple data through ESP-Now. If there is a good opportunity, I will try it. Thank you.
@@ThatProject It would have some practical usage(s).
Could you tell me if to use MSM261S4030H0 mic , is it the same i2s config?
Not sure but it's not that different if it has the same pinout as INMP441.
This is awesome! Thanks for the video! I was able to get my project working with i2s, but when I switched to i2s TDM mode with 4 channels and it doesn’t work anymore 😔
According to the official document, it says, I2S on ESP32-S3 supports TDM mode, up to 16 channels are available in TDM mode. But I have no idea how to set up for having separate settings for each channel. How did you set it up?
This is great, I’m looking for a multi mic streaming system like this - can’t wait to look at your code! Let me know if youd like to collaborate.
I get this error when I went to stream in the navigator : "Cannot GET /client".
Please can you help me ?
Thank you for your response !
There is no path /client. It's /audio. e.g.) 192.168.86.42:8000/audio
hi, a hava a question, i use ADC1 to read analog, it ok, but when i turn on wifi connect, my adc get noise, Can you give me some advices, thanks a lot
This seems to require more low-level signal debugging. I recommend asking questions on the ESP32 forum.
Hi.
Is it possible tu run server in ESP32 instead of Windows or Linux?
I've never heard of ESP32 working as a server. Why don't you go with SBC like RPI series?
Great project ! , my question is related to the audio format broadcasted , the mic outputs PDM format , I dont see any format conversion on your code , how is't converted to PCM ?
If you take a look at my code again, I'm using PCM player for a client Side.
github.com/0015/ThatProject/blob/master/ESP32_MICROPHONE/Broadcasting_Your_Voice/wsAudioServer/audio_client.html
player = new PCMPlayer({
inputCodec: 'Int16',
channels: 1,
//sampleRate: 16000,
sampleRate: 44100,
});
This makes it easy to output PCM data to speakers.
@@ThatProject thanks for the quick response.
Yeah I've seen that client inside js code , but from what i know PDM is different type of audio format than PCM , so I have two hypothesis, maybe the i2s peripheral is doing such conversion from PDM to PCM or maybe the PCM player is doing it.
After some research it turns out that the I²S protocol outlines one specific type of PCM digital audio communication.
And INMP441 is not PDM mic and has no PDM mode. It writes PCM format output signal right away.
@@mohammedbenhadine That's right. That's why I can use it as it is.
We have done the code and html in visual studio code and the server works, but if we open the website and click connect. It doesn't work. The waves are not forming. can you help to resolve this issue?
In this project, I'm using plotly to draw the graph. Are you sure that your internet connection is alive to use CDN?
Can you tell us the settings you did in the arduino tool menu so we can upload it accordingly, because i am confused with a bit of settings and configurations and can't risk the devkit as it takes days to reach to us. your help will be appreciated.
thank you
If you set your ESP32-S3 in your Arduino settings properly, I believe there aren't any issues. There is no risk if you select the wrong settings for this. Just try it.
Thanks for the help just one more question
What should I use FATFS or LittleFS in partition scheme because if i use fatfs and i try to include spiffs it doesn't support and then i might have to use sd card or external storage process and if i use littlefs then it says spiffs partition is too large for spiffs_page_ix type. Please increase CONFIG_SPIFFS_PAGE_SIZE.@@ThatProject
If i understood correct it works from esp32 to browser, or is it worker in both sides?
Browser to esp32
Esp32 to browser?
It's ESP32 to the Local Server and Web browser. You can see the running node.js server. Users can connect to the local server through Web Browser and check live audio streaming.
Great proyect!. I can hear the microphone from chrome on a PC, but is it possible to hear it from an iphone?
Sure. You can access the same from Safari or any other web browser on your iPhone.
In my case it cannot work. I have tried it on three different iphones. It seems that pcmplayer does not play any sound on the iphone (safari, chrome, opera navigators)
wifi connected, but can't connect to the server. can you help me. what goes here: "const char* websocket_server_host =" ?
This will be the IP address of the host running on your PC or server.
thak you for updated and simple code, i have seen your code in walky talky, my queestion, why in mic i2s inmp441 is too over if my voice too high, and this mic very sensitive respon, can add very simple gain for code. thak you, sorry my bad english.
Have you tried this one?
void i2s_adc_data_scale(uint8_t * d_buff, uint8_t* s_buff, uint32_t len)
{
uint32_t j = 0;
uint32_t dac_value = 0;
for (int i = 0; i < len; i += 2) {
dac_value = ((((uint16_t) (s_buff[i + 1] & 0xf)
is it also possible to broadcast your voice from your latop or phone to the esp32 / max98357a?
I think it's possible. After transferring the PCM data obtained from the PC's microphone to the ESP32, writing i2s to MAX98357A seems to play sound.
Can you maybe make a video about it?
it worked for me fine but there are two problems
1. the volume is very low i have to stick the mic to my mouth so i can hear it normally.
2. the stream stops after like 30 seconds and for it to work again i have to reset the esp32 and on the browser click stop playing and start playing.
hopefully you can solve these two problems and it will be the ultimate way to test the microphone
Hello. Have you managed to solve this? I'm having the same problem. The microphone output is very weak and has some weird intensity waves. I don't know if this is normal or not.
@@rbbr08 i didnt tried to do anything because i used this project just to check if my microphone is working.
Would this project work on an ESP32-PICO?
It seems possible because it does not specifically use ESP32-S3-specific components.
Does this server allow for multiple microphone connections to stream to the main server
This server only supports single input and multiple clients.
Hi. thank you for very useful video.
I use ESP-camera board. with sample rate 16000 or 40100 all thing is Ok. but when I change it to 8000 all data in buffer is zero. (I print value of bubber for ratr 16000 and 8000)
I tested only 16k and 44.1K so not sure about the 8k bit sampling rate. But I believe it should be ok with 8K though.
Nice
Thanks a lot for your video :D
Its really Impressive !!! :D Thanks a lot :3
I have a Question 🙏
Is it possible to Record the Audio from the client side ?
How could it be possible ? to get as a result a .MP3 file maybe.
What would be needed to have that fucnctionality ?
Thans for your videos , Right Now Im suscribing :D
www.npmjs.com/package/audiobuffer-to-wav
It looks like you can save the audio data as a wav file on the client side using this, but I haven't implemented it yet.
@@ThatProject Thanks :D
I will check it :D
I have a last question:
In the video you use a ESP32-S3 ... Could it be used a normal ESP32 to integrate this project ? or what changes would be needed to use the code within a normal ESP32 ??
Thanks a lot in advance
And Thank you for your video and your answer 🙏
@@ingenia-tec5194 It seems that the same can be used on ESP32.
hello , great job , i want to ask you how to trasmetted both of video and audio in to local network in one code !
That's super challenging. I've already tried a couple of "possible" ways but all failed. Haha. If I have a good solution for this, I'll share it with you.
@@ThatProjectI made a sketch to achieve this.
@@ThatProject I tried to share Github project link here but message get deleted.
Is it possible to use a normal ESP32 ? ( No S3 Model )
Is it possible ?
You can do this with your ESP32.
Sir i have one doubt is any method to save that audio file...
I think you can add code to save the audio data coming from the Node.js server as a file.
This is awesome!, is there any way to connect it with apps like zoom ,meeting ect...thanks
Unless we use a PC microphone driver, it may be difficult to use in PC applications such as Zoom. I'll take a look into it.
@@ThatProject Thank you, i appreciate it 🙏
Hello ! That Project we had setup this project as it is . we had getting the samples and some what of noise but the voice is not getting in this. can you give some more explaination on my problem
Same issue as others below. As I mentioned to him,
Being able to hear noise clearly means that information is being received from i2s, but I'm still not sure what the exact issue is. How about checking your i2s settings again? Or, check your voice response with serial plotting before sending the data to the server.
@@ThatProject we are also getting the wavefroms in the server also. the exact issue is it recogniging the voice and giving the samples in the server , but we are not getting the voice in the server . is there anything to add to get the voice
@@bathinasumanth6708 Is the waveform of voice data transmitted from the server drawn? If you put a loud noise into the mic, can you see the waveform's pitch change noticeably?
I would like to implement this project. Unfortunately I am not understanding everything.
1. What server are you talking about, I believe I see Linux code, where do I put this code on my Windows Notebook?
2.
1. I ran a local server on my Mac. This operates based on Node.js.
2. No. You can also create and run a local server based on Node.js on Windows. You don't need RPI or Linux for this project.
3. Please google “create node.js server on windows”.
Hello, Please I need your help. When I set with this sample_rate = 8000 and bit_per_sample = 8, I don't hear my voice just noise. Thanks for your help !
Did you change the PCM player's sample rate to 8000?
@@ThatProject I didn't. where in the code I need to do that?
@@ariste3688 Check this one. czcams.com/video/qq2FRv0lCPw/video.html
@@ThatProject I get it thanks !
Hello please try to clarify my doubt!!!
In the client side the delay in my voice is gradually increasing.....initially there will be no delay but as time progress the delay keeps incresing gradually
Could you please analyze and tell me what could be the reason?
Not sure but maybe it's Plotly issue. Have you tried without it?
@@ThatProject yes....i have implemented it without plotly
until now i used your client side code in some other project....
but today i have tried to just replicate your project but the output is white noise completely
what could be the reason?
@@NITHIN-tu7qo Sorry, I have no idea what's going on there. If you think this issue comes from my code, please try it again without it.
Hi sir, I can send file record audio to S3 aws iot core? Thankyou
Amazon IoT Core is based on MQTT communication. The maximum payload you can have here is 128KB. It seems difficult to send voice data as a file with this.
@@ThatProject thankyou repply!!! I'm project send audio esp32 to s3 aws ,you have solution? Thankyou
if we are doing with normal esp32 will it work if we follow your schematics
and code
It is officially recommended to use the ESP32-S3. github.com/espressif/esp-skainet
@@ThatProjectbut I only have esp32
where are the header files in the Arduino code please?
Currently, in order to run ESP-SR, you have to try in ESP-IDF environment.
can i use a esp32- s2 for this as well ?
This project also works on s2.
@@ThatProject Is the delay better or worse in this case ?
@@AmanSinghal-ny3ik There is no difference in this project.
can anyone help which code need to be upload in esp 32
Have you tried this project source from GitHub? You can download and modify them and then try it.
@@ThatProject thanks it is running but the audio volume is very low can u suggest anything.
Ok.I have sound on my mac browser, but no luck on iphone, safarafi chrome and firefox tested on ios. Also graph doesn't show up anything, only sound
That's weird. Have you checked the console of your Mac's web browser for any errors?
@@ThatProject 192.168.0.112:8000/js/worker.js 404 (Not Found) I get that
@@ThatProject I had missing this file. I admit it. Now it's on the place, but still error
@@ThatProject OK, I found what was an issue. I run node from terminal by: node /user/etc/etc. Now I did cd to that direction and then node server.js
@@ThatProject Still no sound on mobile browser thou
Now i know why we use microphone modules for rgb lighting...
😂
i've heard 8bit 8k and thats just what that sounds like. for a 16bit stream it sounds excessively harsh and metallic. reminds me of a mid 90's cheap bluetooth headset.
I think its the hw to blame so 10/10 for getting it working, i'm sure custom hw would be a much bigger pcb with lots of filtersto tweak the audio
Using a professional board with an audio codec will of course give you better audio. ESP32 is a very generic board. Many things are possible, but conversely, many things are lacking. It focuses on improving the understanding of overall system construction.
@@ThatProject I'm not knocking your achievement I couldn't even get NFC to work properly on the 8266 gave up and used a 328 .....
Can we do this in dba
What do you mean, dba?
@@ThatProject sorry decibel{dB}
pls tell fast thx
@@hystorical5009 There are already projects that can get decibels. Check this out. czcams.com/video/IqXVAG8ssjg/video.html
can you also upload schematics
Are you looking for this? czcams.com/video/qq2FRv0lCPw/video.html
is czcams.com/video/m8LwPNXqK9o/video.html same as this videos setup?
I don't think so. Please use this way.
@@ThatProject u r so cool
no youtuber ever replies this much to comments
thx a lot
WiFi Connected, Web Socket connected, Web Socket opened, but the audio is not plotting and not audible at Web Browser
Have you checked the data incoming from ESP32?
@@ThatProject yes verified, data coming from ESP32
@@dr.aswinkumersv5803 The next checkpoint is to check whether any errors appear in the console through the web browser's inspector.
@@ThatProject Voice signal and its plot is not coming, audio and PCM plater errors appear in the console t
@@dr.aswinkumersv5803 im facing the same issue did you figure it out??
every thing is perfect but voice was not coming only noice was coming
how can i resolve the problem??
@@HEMANTHNAGAKIRANKANTU Being able to hear noise clearly means that information is being received from i2s, but I'm still not sure what the exact issue is. How about checking your i2s settings again? Also, check your voice response with serial plotting before sending the data to the server.
sounds terrible. Noisy and distorted.
If you increase the bits per sampling rate, you can get a higher quality of audio input.
I run the code with esp-eye and i had this error in the serial monitor : esp_core_dump_x+͡⸮ Incorrect size of core dump image: 65537
The memory allocation in your code doesn't seem right for your esp-eye board. Check available SRAM.
@@ThatProject Thanks man I will check