Coding a Sound Driver on the ESP32 ULP
Vložit
- čas přidán 28. 08. 2024
- This video shows how to code a sound driver for the ULP on the ESP32 and overcome hardware limitations with some creativity.
Code:
github.com/bit...
Docs:
docs.espressif...
espressif.com/...
en.wikipedia.o...)
Hardware (affiliate links):
ESP32 MINI KIT (~$4.90) s.click.aliexpr...
TTGO ESP32 ($4.90+) s.click.aliexpr...
plz share :-)
Consider supporting my lab on Patreon for some extras and frequent updates:
/ bitluni
I'm also thankful for any donation on PayPal:
paypal.me/bitluni
Twitter: @bitluni
reddit: r/bitluni
#esp32 #ulp #sound #bitluni
Reminds me with the old days of software programming, especially for game programming on the pre-Pentium machines where you need to be a scientist in math, physics and computer hardware to get over the limitations of PC resources available at that time.
and all games were enjoyable back then .
Brilliant work and excellent presented. I really enjoy and learn a lot from your videos about pushing the limits of a Esp32.
Thanks!!
You're doing ESP like the demo scene used to do 80s home computers. I love it! Kinda get the feeling you'll have written your own operating system for the ESP by the time you're done though. :D
Mind blown again Bitluni... I can't keep up with your genius pulling the ESP32 down, to make it do what seems to be - anything!!
Very interesting use of the ULP.
Your projects never cease to amaze me! Would you be able to use this in conjunction with vga output?
It will be adde dto the esp32 lib. but for vga there is still i2s0 free for that task
@@bitluni madness lol genius dude
Very nice work, getting the audio to work with these annoying limitations of the ESP32! The video is great too, you are improving with every upload. Keep it up! Your VGA board is now running happily here, the next version of the board will have an audio jack I assume?
Thanks! the version after has a jack. but I have to test if it works with smd caps properly. You could use the pin directly with a line input since most audio interfaces remove the dc offset anyways but I just destroyed one DAC while plugging the audio jack in w/o the cap. it seems to short for a moment. the dac is very sensitive
Amazing work, I was working with this board to develop a BT a2DP source code, your is brilliant!
have come back to this one having learnt a bit more about the ULP. Still a great one.
Your ESP32 work is brilliant and inspiring. Thanks for sharing with us!
Another great project!! "Congratulations!" from India :) Now you have to consider remaining peripherals of the esp32 to achieve the best performance to price ratio!
Amazing work! I love the fact you've used the ULP to offload processing. And with self-modifying code too! These truly are the oldest tricks in the book, and I'm delighted that you're keeping the concepts alive - because sometimes, they're necessary. While I see the good, I also believe the world has gone bonkers with virtual machines, runtime virtualisation, containerisation etc. VMs inside VMs inside VMs.... WTAF. While tech like Kubernetes / Google Cloud / Azure work for BIG industries, it's costly in human effort to get right, and I know of start-ups that burnt their money trying to "do it the right way" from the outset. I know for a fact that while the "latest" concepts / best-practices mean well, their implementations (or usage of) cause way more issues than they solve. Constraints breed creative and innovative solutions. The Commodore 64 had similar constraints, and timing CPU instructions was critical for stretching its hardware to its absolute limits. Those were the happy days!
Great stuff, I've been intending to move my DACAudio library over to the co-pro, this may be the kick I need to get moving on it.
I was thinking about your stuff while watching. Still intending to try it out some time.
@@WistrelChianti I'm re-writing as I2S and getting the co-pro to do the work. Working on it towards Christmas
Awesome stuff! I'd love to see people doing more with the BLE and possibly BLE mesh
I read the whole week about BLE.. :-D but classic is also very interesting
@@bitluni Do you think it's possible to make the ULP "listen" for specific BLE messages and wake the main CPU when something specific arrives? This would allow it to spare a LOT of battery!
I ordered and received a TTGo 2020 Watch which features an ESP32, however, the battery runtime is only about 8 hours with BLE polling done with the main processor. Bummer...
This is a very fine work with esp32, a total inspiration to start to work with it
Where did you get the audio clip from? The one that says "This is an emergency broad cast, nuclear missile inbound detected, seek shelter, duck and cover. Embrace for impact." ?
I'd like to get this audio clip too.
brilliant! you pushed the limit of this chip again!
This Hack is insane, totally love it!
This was an excellent explanation, and I appreciate the graphics!
perfect for filling the 7 minutes until makercast :)
czcams.com/video/igUd8rgy_0g/video.html
Wow.. Du bist echt ein Brain. Geile Arbeit. Weiter sooo
Haha! I knew you would not give up that easily.
Thanks this is really helpful, i was looking for streaming audio via esp-32. mesh or over esp-now. This one solves one part of the puzzle.
For this, you could use the DMA to write your samples to the DAC instead. The DMA is actually very capable...
Assuming that the DMAs are needed for something else and must use the ULP; I think it is more efficient to "compile" the sample in the main CPU to the ULP machinecode, so that each byte to be written is the actual opcode to write that data out, so plus a call to a delay, and you should be able to get ~4k samples out of it, right?
Write the code where each instruction explicitly writes each sample to the DAC... one instruction for each sample. WOW! Would that actually work - not checked the instruction lengths for that. That's uber-mega-hacky, and I love it! But to be honest, you're completely right with the DMA. That's probably the best way to go.
This is amazing. I could have never created this code, but you've presented it in a way that I should be able to use it in projects. Well done Luni.
I'm curios to know the advantage of this solution vs using DMA and route I2S to DAC. This other way saves CPU and frees the low power core
O_O so this is what the next level looks like...
That was this most brilliant thing I’ve ever seen............
Hey bitluni, I love your work, and this ROCKS! Curious why not use an individual instruction for each possible value you can send to the DAC, only 256 needed, jump table takes 1k? Another 1k for code? Then use bottom 16 bits of the rest of the memory, 2 bytes, so 6k * 2 or 12k sample buffer.
It would be interesting to also see the difference in power consumption :)
Always with great mood, thanks for sharing!
2:10 that was so unexpected I spit out my drink xD
crazy, this video made me forget to drink my morning coffee
This is genius,
I hadn't event considered that the ULP could be fast enough to work with I2S, I'm working on a project that could definitely use this, do you think the the ULP is powerful enough to do a volume level calculation of the i2s stream?
My goal/hope/dream is to be able to wake the main cores using the ULP processor whenever sounds above ambient are picked up with a I2S mic, so i can save as much power as possible.
You are an embedded systems wizard!
3:05 Ok, ok...
Google listener: Yes?
Me: Ghost?
Impressive and cool work, I enjoy your projects.
I got a lot of ESP32 experience with our Arduino ECO Power board, mainly focusing on battery operation and LoRa networking. Maybe we can meet at Maker Faire Berlin next month.
Regards Helmut from Arduino Hannover
Great work! And I really enjoy your moving diagrams in this video (like the one at 1:00), what tools do you use to create them?
exciting! subscribed already!
GreatScott brought me here
Outstanding work!
Nice work and amusing video :). But why not just use the DMA engine to send I2S samples to the DAC?
Can't you use the timer to trigger a DMA transfer from memory to the second DAC's register?
Brilliant work! Makes me wonder how to modify this for ws2812 LEDs...
you got a point! it was always a hassle with the rtos! I MUST try that!
@@bitluni If you do that I will upgrade my lights project to use your library, you have my word! As of right now I'm dealing with the jitter from handling interrupts from a web socket connection. I'm sure many other people who like to torture their esp's processor would appreciate a fix for the jitter!!!
I solved issues with the neopixels on the esp32 using i2s for the pov lightsaber. You should check out the code
Very smart usage of the esp. I noticed you need an upgrade to a 32 bits Bitluni's lab soon... Almost 16 bits of subscribers 😜
Brilliant!!
Masha'allah 🧠🤯
you make me smarter and feel stupid at the same time.
"Oh cool ESP32..."
"Oh shit assembly, what is going on!?!"
Super cool, well done!
Nice video! Congratulations and thanks!
I see that you have a new intro! ;)
Jokes aside, Amazing work!
And I love esp32
I like the idea of having the main ESP32 CPU cores just write the audio samples as a series of move-to-peripheral instructions; no lookup table needs because no look-ups :-)
I wonder if it's possible to build a little FORTH implementation for the ULP. That might be fun, and a more general way to be able to use it to do stuff..
Loved the special effects.
Did you use Maya for that? 😏😀
Nice, thanks for sharing😀👍
nice, I think it is great
keep sharing them =D
thanks for this
Best. Video. Ever! 😁
Takes me back to getting a 640k DOS program written in x86 Assembly to work in just 64k of real memory by swapping out chunks of data in and out of the extended memory... Ah... those were the days.
Woooow amazing!!!!
Simply Genius!!!!
I love you bitluni
He's an awesome youtuber. I really enjoy his content, it's in depth and well explained. Am german too btw :)
Hey bitluni how do you know so many things ? Did you learn all this in college 👀
Just following my interests and gathering infos from the webz as I go
awesome
Hello bitluni, great project nad congratz for solving the riddle! Do you think it would be possible to read adc and drive one pwm from ulp with main cores sleeping?
Can you make a deep dive into esp32, because most videos i see they just consider it as an arduino with wifi/Bluetooth but following your projects it has more stuffs and different pitfalls
thx.
OMG this is extream!
Did the designers have a particular use in mind for the coprocessor? It seems weak to me.
Mostly to be running some small operations during sleep of the main core
You should try hitting up SpriteTM on Twitter maybe he has some tricks to help. Great work on this it's incredible what you can do with these chips.
He has truly some better insights on the esp sitting at the source. I hope to meet him this year. maybe supercon
@@bitluni *waves* Not sure how many insights I personally have , but I can maybe poke the digital team every now and then. Also, if you're going to the Supercon, we should indeed meet up. Edit: Also, forgot to praise you: excellent work, both on the ULP sound driver as well as everything you manage to get out of the ESP32 in general!
Cool
You are super!
I want to make a Theremin using two I2C ToF sensors and an ESP32 can the ULC be easily used as a wave generator with varying wave length and amplitude?
Cool video! Reminds me of writing self modifying for C64 raster interrupt code a long time ago.
Just bought a M5 Stack Core 2 with 2 sensors that seem fast, accurate and sensitive enough. Just worried if it is still ok when controlling playing audio.
In what language is programming? Arduino IDE or mrcroPhyton? Also, what of this two you would recommend us to learn? Thnks and keep going on
What's with the 10 seconds of awkward silence?
my fault... it's being "processed" away by youtube currently
Artistic pause to prepare for catharsis.
Hi dude! Nice video. Could you please tell me a good board with esp32 using Arduino ide? Is Expressif board the best choice or Arduino board is better?
Great Work! Send this immediately to marco reps, so he could forge a stepper-motor controller for us.
---
Schick das bitte sofort an Marco Reps, damit er uns daraus ne nen Stepper Controller bastelt. Super interessantes Video.
True, realtime on the esp is a rarity :-)
You could have done the self-modifying code thing if you were happy with 7-bit samples though, right?
high level stuff
Thank you, for detailed explanation. Is it possible to take mic data via ADC and stream out using ULP ?
Should be possible, the ULP has full access to ADC.
In fact there is a dedicated command to get the current adc value.
Genius!!
Hi bitluni ,can you do a Bluetooth esp32 audio receiver outputting it as is 24 or 32 bit?
Hello dear
I wanna to connect bluetooth audio module to esp32. Can you make library for that ? How much it will coast me?
This is brilliantly executed, but wouldn't it be a lot easier to run an I2S DAC off an available GPIO pin and feed its buffer from the loop? That works very well, even on a single-core ESP8266, and in stereo. I know that it works, because I made a raccoon proof cat feeder that plays the Darth Vader March (VERY LOUDLY) when it closes off the feeding dish, and plays ice cream truck jingle music when it opens. It was funny, at first, but after a few days it became very annoying. But it works: the ESP does almost nothing and the raccoons go hungry.
Hey, they say in datasheet, that there is 16k rtc memory. Is it not the same as ULP memory?
Seems nice! :3
would you consider to code some ULP Fifo buffer as freelancer for us?
Brilliant ! I knew you could do it 👏👏👏👏👏👏👏👏 (8-BIT CLAP for you)
How is the Synchronization done between the two processors?
Patrick Knöbel the youtuber Andreas Spiess has a good video about the ulp
The sample buffer is a ringbuffer. The pointer to the current sample is written back to the memory. The main cpu simply remembers the last pointer it has filled and continues to load samples until it reached the pointer of the ulp. This refill happens each few milisecond s. I'll add some comments to the code. The scetch isn't that long. RTC_SLOW_MEM is an array of 32bit words mapped to the rtc memory so the main cpu can easily read and modify it.
Do you have issues with jitter?
what is that "emergency broadcast" sound bite u used?
Holy shit man
Do you have a link to that sound byte?
Can you not shift the bits in the buffer right 16 bits to get the upper high? From the RTC memory to the register that is. So two sequences. One for the lower 16 bits, then shift, and process the upper 16 bits. If it is out of alignment for some reason you could just shuffle the way you load the data... Hmmm.... dunno.
Shifting is done by the ALU. The source and the target need to be registers. All registers are 16bits only. There is unfortunately no way to get the high 16 bits. Even the technical reference states that.
According to the doc, the write instruction also writes some stuff in the upper 16 bits of the target memory word (the current value of the program counter), as is shown in the video. Doc mentions this can be use by the application 'to know which instruction stored that piece of data here'. So perhaps there is also a way to read the upper 16 bits, but that does not look very practical here if we can not control what we write there :). Unless the main cpu can write the full 32 bits, and we only read ?
What abou the other DAC output??
Started working with ULP assembly about a month ago, it’s pure horror
I saw there is a compiler etc. But I really enjoyed using it with the macros and absolute jumps. The instruction set is so basic. there is not much you can do wrong, except for the limitations
@@bitluni don't get me wrong, I enjoy it, but it takes a lot of blood, sweat and tears...
Hi. Do you also teach to build circuit
would this be useful for making a small sampler with sample playback? or is it better to use only the main cpu?
Once it's included in the sound engine it doesn't really. The dac i2s examples from the espressif docs are sufficient, too
I like the way you use 'retarded'
Nice job...can esp32 work as drum machine!!!!
probably. I might try implement a simple synth on it
spannend wie ein krimi
Simply genius. I think I've understood the concept, but could you tell my, how you came up with 0x1D4C0121 | (i
Black magic. I'm convinced Assembly is black magic.