Whiteboard: How digital FPV systems actually work
Vložit
- čas přidán 25. 01. 2022
- DJI and HDZero both have digital FPV offerings for RC planes and drones. In this video I try to give you some technical insight into how they work so you can better decide which one best suits your needs.
Links:
RCSchim's visualisation of Digital(OFDM) and analog RF systems
• RF in 3D: Analog vs D...
My whiteboard video explaining how analog FPV works:
• Explained: How FPV te...
My Patreon page:
/ rcmodelreviews - Věda a technologie
Corrections: (will explain how you can see these things yourself later)
1. Pure image based video compression like MJPEG is highly inefficient. H.265 and the like are 5-10x more efficient. Your Fatshark DVR uses MJPEG and produces a very low quality video at a very high bitrate. Compare that to any H.264 based DVR which produces a higher quality recording at a much lower bitrate. The difference is MJPEG videos don't rely on keyframes so you can drop any frame, or series of frames, and recover from it instantly. And you'll always capture the very last usable frame before the signal is lost for good, which is useful for a DVR.
2. Yes DJI absolutely uses temporal video compression with periodic key frame packets followed by mostly smaller frame update packets (like H.264, H.265/HEVC/VP9 etc..). This allows them to transmit an entire image update (twice) and still have time left over before the next full frame from the camera, to handle some number of retry packets. Under ideal conditions DJI is on air much less of the time, while HDZero is pretty close to if not at 100% bandwidth utilization all the time. Can see this in spectral density plots.
3. DJI doesn't create a single full sized keyframe for the whole image at once like most temporal compression. Instead, it breaks the whole image down into smaller sub frames (not to be confused with the sub-blocks of all image compression), of somewhere around 5 across by 3 down.. and each has their own keyframe and update packets, which are updated serially. In other words, you can lose a keyframe packet and it won't destroy the whole image at once, but only 1/15th of it (if there were 5 x 3 subframes). And even if the entire signal is lost for a few update cycles spanning multiple frames, not every sub frame is transmitting a keyframe packet at the the same time, so some of the lost packets might kill a a subframe blocks, but most of them are just lost updates which only produce a small visual glitch across the rest of the blocks.
4. DJI uses CDD (Cyclical Delay Diversity) to transmit a copy of every packet twice with a very short delay between them. Rx tries to decode both and uses whichever one works.
5. It uses MIMO (transmitting and receiving data through multiple antennas simultaneously) which allows it to move data through multipath reflections. Proper MIMO can even move *different* data through the primary and reflective paths increasing total bandwidth. I don't know how far DJI's MIMO goes, but I do know that it is *highly* effective in multipath environments where analog and HDZero still suffer. In fact I often know that I can I fly further out of LoS if there are rocks, buildings and powerlines nearby for the signal to bounce off of. The worst place for DJI is just flying behind a row of trees over flat ground where MIMO is neutralized, but with CDD and retries it still does pretty well.
6. This is covered in the video correctly but DJI can retry (resend) missing packets several times per frame *before* it starts to increase latency, and that's partly because of #3. The keyframe and update packets are smaller for the sub frames, than if they covered the entire size of the video frame. Worth noting that the Vista AU only has a single Tx antenna so it can't utilize MIMO from the Tx side but with CDD and additional retries it is still pretty effective. In my experience the dual antenna AU is still more effective in deep NLoS and high multipath environments though.
So how can you see any of this yourself?
Regarding 2, 3 and 6.
Watch someone else fly with your goggles in audience mode. The audience mode goggles are passive and not in the exact same position as the primary goggles so they're losing different packets and cannot request their own retries.
As the aircraft flies further away or into NLoS conditions you'll see the video start to break up, but it will almost never be all at once.
You'll see small glitches which indicate lost update packets, and then sometimes see a block (one of the sub frame keyframes) go completely missing (frozen actually) followed by a bunch of update packets which show smeared motion but they're updating a lost keyframe so the updates make no sense. This is normal breakdown of temporal compression, but at a sub-frame level. When the next keyframe for that sub-frame arrives you'll see it repair the broken area. But notice that you'll almost never see the whole screen blank out at once, but just more and more of the blocks failing and fixing themselves at different times.
Compare to something like Connex Prosight from a few years back. When it lost a keyframe, the entire image was F****d, and it became impossible to fly on the updates alone. Flying behind one tree was usually deadly.
To see how heavily DJI leans on retries, stand close to the pilot's goggles while in audience mode. The retry requests it sends out will start to swamp audience mode signal pretty badly. As you move further away (more than about 4m) audience mode will clear up. Two primary DJI goggles actually do the same thing to each other when they're too close, but they'll both try to patch the image up with more retries (and increased latency) until the signal from the aircraft is weaker then both goggles will suffer. For this reason DJI pilots should always put some decent separation between each other. I often see pilots standing/sitting too close to each other and they just don't realize how much *better* both of their signals would be if they moved further apart. I've tested this extensively with my flying buddy flying long range proxmity/chase.
As for 4 and 5 (CDD and MIMO). Look at the FCC docs for the DJI system. This is tech that they specifically list as in use and the fact that performance actually improves in high multipath environments supports this. MIMO is standard fare for all Wifi standards (it's basically at the core of what makes Wifi viable in indoor high multipath environments). I don't know who all uses CDD but it's fairly well defined, and it's well suited to mobile applications because one of the two packets is more likely to not be blocked by the same small obstacle even a moment later.
On a side note. Several 2.4Ghz RC control systems used equivalent of CDD from day 1. With low signal data rate, low refresh rate, high spreading ratio and high bandwidth available (thus each packet only take a tiny fraction of the total time to send), they'd just always send the same control packet twice per update, often having performed a frequency hop between them.
I missed these educational vids from you Bruce. Thank you for helping me understand the science.
Me too
Yes! He's a great teacher.
I've been waiting for this one for ages! Thanks so much for taking something so complicated and making it simple to understand
Long, but very clear and well explained. Thank you very much for the great informational video! A fitting follow up to the analog explanation video you did. I was waiting for this one. Worth the wait :)
Thank you very much for making this video! The whiteboarding works well and the explanations are very clear. I now have a much better understanding of how digital video systems works in FPV and the differences between HDZero and DJI.
yes for the intro , just who's input iv'e been waiting for. Gday
29:57 received frame buffer that we can access via usb bulk interface (which they added for DJI Smart Controller, but we can “borrow” for DigiView etc.) does contain key frames so I would guess there is some additional smarts related to them on transmitting end as well. I think you did a really good job explaining all of this, as usual, 45 minutes went by really quickly. Thank you.
Awesome explanation Bruce!!
So good to watch your explaining whiteboard videos! Clear and straight forward message. I learned something today, thanks for that. Keep up with the good work!
Thanks Bruce. If I had teachers like you I wouldn't have skipped classes all the time.
👍🏻 THANK YOU FROM AN AMERICAN HAM (KD2MKZ).
Your basic AND highly technical videos are phenomenal!
I have the DJI FPV Racer and a second DJI Goggles v2 for my wife to watch my antics.
I also have the DJI Goggles RE which I originally purchased to used with my Mavic-2 zoom and Mavic-2 pro.
MOST of my “flying” is done way beyond VLOS as I’m kinda stuck in my house due to some physical problems.
(ankylosing spondylitis and PSA which has fused most of my spine and joints)
I fly over a power line easement (a mile wide and 40 miles long where there are no people or vehicles), so safety is not really a concern.
I wasn’t too happy with the distance/range of the Mavic-2 and so purchased the original Smart Controller in an effort to increase range.
Turns out… the original STOCK controller has slightly better range than the Smart Controller.
So, I went and got an external antenna kit for the stock controller and managed to install it without damaging the unit too much LOL.
Slightly better, but still lacking.
So,. Went with a bi-directional (2.5w) 2.4ghz / 5.x ghz amplifier in between the controller and antenna.
Even slightly better, but too cumbersome to use..
After hearing about the advantages of OccuSync 3.0 , and my happiness with the FPV Racer Drone range…
I then purchased the Air-2s for the OccuSync 3.0 feature (really worked well) and was incredibly impressed.
I then bought an RC-Pro once the compatibility with the Air-2s was announced.
With the RC-Pro, the range is significantly improved over the stock controller!!
BUT…. NO GOGGLES !!
The Goggles RE does support HDMI input, so I can connect it to the RC-Pro.
THE PROBLEM-. My spine has degraded to the point that the HEAVY Goggles RE hurts too much to wear.
I really wish DJI would support HDMI input to the Goggles v2.
I tried converting the HDMI output of the RC-Pro to analog composite (inputted via the 3.5mm connector on the Goggles v2)
but… the Quality is TERRIBLE!! (How the heck do you LEGIT FPV flyers deal with the analog lack of visual quality?? )
Does it pay to get the HD0 and go HDMI from the RC-Pro to the digital goggles?
They seem much lighter than the goggles RE..
Anyone with any ideas from the CZcams community?
I am also contemplating selling my Mavic-2 gear and buying the Mavic-3 for the OccuSync 3.0 range and
the larger battery capacity/ flight time.
Any opinions?
Thanks!!
Thanks for your videos.
Wow thanks Bruce!! Very interesting to see a little under the hood of the systems and how the work in basic terms. You did a great job of breaking every step down in simple terms us ding dongs can understand. Appreciate your efforts! +1 Patron
Your channel is one of my favorites. My dad wasn't electronics engineer. So I was a little shocked When I seen the way You had the battery cable wrapped around The DJI goggles We all know that puts stress on the little choirs inside the cable and causes failure down the road
Excellent overview, I had sometimes wondered what these were all about. Thanks!
Brilliant Bruce!
Damn, Bruce delivers again. Excellent lecture
I have a much better understanding .I have the HD zero and I'm very happy with it but I wasn't sure what was the cause of the quality differences. Great job Bruce
Videos like this are why you will always be the best thing thathas happened to the hobby. Thank you for the hard work you put in for our benefit.
These are the videos that show your mind is on fire! Cheers Bruce!!! things like this are why RCModelReviews is on lock for me...I always check to see if there is a new post from this channel...no rant all information, just the facts...were you a professor in the past? fastest 45 minutes I have yet seen...
First video I have seen from you and you seem like a very nice guy. Very informative video, enjoyed it!
G-day Bruce 👍😎
Wow! Thanks for breaking it down Bruce, you are true legend!
This satisfyingly scratched an itch that I didn't know I had but was always there. Great lesson today. Thanks!
This is an awesome video. Don't be afraid of sharing even the deepest most complex details of RF stuff most people will like anyway.
thank you for taking the time to understand and then explain these, huge boon to learning these concepts.
I reckon this took a lot of effort over time to convey these two technologies in just 45 minutes. Kudos.
As usual another well made video from Bruce, thanks for the technical details on this topic. Appreciate it!
content is still king - great whiteboard session
got so much out of this video Bruce, thanks for taking the time to explain all this, really enjoyed it!
You have the best, most informative and useful videos on the topics you cover. I wish we could get some solid state batteries or at least information on them like weight, length of flight time, if and when we could expect to use them.. Thank you very much for taking the time to make these.
Very good explanation. The problem with these complex systems is, that there is no fast and easy way to explain them.
Great explanation Bruce, thanks
A new whiteboard video? Hallelujah indeed
I loved your presentation of the two systems. Thank you for the awesome explanation!
Brillant explaination! Thank you!
Very informative Bruce. There's a few things Carl could clear up, but you've got this explained very nicely. Take a look at my slow motion video tests of the goggles to see how the two systems work at the subframe level. Very interesting stuff! We're so lucky to have these options for FPV
Great to finally see this video being published… Well done Bruce:-)
Great exposition, zounds, who would have ever thought these things were possible!!!
and this is why bruce is the GOAT
What are you talking about? This was a GREAT VIDEO! Love the deep-dive. Keep it up!!
Wonderful stuff, very well explained, even an old timer like me can understand it, keep up the good work.
Thank you it was really good! I have worked with radio systems so I probably have a bit of an edge understanding, but it was really good!
Awesome explanation and a great tool to use for the purchasing decisions thanks
Good one Bruce!
Verry well explained. Thank you!
Great video
15:59 I think you missed something: They almost certainly also add a variable amount of redundancy to the datastream via block-/convolutionial codes, so they can detect individual flipped bits and correct a certain number of them, similiar to how it's done in satellite transmission of digital television. Otherwise you would get multiple frozen/black blocks of pixels in almost every frame that is being displayed. This is also how they can adapt the video quality to the signal strength, they change the coding rate (Send more correction data, send less picture data) when signal gets worse.
Thanks Bruce
I know that HDZero claims no compression is done, but based on the quality of the image, how the image is tiled up, and how break up happens I highly suspect that HDZero is doing some form of S3TC compression. It’s a compression algorithm that doesn’t require buffer accumulation or delta frames and is super fast to process by relatively low power hardware.
I agree, there has to be some kind of "compression" being done, because a true, raw 720p video stream would require way more bandwidth than HDZero is using.
Thanks for the great explanation. I didn't miss the squeaky noise of a pen on your analog whiteboard - tho the old"school" presentation had some value to it back then ;)
Also thanks for the plug - I wish tho that I could have actually shown the OFDM carriers - until now they are still just a myth to me. Next I'll try something easier like WIFI and improve my skillz.
I also watched this one from Blunty where he interviews Davide (FPV Blue): czcams.com/video/KxqS-Uy76QI/video.html
Davide believes that DJI isn't even using normall OFDM but something way more complex. This is handy, I like to believe this is the reason why I couldnt yet analyze it on my spectran v6 :)
About DJI's disadvantage on fast moving blades of gras. ND Filters also on the FPV cam can help quite well here. firstly it "feels" faster, more cinematic already while flying and it helps to save DJI bitrate for more important stuff.
We could also try to "mask" out some blurry spots on the FPV lens (with clear tape?) to have some blurry spots on purpose in the edges (where for example my OSD elements are).
This should help to have more avail clarity there where we need it most...
greets, Mario
Good point about ND filter on FPV lens no one seems to talk about. It's the same principle that makes people use ND filter on GoPros to blur grass and other fast moving small details to get less compression artefacts on CZcams.
Thank you Bruce, More detail..Less latency...less cost, has HDMI out so I can use it with my Skyzones, but lack of availability, so far.
nice video
That was a very interesting comparison of the two.
Recollections from when I used to deal with these types of systems is that one of the advantages of OFDM over single carrier is better proofing against multipath because of the nature of multiple low bandwidth carriers as opposed to one high bandwidth carrier.
Useful when flying through spaces with lots of RF reflective surfaces.
Personally however, a few things will have to change before I consider HD0 over analogue, which is more than adequate for my usage. DJI with its high weight, high cost and variable latency is an absolute non-starter for my use.
I notice though they could implement a fixed (though higher than HD0) latency mode if they wanted, at the cost of degradation with poor signals. Probably not in their use case, though.
I were thinking about digital FPV system a few years ago. I did not consider that uncompressed video might be a practical option.... I was mainly looking at the type of things used for video conferencing. (And likely implementing it on a DSP or FPGA...). HDZero seems like a nice design. Details on the error correction would be interesting (and increasing that would likely help video quality a lot)
Some things that would be nice to mention: "modem" comes from MODulator DEmodulator.
OFDM is used for ADSL as well.
Brilliant 👍🏻
Oh thank gawd now i can stop pretending i know what im talking about, i learned things.
Just great!!!!
Excellent video. Also explains why when flying analog you can lose video when a dji video turns on considering the bandwidth they take up. Got to try an HD0 system soon
Just stay off R6 which is what it always powers on (DJI’s “public” channel 8) and coordinate other channels as usual.
Nice breakdown of the two systems Bruce. I've often wondered what all the fuss was about with HDZero but never bothered looking into it any further. so now I know a bit about both systems to boot!. While I like the idea of DJI, and also HDZero, I think for now at least I'll stick with analog. I wonder with DJI, if it allows for multiple viewers since it is a two way link... at least with analog, and hdzero I presume, you just just give a buddy a pair of goggles and they can follow along...
Definitely, this is the kind of presentation which makes me a follower of your channel. As for sponsorship, is there another venue that one offered by Patreon ? Maybe a PayPal account ?
wow thank you!
Once you go HD...Hard getting back to Analog.
Switched to DJI v2 and can't go back to my Analog for fpv.
Cheers from Auckland, stay safe and Kia Kaha!
You need to take a look at the MAX285x series!
26:17 "that's it, its really quite simple"
Me - 😲
What will happen if DJI VTX will be not able to receive signal from googles ? For ex. if VTX is under RF jamming conditions
Speaking of own HD system, have you thought about transmitting chunks of compressed video via 5.8GHz analog? sure it would take beefy CPU to do this but at least as proof of concept? AFAIK raspberry pi can output analog video, but to test this, I would have to record stream on DVR, then dump raw RGB frames and try if I can decode it. I have this in mind, but have not much time to play with this. IMO at least 15fps HD video should be possible to do this way.
Bit more interesting would be to have many h264 streams encoded in image and sent over analog, this way keyframes are distributed so less "bandwidth" is required, non key frames can then be ultra reliable to transmit. So say worst case, you loose 2 256x256 squares out of 30 for 1 second.
Or am I dreaming?
Could DJI just add a couple of lines of code to enable an HDZero Mode? It seems DJI has every hardware an HDZero needs. Bypass buffer, video compression, and disable Smarts, then HDZero Mode is ON?
One followup question: As we all know there are 60fps high latency and 120fps low latency cameras for the DJI system. Why is that? Is the camera image produced by the 60fps cameras simply larger datawise and thus it takes longer to compress it or is there a different reason, like the camera is physically unable to maintain a high enough datarate for 120fps?
The 120fps ISP that DJI used (and is therefore used in things like Caddx Nebula Pro) was in very short supply for a long time when silicone crisis struck. Originally Caddx made Nebula cameras as a cheap alternative and as a way to have a nano cam (they couldn't fit the 120 ISP into a 14x14mm board and actually they still can't, the new-ish 120fps nano cams from both Caddx and RunCam are actually 14x16mm PCBs).
@@FilipRadelic Interesting, thanks!
I stick to the Analog system because the HDZero doesn't have Audio... Why don't they carry good-quality stereo audio?
hd zero should display the last good packet it recieved instead of random color
I tried the dji and cam kept overheating so returned it.
Nice image while it lasted.. But I can't stand all the interference and crap views of analog. What do I do. I want to see footage that is like a gopro via the goggles.
I have started watching and after half of it I was too tired , bbut will watch it tomorrow, cause there is a lot to understand .
I simply use my DJI goggles, sometimes I switch to analog for an older plane but most likely I will stay with my goggles racing edition and its eco system after the orqa disappointment and Carl also not being ready - looks like an endless game or race except DJI crushes them with goggles V2 or V3 which I would prefer.
Hdzero is a good balanced system (quality/cost)
these are custom chips who own the IPs don't pay 300 USD per unit I think fathsark own them or have ASIC license some of ip licensed from analog devices company inside DM5680 there is AD936X IP along with others
The ad9361 is expensive right now because of the lead-time. There's a global chip shortage. Supply and demand. Normally the chip is well under a hundred US dollars.
Drinking game: Every time he says "magic sauce"... Very fast way to get plastered...
no discount for bulk buying then
Really good videos. Very informative but please work on your drawing skills or open up a patreon and buy a whiteboard 👍🏼
BR
Beem me up scotty Next stop it will be rats n mice but I don't know how they will manage how to squeeze a mouse down the antenna 🤣😁👍
But I want my stickers!
Actually no... never understood why people ruin stuff with stickers.
How infomative
why in era of quick digital transmissions and when all modern googles have separate screens for two eyes, we are not widely using 3D-vision which would promise an outstanding experience when flying a drone? What is needed is just to add another camera, and use dual channels transmission (maybe technically could be on same channel), then you get a stereo-video on your goggles.. Quick googling brought me to an article where it says there were some trials, but all of them gone now, and basically no commercial offer for today. Again thinking what level of realism and presence it could bring, if you get binocular rather than monocular vision.
Sorry To Late already Drinking 👍👍👍👍👍👍
I am fairly sure jpeg doesn't send by area, but instead does an Fast Fourier Transform on the image and then throws away a ton of the high frequency stuff as most of the image is represented in low frequencies. A low pass filter on an image FFT shows up as blur the more you throw away - but the analogy still works, you're still throwing away all the redundant information :)
Yes, image compression is a complex process beyond the scope of this video so i tried to get across that it removes redundant information and fine detail.
czcams.com/video/0me3guauqOU/video.html
There are a whole slew of strategies within the overall jpeg family that could be useful. Progressive decoding can provide a better than nothing low res image if the full data was not received in time. The JPEG2000 spec allows for processing sub-areas as individual tiles for multi-threading. In a very time and data constrained application like this it's feasible that some tiles could be deemed too low information to bother sending, and this threshold could be adjusted on the fly.
The cost is a bummer, but the DJI approach offers a huge range of tweaking possibilities for whatever the user needs. I dare say they could even skip the framebuffer stages and also do what HD zero does.... maybe
... which means the latency numbers on the DJI systems are highly irrelevant...
RF (radio frequency COMMS) and digital video compression are probably the most complicated things humanity has developed this far so let us appreciate the pushing of the boundaries that FPV has forced! 😁
I can look like a bug if I want to 🤣
Why isn't anyone making these HD systems on 2.4ghz?
The DJI system in the DJI FPV drone can and does use 2.4ghz ( and the v2 goggles can receive 2.4ghz). Unfortunately they didn't fully include that in the DJI FPV system or v1 goggles.
@@5zero7RCexactly. That's what makes me even more frustrated. 2.4 has much better performance (away from congested wifi at least). Dual band is even better.
Because the bandwidth on 2.4 is much narrower than on 5.8. Because the frequency is lower, the frequency band is narrower, means you can fit fewer “modulation streams”, means the data rate will be lower.
@@sstadnik I didagree. 1. dual band can and does solve the problem you mentioned. 2. I think the the answer to my question is not a technical one, but rather a financial/market or regulatory one. I think DJI and other manufacturers were led to believe 2.4ghz is the main control link frequency used in the hobby and therefore 5.8ghz is the best frequency for video to avoid interference. And they don't want to take a risk on making both.
I think that you may be confusing younger viewers by drawing a camera like that.
Ana-log works for me...have no use at this time for anything else. 😁
5 min into video and still no info
Real men fly analogue. Only girls who are scared of ghost branches use HD fpv.
lol.....😅
Lost Wing would love this!
Not taking this seriously until it’s sent from an analogue phone 😂