how NASA writes space-proof code
Vložit
- čas přidán 2. 05. 2024
- I've been told the worst thing that can happen to a developer is their code crashes in production? Well.... what happens if that production environment is outer space?
Safety critical systems require strict coding standards. In this video, I discuss how NASA's power of ten helps them write space-proof code.
🏫 COURSES 🏫 Learn to code in C at lowlevel.academy
📰 NEWSLETTER 📰 Sign up for our newsletter at mailchi.mp/lowlevel/the-low-down
🙌 SUPPORT THE CHANNEL 🙌 Become a Low Level Associate and support the channel at / lowlevellearning
🔥🔥🔥 SOCIALS 🔥🔥🔥
Low Level Merch!: lowlevel.store/
Follow me on Twitter: / lowleveltweets
Follow me on Twitch: / lowlevellearning
Join me on Discord!: / discord - Věda a technologie
learn to code in C at lowlevel.academy ! 😎
never went to moon. habla habla. freemason scammers!
42 likes, let's go
@@FIREINTHEHOLE446 one like, let's go
It is very similar with the rules used in the embedded development for automotive industry.
I was pretty impressed with the Tesla security presentation at Defcon; iirc it compared Tesla's decisions against other automakers' in a decent way
_[edit]: Due to questions below, here are search terms: _*_DEF CON_*_ 23 Marc Rogers Kevin Mahaffey _*_Tesla_*_ Model S_
Maybe for the engine computer, otherwise all bets are off IMH
Big rocket needs to go boom boom but code not
Yep. The rules are known as MISRA, and they are used not just in automotive industry.
banning function pointers is like banning knives just because someone got hurt whilst opening a can with one.
i did misra for years, but i basically also wrote functional C code. so either i did misra wrong or they are doing it wrong.
misra 104 disallows calculating function pointer address with like... math and such - but passing references is completely ok!
rule nr. 11: always annotate the unit of measurement wherever applicable. private aerospace contractors could otherwise introduce imperial units into your code without noticing 🙈
I agree; units are essential where they are relevant.
Units should be a part of type not a comment next to a value. Your code should not compile when you mix units. You can add explicit conversions if you want to. Create a type LengthInMeters and use it everywhere.
@@rafazieba9982 yep, that’s a great standard!
@@rafazieba9982 Assuming the language supports it and that one can spare any memory/time cost, I agree.
@@lissythearchitect I think they meant creating user types corresponding to units, not a language's built-in types
I am a software engineer working on the mars helicopter with JPL. Most of these concepts are already familiar to me, but a couple are new as well. The ban on using heap memory is one that I very strongly adhere to and a lot of new embedded developers are often surprised by. When the IDE supports it, I set the heap size to 0 right away.
Do you have the stack size set to a large number in production?
Huh, so basically the whole state of the app is either persisted or is located on stack? Can’t even imagine how hard it is to code this way
@@stevenhe3462 I have not explicitly messed with any of the settings for the stack, but I also haven’t encountered any bugs related to the stack overflowing/etc. I also write applications for various tests rather than production code, though I can’t imagine that impacts this particular point.
@@ShimonDanilov It’s really not that difficult at all. In bare metal embedded systems (no operating system) the applications very rarely have any functionality that would particularly warrant dynamic memory allocation. You can write incredibly complicated control algorithms and signal processing without malloc ever even coming to mind.
When you need a bunch of memory to mess around with, you usually just use a buffer of some kind. Just an array of bytes big enough to store whatever data you’re anticipating. For example that’s super typical with serial communication where you’ve got a bunch of bytes coming in and you toss them into the array until the full message has arrived, then if you want you can cast that array to a pointer and treat it most ways you would any pointer. You can even cast that same array to different kinds of pointers based on what type of message came in.
That’s all in the context of C/C++.
Mars helicopter?! So cool!
00:00 Introduction
00:44 #01 Simple Control Flow
01:08 #02 Limit All Loops
01:40 #03 Don't use the Heap
02:15 #04 Limit Function Size
03:01 #05 Practice Data Hiding
03:27 #06 Check Return Values
04:17 #07 Limit the Preprocessor
04:57 #08 Restrict Pointers Use
05:33 #09 Be Pedantic
05:47 test test test
05:55 Interaction Reminder
Why did the creator like this but not copy it and add It to the description so that chapters would show? 😞
He doesn't code at NASA 😂
@@BraydenPrice30 programmers are lazy
Bro I thought it was power of 10. There are only 9 rules lol
@@carriagereturned3974 In there words you describe the root cause of the vast majority of software failures 🙂
I went to college in Huntsville Alabama where a lot of the aerospace engineering companies are contracted for NASA so many of my computer science professors had worked on live code for many major space projects like the James Webb Space Telescope. They used a lot of Ada and Fortran. Ada because it's so strict on typing and helps eliminate many mistakes, and Fortran because the routines and functions have been tested for decades to eliminate any bugs. They have to know everything in the code is deterministic and will never deviate from the designed logic.
Makes a lot of sense especially considering the amount of processing power we have nowadays, for those applications you don’t need the last, top notch algorithm or asm library in your code. As long as it’s safe, predictable that’s all that matters.
I wrote some code for the space shuttle program for day of flight operations, but it ran on ground computers, not the shuttle itself.
I found a chunk of code that called from Fortran into assembly to do a branch, only the code that took a parameter had every case commented out over time, so the assembly only ever branched to one place.
I offered to fix it and my boss pointed out our 'C' code implementation of the Fortran and assembly would be completed before we could get the patch through code review, because the latter took about six months.
So even for support code running on ground computers without the big ten rules applied, the process was pretty strict.
@@davidneal1127 im very interested in working for nasa on embedded systems, im a SWE with full stack experience but not a lot of writing for bare metal applications. Mainly self taught after i went to a boot camp. I was thinking of going back to school so i can jump into the aerospace industry. My dream is to work for NASA. Do you have any tips? I have about 2 years professional full stack experience but im not sure if im a strong enough candidate to apply right away, especially considering my lack of formal education. Any thoughts?
Ada is cool
@@DavidGarcia-yx2hu I'm not saying it's impossible, but it is very difficult to get hired directly by NASA because they don't expand staff very often and when they do, they usually immediately hire top talent from big aerospace engineering companies. If you want to work for NASA, get a masters degree in computer science, work for an aerospace engineering company for about 10-15 years and then maybe you'll be hired.
When dealing with extreme scenarios is necessary to implement extreme measures. Not feasible on regular projects however it is a rock solid approach for coding.
Now I want to work on nasa. Not that I want to torture myself, but opposite of that. _ALL_ of their code is crystal-clear and it should be very satisfying working on a codebase that for many years follows this strict set of rules, where you can read and actually understand what's going on without ever needing to run that code. It may be a little hard to work on such a project at the start, but there's nothing a few code reviews can't fix :)
@@IqweoR But a while ago even NASA got a constant wrong and literally caused the rocket/space craft to be 180 degrees off, I think it was a minus sign. It happens to the best of them.
Personally, I disagree that such code practices are infeasible for more mundane projects like financial software or video games. Rules like these help you and other programmers to understand what your code is doing more easily. They reduce the odds of undefined behavior and memory leaks. They also help you catch design oversights before they reach production.
@@raylopez99 No amount of care in coding is going to fix a logic error.
@@megabyte01 I think everything is a trade-off, some applications are not as crucial as others and sometimes a faster development is more desirable than a more solid one.
I understand your point, don't get me wrong, yet I see it more like using a tank for killing a mosquito.
For example: a correct use of Heap can be correctly deal with no problems in a production environments (pretty much everyone does it), this code guides asks to remove such a useful tool which may be more troublesome for applications that are not that crucial.
That's honestly how almost all embedded code should be written. The code base I worked on recently (inherited from another company), breaks almost every single of these rules, it's a miracle it doesn't crash more often
It's kinda funny to me how the "3 levels" of programming are related. I find beginners and expert programmers have a lot in common, but for very different reasons. Loads of emphasis goes to writing code that is easy to read and edit. The beginner avoids complexity because they're afraid of using advanced features they don't understand. The expert avoids complexity because that makes debugging and testing easier, increasing reliability.
It's really the intermediates you have to watch out for. Those are the absolute menaces. Just enough knowledge and confidence to make a mess. Using advanced features in functional but unsafe ways, writing extremely long functions, allocating tons of dynamic memory, etc.
@@asm_nophorseshoe
@@asm_nop I've never been so offended by something I 100% agree with.
@@asm_nop "Just enough knowledge to be dangerous." AKA "A little knowledge is a dangerous thing!" AKA "If you think hiring an expert is expensive, try hiring a non-expert." xD
On additional issue with heap usage is memory fragmentation. Repeated allocation and freeing of different sized blocks of memory can leave gaps of free memory between allocated chunks that are too small to be used in any following allocations. This memory is effectively lost until the program restarts. Usually there is a cyclical pattern to allocations that generates more and more of these gaps the longer the program runs. Normally this is not a big issue, but when your program runs continuously for years without restart, these unusable gaps can slowly accumulate until you run out of memory.
This depends on which memory allocator the program uses, and the kernel on which it runs
Its not truc ! Modern memory allocator search for free hole and flat hole arround on free process. Free and flat process Can bé handled easyly by thread smart service at a bit cost..
Slab allocators on top
Some memory allocators can defragment the memory over time.
Or in one go if its mean enough.
Another alternative for space applications is to save the program state and reboot the software
@@Mallchadthat would mean that the space ship /satellite etc temporarily remains uncontrolled and that can leave your system literally unstable.
I would also add, when an error occurs in your code, error recovery should bring back the code in a "predictable state".
That's not always feasible. Fall back measures or safe state restarts close the gap.
Something they could do, but they probably don’t is to sanity check different states, but it would probably be prohibitive to think about all edge cases that should not happen and hence could be signs of corrupt state.
I am thinking if a is the case, b should never happen, etc.
It would probably be too complex and too much overhead and possibly create opportunities for more things to go wrong in the rules for sanity checking itself.
@@roygalaasen Interesting thought.
@@roygalaasen what happens if you force (nearly) every error to be pedantically handled you can see in rust. The rust system of locking the result of a function behind the check of rather the function was successful or not and nagging you if you don't check the error on a "void" function is creating a system similar to what you imagine. Except for catastrophic failures (that could theoretically also be handled the same way but aren't for simplicity) that crash the program rust in every case either handles an error or forwards the decision to the calling function creating a perfectly collapsing error pyramid that is 100% statically provable to handle all cases.
Uncaught exceptions and the c lastError() system were a huge mistake and should never have been created.
Handling or propagating all errors isn't as impossible of a task as you imagine it if every function is handling it's own stuff.
What the first two replies are forgetting is that this program is *LAUNCHED TO GODDAMN SPACE* and you don't have the little privilege to rerun a crashed process... that's not always feasible or might be prohibitive to think about? Then why oh why does 350 million peoples' tax money and numerous enormous grants fund NASA exactly???
1. Simple Control Flow - don't use goto, setjmp, longjmp, recursion
2. Limit All Loops - hard limit the number of iterations in all loops
3. Don't Use the Heap - use only stack memory, don't use malloc or free
4. Limit Function Size - function should be no longer than 60 lines
5. Practice Data Hiding - declare variables in the lowest scope required
6. Check Return Values - explicitly cast all ignored return values to a void type
7. Limit the Preprocessor - limit the use of the C preprocessor to file inclusions and very simple conditional macros, don't use conditional compilation
8. Restrict Pointers Use - pointers should not be able to be dereferenced more than one layer at a time, also don't use function pointers
9. Be Pedantic - compile with all warnings enabled and in pedantic mode, analyze the code with multiple static code analyzers with different rulesets, also unit test the code
Thanks, but you forgot "TEST! TEST! TEST!" :P
@@chinoto1 dude parsed timecodes but didn't check the return lmao
1. You can write simple control flows that involve gotos and labels. You can also impose fixed upper bounds on recursion depth, just like you can with loops.
2. Some problems can only be solved using loops that have no fixed upper bound. Space probes, automobiles, etc. will eventually need to be able to solve these problems as we require more sophisticated behavior from them. I also wonder how code that, for example, searches a large structure reacts if the search fails because it hit its hard limit even though the desired value is (deep) in the structure.
6. You cannot make a developer verify that a return value can be safely ignored. You can only make him code “(void)” in front of a function call to stop the static analyzer’s warning. This is also true of many compiler warnings, like comparisons between signed and unsigned integers, assigning a value of a large integer type to a smaller integer variable, and not having cases for all enum values in a switch statement.
9. I have never seen a pedantic warning that indicated a potential bug. I’m ambivalent about promoting warnings to errors (-Werror). If you don’t promote them, warnings fly past the developer in the compiler’s output and he ignores them. If you do promote them, developers add “ignore warning” directives to the code rather than investigate each warning to determine if it indicates a bug. This is partly because about 99% of compiler warnings are false positives; they do not indicate a bug. And you cannot simply ban “ignore warning” directives because, occasionally, they are necessary.
What is conditional macros and conditional compilation?
@@richardblain4783
1. You can write every recursive code in iterative way. Machines can always convert them so can you. You may need to have a stack available to do it though.
2. Just add the upper limit of 10^9 or even more and log something if it happens. It is usually (not always) better to fail small than to freeze forever.
6. True. Just tell developers that they need to write a 10 line explanation why they are ignoring it or have two more approvals before commit with the names in the code every time they do it and that it cannot be ctrl+c, ctrl+v.
9. Same here. If you want to ignore warning you need to work for it. I guarantee that there will be no warnings and no ignores in the. They will quickly learn how to avoid warnings doing common tasks.
I'm not saying all such rules are good and that you should restrict your developers to only basic constructs. I would never enforce rule 2 for example but 1, 6 and 9 - always.
No need to use pointers and recursion, that's the dream job
I'm an electronics engineer at NASA GSFC. As a hardware guy, I am not familiar with the software design standards, but these that you list in this video seem to be functionally similar to the ones I experience in electronics design, in that, they are aimed at making the sure the system is reliable beyond a shadow of a doubt. Given we work on space systems, it would obviously be catastrophic to have an undetected failure or bug show up mid-flight.
One example of a relevant electronics design standard, would be our part derating. All electronic components that are bound for spaceflight, must have their maximum power/voltage/current requirements derated to some level below what the manufacturer states. Ceramic capacitors, for example, must have their maximum voltage derated to half the manufacturers level. So if we have a ceramic capacitor that is rated for 10 volts by the manufacturer, then we are only allowed to apply 5 volts maximum across it as part of our design process. This helps guarantee reliability and the lifetime of the mission.
Interesting... I was wondering about the hardware requirements too.
I’m a computer engineer and I had few courses on digital and analog circuit design. can you suggest me some books or documentation about the design of electronic safety critical systems?
They call it the Power of Ten, but only 9 are listed.
#10 - Off-by-one errors
😀
10 is testing
He skipped one rule, which is "use a minimum of two runtime assertions per function".
JFGI
@@sousahenrique what does "runtime assertions" mean? Guard clauses?
@@redcrafterlppa303 to see if a function return the desired value.
#include
int add_one(int x) {
return x + 1;
}
int main(void) {
int foo = add_ond(5);
assert(foo == 6);
In this example, if foo is not equal to 6, the program will crash and print an error.
@@rj7250a ok I understand now but aren't those short circuiting crash operations bad for runtime code stability? I mean for example a calculator that is asked to divide by 0 should print a warning and return to a stable state instead of shutting the program.
My philosophy for my code is, never crash (outside main). Of course sadly I can't enforce that rule on code written by others.
As a function always inform your callee of your failure and allow him to deal with it in his context gracefully.
Sounds like sound advice. Making a medical thingy nowadays and since we've decided to have MISRA shout at us we're compliant with most of these rules :)
Very annoyed at GCC not enabling all warnings with `-Wall` or even with `-Wextra`. A buuunch of extra warnings to enable.
I leave off -Werror just as a developmental thing. Nothing even makes it into an archive library w/o being warning-free. But sometimes during developing half-finished stuff - yes, there are interim warnings. In those cases I don't want the compilation to fail because the coding hasn't been completed.
MISRA is pretty crucial rule series.
It turns out gcc enables many of its warnings only when optimizations are turned on, because that’s the only time it does the analysis needed to detect the warnings.
If you're doing code developmrnt for Medical, take a look at IEC 62304 too. Personally I think it's very light on technical requirements though and focusses too much just on process. I can understand why you've adopted Misra. Also think about using some other static code analysis tools. Lint isn't bad. Depending on what you're developing and how much resource you have, you could also look at dynamic analysis and tools for version control and requirements tracking.
@@richardblain4783 Oh, I've never heard about this before. Very interesting, but I suppose some things "look" different when aggressive optimizations are being done.
1:58 Another classic problem of using the heap is that it makes the code non-deterministic, because the base locations of, well, everything, change from run-to-run.
@@dandymcgeeNASA certainly doesn't use windows. All of that is embedded CPUs that run some kind of RTOS which certainly doesn't use ASLR
And memory fragmentation. I'm a little surprised it wasn't mentioned.
I love how for every rule, you give the most palatable example that also best illustrates what the problem with not using the rule is. And you got through this in 6 minutes 🤯 Bravo!
Meanwhile I can’t write safe C++ code that handles user’s string input
C++ be like
@@LowLevelLearning 🥲
lol =) well, there have been codebugs in rocket launches, like Ariana 5 in the 90ies, which had software done in ADA, and the code tried to push a 64bit floating point number into a unprotected 16 bit unsigned integer register. So when the rocket reached a certain height, the register could not store the data anymore, and sent the rocket down to earth. I think its called the worlds most expensive software bug.... not rocket launch, but there was also a coding error in a x-ray machine that hospitals used, which unfortuntatly led to people got killed. They used a register which was too small to keep the data, which led to the machine giving the patient a way way higher dose of x-rays then intended. I am lucky I only do simple applications :D
who needs strings? :)
- calculation amatuer
@@NoNameAtAll2 Oh no, a string bug "amatuer" ... eeeek!! Help! 🤣🤣🤣
A colleague of mine worked on VxWorks for many years writing OS code - it's the operating system that the mars rover uses. It's quite fun hearing about his stories. Apparently there was a bug in the code for the rover, but he determined it wasn't an OS bug but rather the programmers at NASA who had introduced it ;)
Was it the Pathfinder priority inversion?
@@rayc1557 how you know? ;?
@@NoTraceOfSense I think so, yeah! Sounds similar to what he described.
@@NoTraceOfSense Interestingly the priority inversion issue was not because the code was badly written but to well written. It changed precedence constrains on a schedule that had been analysed and proven feasible. Well the code went faster than it should of and overran one of the modules controlling the bus master or something like that. But the thing is it wasn't because of bad software practice or anything and more of lack of knowledge in real-time systems theory which was a new field at the time. Precedence constrains proved to be capable of creating priority inversion if not preserved. People assumed that best case scenarios should not be problematic for a real-time system but they are.
@@reiniertl yap yap yap
I'm taking the void casting rule to my own list, thanks! Checking every return makes a ton of sense. If you have many independent calls needed for some kind of setup, it only makes sense to check their return values in one place (probably followed by an early return). Once that is done, you kinda gravitate towards bubbling those setup calls up the function body. great, now you have all you need to potentially move the setup outside as soon as you realize the function is called in a loop or during traversal... an optimization that came naturally through good practices.
#3 is still widely used in the defense industry. It's not a requirement anymore, but there is a general bias against using the Heap without good reason to.
#6 is just good practice in general. Our internal compiler flags generate warnings for any function that returns a value that is ignored.
One more that many programs follow: All Warnings get elevated to Errors, forcing the developer to actually address them.
I had to do this when I worked on pump controllers, the rule about not using free etc. if you mess up you flood a room because of pressure relief valves in some cases don’t outflow to safe places because they need to be at x point and each install is different and fudged up.
A good tool to mention is WDT (watch dog timers)
This is an extremely well put together video! It's interesting, informative, entertaining, and it's extremely well produced - very well done!
It sounds very nice. However, I have written a fair amount of code for the International Space Station (Ada) and for the Shuttle Cockpit Avionics Upgrade (C++). I never heard of any of the things this video calls the Power of 10. It sounds like a lot of hot air to me.
This is great, thank you for uploading this. As someone with a computer science background and a love for C/C++, I like looking at different ways of writing clean and efficient code.
Very wise advice. I used to program devices that were part of military equipment, and we took much of the same precautions. We didn't want to have a software failure lead to a military loss. A minor crash in civilian life could be a devastating one in combat.
I love that we have this discussions over safety. There is also a second debate to be discussed, about how minimal is a language. Because C is very small. Other languages are bigger in features. These may be pros or cons depending on context. For example, there is always a big fuss about Python being "bloated" when comparing it to Lua. The size of a language, it's ease of use are also important when dealing with topics like safety and performance.
Rust.
I give you a video idea: do embedded c++ that does not use standard library by using -fno-exceptions and -fno-rtti
-no-stdlib
std still works without exceptions and rtti
it just terminates if those are called
Embedded C++ is just a C, because all what C++ does is creates syntax sugar around v-tables (i'm exaggerating of course), and if you will try to avoid everything that consumes CPU cycles and memory for no reason other than "clean code", you will be program in C++ like in C. But, you will not have access to restrict keyword, and there was something about unions in C++ which I don't understand, but for some reason they are not just a multiple types for one address.
@@rogo7330 Most C++ compilers should have a __restrict keyword anyway.
@@rogo7330 Well no, since C++ and C are far from the same language despite the differences between their respective standard libraries. And the union case you're talking about is that C allows type punning through unions, while that is undefined behavior in C++ (for very good reasons), instead you go with the `std::bit_cast` or `std::memcpy` route if you want to reinterpret some memory, as that deals with lifetimes properly.
on form of testing that they would be looking for is "Modified condition/decision coverage (MC/DC)" its very useful for making sure your program is tested for the different variation. i've even seen some compiler bugs found during this style of testing
Just got a new subscriber. Thank you for this. Im researching autonomous cars and im experiencing some issues by not following those rules, even with extensive testing
The fun part in these rules that they won't allow you to cut corners and do coding hero stuff.
These rules simply force programmers to think over what's being done and punishes premature optimization (mostly done by people creating technical debt in exchange for personal interests and 20/80 pitfalls for entire teams).
In an interview Bjarne Strostrup mentions that in Airplane code the rule is no heap allocation after take off
Many years ago, I worked at Oak Ridge National Laboratory (ORNL). All three sites in Oak Ridge (ORNL or X-10, Y-12 [nuclear weapons plant], and K-25 [gaseous diffusion plant]) were operated at that time by Martin Marietta Energy Systems. I was one of two representatives from ORNL that were part of a task group working on a "Systems Development Methodology" that would be used at all three plants for software development. At some point during our efforts, persons from Martin Marietta near Denver that had worked on their software for the space program were touting how their line-by-line code reviews eliminated all significant errors. Shortly thereafter, the Mars Lander crash occurred due to the mismatch between English and metric units that was not caught during their code reviews. We never heard another word about how good their code review process was.
Good video! The whole "limit the preprocessor" rule is one I need to learn. I was writing some fun code to solve a cpu intensive problem. I kept coming up with ideas that might speed up the code; said code I could select via the preprocessor. I kept leaving in old ideas with new ones until my .h files were an ugly nested mess of preprocessor switches.
Probably better to just use seperate functions when your trying to optimize code performance. Its pretty easy to just copy & paste a function and than edit the copy to clean up it up or make improvements. You can even just add a version number to the functions as you make updates.
When you’re experimenting and trying various techniques on a problem, that’s fine. But it’s completely different from building a production embedded system.
All of these things are valid in aeorspace and automotive. Safety is the first principal for the code development in these areas. Interesting thing is whenever you apply for a job they ask you recursion :D
Recursion will be asked so that we should aware of not to implement recursion
kim soruyor recursionu, hangi firma ?
@@shellohd8421 girdiğim hemen hemen çoğu interviewde vardı. Kullanılıp kullanılmamasından bağımsız olarak soruyolar.
Railways as well.
This is something I have always wondered. I'd also love to know how issues are fixed, what backup systems they have and what their coding environment looks like
It's heavily dependent on the mission, project, and even NASA center. Most systems support atomic over-the-air updates with multiple redundant copies of the flight software and fail-safe mechanisms if an update renders a vehicle unusable. For example, the system might fallback to a previous image or golden image if the spacecraft isn't contacted for x number of hours.
I would imagine them using C as it is the staple of stable code. Meanwhile rust might be a good contender in the future as it features many compile time static checks that make it nearly impossible to write bugs. Which has a similar mindset to the here presented rules I themselves.
@@redcrafterlppa303 Yeah tbh, applications with no room for error would probably be the first to Rewrite It In Rust :)
@@juniuwu I don’t see that working, i know code c for embedded automotive, don’t think rust can be used there, cause everything is very time critical.
@@hetstandaardkanaal7167 Why not? Rust isn't any inherently slower than C is.
i'm really happy you respond to that question, i always asked myself but never searched
When doing my CS degree back in '07 I did a course on program correctness, where published material from NASA was heavily used. At the time there really wasn't an entity out there that had as long a history of rigorously taking care of their software practices and that publicly made that material available. There were statistical analyses over various sized code bases and deep dive articles on do's and dont's (similar to Google's famous 'Testing on the toilet' documents). Sadly all of those links have rotted away, and I've yet to find all that material online in the years since.
This is a great insight. Car electronics have been on my mind, the programming that goes into making them totally reliable
As an end-user, no, no, it's so incredibly unreliable. Maybe it's just a recent development of cars no longer being treated as embedded systems, but the amount of stuff that can go wrong between modules or just with CANbus itself - not to mention the insane startup times for what ought to be barebones embedded systems, which makes getting startup data impossible - makes them an absolute nightmare to work on sometimes. Not just electric cars, either.
@@tildessmoo Agreed. The level of garbage allowed by the DOT is insane, manufacturers don't even isolate vital control systems from the children's backseat entertainment.
@@tildessmooI miss when the computer in a car had one job and it was to be a carburetor
Must be an incredible feeling for a NASA Developer when the code finally runs without any error and warning and all tests pass and the rocket doesnt explode😂
You know that these are just the preventative measures during the coding phase, right? After this level, there's getting the actual subject-matter logic correct. Then there's getting the engineering correct (task time constraints, relative task priorities, correct constants, correct engineering units, etc.) The weeds get deeper as you go further.
What if the rocket does fail, and later the cause is determined to a be software problem?
@@guytech7310 you just gotta hope. you would probably be fired for that but most of the time it isn’t a manned rocket. (There is a lesser risk of one getting hurt)
It is. ☺️
@@guytech7310 add the lessons to the repository and take care not to repeat the mistake?
I love it! Especially as an embedded programmer. I never considered checking if NASA had programming guidelines.
These are very similar to the rules that we had to follow when I worked for the government. I was working on a system that was inside of airplanes so it needed to be very resilient and fault tolerant. We did write it in common lisp so we didn't necessarily have to worry about pointers and control flow problems but the other rules did apply (there are pointers in clisp but they are obfuscated so you rarely have to worry about them). The flavor lisp that we were using was Jit compiled and it didn't have a garbage collector. It was this extremely small compiler that was packaged with the program because we didn't really have a lot of space to work with.
I really like programming in LISP. It's so simple and hard to screw things up. That's how I know something is wrong with me.
I always thought it was interesting that you can both call a function as is, and call it as a means to set a value to a variable. Never occurred to me that when doing the former, the return value could turn into junk data in extreme edge cases.
Yep. I work in Delphi. While it's a significantly flawed language, one of its uncommon strengths is its division of routines into _functions_, which return values, and _procedures_, which do not. Calling a procedure on the right side of an assignment statement results in a compilation error, so there's no chance of creating junk data that way.
As much work as this sounds. I'd love to learn how to code like a NASA Engineer. A lot of these steps might be a bit extreme for low risk civ stuff, but just having these skills would be a great boost in your ability.
Simulink code generator to generate the control logic.
Architecture tools like Vector Davinci to generate the basic software.
The code generators themselves abide by the rules you mentioned (no dynamic memory allocation, no recursion, etc.)
Excellent use of time. 6 minutes and not a second wasted. Just a list of useful helpful facts. Well done.
As someone who has done programming for >13 years, ranging from low level to high level, I am actually quite fond of these rules. Any software dev reading this has had that one moment where something odd is going on because of a silly mistake that has been right in front of you the entire time. This technique prevents that from happening, or at least it makes it easier to see where it happens, because it sticks out like a sore thumb.
Sure, but it comes at a considerable cost in terms of time and, therefore, money. For a money-is-no-object organization like NASA, they make sense. They don't make sense for most businesses.
@@KG4JYS You must be talking about some alternate universe NASA, the NASA in our universe is constantly getting underfunded and grappling with saving money.
@@KG4JYS The difference isn't money, it's stakes. The tiniest mistake might cause an annoying exception popup in your C# business app, but might kill 6 people in your rocket control program.
These rules look very similar to MISRA, with a stronger restriction on pointers and function pointers.
One of the big differences is allowing multiple return statements in a single method, which is arguably a very good thing. That particular restriction is really quite annoying in MISRA code.
Might be a good video idea to cover MISRA. It’s enjoyable most of the time and makes for some decent clean code.
i had never heard of it... but thanks
Yeah I was thinking about the exactly same thing when I watching it. The single-return rule in MISRA-C is the reason why I highly opposed to apply all the rule given by misra-c to our production code, which is once required by our boss.
I am happy that NASA don't do that BS😂
I have been recently introduced to the BSW architecture in AUTOSAR, which uses MISRA c. However, I noticed that a large percentage of the MISRA c generated code abuses the preprocessor #define, followed by a function that takes multiple arguments. I'm not sure why this is the case. Even when compiling, the number of warnings is scary.
Rules at my workplace (aviation equipment) are based on MISRA so that's also what first came to mind
Interesting. I remember many of those being promoted, especially function code size, way back when (1970s). I was working in commercial programming, i.e. COBOL back then. The main portion of the code was just a series of PERFORM statements with SECTIONs being the equivalent of subroutines. The only GOTO allowed was a GOTO to the EXIT paragraph to leave the SECTION.
Excellent. Also check the value range for the arguments provided to a function, assume the caller can make some mistake.
Another thing is to have a hierarchical up-down order in the calls, no sibling functions calling each other or up.
The only thing I don't follow from the video is the use of function pointers, I use callbacks for when low level peripherals finish a task 🤔
The heap is a big problem generally. Most ways I see of properly using the heap involves using it as little as possible.
Most modern languages only have heap allocated objects though. But those languages aren't used in this type of software.
@@Evan-dh5oq The peasants get the mediocre stuff, the pro OGs get the good stuff.
Big problem generally? Nope, not at all. In specific cases sure. If "generally" would be true then heap allocation would even be a problem in simple things as window managers, calculators, note taking apps etc etc etc etc * 1 billion examples, none of which allocating heap is a problem...
Heap allocation and deallocation causes memory fragmentation, which results in failed allocations due to lack of contiguous memory. Searching through the heap also takes longer. These problems get worse the longer the program is running. People who program PCs just ignore this issue, letting the program crash then restarting it. You don't have this luxury if your software has to run for years at a time without human intervention. A lot of equipment becomes sluggish after running for just a few days.
@@sexygeek8996 _cough_ Windows _cough_
Very interesting video. I think I'll apply the "Limit All Loops" rule to some of my code (not C/C++). I experienced some deadlocks where we wait in a loop until other processes are done.
Worse is calling blocking APIs that force you to wait. I try to use only non-blocking methods to avoid deadlocks. The only way I came up to deal with blocking API deadlock issues is to put it in a separate thread, that you can terminate if necessary, but this can cause memory\handle leaks.
NASA mainly uses Fortran
In re. #4: I was told by a wiser friend long ago that any function should be no more than a screenfull from beginning to end. And this was at a time when the most advanced display (CRT) resolution was 800x600, so we're talking somewhere between 30 and 40 lines of code. That rule has served me well.
thank you for this. this is a new way I could approach learning about best practices.
I'm curious about the details of [potentially nonexistent] circumstances where they decided, after careful careful consideration, that violating one of these rules was fine.
I believe this is potentially existent circumstance to violate one of the rule - is to make the function more than 60 lines of code, specially the main function.
@@imdanielmartinez Hm, although it is possible to maintain that by creating sub-functions... if n is main's LOC, you'd need ~log60(n) of them... 😂
If NASA write a programming language based on C that followed their standards, they should call it GalaxC - pronounced like galaxy!
You've finally convinced me there is a legitimate reason someone would want to use C. It's imaginary but still, it counts.
I was always wondering what they were doing differently. I heard about MISRA C, and using ADA (never heard or seen anything that uses ADA). Thanks for covering this topic.
ADA and the extension "Spark"
Ada is used in many planes, specially the jet fighters
Arianne 4, 5, 6
That is very nice set of guides - as I do a lot of mcu (atmega/xmega) projects, I wrote my own stack use estimator to see whether a program would have a potential of crash - and the things pointed out here (recursion, function pointers) indeed make such static analysis (which I did always on the final binary that was disassembled) a lot harder and exceptions had to be added.
@1:28
just a small correction, the function is supposed to return the predefined type (list_t) instead of int
as the purpose of the function is to return the node that holds the value being searched for, not the value itself
This has a lot of general best practices , great video
It sounds quite easy once you acquire all the things that The Power of Ten prohibits.
My uni actually is researching a tool to do proofs even on the heap.
It uses separation logic as an extension to Hoare logic, allowing for more complicated aliasing from pointers. They actually made a tool to do proofs in C called SecC by university of Melbourne
So, you write programs to generate low-level programs with complile-time proven properties?
haha SecC
Writing code in C is like eating your own legs, you shouldn't do it on purpose.
I agree with most. Some are impossible in some condition (you might need to target multiple architecture/OS...
The new virtual function, I agree with the point, but it can help abstraction and unit tests, and it might be a more convenient way to reduce code bloat. (Think reading IO, it allows abstraction on the where from (input stream, file, network) or the lower level stuff (does it have a V1 or V2 of the document).
Fine for application specific where you can safely control all the path and reduce to amount of path possible.
The function should do 1 thing I totally agree. One thing at one abstraction layer. Though not of the line limit. But generally, you should have short function if you respect that, and use some design pattern for "big cases" (like factories)...
Great explanation of writing more reliable and obviously testable code for mission critical software. Most rules should apply to normal commercial or hobby programming except I can't imagine any large program not using the heap or function pointers for callbacks. Preprocessor #ifdef can be invaluable in the real world for preventing multiple inclusion or definition errors. Not sure how they manage without that.
Real interesting. I love coding and space, but the thought of my code crashing in space is just terrifying
I was wondering why the hell our school made us code with somewhat the exact style that is shown in the video, I think we are lucky to code this way 🎉
What school have you been?
poor kids
@@flobuilds alx africa its like holberton school
@@user-tk2jy8xr8b Many of us appreciate the teaching methods, even when we find them challenging. The difficulties we face during our learning days serve as valuable lessons, preparing us to handle the challenges we may encounter when we begin our careers. As a result, we are less likely to become easily frustrated, having already experienced and overcome various obstacles during our educational journey.
Very similar to the way the company I work for approaches PLC programming. (Code for industrial machines). The program language we use is fixed memory only though, so rule 3 is free.
A video on what state of the art static analyzers are used by NASA would be very interesting.
I guess that the main loop in embedded systems are an exception to the rule 2 as microcontrollers cannot operate without a main loop, but obviously all the other loops will have a hard limit.
So, I imagine there are exceptions to these rules bc you need setjmp like operations in order to yield a thread or to enter an interrupt. Similarly, function pointers can be typed so that threading can occur. Based on the restrictions presented in the video they couldn't use a normal RTOS statically. I would argue within context of context switching that they would have to use them then and only then.
It could very well be that they don't use a RTOS which doesn't make use of these rules either. Considering their environment, that would be very understandable.
So it could very well be the case that they do in fact have no exceptions to these rules.
I work in aerospace, where we enforce MISRA analysis of our code. You're right that these rules make threading extremely difficult (I won't say impossible), which is why we don't use threading. We also don't use an OS in the first place, so each processor is incredibly limited in available resources.
What makes it possible to do anything more complicated is, in my opinion, the use of distributed processors and specialized hardware for communication between them.
If we've got to control 7 actuators, we don't have a single multithreaded processor running an RTOS to process any events that might occur. We'll have 7 tiny processors sending their state to a main control unit, which will then react to (and filter) external input and send discrete commands to each actuator.
In terms of control flow it's not significantly different. Roughly the same information ends up at the actuators, and at the central unit, achieving the same results. But it allows us to use much cheaper & more modular units, while providing a better framework for testing each individual unit in isolation.
Of course the flipside is integration hell, or what happens when all these individual units get put together and we discover that these 'perfect' units don't speak the same language in the first place 😂
@@aidanbecker9758 misra standards I get, but I've still seen things like vxworks used across industry for aerospace applications. When you're talking multiple hardware devices you can get away with it by having them easily talk over a bus , but that drives up cost and ultimately you still run interrupts on a bare metal system.
Like I could see getting away without set jump contact saving your functions which means whatever running has priority except interrupts. Which if I remember properly is a cooperative operating system. You can get away with that as long as your function has exit points for that and reevaluates system state. However, task handles would still need to be typedef function pointers in order to implement a scheduling system unless you have everything based on context state and not event based. In that case, you could run everything in a central while loop.
This article is a great relief ! having excessive checks not abnormal behaviour at all !
It might be worth mentioning that while heap memory is banned, statically preallocated memory isn't. Usually you can still use memory pools since those are not subject to memory fragmentation
I am really going to start to define max upper bounds for loops, I've never thought of that...
Hm, where would it be useful (if you can say)?
We weren't taught any coding like this at all, in fact most teach this almost in an opposite manner yet this all makes sense. This will truly change how I code.
I generally use a very simple state machine where every state is decoupled but can use same libraries. This way i can use my libraries in other projects if need be. This adds some code duplication but good enough for my embedded projects
Also note that these rules entirely with the purpose of NASA projects. It’s a mistake to generalize this to other areas like web development for example without experimenting. Code that can only be checked statically, for example, is great because you don’t have have to worry about wireless data transfer. With web development, size of your code bundle matters so there needs to be investigating there if static code philosophies work given that new variable at play. Not saying not possible by lazy loading small cdns when needed and stuff like that but this runs into my point exactly. There needs to be experiments before generalizing to all coding practices.
I think you're missing a key part of #3. He says avoid heap allocations, not to always use the stack. You can always use statically allocated memory not on the stack, anything with static storage is still fine. By exclusively using the stack you could run into a different problem, the stack overflow.
This is something I've followed for many years. If I write a server or any other program that must run for extremely long periods of time, it's easier to eliminate the possibility of memory leaks where possible than being hyper-vigilant about malloc and free. It's sometimes inconvenient, I sometimes statically allocate things like buffers much larger than I need them and that becomes an absolute hard limit. But that's okay, the OS will swap out all those upper pages I never use, and indeed probably never mapped them in the first place.
"By exclusively using the stack you could run into a different problem, the stack overflow."
If you not allow recursion (and no function pointers) you can determine the calling hierarchy of all functions at compile time and also how much stack space is allocated by each function.
So you can also determine the maximum allocated stack at compile time - ensuring that no stack overflow could happen.
@@elkeospert9188 That is true, but you may run into edge cases where the call stack goes deeper than anticipated and a stack overflow is triggered by a user. Or you may trigger a stack overflow during development and that's a sudden and nasty complication that may have no easy solution. Or you may have a lot of trouble porting the program to a system that uses a smaller stack. It's mostly an issue for small systems these days, but it pays to keep things off the stack if they don't need to be on the stack.
There's just not much reason to put _everything_ on the stack. Judicious use of the static keyword on non-reentrant functions can almost eliminate this problem. Imagine you need to form a filename from a set of strings and integers and other values. The system has a maximum path and filename length of 4096 and you must be able to generate all legal paths so if you aren't using VLAs or dynamic memory then you need a 4096 byte buffer. Slamming that onto the stack just might not be an option on a small system, but using a static buffer will not have that problem.
@@bumfuzzledgames548 "That is true, but you may run into edge cases where the call stack goes deeper than anticipated and a stack overflow is triggered by a user. "
As long as you not have direct or indirect recursive functions and you not put data with variable length on the stack it is possible to determine the maximum used stack space at compile time by creating a tree starting at "main" and having nodes for each call to a subroutine and when sum up the space needed for each branch in this tree..
"The system has a maximum path and filename length of 4096"
Typically strings are passed as references - so only the address of the String is pushed on the stack - some languages even not allow to call a subroutine and use a string as a value parameter.
The 6502 architecture is "special" because it limits the stack to 256 bytes (even the 8080 had a 16 bit stack pointer) which is eg. for compilers a problem as even with smaller programms the compiled code will crash with a stack overflow - only thing to resolve this would be to implement another stack in software having a 16 bit stack pointer but that slows down the code a lot
It's interesting that ensuring the most reliable code also involves letting go of many typical conveniences. I imagine the future of coding lies not in runtime analysis but rather in easily statically analyzed languages, like Rust.
If I was an astronaut and I visited NASA programming centre and saw that somebody uses 4 spaces instead of a tab, I would personally send them into orbit to check for asteroids ahead of me.
I hear (from them) that they are moving away from c all together, and looking at moving to Haskell for their flight controller code.
They have made a library called copilot which is a DSL in Haskell. You can easily formally verify your Haskell program, and then when you run it, it outputs C code to upload to the device. Much smarter than using C for important things
but they don't use C at all. It is a small impact. NASA mainly uses Fortran
Rust helps with (surprisingly) *every* rule:
1. Iterator adapters and Result/Option methods massively simplify control flow while making it clear what kind of control flow you are doing.
2. You can `.take(N)` an iterator to limit it.
3. Rust's ownership and Option enum mitigate the issues with using the heap, but I agree it's best to avoid when you can because the stack is faster and more obvious when you waste it.
4. Functions can be much tidier because syntax sugar and convenient methods hide the boilerplate.
5. Everything starts as private and must explicitly be made public.
6. You can declare that you `#[must_use]` a function's output, which means less cruft at call sites that don't need to be used.
7. Rust's macros are much more strict and "hygienic" than C's preprocessor.
8. Similar to 3; Pointer safety is enforced by the compiler, so layers of indirection is harder to get wrong. The logic behind avoiding function pointers is sound regardless of language, but you could make an enum with some methods on it so you can get static verification and some restricted dynamism.
9. Rust is pretty pedantic by default in comparison to C, gets more pedantic with clippy, and even more with clippy::pedantic.
10. Testing is built right into cargo and making a test is as simple as marking a function with the test attribute.
why are you comparing C and Rust? NASA uses Fortan as the main programming language and maybe also Ada to some lesser extent
@@somebody1241 I only compared to C in points 7 and 9, and that's likely influenced by every example in the video being of C code.
Rule 3 also protects you from memory fragmentation. Which shows up, in some cases, on software that is running for a long time. A fun interview question for principle programmers is to ask them why some times you can't find a parking spot, although the cars are far apart with gaps smaller than your car!?!
I am thinking of space probs that takes years to reach their destination can suffer from that.
Fragmentation can happen while there is NO BUG!😮 in the code
Yeah, write functions so they do one thing without side-effects. You don't want a function that has a errant write operation when all it's suppose to do is read.
60 line rule is interesting - if the function can be broken down into smaller functions then break it down so it's not only easier to read but also to reduce duplicate or boiler-plate code.
Interesting that they don't include "don't pass-by-reference" but maybe that goes under declare variables only in local scope because you can definitely have an erroneous value if you do pass-by-reference a lot.
Great video!
I genuinely appreciate that the title of this video isn't something like "how to write code like NASA"... It's a small difference, but a lot of other coding channels tend to gravitate toward those kinds of titles, and they often dissuade me from watching their videos.
I agree. I would not have clicked that. The declarative statement is what drew me in.
I feel like they would like Rust
Next step and much better than the requirement for safety relevant software in the IEC standards.
Love the content and format of this channel
Writing software for moon exploration is my ultimate goal.
Uhm, this feels like i asked a question in some discord, and you answered it 2 days later on youtube. thanks for the upload, this was a bit above my capabilities, but still very interesting. If despite all those precautions something went wrong, could you think of any additional measures that would make the spacecraft "reset"?
In planes systems can have "watchdogs", which are hardware features which periodically restart tge software, unless the software periodically resets the watchdog. This means that even if an endless loop occurs, it can be recovered from.
Good points. As a high-level developer the most important I think, even in a very fast start-up environment are:
#2 Limit All Loops - 1:10
#4 Limit Function Size - 2:18
#8 Restrict Pointer Use - 4:59 (in fact, TypeScript really helps deal with this)
#9 Be Pedantic - 5:34
im an 8th grader and i wanna learn this stuff. but i have no idea where or how to start :(
like... if you were starting to code all over again, howd you do it? i just wanna learn a couple languages or smth ;; but its all confusing
ty
Honestly, aside from the pedantic mode one and possibly the return value one which I don't know anything about, these are just good principles for promoting good code overall (the no function pointer one is more a regrettable reality than plain bad, though). Keeping strong principles in place can have a snowball effect on your codebase's integrity.
I guess one of the downsides of some of these rules is it likely generates pretty inflexible code, where even a small change to functionality may require a lot of lines of code to be rewritten. This is probably fine when you're creating a very conservative program with limited functionality that's unlikely to be regularly updated if at all, and especially if you've got a lot of time with a small team with extremely close collaboration and aren't working on too may things in parallel.
Thats what i was thinking 🤔🤔
I see the "Implementation of Server" in almost all your video thumbnails. I hope it changes
as several others have mentioned, embeded systems also use these rules,
I write firmware code that goes in a camera and I think, the lower level you go,
the more your code will use these rules by default.
I know I've been coding like this for the longest time, and just assumed everyone else did too..
I figured the people who ignored these rules work for game companies
You had me at NASA. 🤩
Wow, I haven't heard anyone mention a certain red crab in the comments 😅
This seems like an excellent use case for Rust, zero overhead so all the performance of C/C++, while also having incredible static analysis + type safety.
but anyways NASA's main programming language in use is Fortran
Great talk :) The conflation of static typing with setters/getters triggered me pretty hard ngl. [Shakes fist at cloud] 😂
alien reverse engineers are going to think we're pretty dumb
I wonder if they'll switch to rust
I've always coded on the "don't make your functions longer than a page" rule. Sure we all do it, but you aught to get that feeling that the function is ripe for a restructuring. Similarly code which is all functions with only 2-3 lines in them can be difficult to understand. I tend to do this only when the either the function is used a lot or for certain aesthetic reasons so it is more readable.