Make your Data Type more Abstract with Opaque Types in C
Vložit
- čas přidán 5. 04. 2021
- Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.thinkific.com
Website ➤ www.jacobsorber.com
---
Make your Data Type more Abstract with Opaque Types in C // When you think about abstract data types (ADTs) you probably think about object-oriented languages, like C++, java, ruby, or python. But, even though C isn't object-oriented and doesn't have a private keyword, you can still limit data type visibility using opaque types, in the same way that LibC does with FILE pointers. This video shows you how.
Related Videos:
Queue Example: • Implementing a Circula...
IFDEF guards: • Header Issues: Guards,...
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorber.com
people.cs.clemson.edu/~jsorber/
persist.cs.clemson.edu/
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
Source code is also available to Patreon supporters. --- [jsorber-youtube-source.heroku...]
I have used opaque types in C for years, but I didn't know that they where called opaque types until today. Thanks
Thank you for explaining not only what opaque types are and how to implement them, but also why one might want to use them. I've only just begun learning C, but you've helped me to appreciate C for what it is and to leave my OO baggage behind.
This is perfect thank you. Top of my OS class thanks to all these. We NEVER learn anything about writing good C!
You're welcome. Happy to help!
Honestly your videos are the best! Keep posting!
Thanks. Glad you like them!
One reason that incomplete data types are allowed is that they are sometimes necessary. You might have two structs that reference each other with pointers, and then you must first put an incomplete declaration of one of the structs, then a complete declaration of the other, and lastly the complete declaration of the first one.
To me, this is the best pattern I learned for C. This makes everything better!
Very well explained ! First time on this channel and instant subsciption !
Excellent explanation. They key was the forward declaration of the struct
Thank you, sir! This is exactly what I'm waiting for!
Heard a lot about incomplete data types but your video explains perfectly. Thanks.
Welcome.
This was very helpful. Thanks!
thanks for this,, it is awesome how you conveyed this information
Very nice video, thank you.
I'd like to see a video on virtual function tables on structs. I am a Go programmer and it's convenient having function members that tell me exactly what my "object/struct-instance" can do simply by typing a period. In Go, struct methods are just syntactic sugar for "mystruct.mymethod(&mystruct, somevar);". However in Go, "&mystruct" is passed implicitly into the function if it is a pointer receiver. Go doesn't have inheritance either, and IMO it's very similar to C. I am new to C but after watching a few of your videos I wrote a simple Vector struct but I wanted to emulate go so it worked like:
Vector* my_vector = new_vector(100);
my_vector->add(my_vector, 1);
etc...
Very good information! Thanks.
You're welcome. Glad it was helpful!
hi, instead of using 2 malloc (one for meta data and data), i'd suggest using array inside the allocated data
typedef struct {
int size;
int num_entries;
int head;
int tail;
int values[0];
} queue;
you can allocate them using
queue * q = malloc(sizeof(queue) + max_size);
the structure can be free using single free, just suggestion, i usually use this myself
this is clever and does work well but if you intend to use realloc (or a similar custom function) to resize the array then you have to be able to reassign the value of the original pointer. This can potentially create problems that are avoided by just having a pointer to separately alloc'd memory in the struct, since in that case you can realloc that pointer and the leave the struct pointer untouched.
This is great. Who needs C++?
Thank you very much for teaching C to us!
My pleasure!
Can you make a video on polymorphism and casting a pointer of one struct type into another like we do `struct sockaddr_in` to `struct sockaddr` and other places and how pointer casting between structs can give us polymorphism behavior in C? It will be really helpful if you can make a video on this specific topic. Anyways, amazing video as always :-)
Thanks. Good topic idea. I'll see what I can do.
Keep in mind that this is a violation of strict aliasing, unless you have all those structs in an union. If you put sockaddr_in and sockaddr into an union, you can alias both with the union type wihout violating strict aliasing. You just can't alias plain sockaddr_in and sockaddr with each other. You can include sockaddr_storage in the union, and it should have enough space for any of the sockaddr types.
I love a channel. Thanks Jacob, u are the man.
You're welcome. Glad you're enjoying it.
Wow! Great video!
Thanks!
Awesome!
great video, thanks!
You're welcome!
Love it!
Thanks, Mark.
And there's a step more on top of this. Using a handle or a void * to the struct so you don't even expose the struct type. Really handy for libraries.
Amazing, I already know all of this but glad to see that you're making these high quality beginner videos.
I've heard a few compelling arguments about not abusing typedefs and polluting the global namespace. I prefer to typedef only in the .c where it can help to clean up some of the verbosity of the implementation. But in the .h keep the "interface" clear on exactly what is what.
Is the typedef queue* technique similar to a forward declaration in C++?
Thank you for the video, I was implementing all my libraries but without the last step (an important one :P). However I have a question.
I don't really understand the difference between:
void queue_init (queue_t* queue, int max_size) vs queue_t* queue_init (int max_size). Could someone explain me why one is better that another
Can you do video on Implementing simple custom malloc? Great video btw thanks!
try randal bryant's and o'hallaron's computer systems book, there is an excellent section on memory allocators (and an exercise that has you writing a _really_ solid one from scratch)
Best channel ever
Thanks.
Is there any benefit in repeating the typedef in the source file?
Just declaring the `struct myq { int *values; /*...*/ };` and including the header (which has the typedef) should still compile, right?
niceee!
Is it possible to baboozle the linker by guessing the signature for something that isnt in the .h of a library? I know that the names of functions aren't in the compiled code, but I have been wondering about this.
I'm learning in the land of game modding, and we call the games code all the time, but I think usually you want to find a pointer to that function, but can you theoretically call a function just by guessing the signature?
Ive used Microsoft detours to intercept a games call to the windows API (Dark Soul Remastered changes your PC mouse settings and I made a mod to stop it 😂) and that seems to just use the function signature.
Any thoughts? Thanks for the great videos!
I'd like to see a video about benefits of making header files standalone-compilable, so they contain all other headers required to ensure the header can be compiled on its own, and why this is a good thing (or downsides, if there are any).
Thanks! I'll ad it to the list.
Is it possible to make parts of the struct accessable and others not ?
Is the burden of defining the struct on the users of the header interface then? Or if it’s defined in the my_queue file how can a user properly create a queue struct and assign all/correct attributes?
Do you do code review, like reviewing my C library and such? If so, how can I approach you?
I do, occasionally. More instructions here: czcams.com/video/k2K2HVg4Arc/video.html
Sooo if you can't see the size of it - you can't malloc them?
Well explained. However, there was a good opportunity for mentioning "getters" and "setters" which I would have taken :)
Right in time! I just started a project that needed opaque pointers. I'm wondering if it is possible to declare the struct in the main file, not just the pointer and then pass the address (&struct) to the functions. I'm working with embedded devices and malloc is not supported. Thanks!
I don't know if i'm understanding question right, but i'll try to answer.
You will still need to either allocate memory somehow or make global variable for structure, because if it's created in stack of some function, it will disappear once that function ends. You probably can do some crazy workaround like keeping function that creates objects in separate tread, which is mostly sleeping, so function won't end and free the stack, and treat that as a "malloc", but it is way too much overhead.
Got another idea, that is probably closer to what you need.
You can create another struct, of same size. Creation function will return this "faux" struct as value, and all functions that need to work with your struct will take pointers to faux struct. Since sizes are same, you will need to convert the pointers, but other than that, everything should be fine.
This allows you to store struct without malloc in a way that allows it to persist. So for example you have vector type, of size 42 (for some unknown reason), it will look somewhat like this
header:
typedef struct {
char[42];
} fauxVec;
float norm(fauxVec* vec);
code:
typedef struct {
} vec_t
float norm(fauxVec* vec) {
vec_t* trueVec=(vec_t*)vec;
}
@@wumi2419 Hi, thanks for answering. Maybe I wasn't clear enough. Following the queue example, what if I want the queue instances to be declared at compilation time. I don't need a function that allocates the struct and returns the pointer. The user should be able to create as many instances as wanted when using the API but again everything is allocated at compilation time, nothing dynamic. Something like:
#include "queue.h"
Queue q1;
Queue q2;
int main() {
foo(&q1, args);
}
@@00zuu00 Nope this wont be possible as you cannot create an instance for an opaque type in main. If your concern is regarding usage of dynamic allocation, you can use a large object pool of required structs at compile time and when queue_create is requested, pass on the index from the object pool as the address of the queue.
@@deepakr8261 Ok, thanks! It makes sense if I think of it now, just wanted to know if there was a way to do it that I was missing. The object pool idea is not bad, nevertheless my objects are "big" compared to the available memory, I'll investigate further for alternatives.
I have a couple of questions:
1. Do opaque types have to be a pointer/address of that struct?
2. Is there anyone to declare “public” variables in the .h but still have other “private” variables declared in the .c file?
1. At compile time, the compiler cannot tell the size of the struct. This is why it cannot create memory on the heap, and instead we need to `malloc` the appropriate amount of memory.
2. Yes, any variable that isn't exposed in the header file, is limited to the scope of the translation unit it's declared in (aka the C file).
I think that that is default behaviour, and you can use `static` to make it explicit, but I am not sure about that.
@@anon8510 I think Samuel changed his question.
is there a way of doing this if the struct is not passed as a pointer to a function declared in the .h file?
I'd say nope. A pointer to 'anything' is the same size 4 bytes for x86 8 bytes for x64. This is the power and the pain of C type languages. As long as you know what your doing the compiler will go along with you but... you'd better be right
@@spudgossie I agree - thank you for your answer!
I thought the concept of 'opaque types' in C referred to void * pointers, typedefed to a meaningful name such as 'queuehandle_t', and then typecasted to (or assigned to a pointer variable of) the correct type inside of the function. I guess this is more typesafe and i need to do some refactoring now :P.
Awesome video!
Often times C would lack features and one would have to either live with the fact that these features don't exist or make crazy workarounds that are questionable and often insane.
However, in the case of abstraction, I think C does it the best. I code C++ but I still make my module interfaces the C way. C really does hide all the things that don't need to be shown, while pretty much every other language would show these things but with a special keyword like 'private' which introduces dependency issues. So now the C++ people do workarounds to imitate the C way of doing things, and do things like the 'pimpl' idiom.
In short, C interfaces win over pretty much every other language.
I totally agree. Encapsulation in C++ is broken by design.
@@sghsghdk I see a problem with the need of declaring all the class in the same file. I think it would be better if the header had the public part and the cpp, the private part. The protected maybe it would be better placed in the header... but I don't know enough about C/C++.
What if I cannot use malloc()? In embedded software development, especially when it comes to bare metal, dynamic memory application is a no-go and so I have no choice but to expose my implementation in the header file.
Yes, at which point the easiest way to do it is just exposing the struct but have underscores prefixed to every member variable. By convention people don't mess with member variables that begin with underscore.
Hey! Would you mind telling me more about why we shouldn't use dynamic memory allocation in embedded systems?
Good idea for a future video. Thanks. Short answer is, it introduces nondeterminism, which can introduce bugs, especially when you don't have much memory to work with.
Also, one option is to use an object pool - acts like dynamic memory allocation, but with much less nondeterminism.
@@lordadamson I personally prefer subtle comments, like // DO NOT MESS WITH THE INTERNALS, USE THE DAMN API OR I WILL DISABLE DARK MODE ON YOUR IDE!!@#$%& xD
you don't need the second typedef, just the struct definition
12:15 I was, yes :)
Looks like there's a bug in your queue or in your test. The test queues 6 values but only prints the first 5.
I undestand how to use this shema for encapsulation... But i still cant understand how to use it for polymorphysm or inheritence, unless it is not a case of Opaque Types
You can use composition to do inheritance with this. If you have struct foo, you can have another struct bar whose first member is a struct foo. Now you can cast a pointer to a bar to a pointer to a foo and it will work fine because both structs begin at the same time.
Polymorphism is a little more complicated, but there’s a way to do it with function pointers as members of the parent class.
I understand that an interface where implementation details don't need to be obvious to the user is important. I also know that public member access is "wrong" in most cases, but I don't know why. Could anyone explain why?
In this case it's because the attributes in the queue struct are used to represent the queue's state. If you fiddle with these directly, e.g. you decrease the "head" variable directly, then the next time you try to enqueue an element you would end up overwriting data.
The top g strikes again.
Another way how to hide implementation just provide *.h file and library *.so instead of *.c file
That's not so much "hiding" implementation but just making it consumable by dynamically or statically link to a binary which contains the implementation and the header file as a reference for what is exposed. The concept that this video talks about is about API surface area to prevent unintentional behavior from changing internal state.
The example in the video, where he was able to change head field would not perform the necessary steps to ensure the queue works as intended.
Please remove the bottom right red icon... It actually hinders with the overall theme of the video.
I really like your videos. Which university do you teach in? I would love to get in and have you as my teacher :)
I teach at Clemson University. See you in class. 😀
@@JacobSorber Already researching about it 😃
But technically I am still 15.. Maybe I should focus on High School for now. 😅
I think your definition of abstraction is not entirely correct.
I can make abstraction that doesnt hide anything from user. If you define abstraction as simply "naming something" then it makes more sense even outside of CS. I think it may be beneficial to think of abstracting data as naming it. If I make a struct holding int x and int y and name it Point, ive added more abstraction to the code.
First!
dude just use python already
i suffered for years with strict datatypes until there was python and i could not believe what the gang of two did ... 10 years now i have never touched c again in my life
@@JusticeNDOU But what if i want performance instead of easy to type, slowness of python?
@@nonnullptrhuman504 depends on what you are coding, what are you coding ?
@@JusticeNDOU kernel development, opengl
I used python and got overwhelmed by the standard library
Why not have queue in the header file be void? That way the pointer would be a void pointer.
You definitely could do that. I'm not sure what the advantages would be, though. It would make the type a little more opaque.
@@JacobSorber Oh I do that just so I don't have to name my struct before I typedef it. Other than that I don't think there are any real advantages
My concern with using void* as the argument is that any pointer is implicitly casted to void* . So if you accidentally pass a different pointer to the function it will compile without warning. I had to work in an environment like that, and I used "magic id" as a first field to be sure I have the correct struct, but that is a runtime check. I prefer to have a compile time warning/error.