Linux Kernel wtfness



  • So, for some reason (masochism?) i'm trying to understand how to deal/write Linux Kernel Drivers.

    Of course, trying to understand the way it works by reading the code is useless, we know that.
    There is some documentation (txt files in the source tarball) but it sucks and we know that too.
    I'm now reading a book written by some big name(s) and... well, just read:

    We start,however,with a bottom-up presentation of the device model. The complexity
    of the device model makes it hard to understand by starting with a high-level
    view. Our hope is that,by showing how the low-level device components work,we
    can prepare you for the challenge of grasping how those components are used to
    build the larger structure.
    For many readers,this chapter can be treated as advanced material that need not be
    read the first time through. Those who are interested in how the Linux device model
    works are encouraged to press ahead, however, as we get into the low-level details.

    So... you think it's better to start bottom-up (??) but you don't recommend it. Nice start.

    ...
    As an example,let’s look back at struct cdev,which we encountered in Chapter 3.
    That structure, as found in the 2.6.10 kernel, looks like this:
    struct cdev {
    struct kobject kobj; <------ (my annotation)
    struct module *owner;
    struct file_operations *ops;
    struct list_head list;
    dev_t dev;
    unsigned int count;
    };
    ...
    The end result is that a structure protected by a kobject cannot be freed at any single,
    predictable point in the driver’s lifecycle,but in code that must be prepared to
    run at whatever moment the kobject’s reference count goes to 0. The reference count
    is not under the direct control of the code that created the kobject. So that code must
    be notified asynchronously whenever the last reference to one of its kobjects goes
    away.
    This notification is done through a kobject’s release method. Usually,this method
    has a form such as:
    void my_object_release(struct kobject *kobj)
    {
    struct my_object *mine = container_of(kobj, struct my_object, kobj);
    /* Perform any additional cleanup on this object, then... */
    kfree(mine);
    }

    So, kobject is embedded in another structure and controls its lifecycle. container_of calculates the offset of kobj field in the container structure to get address of the structure itself. WTF. Why not add a damn pointer INSIDE kobject to the target structure? This raises a WRONG WAY road signal in my head. This seems to only spare 1 more malloc() imho.

    Then, there are ksets which are sets of kobjects...

    Adding a kobject to a kset is usually done when the object is created; it is a two-step
    process. The kobject’s kset field must be pointed at the kset of interest; then the
    kobject should be passed to:
    int kobject_add(struct kobject *kobj);

    .... ARE YOU FUCKING KIDDING ME? Was kset_add(struct kset *kset, struct kobject *kobj) too easy to understand?

    There is a convenience function provided by the kernel:
    extern int kobject_register(struct kobject *kobj);
    This function is simply a combination of kobject_init and kobject_add.

    ...so in that case, again, the kset field must be set manually. It's not getting better, only more confusing.

    A kset also has a name,which is stored in the embedded kobject. So,if you have a
    kset called my_set, you would set its name with:
    kobject_set_name(&my_set->kobj, "The name");

    Embedded kobject? What's that? It's the first time this is mentioned, no idea what it is used for. And WTF, i want to give a name to the kset so, again, even if it makes sense using that embedded shit just to keep the name, was an abstraction like kset_set_name(my_set, "The name") too obvious?




  • @Zmaster said:

    So, for some reason (masochism?) i'm trying to understand how to deal/write Linux Kernel Drivers.


    If you're looking for a truly masochistic experience, try writing an EFI device driver. Or even reading one. Awful coding style, and an obnoxiously verbose, bureaucratic and unreadable clusterfuck of an API.

    @Zmaster said:

    Of course, trying to understand the way it works by reading the code is useless

    Have you tried that even? Call my judgement skewed, but compared to EFI I find reading Linux kernel code a pleasant experience.

    @Zmaster said:

    This seems to only spare 1 more malloc() imho.

    Sparing 1 malloc 34123094 times means 34123094 spared mallocs. Which may have huge performance impact. And everyone knows how Linus cares about performance.

    @Zmaster said:

    Adding a kobject to a kset is usually done when the object is created; it is a two-step
    process. The kobject’s kset field must be pointed at the kset of interest; then the
    kobject should be passed to:
    int kobject_add(struct kobject *kobj);

    Your book seems out of date, [url=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/lib/kobject.c?id=43968d2f1648f4dc92437dc0363a3e88377445b3#n160]kobject_add had this prototype in 2008[/url], it has [url=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/lib/kobject.c#n312]changed[/url] since, two times I think.

    (And I am not even that knowledgeable about the kernel.)



  • @spamcourt said:

    @Zmaster said:
    Of course, trying to understand the way it works by reading the code is useless
    Have you tried that even? Call my judgement skewed, but compared to EFI I find reading Linux kernel code a pleasant experience.

    Yes. I'm not sure if i'm good or bad at doing it anyway. But while i think that having the source available can get handy, i'm against using it as the primary documentation source (autodoc is ok of course).

    @spamcourt said:

    @Zmaster said:
    This seems to only spare 1 more malloc() imho.

    Sparing 1 malloc 34123094 times means 34123094 spared mallocs. Which may have huge performance impact. And everyone knows how Linus cares about performance.

    AFAIK, kobjects are only used to build sysfs and i think this is not performance critical.

    @spamcourt said:

    @Zmaster said:

    Adding a kobject to a kset is usually done when the object is created; it is a two-step
    process. The kobject’s kset field must be pointed at the kset of interest; then the
    kobject should be passed to:
    int kobject_add(struct kobject *kobj);


    Your book seems out of date, kobject_add had this prototype in 2008, it has changed since, two times I think.

    (And I am not even that knowledgeable about the kernel.)

    Of course! The book is open, so yeah it's outdated: http://lwn.net/Kernel/LDD3/
    That "third edition" did trick me, it's from 2005.
    I'm not sure if the fact that the changed the API is a WTF or the RWTF is that is was so bad before. Oh wait, it's still as bad: no improvement regarding the kset.

     



  • @Zmaster said:

    AFAIK, kobjects are only used to build sysfs and i think this is not performance critical.


    container_of is quite widely used for hopping through various data structures, not just kobjects. So since it's already there, why not use it?

    @Zmaster said:

    Of course! The book is open, so yeah it's outdated: http://lwn.net/Kernel/LDD3/

    Ah. So, back to your original post…

    So... you think it's better to start bottom-up (??) but you don't recommend it. Nice start.
    You must have a quite unique definition of "start" if the fifth-from-the-last chapter falls under it.
    Embedded kobject? What's that? It's the first time this is mentioned, no idea what it is used for.
    Earlier in the chapter… @The Book said:
    Therefore, the main function of a kset is containment; it can be thought of as the top-level container class for kobjects. [b]In fact, each kset contains its own kobject internally, and it can, in many ways, be treated the same way as a kobject.[/b] It is worth noting that ksets are always represented in sysfs; once a kset has been set up and added to the system, there will be a sysfs directory for it. Kobjects do not necessarily show up in sysfs, but every kobject that is a member of a kset is represented there.
    Think: inheritance.
    And WTF, i want to give a name to the kset so, again, even if it makes sense using that embedded shit just to keep the name, was an abstraction like kset_set_name(my_set, "The name") too obvious?
    Short answer: Yes. Long answer: I guess the reason is that the Linux kernel developers prefer to immediately see what operations are involved to estimate their cost. Abstraction takes that away. [url=http://www.joelonsoftware.com/articles/LeakyAbstractions.html]And that's terrible.[/url] (Also, look up Torvalds comparing C++ with C, I remember him mentioning something similar.). This particular one wouldn't be actually very expensive (just a macro, or equivalently a static inline function, so it would just compile quarter of a second longer), but it's the principle that matters. Also, if kobjects gain fifty new routines operating on them, should ksets gain fifty wrappers around them? And the other hundred structures which in this way wrap/extend a kobject? Occam's razor, basically.
    That "third edition" did trick me, it's from 2005.
    I'm not sure if the fact that the changed the API is a WTF or the RWTF is that is was so bad before. Oh wait, it's still as bad: no improvement regarding the kset.
    [url=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/stable_api_nonsense.txt]Documentation/stable_api_nonsense.txt[/url]. The reasons listed there might not necessarily be directly relevant here, though. On closer inspection this particular interface hasn't changed very much. I guess this particular change was motivated by convenience, since all the consolidated operations here have been probably done together, and a desire to keep the number of kernel interfaces to minimum for the sake of maintainability - so instead of creating a new call, they modified the existing one.


  • If you feel bad about UEFI, don't worry, in just two or three decades we're gonna switch to another standard.



  • @Zmaster said:

    Of course, trying to understand the way it works by reading the code is useless, we know that.

    My experience is that trying to understand Linux kernel by reading the code is both possible and actually the only way, because the documentation is usually outdated. @Zmaster said:

    So, kobject is embedded in another structure and controls its lifecycle.

    There is one key thing you need to understand. First member of structure in C is it's base class. Sometimes later members too when multiple inheritance is needed. Linux kernel is heavily object oriented. @Zmaster said:

    container_of calculates the offset of kobj field in the container structure to get address of the structure itself. WTF.

    That's exactly what upcast in C++ compiles to. Except this is C, so one has to write it out. Definitely not a WTF. @Zmaster said:

    Why not add a damn pointer INSIDE kobject to the target structure? This raises a WRONG WAY road signal in my head. This seems to only spare 1 more malloc() imho.

    That would be two pointers and extra allocation overhead and breaking reference locality and cache alignment. And it would be totally useless. You are usually taught that you shouldn't care about these things much. And for application development it is usually true. But kernel is the odd man out here. Kernel performance affects a lot and it still has to be usable on slow hardware (like various embedded controllers) so in kernel it matters.

    And it would actually be wrong way. Because your structure is a kobject. So it really should embed it by value. Your wrong way flag is wrong this time. @Zmaster said:

    The kobject’s kset field must be pointed at the kset of interest; then the
    kobject should be passed to:
    int kobject_add(struct kobject *kobj);

    .... ARE YOU FUCKING KIDDING ME? Was kset_add(struct kset *kset, struct kobject *kobj) too easy to understand?

    While I don't see the reason behind this one, I am sure the author did have one. Perhaps the two steps are sometimes done separately for some reason. Or have to be done under different locks or the later can't be done in some context where the former sometimes has to be or something.

    You see, Linux kernel has two types of locks, semaphores and spinlocks, that interact in funny ways. Especially there are many things not allowed while you hold (any) spinlock. Like locking semaphores or allocating memory. I bet kobject_add does both.

    Linux kernel also has these various context. In the process context, that is in code triggered by system call or in kernel thread, you can do almost anything. But than there are interrupts, bottom halves and tasklets and you can't do many things in them. Like locking semaphores or allocating memory (there is a special flag to kmem_cache_alloc which allows calling it under those contexts, but that you have to be prepared to deal with failure even if there is plenty of memory available...). @Zmaster said:

    Embedded kobject? What's that? It's the first time this is mentioned, no idea what it is used for.
    Remember. First member of C structure is it's base class. kset is a kobject. @Zmaster said:
    And WTF, i want to give a name to the kset so, again, even if it makes sense using that embedded shit just to keep the name, was an abstraction like kset_set_name(my_set, "The name") too obvious?
    So you override all methods just to provide abstraction, even if they just call the base class? Kset is a kobject, kobject has a method to set name. Kind of obvious. kset_set_name would look like it's something else than the base class method. That would be anti-abstraction.


  • In short,the "real WTF" would be trying to do object-oriented programming with inheritance in C instead of C++. And even that is probably justifiable by context.



  • @Medinoc said:

    In short,the "real WTF" would be trying to do object-oriented programming with inheritance in C instead of C++. And even that is probably justifiable by context.

    What about "a book about the subject leaves out several extremely important concepts required for understanding the subject"?



  • @blakeyrat said:

    @Medinoc said:

    In short,the "real WTF" would be trying to do object-oriented programming with inheritance in C instead of C++. And even that is probably justifiable by context.

    What about "a book about the subject leaves out several extremely important concepts required for understanding the subject"?

    Works for me. A book should clearly define such concepts at the start, especially in a "bottom-up" approach.

     



  • @blakeyrat said:

    @Medinoc said:

    In short,the "real WTF" would be trying to do object-oriented programming with inheritance in C instead of C++. And even that is probably justifiable by context.

    What about "a book about the subject leaves out several extremely important concepts required for understanding the subject"?

    By which you mean specifically…?



  • Read Zmaster's original post, where he didn't understand what an "embedded kobject" was, presumably because it wasn't covered in the book before being casually mentioned as something you should already know.



  • @blakeyrat said:

    Read Zmaster's original post, where he didn't understand what an "embedded kobject" was, presumably because it wasn't covered in the book before being casually mentioned as something you should already know.

    Read the third reply. It was, a few paragraphs earlier, they just used different phrasing. "Embedded kobject" means exactly a kobject which is embedded. This is basic reading comprehension.

    TRWTF: People who barely skim texts about complex topics and later complain that they cannot understand them.



  • @spamcourt said:

    "Embedded kobject" means exactly a kobject which is embedded. This is basic reading comprehension.

    Duh! A frobbed dingleheimer means a dingleheimer which was frobbed! This should be clear to everyone!



  • Well poopity poopity poo.

    If the book is going to use the term "embedded kobject", it needs to define it somewhere. The quoted text in the OP didn't. I dunno. Maybe it's there. I'm trusting the OP gave us all the relevant stuffs.

    Hey here's a thought: maybe the author could have written, "a kobject that is added to a kset can be referred to as an 'embedded kobject'." Look! Now it's crystal clear! No confusion!!!!!!!!!!!!!!!!!!!!!!!!!!



  • @blakeyrat said:

    Hey here's a thought: maybe the author could have written, "a kobject that is added to a kset can be referred to as an 'embedded kobject'." Look! Now it's crystal clear! No confusion!!!!!!!!!!!!!!!!!!!!!!!!!!

    First, it means "the kobject which is a structure member of the kset". Second, a quick glance at the relevant source code would clear this up. Third, there are two paragraphs devoted to the subject four pages earlier. If only someone read them.

    [b]Embedding kobjects[/b]

    Before we get into the details, it is worth taking a moment to understand how kob- jects are used. If you look back at the list of functions handled by kobjects, you see that they are all services performed on behalf of other objects. A kobject, in other words, is of little interest on its own; it exists only to tie a higher-level object into the device model.

    Thus, it is rare (even unknown) for kernel code to create a standalone kobject; instead, kobjects are used to control access to a larger, domain-specific object. [i]To this end, kobjects are found embedded in other structures. If you are used to think- ing of things in object-oriented terms, kobjects can be seen as a top-level, abstract class from which other classes are derived. A kobject implements a set of capabilities that are not particularly useful by themselves but that are nice to have in other objects. The C language does not allow for the direct expression of inheritance, so other techniques—such as embedding one structure in another—must be used.[/i]

    Did they use this specific phrasing? No, but someone who [i]reads[/i] instead of randomly skimming will not care.



  • @blakeyrat said:

    Hey here's a thought: maybe the author could have written, "a kobject that is added to a kset can be referred to as an 'embedded kobject'." Look! Now it's crystal clear! No confusion!!!!!!!!!!!!!!!!!!!!!!!!!!
    Personally, I prefer if my technical books aren't written in legalese. While it may be a form of english, it's overly verbose an the nitpickyness of it gets in the way of reading comprehension, which is especially important in a technical manual.



  • @Circuitsoft said:

    Personally, I prefer if my technical books aren't written in legalese. While it may be a form of english, it's overly verbose an the nitpickyness of it gets in the way of reading comprehension, which is especially important in a technical manual.

    Well an even better solution would be for the author to pick one, and only one, term for the thing and consistently use it everywhere. Then you wouldn't need that awkward sentence saying "oh BTW, sometimes I call X Y".

    This reminds me of that discussion we had about Heroku and whether their "getting started" page was intended to guide a person through getting started *using Heroku* or getting a new Heroku project started.



  • @blakeyrat said:

    Well an even better solution would be for the author to pick one, and only one, term for the thing and consistently use it everywhere. Then you wouldn't need that awkward sentence saying "oh BTW, sometimes I call X Y".

    oh BTW, sometimes I call idiots who blither "blithering idiots".

    @blakeyrat said:

    This reminds me of that discussion we had about Heroku and whether their "getting started" page was intended to guide a person through getting started using Heroku or getting a new Heroku project started.

    To the extent that both of them involve you seizing a perceived opportunity to launch a spurious attack on some perfectly clear document as unfair discrimination against people who can't be bothered reading it: I agree.



  • @blakeyrat said:

    Well poopity poopity poo.

    If the book is going to use the term "embedded kobject", it needs to define it somewhere. The quoted text in the OP didn't. I dunno. Maybe it's there. I'm trusting the OP gave us all the relevant stuffs.

    Hey here's a thought: maybe the author could have written, "a kobject that is added to a kset can be referred to as an 'embedded kobject'." Look! Now it's crystal clear! No confusion!!!!!!!!!!!!!!!!!!!!!!!!!!

    You should write a book about Linux.

     



  • @El_Heffe said:

    You should write a book about Linux.
     

    Subheading "Poopity Poo"



  • @dhromed said:

    @El_Heffe said:
    You should write a book about Linux.
    Subheading "Poopity Poo"
     


     


Log in to reply