Bitfield wtf



  • So we have some component production equipment that runs some tests on every part that goes through and stores the data in a text file. I wrote a script that parses this data and plugs it into a database for reporting. We're ramping up production to where it's going to be pumping out a massive amount of data and since they were updating the software to handle this I asked if they could cut some of the data down by combining a bunch of related boolean fields into a single bitfield. Their solution was to give me a field with data like
    "122122". They basically just concatenated all the booleans into a string field and decided to go with 1s and 2s instead of 0s and 1s for whatever reason. I know it's not that big of a wtf and i can just convert to bitfield on my end but I'm irritated that we're paying this outside contractor ungodly amounts to write this stuff and he apparently doesn't understand the concept of a bitfield, or maybe even binary. /endrant


  • Discourse touched me in a no-no place

     maybe they're Postgresql users. Last I tried to use that sorry excuse for a database, the .net connector library didn't support bitfields - so you HAD TO use a string in the database if you were going to allow .net access.



  • I'm on their side.Text files - traditionally ASCII but today perhaps UTF-8 - are a marvellous way to transfer data from one system to another. You can read it, you can edit it, you can parse it, you can easily create test data. I love XML, for just this reason. It's not only data, it is VISIBLE data. If you've ever worked with RTF files and DOC files you can appreciate the difference.

    Dump a bit field into that and it's no longer text. Notepad can't read it, you can't print it, you can't even display the first lines to check the format. You can't use 'wc' or 'head' or 'tail' or 'awk' or any of these marvellous tiny tools to examine and work with the data. Suddenly the only way to work with the file is to use your code, which is of course perfect (ha ha). Lots of trouble. Better to keep text files as text, even at the cost of seven extra bits per data bit.



  • @AndyCanfield said:

    I'm on their side.Text files - traditionally ASCII but today perhaps UTF-8 - are a marvellous way to transfer data from one system to another. You can read it, you can edit it, you can parse it, you can easily create test data. I love XML, for just this reason. It's not only data, it is VISIBLE data. If you've ever worked with RTF files and DOC files you can appreciate the difference.

    Dump a bit field into that and it's no longer text. Notepad can't read it, you can't print it, you can't even display the first lines to check the format. You can't use 'wc' or 'head' or 'tail' or 'awk' or any of these marvellous tiny tools to examine and work with the data. Suddenly the only way to work with the file is to use your code, which is of course perfect (ha ha). Lots of trouble. Better to keep text files as text, even at the cost of seven extra bits per data bit.

    I'm assuming he meant an ASCII representation of a bitfield, either decimal or hex, ie 170 or AA.



  • @error_NoError said:

    Their solution was to give me a field with data like
    "122122". They basically just concatenated all the booleans into a string field and decided to go with 1s and 2s instead of 0s and 1s for whatever reason.

    That suggests that they're using integers instead of strings. Possibly internally, if they're returning strings to you.

    If you have a string like 000110, it will get converted to an integer 110, which is ambiguous. Does it consist of just 3 fields? If the fixed width is 6 fields, does it have leading zeros or following zeros?


    I'd say they're using too many casts and conversions not to be a wtf.



  • That's just future proofness.

    1 means TRUE, 2 means FALSE (or the other way around), and the other character codes (eg. 0, Z, and ÷) are reserved for things like ENOENT, EMAYBE, etc.



  •  @flop said:

    That's just future proofness.

    1 means TRUE, 2 means FALSE (or the other way around), and the other character codes (eg. 0, Z, and ÷) are reserved for things like ENOENT, EMAYBE, etc FILE_NOT_FOUND.

    FTFY

     



  • @Shondoit said:

    If you have a string like 000110, it will get converted to an integer 110, which is ambiguous. Does it consist of just 3 fields? If the fixed width is 6 fields, does it have leading zeros or following zeros?

    I'd say they're using too many casts and conversions not to be a wtf.

    Seriously?  I mean seriously?  Do you know how things really get represented or are you really that clueless?

    000110 = 6 when converting from base 2 to base 10.  If you save it for human consumption you save it as 6 which is the sum of all the on bits. 6 is the value of that field regardless if you are displaying in base10,  base2, base16, or whatever base you are displaying the value.  Setting values your OR the bit field with the value of the field you want to set, your result is still the sum of all the on bits.

    Remember, there are truely only 10 kinds of people in this world.

     



  • @KattMan said:

    @Shondoit said:

    If you have a string like 000110, it will get converted to an integer 110, which is ambiguous. Does it consist of just 3 fields? If the fixed width is 6 fields, does it have leading zeros or following zeros?

    I'd say they're using too many casts and conversions not to be a wtf.

    Seriously?  I mean seriously?  Do you know how things really get represented or are you really that clueless?

    [...]
    Seriously? I mean, seriously?
    Ofcourse I know how bitfields are stored. I know how they can be represented as numeric values, byte values, base64, whatever. Did I not state that they overused casting and conversions?
    Like I said, they probably, wrongfully represented the booleans as a concatenated string, then converted it to integers and finally converted it back for textual representation.

    If I wasn't clear enough: it's a WTF, I agree. I was merely explaining a possible cause.
    If you have a possible cause, one can suggest a possible solution to possibly educate the ignorant.


  • @Shondoit said:

    If I wasn't clear enough: it's a WTF, I agree. I was merely explaining a possible cause.
    If you have a possible cause, one can suggest a possible solution to possibly educate the ignorant.

    lol, ok.

    Communication WTF's are wholly shared by both parties.



  • @Weng said:

     maybe they're Postgresql users. Last I tried to use that sorry excuse for a database, the .net connector library didn't support bitfields - so you HAD TO use a string in the database if you were going to allow .net access.

     

    Erm? I've been using PgSQL since what is it, 2005 or so, and it (of course) supports bitfields fine - you just store them as an integer (same as you would in MySQL). I'm assuming you come from MSSQL which - presumably - has a dedicated bitfield type your connector - obviously - didn't understand?

     



  • Ok maybe this wasn't as much of a wtf as I thought it was. As it turns out this is being written in a GUI language which may not be so easy to implement a bitfield. I had also fumed past the fact that a 1-byte bitfield could result in non-printable chars in this plain text file and using a string representation of its decimal value would only cut the bytes by half.

    There have been so many wtfs with this project I just kind of flipped when I saw my bitfield request turned into a string of 1s and 2s. It would have been easier to parse if they had left it as separate fields of 0s and 1s. I still don't think he understands the concept of a bitfield from the verbal conversations I've had with him.



  • @error_NoError said:

    I still don't think he understands the concept of a bitfield from the verbal conversations I've had with him.

    If he's database-y, just refer to them as different columns of type bit.



  • @error_NoError said:

    Ok maybe this wasn't as much of a wtf as I thought it was. As it turns out this is being written in a GUI language which may not be so easy to implement a bitfield. I had also fumed past the fact that a 1-byte bitfield could result in non-printable chars in this plain text file and using a string representation of its decimal value would only cut the bytes by half.

    There have been so many wtfs with this project I just kind of flipped when I saw my bitfield request turned into a string of 1s and 2s. It would have been easier to parse if they had left it as separate fields of 0s and 1s. I still don't think he understands the concept of a bitfield from the verbal conversations I've had with him.

    So your username is unexpectedly apt? :)


  • Discourse touched me in a no-no place

    @Monomelodies said:

    Erm? I've been using PgSQL since what is it, 2005 or so, and it (of course) supports bitfields fine - you just store them as an integer (same as you would in MySQL). I'm assuming you come from MSSQL which - presumably - has a dedicated bitfield type your connector - obviously - didn't understand?
    I just looked back into my notes - I was in fact using pg for that project, and it wasn't a bitfield datatype - it was fucking bit[4]. Fucking bit wasn't supported by the .net connector. Do you have ANY FUCKING IDEA how stupid that is?

    bit [ (n) ] fixed-length bit string


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.