Byte Array To String



  • About 6 posts in.

    Related: Anyone have any idea why my code crashes on the sprintf_s line? I'm trying to convert a byte array to... a string.

    char* hashstring = new char[(len * 2) + 1];
    for (DWORD i = 0; i < len; i++) {
    sprintf_s(&(hashstring[i * 2]), 2, "%.2X", hashdata[i]);
    }
    hashstring[len * 2] = 0;

    hashdata is a byte array of length len (len is a DWORD).

    Also if there is an easier way to do this I would appreciate a code sample.



  • I just love forums that require HTML, and have edit limits, don't you? Cutting off my text because I forgot to escape an angle bracket is a bonus.

    Oh yeah, and it crashes on the first call to sprintf_s. I don't see anything wrong with it though. :(

      char* hashstring = new char[(len * 2) + 1];
    for (DWORD i = 0; i < len; i++) {
    sprintf_s(&(hashstring[i * 2]), 2, "%.2X", hashdata[i]);
    }
    hashstring[len * 2] = 0;

    Nevermind, looks like sprintf_s explodes if you don't give it room for a null. I fixed it.



  • From the thread linked above:

    @gurhall said:

    If I caught anyone using anything but BitConverter.ToString and trying to roll their own solution, I would fire them on the spot.

    Thanks, I'll remember that.  Next time I need to generate a hex string in C++, I'll have my app fire up a .NET VM and call that function.  Better yet, I'll host a C# server with that functionality :)

    I'm kidding, of course.  gurhall has a good point.  If I did need to do it in C/C++, though, I'd use a function like this:

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {
        const unsigned char *b = (const unsigned char*)bytea;
        const char *h = "0123456789ABCDEF";
        for (;len--; b++) {
            *hex_out++ = h[*b >> 4];
            *hex_out++ = h[*b & 15];
        }
        *hex_out = 0;
    }


  •  Well? People post code of varying quality, don't read the thread before posting, promise to fire every programmer who doesn't share their preferences and complain about coders using incomprehensibly complex methods (like bit shifting or modulo), just like everywhere else. It's good for a giggle, but anything WTF-like has escaped me.



  • @Ilya Ehrenburg said:

     Well? People post code of varying quality, don't read the thread before posting, promise to fire every programmer who doesn't share their preferences and complain about coders using incomprehensibly complex methods (like bit shifting or modulo), just like everywhere else. It's good for a giggle, but anything WTF-like has escaped me.

     

    void bytea_to_hex(const void *bytea, size_t len, char hex_out[]) {
    const char *h = "0123456789abcdef";
    size_t i, out_i;

    //ugly hack to allow us to access the bytes of input individually
    const unsigned char *b = (const unsigned char*)bytea;

    for (i = 0; i < len; i++, out_i+=2) {
    hex_out[out_i] = h[b[i] / 16];
    hex_out[out_i+1] = h[b[i] - (b[i] / 16) * 16];
    }
    hex_out[len * 2] = 0;
    }

    There, I removed most of the "incomprehensibly complex" methods.  It should be easier to understand now ;-)



  • I once found the below class in source code.  Its a horrible abomination and a cornucopia of pain and agony... It was thankfully removed without incident.

     

     

    '*************************************************************
    '*************************************************************
    '***This Class Handles bit level flags. allowing you to    ***
    '***Access individual bits within a variable               ***
    '*** You can check the status and modify the bit pattern   ***
    '***                                                       ***
    '*** This Class wille even pull values from web controls        ***
    '*** in order to create a single byte to store in sql server     ***
    '***                                                                                 ***
    '*** Use with care and remember that modifying a single    ***
    '*** bit WILL drastically alter the value of any variable  ***
    '*************************************************************
    '*************************************************************

    ''' -----------------------------------------------------------------------------
    ''' Class  : BitProcessor
    '''
    ''' -----------------------------------------------------------------------------
    ''' <summary>
    ''' Provides a way to work with individual bits as flags
    ''' </summary>
    ''' <remarks>
    ''' This Class Handles bit level flags. allowing you to
    ''' Access individual bits within a variable You can check
    ''' the status and modify the bit itself. Use with care and
    ''' remember that modifying a single bit WILL drastically alter
    ''' the value of any variable
    ''' </remarks>
    ''' -----------------------------------------------------------------------------
    Public Class BitProcessor

        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''
        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''
        '' Fields - Implementation variables
        ''
        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Integer count for error checking
        ''' </summary>
        ''' <remarks>
        ''' Because the bitproccessor can only handle 32 bits it is important to make sure
        ''' that we do not run over.
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Private _index As Integer = 0 'mind the underscore at the beginning

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' The value whose bits to check/modify
        ''' </summary>
        ''' <remarks>
        ''' This variable holds the value associated with the MyByte property
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Private _MyByte32Bit As Integer = 0 'This is our control byte 32 bits of pseudo unsigned goodness woot!

        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''
        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''
        '' Properties                                         ''
        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''
        'Public Property index() As Integer
        '    Get
        '        Return _index
        '    End Get
        '    Set(ByVal Value As Integer)
        '        If Value < 32 And Value >= 0 Then
        '            _index = Value
        '        End If
        '    End Set
        'End Property


        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' an Indexed boolean property that allows access to the individual bits
        ''' </summary>
        ''' <param name="index"></param>
        ''' <value></value>
        ''' <remarks>
        ''' This property applies a mask to the bits of our value
        ''' to either set, unset or read the bit value.
        ''' The bit mask used is 1<<index where index is the bit number to check/modify
        '''
        ''' The Bit Mask is used in conjunction with the following logical algorithms.
        '''
        ''' Check bit: MyByte AND mask
        ''' SET Bit:  MyByte OR mask
        ''' UnSET Bit:  MyByte AND  (Not mask)
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Default Public Property Bits(ByVal index As Integer) As Boolean
            Get
                If (index <= 31 And (index >= 0)) Then
                    'lets check our bit
                    Dim mask As Integer = (1 << index) 'set our mask
                    If (_MyByte32Bit And mask) = False Then
                        Return False
                    Else
                        Return True
                    End If
                Else
                    'our bit index is out of range so Crash and Burn Baby!!!!
                    Throw New HttpException(9002, "Bit Proccessor encountered an error: Bits Index out of range", 9002)
                    'Err.Raise(9002, "BitProccessor", "Bit Index out of range")
                End If
            End Get
            Set(ByVal Value As Boolean)
                Dim mask As Integer = (1 << index)  'set our mask
                If Value Then 'we set the bit
                    _MyByte32Bit = _MyByte32Bit Or mask
                Else 'lets unset the bit
                    _MyByte32Bit = _MyByte32Bit And (Not mask)
                End If
            End Set
        End Property

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Provides access to the to a 32 Byte Integer to work with
        ''' </summary>
        ''' <value></value>
        ''' <remarks>
        ''' The above sums it up
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Public Property MyByte() As Integer
            Get
                Return _MyByte32Bit
            End Get
            Set(ByVal Value As Integer)
                _MyByte32Bit = Value
            End Set
        End Property

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Creates a bitpattern from a checkbox/checkboxlist,radiobutton,radiobuttonlist control.
        ''' </summary>
        ''' <param name="myControl">A webControl to make a bit pattern from</param>
        ''' <returns>a string containing the type name of the control.</returns>
        ''' <remarks>
        ''' </remarks>

        Public Function addControlValues(ByRef myControl As WebControl) As String
            Dim strControlType As String
            strControlType = myControl.GetType.Name().ToLower

            Select Case strControlType
                Case "checkboxlist"
                    'run checkboxlist function
                    addCheckboxList(myControl)
                Case "checkbox"
                    'run checkbox function
                    addCheckbox(myControl)
                Case "radiobuttonlist"
                    'run checkboxlist function
                    addRadioButtonList(myControl)
                Case "radiobutton"
                    'run checkbox function
                    addRadioButton(myControl)
            End Select
            Return strControlType
        End Function

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Adds a checkbox to the bit pattern
        ''' </summary>
        ''' <param name="myControl">a checkbox control whose value to add</param>
        ''' <remarks>
        ''' nope
        ''' </remarks>
        Private Sub addCheckbox(ByRef myControl As CheckBox)
            If _index <> 32 Then
                If myControl.Checked Then
                    Bits(_index) = True
                Else
                    Bits(_index) = False
                End If
                _index += 1
            Else
                Throw New HttpException("Bit Processor Index out of range")
            End If
        End Sub

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Adds a checkboxlist to the bit pattern
        ''' </summary>
        ''' <param name="myControl">a checkboxlist control whose value to add</param>
        ''' <remarks>
        ''' nope
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Private Sub addCheckboxList(ByRef myControl As CheckBoxList)
            Dim myItem As ListItem

            If (_index + myControl.Items.Count) < 32 Then
                For Each myItem In myControl.Items
                    'set me?
                    If myItem.Selected = True Then
                        Bits(_index) = True 'Victory is MINE!
                    Else
                        'nahh another time perhaps
                        Bits(_index) = False
                    End If
                    _index += 1
                Next
            End If
        End Sub

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Adds a radiobutton to the bit pattern
        ''' </summary>
        ''' <param name="myControl">a radiobutton control whose value to add</param>
        ''' <remarks>
        ''' nope
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Private Sub addRadioButton(ByRef myControl As RadioButton)
            If _index <> 32 Then
                If myControl.Checked Then
                    Bits(_index) = True
                Else
                    Bits(_index) = False
                End If
                _index += 1
            Else
                Throw New HttpException("Bit Processor Index out of range")
            End If
        End Sub

        ''' -----------------------------------------------------------------------------
        ''' <summary>
        ''' Adds a radiobuttonlist to the bit pattern
        ''' </summary>
        ''' <param name="myControl">a radiobuttonlist control whose value to add</param>
        ''' <remarks>
        ''' nope
        ''' </remarks>
        ''' -----------------------------------------------------------------------------
        Private Sub addRadioButtonList(ByRef myControl As RadioButtonList)
            Dim myItem As ListItem

            If (_index + myControl.Items.Count) < 32 Then
                For Each myItem In myControl.Items
                    'set me?
                    If myItem.Selected = True Then
                        Bits(_index) = True 'Victory is MINE!
                    Else
                        'nahh another time perhaps
                        Bits(_index) = False
                    End If
                    _index += 1 'increment by 1
                Next
            End If
        End Sub

    End Class

     

     



  • @joeyadams said:

    @Ilya Ehrenburg said:

     Well? People post code of varying quality, don't read the thread before posting, promise to fire every programmer who doesn't share their preferences and complain about coders using incomprehensibly complex methods (like bit shifting or modulo), just like everywhere else. It's good for a giggle, but anything WTF-like has escaped me.

     

    There, I removed most of the "incomprehensibly complex" methods.  It should be easier to understand now ;-)

     

    Uh, I don't know whether you're now sharing a joke with me and I don't get it or whether you completely misunderstood me and think I'm far more stupid than I really am. I was referring to

    @timster at home said:

    Well done sandeeprwt - I have used this in my code and it works brilliantly.  Simple, elegant and efficient - [b]what is it that makes developers go for the most complex solutions[/b] when you can do something like this.

    and the "goofy modulus math" [url=http://thedailywtf.com/Articles/nice_num,-mean_programmer.aspx]lamented here[/url].



  • @joeyadams said:

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {

        const unsigned char *b = (const unsigned char*)bytea;
        const char *h = "0123456789ABCDEF";
        for (;len--; b++) {
            *hex_out++ = h[*b >> 4];
            *hex_out++ = h[*b & 15];
        }
        *hex_out = 0;
    }
    I'd fire you on the spot for using such a table! What a waste of RAM! You should instead use '0' + offset, unless offset > 10, in which case it should be '7' + offset.

    {
        const unsigned char *b = (const unsigned char*)bytea;
        for (;len--; b++) {
            *hex_out++ = (*b >> 4) < 10 ? '0'+(*b >> 4):'7'+(*b >> 4);
            *hex_out++ = (*b & 15) < 10 ? '0'+(*b & 15):'7'+(*b & 15);
        }
        *hex_out = 0;
    }



  • @joeyadams said:

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {
        const unsigned char b = (const unsigned char)bytea;
        const char *h = "0123456789ABCDEF";
        for (;len--; b++) {
            *hex_out++ = h[*b >> 4];
            *hex_out++ = h[*b & 15];
        }
        *hex_out = 0;
    }

    If you have a real programming language anyway, you can speed this up by reading, converting, and writing several bytes at once, i.e.

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {
        const unsigned int *b = (const unsigned char*)bytea;
        unsigned int *o = (unsigned short*)hex_out;
        const unsigned short *h = /* lookup table */;
        int l = len / sizeof(*b);
        for (;l--; b++,o+=2) {
            if (sizeof(*b) == 4) {
              o[0] = h[(*b    ) & 0xff] + (h[(*b>>8) & 0xff] << 16);
              o[1] = h[(*b>>16) & 0xff] + (h[(*b>>24) & 0xff] << 16);
            } else { /* other cases */ }
        }
        /* handling of rest bytes and string termination */
    }
    

    or

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {
        const unsigned int *b = (const unsigned short*)bytea;
        unsigned int *o = (unsigned int*)hex_out;
        const unsigned int = /* lookup table */;
        int l = len / sizeof(*b);
        for (;l--; b++,o+=2) {
            if (sizeof(*b) == 4) {
              o[0] = h[(*b    ) & 0xffff];
              o[1] = h[(*b>>16) & 0xffff];
             } else { /* other cases */ }
       }
        /* handling of rest bytes and string termination */
    }
    

    , depending which lookup table (byte order dependent like the code) better fits into the L1 cache.



  • @Ilya Ehrenburg said:

    @joeyadams said:

    @Ilya Ehrenburg said:

     Well? People post code of varying quality, don't read the thread before posting, promise to fire every programmer who doesn't share their preferences and complain about coders using incomprehensibly complex methods (like bit shifting or modulo), just like everywhere else. It's good for a giggle, but anything WTF-like has escaped me.

     

    There, I removed most of the "incomprehensibly complex" methods.  It should be easier to understand now ;-)

     

    Uh, I don't know whether you're now sharing a joke with me and I don't get it or whether you completely misunderstood me and think I'm far more stupid than I really am. I was referring to

    @timster at home said:

    Well done sandeeprwt - I have used this in my code and it works brilliantly.  Simple, elegant and efficient - what is it that makes developers go for the most complex solutions when you can do something like this.

    and the "goofy modulus math" lamented here.

     Sorry for the ambiguity. My point was that "goofy modulus math" is, in some cases, much clearer than finding alternatives some people (such as the poster of nice_num, mean_programmer) deem "simpler". Hence, I certainly agree with you.


  • @Mole said:

    @joeyadams said:

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {

        const unsigned char *b = (const unsigned char*)bytea;
        const char *h = "0123456789ABCDEF";
        for (;len--; b++) {
            *hex_out++ = h[*b >> 4];
            *hex_out++ = h[*b & 15];
        }
        *hex_out = 0;
    }
    I'd fire you on the spot for using such a table! What a waste of RAM! You should instead use '0' + offset, unless offset > 10, in which case it should be '7' + offset.

    {
        const unsigned char *b = (const unsigned char*)bytea;
        for (;len--; b++) {
            *hex_out++ = (*b >> 4) < 10 ? '0'+(*b >> 4):'7'+(*b >> 4);
            *hex_out++ = (*b & 15) < 10 ? '0'+(*b & 15):'7'+(*b & 15);
        }
        *hex_out = 0;
    }


     

    Haha, nice.  Better yet, we could pre-generate the hex character table, thus only requiring 16 bytes temporarily:

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {
    const unsigned char *b = (const unsigned char*)bytea;
    char *h = new char[16];
    int i;
    for (i=0; i<16; i++)
    h[i] = '0'+i;
    for (i=10; i<16; i++)
    h[i] += 7;
    for (;len--; b++) {
    *hex_out++ = h[*b >> 4];
    *hex_out++ = h[*b & 15];
    }
    delete[] h;
    *hex_out = 0;
    }

     



  • @Mole said:

    @joeyadams said:

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {

        const unsigned char *b = (const unsigned char*)bytea;
        const char *h = "0123456789ABCDEF";
        for (;len--; b++) {
            *hex_out++ = h[*b >> 4];
            *hex_out++ = h[*b & 15];
        }
        *hex_out = 0;
    }
    I'd fire you on the spot for using such a table! What a waste of RAM! You should instead use '0' + offset, unless offset > 10, in which case it should be '7' + offset.

    {
        const unsigned char *b = (const unsigned char*)bytea;
        for (;len--; b++) {
            *hex_out++ = (*b >> 4) < 10 ? '0'+(*b >> 4):'7'+(*b >> 4);
            *hex_out++ = (*b & 15) < 10 ? '0'+(*b & 15):'7'+(*b & 15);
        }
        *hex_out = 0;
    }


     

    Just out of curiosity, I generated code listings for both routines just to check my hunch. And sure enough, Joeys code uses 77 bytes including the array, and your code wastes 98 bytes (MSVC++ 8). He does waste one byte (or actually three, I guess, because of alignment) by having the array zero-terminated. To reclaim that byte, use this:

    const char h[16] = { '0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F' };

    In addition to wasting more space, your code uses a lot of tests, which kills performance. The original does not have any conditions other than the loop index, and thus is a good candidate for unrolling and speculative execution by the CPU. I don't remember offhand how many execution paths a modern CPU can follow simultaneously, but there's a good chance your code will stall on every iteration.

    If you want to optimize code, you have to really know what you're doing....



  • @boh said:

    If you want to optimize code, you have to really know what you're doing....

    If you want to micro-optimize code (which is appropriate in some cases) you have to really know what you're doing. Most code is perfectly fine being optimized at a much higher level, which takes common sense and a lot less minutiae.



  • @joeyadams said:

    My point was that "goofy modulus math" is, in some cases,
    sed s/some/most/

    much clearer than finding alternatives some people (such as the poster of nice_num, mean_programmer) deem "simpler".
    Yes, I [i]love[/i] some of the comments there. Inserting characters into a string is 'simpler' than dividing by 10 and counting digits - really, in which language would that have a half-chance to be true?



  • @negativeview said:

    @boh said:
    If you want to optimize code, you have to really know what you're doing....

    If you want to micro-optimize code (which is appropriate in some cases) you have to really know what you're doing. Most code is perfectly fine being optimized at a much higher level, which takes common sense and a lot less minutiae.

    If you're ever in a situation where you really really really do have to convert millions of digits from hex to ascii every second, you should worry about optimising this routine, and you should not fart around counting bytes or cycles, you should go straight for a proper SSE-based algorithm, which will tear through it so fast you probably won't even need to bother about microoptimising the exact instruction sequence.  For any smaller throughput, you probably shouldn't care.



    1. Don't optimize.
    2. Don't optimize yet.
    3. Do it differently.


  • @dhromed said:

    1. Don't optimize.
    2. Don't optimize yet.
    3. Do it differently.

    Real Optimizers do it Algorithmically!



  •  I make love to my woman Fibonacci style, for best results.



  •  I can't tell if everybody else is intentionally ironic or not, but surely .NET has a sprintf() clone?



  • @savar said:

    I can't tell if everybody else is intentionally ironic or not, but surely .NET has a sprintf() clone?

    Solution using sprintf():

    void bytea_to_hex(const void *bytea, size_t len, char *hex_out) {
        if ( len == 0 ) {
            sprintf( hex_out, "" );
        } else if ( len == 1 ) {
            sprintf( hex_out, "%02x", bytea[0] );
        } else if ( len == 2 ) {
            sprintf( hex_out, "%02x%02x", bytea[0], bytea[1] );
        } else if ( len == 3 ) {
            sprintf( hex_out, "%02x%02x%02x", bytea[0], bytea[1], bytea[2] );
        } else if (...snip...)
        } else {
            sprintf( hex_out, "ERROR" );
        }
    }

     



  • @boh said:

    Solution using sprintf():

    You forgot to check the return value of those sprintfs.


  • Discourse touched me in a no-no place

    @negativeview said:

    @boh said:
    Solution using sprintf():

    You forgot to check the return value of those sprintfs.

    I'm assuming that's sarcasm?



  • @PJH said:

    I'm assuming that's sarcasm?

    My sense of humor is dryer than I thought. I was just wanting to see how many more lines could be added to the already way too complicated example.


  • Discourse touched me in a no-no place

    @negativeview said:

    @PJH said:
    I'm assuming that's sarcasm?
    My sense of humor is dryer than I thought. I was just wanting to see how many more lines could be added to the already way too complicated example.
    Which is what I wondered. Not seen too many of your replies to 'place' you.



    My original (typed but not posted) reply postulated on the possible returns of sprintf() and the chances of it being <0 (or indeed 2) under this particular circumstance.



  • @dhromed said:

     I make love to my woman Fibonacci style, for best results.

    So: first couple of times you make love to her she gets pregnant, then next time it's twins, then triplets, then quints, octuplets, thirteen-tuplets...

    Man she's gonna hate you by around the 20th time or so...

     



  • @DaveK said:

    @dhromed said:

     I make love to my woman Fibonacci style, for best results.

    So: first couple of times you make love to her she gets pregnant, then next time it's twins, then triplets, then quints, octuplets, thirteen-tuplets...

    Man she's gonna hate you by around the 20th time or so...

     

    And here I thought he just meant, the first two days, he just made love to her once, and then twice on the third day.  Of course, given the shorter cycle time, that gets infeasible much quicker...



  • @tgape said:

    @DaveK said:

    @dhromed said:

     I make love to my woman Fibonacci style, for best results.

    So: first couple of times you make love to her she gets pregnant, then next time it's twins, then triplets, then quints, octuplets, thirteen-tuplets...

    Man she's gonna hate you by around the 20th time or so...

     

    And here I thought he just meant, the first two days, he just made love to her once, and then twice on the third day.  Of course, given the shorter cycle time, that gets infeasible much quicker...

    I think he meant that he did it twice on the first day, then once on day 2, then once on day 3, then on day 5, day 8, day 13,..



  • @Spectre said:

    I think he meant that he did it twice on the first day, then once on day 2, then once on day 3, then on day 5, day 8, day 13,..

    That's how I interpreted it.



  • @morbiuswilters said:

    @Spectre said:

    I think he meant that he did it twice on the first day, then once on day 2, then once on day 3, then on day 5, day 8, day 13,..

    That's how I interpreted it.

    All the simple textbook explanations I remember from my schooldays of the fibonacci series use the example of a pair of breeding rabbits to illustrate the progression of the series, so I couldn't help but think it was a reference to offspring.  (I see it's still one of the standard examples.  ROFL, now I come to think about it, that is based on the somewhat dubious assumption that rabbits partner for life, rather than about fifteen seconds.) 



  • @dhromed said:

    1. Don't optimize.
    2. Don't optimize yet.
    3. Do it differently.

     

    I hate that quote. What do you think optimization actually is? If it's the same then it's not optimized.

    The second thing about this is it means that I have to swim through an absolute mire of code because people don't understand the basics of optimized coding.

    I point out basic coding "tricks" to speed up the code and you get, "that's optimizing" and they then ignore it. It mean I have a GUI (embedded STB) that takes 10-30 seconds to load, and people are now looking at me asking why it is not quicker.

    *splode*

    [Apologies if this reply sounds harsh. Currently looking for a new job, where embedded engineers don't change code that uses memcpy() to back to strcat() because they don't understand why I changed it.]



  • @random_lurker said:

    @dhromed said:
    1. Don't optimize.
    2. Don't optimize yet.
    3. Do it differently.

    I hate that quote. What do you think optimization actually is? If it's the same then it's not optimized.

    The second thing about this is it means that I have to swim through an absolute mire of code because people don't understand the basics of optimized coding.

    I point out basic coding "tricks" to speed up the code and you get, "that's optimizing" and they then ignore it. It mean I have a GUI (embedded STB) that takes 10-30 seconds to load, and people are now looking at me asking why it is not quicker.

    I believe the 'do it differently' means, perform an algorithmic optimization, rather than just doing little tweaks that speed up individual statements.  When it's viable, it does work much better than the former.

    That having been said, I personally feel there's a whole world of difference between writing code in a suboptimal way for clarity initially, and only worrying about optimization later, and writing code like a festering wound initially because you can't be bothered to learn reasonable coding, and then performing monkey code "optimization" passes until you find something that appears to be faster.  Most of what I see for optimization falls in the latter category, sad to say.

    For example, the classic case I see, as a Perl programmer, is someone who wrote a shell script using Perl as a shell, where nearly every non-blank line contains either a system call or a backtick invocation (or both); the only exceptions are flow control and close curly bracket only lines.  Typically, I can get about a 100 to 1000 fold performance increase1 by converting those things to actual Perl code, before even doing algorithmic improvements.

    Of course, as a programmer for embedded systems, I would *hope* none of the code you get is in Perl, and none of it uses system(3) calls in innermost loops.

    1 For those who don't understand how this could be, who have access to a unix or Linux system, strace, truss, dtrace, or the equivalent, a simple invoke of /bin/true.  Other than one system call, *that* is what is being skipped on *every* statement executed in Perl, rather than being executed via system or ``.  That Perl itself is so suboptimal that one can sometimes get the same kind of performance improvement from converting from Perl to C is, IMHO, kinda scary.  For those who only have access to Windows or MacOS systems (which are mostly unixes, but in this instance, fail), I don't know how to trace processes on your platform.



  • @tgape said:

    @random_lurker said:
    @dhromed said:

    1. Don't optimize.
    2. Don't optimize yet.
    3. Do it differently.

    I hate that quote. What do you think optimization actually is? If it's the same then it's not optimized.

    The second thing about this is it means that I have to swim through an absolute mire of code because people don't understand the basics of optimized coding.

    I point out basic coding "tricks" to speed up the code and you get, "that's optimizing" and they then ignore it. It mean I have a GUI (embedded STB) that takes 10-30 seconds to load, and people are now looking at me asking why it is not quicker.

    I believe the 'do it differently' means, perform an algorithmic optimization, rather than just doing little tweaks that speed up individual statements.  When it's viable, it does work much better than the former.

    That having been said, I personally feel there's a whole world of difference between writing code in a suboptimal way for clarity initially, and only worrying about optimization later, and writing code like a festering wound initially because you can't be bothered to learn reasonable coding, and then performing monkey code "optimization" passes until you find something that appears to be faster.  Most of what I see for optimization falls in the latter category, sad to say.

    *snip*

     

    I agree with most of what you say. But, the timescales within the embedded space are minute. To go back and re-write (and re-test) a whole stack of code is just not going to be done. The best you will get is the worst hacked in optimisations during bug fixing before delivery*.

    Also, code optimization should be done at design. You know how often a function is going to be called, if it's in a frame update function called 30 times a frame, or if it is called once every hour. You can spend time writing the code to the proper level of performance (OK, you wont know how it actually performs to you run for real, but...).

    The sort of problems I have with programmers that don't think about optimization is that they write systematically sub-optimal code. Structures that can't be optimised, dynamic data structures (on embedded systems), the use of libraries that use an unspecified amount of resources and the best one - "future-safe" or "standard" use of libraries (SQL on an STB is my favorite of all** - it comes up so often - Que the flame war).

    These can get so deep into a system there is no way to get them out.Trying to optimize this code can mean hours of test runs trying to find out under load where the problem lies. It's tedious and takes far more time than a little extra thinking at coding/design time.

    But, mostly that phrase can be used to defend the not thinking about optimization/performance, which means we don't get either.

    *I have been responsible for the most Orgasmically awful speed up/stability hacks in the MINUTES before delivery. Got to love products with fixed delivery dates that can't slip.***

    **Most data structures on STBs are defined by standards agencies and broadcasters and don't change often and are fixed size. You do not need a full DB engine to handle them. If they do change then, Menus, InfoBanners and lots of the system (inc. hardware) may have to change, so a new full release will have to be done. You can't future proof that.

    ***It is why I still do this. No better fun (at work) can be had, than while trying to get a system out that has to be on the plane at 5:30 and someone finds a serious bug at 2am, and we don't get paid unless it makes the plane.


Log in to reply