Stumped on memory leak ODBC connection



  • Ok, I'm just completely stumped.

    There's a memory leak in a program. The leak is about 1 kilobyte per minute. I take lines out of the program until I isolate the behavior.

    The result.

    An ODBC connection is open for the entire duration of the application, opened by a static constructor as a static member of the same class.

    So, I close it.
    Still have leak.

    So I write a new program. I open it and close it and dispose it, all explicitly.
    Still have leak.

    So I call ReleaseObjectPool()
    Still have leak.
    I force garbage collection.
    Still have leak.
    So, I take out the code that opens it.
    Still have leak.

    So, I take out the code that creates a new Connection.

    Leak is gone.

    Why when I open, close, dispose, finalize, and garbage collect an ODBC collection, it persists and leaks data over time?

    What is my code doing after the connection is closed?
    It enters an infinite loop of Sleep(50,000,000);


  • Impossible Mission Players - A

    @xaade said:

    the code that creates a new Connection.
    Is that checking if the connection exists before creating it?
    Might be something in the constructor that's not actually static?
    Or, because it's static, it's not clearing out prior instances (because they might be used later) and it's refusing to re-use the static reference?



  •                      adoConn = new OdbcConnection(connStr);
            //            //if (adoConn == null) adoConn = new OdbcConnection(connStr);
            //            while (adoConn.State != ConnectionState.Open)
            //            {
            //                Console.WriteLine("adoOpen");
            //                //adoConn.Open();
            //                reTries++;
            //                if (reTries >= 5)
            //                {
            //                    return;
            //                }
            //                System.Threading.Thread.Sleep(500);  //We are locking this so we should probably not lock for up to a minute
            //            }
                        //Console.writeline("ado close");
                        //adoconn.close();
                        adoConn.Dispose();
                        adoConn = null;
                        OdbcConnection.ReleaseObjectPool();
                        GC.Collect(); GC.Collect(); GC.Collect();
    

  • Impossible Mission Players - A

    So in other words, the object you're creating is responsible for the leak?
    Do you know what happens inside the Open function?

    Does it happen if you instance the object and immediately dispose it?



  • That's what the code does.
    The open statement is commented out. The only thing it's doing right now is creating a new object with the connection string, and then disposing and setting to null, then it forces garbage collection.
    So that object has no excuse for it to keep leaking.

    So, I'm looking and it's leaking handles over time.


  • Impossible Mission Players - A

    Seems like it. Time to switch libraries? 🍢



  • It's ODBConnection as in System.Data.Odbc
    From Microsoft.

    I don't see an article for a leak on simply creating a new one.


  • Impossible Mission Players - A

    Hmm.... Well.... Damn. 😗
    Not sure what to say on that then.



  • File a bug report? It certainly seems like it's a library issue. Microsoft have been extremely good to me with reporting bugs, even when it turned out to be my fault...


  • Discourse touched me in a no-no place

    @xaade said:

    Why when I open, close, dispose, finalize, and garbage collect an ODBC collection, it persists and leaks data over time?

    Frequently this is caused because you're leaking something else, like recordsets or something (as opposed, or perhaps in addition to, to the connection itself).



  • What about the ODBC driver? Although I guess a memory leak there wouldn't leak into your process... hm.



  • What happens if you leave out the ReleaseObjectPool()?


  • Discourse touched me in a no-no place

    @blakeyrat said:

    What about the ODBC driver? Although I guess a memory leak there wouldn't leak into your process... hm.

    An ODBC driver is probably a DLL so it probably IS in his process.



  • It's almost certainly the ODBC driver, since almost no one else has the same problem.

    May I ask why you are using ODBC, which was superseded in 1996 by OLEDB, which was superseded by .Net Providers at least ten years ago?



  • Does it leak if you wrap the connection in a using statement? (See: http://stackoverflow.com/questions/887831/how-does-the-using-statement-translate-from-c-sharp-to-vb for syntax*)

    Don't manually gc, and definitely don't gc three times.

    *where stream is your ado.net connection


  • Discourse touched me in a no-no place

    @Jaime said:

    May I ask why you are using ODBC, which was superseded in 1996 by OLEDB, which was superseded by .Net Providers at least ten years ago?

    My first guess is "because the DB vendor hasn't released an OLEDB or .Net provider".

    For example http://knowledgebase.progress.com/articles/Article/000028572
    "There is no .NET native dataprovider or OLEDB dataprovider available for the OpenEdge database.

    The OpenEdge database does support ODBC connections, so a .NET application using the ADO.NET can use the ODBC dataprovider to connect to the OpenEdge database. "

    🤦 undefined


  • Discourse touched me in a no-no place

    @Matches said:

    Don't manually gc, and definitely don't gc three times.

    Correct me if I'm wrong but you're not supposed to need to call Dispose, either.



  • You see all the code.

    @blakeyrat said:

    What about the ODBC driver?

    The connection string only has DataSource defined, not even the driver.

    @flabdablet said:

    What happens if you leave out the ReleaseObjectPool()?

    Same thing. Leak happens.

    @Jaime said:

    May I ask why you are using ODBC

    Because I'm connecting to a proprietary database, that is usually accessed through API, but this option was created (probably at that 10+ years ago point).

    @Matches said:

    Does it leak if you wrap the connection in a using statement?

    Yep

    @Matches said:

    Don't manually gc, and definitely don't gc three times.

    Just for the purposes of ensuring the leak isn't due to delayed GC. Not in production code.

    @FrostCat said:

    My first guess is "because the DB vendor hasn't released an OLEDB or .Net provider".

    undefined

    @FrostCat said:

    Correct me if I'm wrong but you're not supposed to need to call Dispose, either.

    You should, unless you use a using statement.



  • I thought that the driver was doing it, but I removed the driver from the connection string, and it still leaks.



  • Calling dispose signals you're done with the connection and it can be freed (though you can still leak recordsets if not handled)

    I think in vb calling dispose and =null is the same thing, provided you called close first, but I'm not 100 percent on that and would have to look it up.



  • Today I have my personal laptop, and I'll try the same code on it.



  • How are you making your connection string? Maybe the leak is actually happening there if you're using the builder in a weird way.



  • static string, defined once.

    I can try to change it to const.



  • Also, where are you calling close? All i see is dispose (specifically, you've commented out close.)



  • I have both open and close statements commented out.

    I'll try uncommenting them again, but the connection object shouldn't leak simply because I haven't opened a connection yet.



  • Are you sure it's also an actual leak, and not a case of 'app just hasn't gotten around to cleaning up yet'? (Have you tried running in release mode instead of debug)



  • I've watched it for 6 hours straight, and handles are leaking too. Went from 180 handles at start to 1000+ handles and from 2k start to 30k+ private working memory.

    Consistently adds 4-5 handles, releases 1-2.



  • @Matches said:

    (Have you tried running in release mode instead of debug)

    I'll try that.


  • Discourse touched me in a no-no place

    @Matches said:

    Calling dispose signals you're done with the connection and it can be freed

    Isn't that what Close() is for?



  • Dispose calls close too.
    And if in a using statement, dispose is automatically called.



  • Possibly relevant, though i don't know what Filemaker is

    (More referencing the reuse connection)



  • @Matches said:

    I think in vb calling dispose and =null is the same thing, provided you called close first, but I'm not 100 percent on that and would have to look it up.

    Nope. VB handles destruction almost identically to C#.

    @FrostCat said:

    Isn't that what Close() is for?

    Close() frees the connection to the connection pool. Come to think of it, even Dispose only does this - the ODBC connection pool has its own timeout that's completely independent of the object lifecycle in .Net. @xaade's problem may be related to connection pooling. Check here.



  • Last question before work, if you make a completely empty project with that code, does it leak?



  • That's what I've been doing.



  • FILEMAKER!

    That brings back memories. Welcome to 1993!



  • That... actually explains a lot for your hatred of the world.



  • I liked Filemaker, it was like Access but much much better.



  • Low bar.



  • Ok, so I tried the same code on my personal laptop.



  • @xaade said:

    Ok, so I tried the same code on my personal laptop.

    And what happened?



  • still leaks handles.



  • @xaade said:

    still leaks handles.

    Cool.
    Was asking for @blakeyrat as he can't read minds.



  • I suppose I would have been more emphatic if it didn't leak.

    Like

    "I just.... and it didn't leak! undefined"

    But fair enough.


  • Impossible Mission Players - A

    Well, if all in-house code is ripped out (because you started a new project), you can disclose the full source then, right? 😛



  • that's up at top, like 3rd post.

    @xaade said:

    ```
    adoConn = new OdbcConnection(connStr);
    // //if (adoConn == null) adoConn = new OdbcConnection(connStr);
    // while (adoConn.State != ConnectionState.Open)
    // {
    // Console.WriteLine("adoOpen");
    // //adoConn.Open();
    // reTries++;
    // if (reTries >= 5)
    // {
    // return;
    // }
    // System.Threading.Thread.Sleep(500); //We are locking this so we should probably not lock for up to a minute
    // }
    //Console.writeline("ado close");
    //adoconn.close();
    adoConn.Dispose();
    adoConn = null;
    OdbcConnection.ReleaseObjectPool();
    GC.Collect(); GC.Collect(); GC.Collect();


  • Impossible Mission Players - A

    Wait, seriously? That's it?
    Well..... that's definitely a stumper....



  • Here's code I wrote in VS 2015 Community Edition, targeting .NET 4.5.2 that opens an OdbcConnection to SQL Server 2014 Express LocalDB, I don't get any leakage.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    using System.Data.Odbc;
    using System.Threading;
    using System.Diagnostics;
    
    namespace OdbcTest
    {
        class Program
        {
            static void Main(string[] args)
            {
                string ldbConString = @"Driver={SQL Server Native Client 11.0};Server=(localdb)\MSSQLLocalDB;Trusted_Connection=Yes;";
                for (int i = 0; i < 100; i++)
                {
                    using (OdbcConnection con = new OdbcConnection(ldbConString))
                    {
                        Console.WriteLine("Opening connection: {0}", i);
                        con.Open();
                        Thread.Sleep(5 * 1000);
                        using (Process proc = Process.GetCurrentProcess())
                        {
                            Console.WriteLine("Current handle count: {0}",proc.HandleCount);
                        }
                    }
                    OdbcConnection.ReleaseObjectPool();
                }
                Console.ReadLine();
            }
        }
    }
    

    Sounds like the underlying driver is a POS and crapping handles all over the place.



  • This is with the open statement commented out.





  • And the process keeps executing, as if there's a thread still going. Let me see it's threads.

    It's going back and forth between 8 and 9 threads.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.