Simple program with 1000 connections.

xaade

Does anyone know if there's a simple program that can handle multiple connections in such a large magnitude?

I'm having a problem with a inhouse service trying to re-establish connections after being handed over control, and the problem case occurs when >1000 connections are configured.

In theory I should just be able to run 1000 copies of a simulator using a different port for each, and well, that... did everything but BSOD' my VM.

So, I'm thinking one program basically handle 1000 connections from the service by itself, so I was wondering if such a thing already existed before I go figure out how to code such a thing.

lucas1

@xaade You are most likely running out of threads. If the program can be run in an event loop you can handle more.

kt_

@lucas1 said in Simple program with 1000 connections.:

If the program can be run in an event loop you can handle more.

What do you mean?

masonwheeler

@xaade You're getting to the point where I'd start up multiple instances behind a load balancer.

RaceProUK

@xaade I think the first thing we need to know is what tech stack are you using e.g. NodeJS, .NET, whatever. Once we know that, we can start coming up with some ideas.

xaade

@RaceProUK .NET, windows VM (10 Enterprise 2016).

Although the connecting service is C++.

xaade

I'm trying to look into alternatives.

The problem is that there is a primary and backup system, and during a switch to the backup system, the service managing the connections thinks it didn't connect to all of the connections, even though the sockets are still active. This only happens if 500+ connections exist, with increasing likelihood with more connections.

Apparently the "workaround" that the integration team offered changes the code to close the connection when the connection's entry in the cached table is deleted. This can happen for various reasons, including mismatched data for the cached entry.

Immediately, my thought is how this hasn't always happened.

lucas1

@kt_ Some workloads work better in an event loop. It depends what the workload is and even if it is possible.

xaade

@lucas1 I don't have to do any work, I just have to open the socket connections. The simulator does it for me on a one-by-one basis, but I can't open that many instances of the simulator.

Theoretically, I don't even have to have valid communication, it just has to hold the connection, and the service has to just attempt losing and gaining that many connections as fast as it does this already.

cvi

@lucas1 said in Simple program with 1000 connections.:

@kt_ Some workloads work better in an event loop. It depends what the workload is and even if it is possible.

I've been able to open up 1k+ (I don't remember the exact limit - this was some time ago) inbound or outbound connections using the basic socket APIs (more or less) of various platforms. Key is using the event based IO models, as @lucas1 says.

Edit: @xaade -- so have you just tried firing 1k async connect()s from a single program? Shouldn't be too many lines of code to try that.

lucas1

@cvi @xaade I think @cvi understand the specifics more than I do. I can do thread based programming but only in .NET and older versions of .NET since I understand how their thread based Async works.

RaceProUK

@xaade said in Simple program with 1000 connections.:

.NET

Ah, in that case, the Task Parallel Library is your friend ;)

xaade

@cvi

Just to be sure, the program isn't starting the connections, it's responding to the service that's opening the connections.

The host server, service, is opening connections to the simulators.

ben_lubar

@xaade said in Simple program with 1000 connections.:

did everything but BSOD' my VM.

Without knowing any of the symptoms, I'd guess you mean something along the lines of "slowed to a halt and constantly swapped memory from RAM to disk and back"?

Because that's not a problem with connections, that's a problem with the overhead of whatever's handling them being too much to fit in physical memory.

:asspull:

cvi

@xaade Ah - OK. Nevertheless, you should be able to accept (well, accept() even) 1k incoming connections too. There's a couple of issues you can run into, like the listen() backlog being too small if you can't accept the connections quickly enough (IIRC -this was a while ago- Windows used to have a somewhat limited upper bound for that). But maybe you can get away with just being fast enough, especially if you don't need to do anything with the connection.

If you're not averse to using Boost, there's Boost.ASIO. Haven't used it that much, but it might be nicer than using the old-style socket API or the Winsock one directly. It'll probably use the most modern/efficient API that your platform of choice has to offer as its backend...

wharrgarbl

@xaade in my home network, it's my cheapass router that would misbehave before the 1000 connections happen

xaade

@wharrgarbl Well, the real life scenario is a massive redundant system that monitors/controls thousands of devices in the field.

kt_

@lucas1 said in Simple program with 1000 connections.:

@kt_ Some workloads work better in an event loop. It depends what the workload is and even if it is possible.

Never knew that. Guess I'll need to read up about that.

PleegWat

Yeah, this needs a single-threaded event loop with async IO. Launching thousands of threads tends to cause extreme load due to task switching in the OS.

xaade

@PleegWat said in Simple program with 1000 connections.:

Yeah, this needs a single-threaded event loop with async IO. Launching thousands of threads tends to cause extreme load due to task switching in the OS.

How can you do an event loop for socket connections? I see that there's a blocking wait for connection, but I can't just block them in order of port number... sigh

xaade

@PleegWat

Could I just call AcceptTcpClientAsync() for each connection, and just ignore the event?

PleegWat

@xaade Unfortunately I don't know how this works on windows. I'm a linux/C guy, and there I'm 99% sure you can just select/poll/etc for the relevant sockets.

anotherusername

I knew there was some thread where we figured out how to force new connections using socket.io... (I just couldn't remember how to spell "shawarma" at first)

@anotherusername said in Shawarma Spin:

I'm pretty sure I killed the server by running this in a few tabs:
for (var i = 0; i < 100; ++ i) {
    setTimeout((function (a) {
        var sock = io.connect('http://ws.shawarmaspin.com/',
            {'force new connection': true, timeout: 10000});
        sock.emit('set_team', 'WTF');
        sock.emit('set_initials', ('___' + a).slice(-3));
    }).bind(0, i), 500 * i);
}
There were already 700+ SPM in the top 2 rooms combined, so I don't know why a couple hundred more would've killed it.

PleegWat

@anotherusername said in Simple program with 1000 connections.:

(I just couldn't remember how to spell "shawarma" at first)

I tend to spell it shoarma.

Jarry

@xaade you need socket connections or ports open?

xaade

@Jarry said in Simple program with 1000 connections.:

@xaade you need socket connections or ports open?

It needs to be able to accept the connection. The service has to know it successfully connected. I don't think it needs to get back anything specific, because the remote object will report invalid message due to the reply not being parsable.

Jarry

@xaade
then i think you need only one socket a listen and an accept loop.
last time i did something like that was for a uni project. it handled about 2000 connections fine. so it should be able to work.

Pseudo code:

connected_socks = [];
socket  = new socket(port:8000);
listen(socket);
while(true) {
connected_socks.push(accept(socket));
}

or something like that. maybe the c#'ers of the forum can turn that into valid code.

anotherusername

@PleegWat said in Simple program with 1000 connections.:

@anotherusername said in Simple program with 1000 connections.:

(I just couldn't remember how to spell "shawarma" at first)

I tend to spell it shoarma.

I tend to spell it "gyros", because I can remember how to spell that.

Or, even better, "tacos". I can remember how to spell that and pronounce it...

cvi

@xaade If you need to accept from several sockets (i.e., from several distinct port numbers), you can do that by waiting for write events on the listening sockets (IIRC). Once you get a ~~write~~read event, you can accept a single incoming connection from that socket (I'd set non-blocking mode either way, makes it easier to debug if you handle the would-block errors).

edit: On windows, you can even do the whole thing via a simple select()-based program (you should be able to find examples of that fairly easily). You can redefine FD_SETSIZE [MSDN] to a larger number, just do so before including the relevant headers. Then, essentially, you need to set up your listening socket(s) via (from memory) socket+bind+listen. Then, in a loop, you add each to an fd_set, which you pass as the read-set to select(). Each time select returns, you can accept one or more sockets with accept, at which point the connection will have been established.

(Select doesn't scale well for large numbers of sockets, because checking for the sockets is O(N), but I don't think that's a problem here.)

xaade

This is what I got so far. Have no clue if this is going to work. (WELL, obviously not since I'm not done with it).

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;

namespace TcpMultiListener
{
    class SocketInfo
    {
        public SocketInfo(int port)
        {
            Port = port;
        }
        public int Port { get; private set; }
        public Socket Listener { get; set; }
    }
    class Program
    {
        const string DEFAULT_PORTS = "6001-6099";
        static string portsetlist = DEFAULT_PORTS;
        static IPAddress localaddress = IPAddress.Parse("127.0.0.1");
        static BackgroundWorker connectionsThread = new BackgroundWorker();
        static void Main(string[] args)
        {
            if (args.Length >= 0)
            {
                portsetlist = args[0];
            }

            connectionsThread.DoWork += ConnectionsThread_DoWork;
            connectionsThread.WorkerSupportsCancellation = true;
            connectionsThread.RunWorkerCompleted += ConnectionsThread_RunWorkerCompleted;
        }

        private static void ConnectionsThread_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
            throw new NotImplementedException();
        }

        private static void ConnectionsThread_DoWork(object sender, DoWorkEventArgs e)
        {
            var socketInfoList = Ports(portsetlist);
            foreach (var socketInfo in socketInfoList)
            {
                socketInfo.Listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                socketInfo.Listener.Bind(new IPEndPoint(localaddress, socketInfo.Port));
                socketInfo.Listener.Listen(10);

                socketInfo.Listener.BeginAccept(new AsyncCallback(OnConnectRequest), socketInfo.Listener);
            }
        }

        static void OnConnectRequest(IAsyncResult ar)
        {
            Socket listener = (Socket)ar.AsyncState;
            listener.EndAccept(ar);
            listener.BeginAccept(new AsyncCallback(OnConnectRequest), listener);
        }

        static IEnumerable<SocketInfo> Ports(string ports)
        {
            var portsets = ports.Split(',');
            foreach (var portset in portsets)
            {
                var portbounds = portset.Split('-').Select(x => {
                    int boundint = 0;
                    if (int.TryParse(x, out boundint))
                    {
                        return boundint;
                    }
                    return -1;
                }).ToArray();

                if (portbounds.Length > 1)
                {
                    for (int i = portbounds[0]; i < portbounds[1] + 1; i++)
                    {
                        yield return new SocketInfo(i);
                    }
                }
                else if (portbounds.Length > 0)
                {
                    yield return new SocketInfo(portbounds[0]);
                }
            }
        }
    }
}

xaade

@cvi Will this do? How do I close the listener? I wanted it on a separate thread, but it looks like that's not necessary.

cvi

@xaade Ah, sorry, I though you were writing in C++, but I rechecked your previous post, and apparently it was the original service that was C++. :-/

xaade

@cvi Still looks like I'm following the socket bind listen pattern.

cvi

@xaade Yeah ... seems similar enough. I don't really know how the BeginAccept() works. The docs say asychronous, so that's good but they also mention that the system may use a thread in the background for that, so you potentially still end up launching a bunch of threads.

xaade

@cvi lol, the program immediately terminated....

xaade

Well, here goes nothing

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;

namespace TcpMultiListener
{
    class SocketInfo
    {
        public SocketInfo(int port)
        {
            Port = port;
        }
        public int Port { get; private set; }
        public Socket Listener { get; set; }
    }
    class Program
    {
        const string DEFAULT_PORTS = "6001-6099";
        static string portsetlist = DEFAULT_PORTS;
        static IPAddress localaddress = IPAddress.Parse("127.0.0.1");
        static BackgroundWorker connectionsThread = new BackgroundWorker();
        static void Main(string[] args)
        {
            if (args.Length > 0)
            {
                portsetlist = args[0];
            }

            var socketInfoList = new List<SocketInfo>();
            var socketInfos = Ports(portsetlist);
            foreach (var socketInfo in socketInfoList)
            {
                socketInfoList.Add(socketInfo);
                socketInfo.Listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                socketInfo.Listener.Bind(new IPEndPoint(localaddress, socketInfo.Port));
                socketInfo.Listener.Listen(10);

                socketInfo.Listener.BeginAccept(new AsyncCallback(OnConnectRequest), socketInfo);
            }

            while(true)
            {
                Console.WriteLine("Type exit to end application");
                if ("exit" == Console.ReadLine().ToLower())
                {
                    break;
                }
            }

            foreach(var socketInfo in socketInfoList)
            {
                socketInfo.Listener.Shutdown(SocketShutdown.Both);
                socketInfo.Listener.Close();
            }

            //connectionsThread.DoWork += ConnectionsThread_DoWork;
            //connectionsThread.WorkerSupportsCancellation = true;
            //connectionsThread.RunWorkerCompleted += ConnectionsThread_RunWorkerCompleted;
        }

        private static void ConnectionsThread_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
            throw new NotImplementedException();
        }

        private static void ConnectionsThread_DoWork(object sender, DoWorkEventArgs e)
        {
            var socketInfoList = Ports(portsetlist);
            foreach (var socketInfo in socketInfoList)
            {
                socketInfo.Listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                socketInfo.Listener.Bind(new IPEndPoint(localaddress, socketInfo.Port));
                socketInfo.Listener.Listen(10);

                socketInfo.Listener.BeginAccept(new AsyncCallback(OnConnectRequest), socketInfo);
            }
        }

        static void OnConnectRequest(IAsyncResult ar)
        {
            SocketInfo socketInfo = (SocketInfo)ar.AsyncState;
            Console.WriteLine("Connected to port {0}", socketInfo.Port);
            socketInfo.Listener.EndAccept(ar);
            socketInfo.Listener.BeginAccept(new AsyncCallback(OnConnectRequest), socketInfo);
        }

        static IEnumerable<SocketInfo> Ports(string ports)
        {
            var portsets = ports.Split(',');
            foreach (var portset in portsets)
            {
                var portbounds = portset.Split('-').Select(x => {
                    int boundint = 0;
                    if (int.TryParse(x, out boundint))
                    {
                        return boundint;
                    }
                    return -1;
                }).ToArray();

                if (portbounds.Length > 1)
                {
                    for (int i = portbounds[0]; i < portbounds[1] + 1; i++)
                    {
                        yield return new SocketInfo(i);
                    }
                }
                else if (portbounds.Length > 0)
                {
                    yield return new SocketInfo(portbounds[0]);
                }
            }
        }
    }
}

Dreikin

Not sure if this'll help you, since I know naught about socket programming yet, but I found this:

IEvangelist

Use Sockets to send and receive data over TCP - .NET

Learn how the Socket class exposes socket network communication functionality in .NET.

cvi

So ... I guess it was finally time to write my first C# program ever. Here goes nothing:

using System;
using System.Net;
using System.Net.Sockets;
using System.Collections.Generic;

namespace net_accept
{
    class Program
    {
        static void Main(string[] args)
        {
            var sockets = new List<Socket>();

            IPAddress local = new IPAddress(0);

            const int start = 6000;
            for (int i = 0; i < 1000; ++i)
            {
                var s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                s.Bind(new IPEndPoint(IPAddress.Any, start + i));
                s.Listen(10);
                s.Blocking = false;
                sockets.Add(s);
            }

            var active = new List<Socket>();
            while (true)
            {
                var list = new List<Socket>(sockets);
                //list.AddRange(active);

                Socket.Select(list, null, null, -1);

                foreach(var s in list)
                {
                    Socket incoming = s.Accept();
                    active.Add(incoming);

                    Console.Write("New connection: " + incoming);
                }
            }
               
            // technically: Close() all sockets. But we never get here.
        }
    }
}

AFAIK, manages to accept a connection or two. The ~~printf~~ Write doesn't really print anything useful. Also, this never closes any of the incoming or listening sockets, because lazy.

dkf

@cvi said in Simple program with 1000 connections.:

AFAIK, manages to accept a connection or two.

You need to test with at least 65 connections, and not just having them connected; some simple echo service needs to be in operation. This is because, if the underlying system call used to do the management of the waiting for activity on the sockets is WaitForMultipleObjects, you'll have a hard limit of 64 handles that it can wait on. OTOH, if it is using IO Completion Ports then you can scale up a lot larger.

For comparison, I'd expect a simple single-threaded async service on Unix to get up to around 1000 before the first equivalent problem crops up (i.e., that can't be fixed by just recompiling with the right #define).

cvi

@dkf Quite right. But: lazy. And didn't have netcat handy (in fact, I tested by using browser tabs to initiate the connections [+netstat to see that they stick around] -- it doesn't get much more lame than that).

MSDN mentions that it you can redefine FD_SETSIZE, since the structure is technically dynamic-size but looks fixed-size for POSIX compat. The default value is already 64, and their example lists a larger number. SO (heh heh) says that people have gotten it to work with 5000+. I don't think the .NET version would be more limited than the good old select, because that would possibly be even more lame than using browser tabs to establish TCP connections for testing.

The Select already operates on a 1000 sockets, although all of them in listening mode. So it's unlikely that the implementation uses WaitForMultipleObjects with one handle per socket.

But, alas, yes -- you'd want to do proper testing. I'll also admit that I didn't really read the .NET docs about the stuff, but skimmed at best (and just used the auto-completion hints for the rest).