Github - TDWTF-Forums
-
If you are interested in the project I'm going to be working on, it will be located here:
Open source, because obviously it will be shit software <according to the rat>
Software of choice? Visual studio 2015 community edition, virtual box, and a windows server 2012 ISO. For testing I will also be setting up a micro linux server for load balancing and general purpose runner boxes for testing. YMMV, but I will provide complete setup instructions to replicate the environment on the wiki.
How dedicated will I be? Spare time only.
License? IDK - Apache 2.0, Don't like it? Propose something better. The current criteria is mostly "Throw a noodle" tactics.
Linux? Going to be targeting .NET 4.5 and 4.6 depending on stability of features. Theoretically Mono is going to be plug and play - I'm not going to be actively supporting Linux, but if something is broken and a patch is recommended I'll pull it in.
Can I do some feature? Sure, I don't care. It will have to meet criteria about stability (Doesn't crash, Doesn't completely wreck server resources), and security though.
Call me again in 10 years? Probably not, but you can check on the progress if you're bored!
But nothing is there yet? Yep, going to be updating this post sometimes. This is just a repository stub that I'll be committing to as I get work done. I'm going to mostly be a weekend warrior for the time being though. Clone it locally in source tree (or w/e) to get updates.
I want power! If you're skilled in the eco system, you'll have it. Otherwise, fuck off.
Html Markdown Other? No. Supported tags will either be HTML or BB Code. HTML tags would need to be white listed, and would be put in a content container sandbox so that (in theory) it won't spill out. (So FA-Spin would only be able to spin your own post, provided it's white listed.) - Full exact rules TBD, but the primary requirement is that all input generates expected output, and that preview output is exactly what will actually be rendered.
I don't know the language, but I'm willing to learn? You can either take grunt work tasks and be helped to understand how it fits in to the larger picture, and have your questions answered - Or you can go learn stuff by yourself and come back to the project later. Insecure, bad performing code stays out of the master branch.
Shared host support (IE: Godaddy) Probably not. Access to run arbitrary access is required, so at minimum you're probably looking at a VPS / AWS EC2 instance.Technology stack / organization:
- C# .NET 4.5+ MVC front end (+jquery +jquery-ui +signalr possibly +knockout)
- C# .NET 4.5+ job runner (Think console app)
- C# .NET 4.5+ Back end
- C# .NET 4.5+ Monitor (Think console app) - Validates everything is healthy / running
- SQL Server database (Express for micro/small sites, standard/enterprise for medium-large. Exact requirements TBD) - There is a reasonable chance that postgresql will be supported once the core framework stabilizes.
- Load balancing supported via Nginx routing, allowing for true 24/7 up time using 2+ servers (Small print here about stuff)
- Small scale should run on a 256/512mb machine pretty reliably (Exact stats still TBD - TDWTF is probably somewhere between small and medium scale, where it's a small amount of users creating ridiculous server load)
- If the project gets large enough to require message queuing, it will probably be RabbitMQ, but I think there's a pretty strong possibility that would be extreme overkill for anything but the largest of sites.
- Docker? Maybe in the future, microsoft is doing something with it but I'm not really sure where they want to go with that.
There's more, but I'm hungry.
TL;DR; Fuck discourse, it's time for something that works and doesn't take a long ass time to load basic discussions. You're welcome to ask for an invite/invite yourself to the github, or just eat some popcorn.
-
If you're using Visual Studio as your IDE, why the hell would you use Git instead of Visual Studio Online where SANE PEOPLE could access it via TFS? (And insane moron retard people could still use Git.)
-
Because I don't want to use TFS, and since it's my project, I get to do whatever the fuck I want because open source.
<also, because it pisses blakeyrat off>
-
Visual Studio 2015 has built-in support for Git, not sure why you're complaining.
-
My mouth supports eating shit, that doesn't mean I had shit for dinner.
-
Other than just trolling you, the real answer of "Why git over TFS" is because TFS is only free up to 5 team members. Github is completely free for public projects (which this is) or I could have used BitBucket (my normal go-to for private repos) but github is more "public".
-
My penis is public, I don't put it up on Github.
Although...
-
-
-- Problem: there are 2 competing TDWTF in-house forum platforms
-- I know, I'll make the ultimate Discourse replacement forum, to supplant all the others
-- Problem: there are 3 competing TDWTF in-house forum platforms
-
There are 104385793 open source forum platforms already. Maybe we should fork an existing one and adapt it instead of making new ones?
Just saying.
-
Out of the air, how much would we have to pay Microsoft to handle this sort of load with all those fancy technologies you threw there? And some of those are far from free (as in beer)
Also, I could do the same with a bunch of Oracle and IBM products... Any AS400 specialist around here?
-
Any AS400 specialist around here?
Not quite but if you're in London there are quite a few places if you have the experience. It's on my cv from college and I got six recruiters looking specifically for it. I promptly removed it.Just out of curiosity, how many people are working on forum software at the moment?
-
-
Out of the air, how much would we have to pay Microsoft to handle this sort of load with all those fancy technologies you threw there? And some of those are far from free (as in beer)
I was wondering that. We put a tiny SQL Server database into Azure to farm out instructions to worker nodes and quickly discovered that the tiny / small instances simply stop responding when given "too many" requests at once (where "too many" = maybe 100 connections from 100 remote VMs via an Azure hosted IIS app). We ended up having to move it from the ~$20 a month plan to one at several hundred dollars per month, all for a database that only has around 200MB of data but sporadically needs to handle hundreds of simultaneous requests.
So don't underestimate the cost of doing this all on a Microsoft stack.
-
My penis is public
Actually, someone invented something to fix that.
It's called "clothing."
-
Does that include licensing or you can scale in Azure to more enterprise products (MSSQL Express -> MSSQL PIG) without paying any fees?
-
Just a sidenote: wouldn't this be better in
Programmers' Testing?
-
Does that include licensing or you can scale in Azure to more enterprise products (MSSQL Express -> MSSQL PIG) without paying any fees?
It doesn't really talk in terms of editions as we know it on-premise. (express/standard/enterprise, etc). There are tiers, which then get scaled with vcpu and RAM, but also these ethereal "Database Throughput Units". It looks like the latter that was hurting us and the only way to increase the Database Throughput Unit pixie dust is to scale up the tier, vcpu and RAM.
You could pay up to $7k a month for a database...
-
With this amount of competition, I wouldn't be suprised if it ended up in
Games/Mafia.Filed Under:
-
I said
Programmers' Testing, notMigration....##vote @Matches ....
-
Aws medium server ec2, about 40..60 mo depending on how beefy you want, then add another 40 to 150 for the db. (Storage requirements is the primary question)
My guess is tdwtf could easily get by on the 80 to 100 range when the server requirements drop for this bloated piece of shit. The primary reason i would support postgres is to remove the giant cost of sql server.
Aws, godaddy, and the current host all have reasonable hosted solutions.
-
Forgive me if i don't think a nodejs forum is a good idea. But i think by the time it starts getting to maturity for use, it's going to suffer from performance issues and be very difficult to support back end jobs.
I'll be starting with the back end work, so it's not impossible to merge some ideas later, since the primary delivery of Ajax data will be json. That said, the actual needed Javascript will be relatively light compared to this thing (which isn't saying that much i suppose)
-
You did it wrong (and pricing out hardware is part of what i do, and i work with a lot of big data (ie: billions of rows per day)
If you need help finding reasonable cost servers and correct performance let me know and i can help.
-
I'm not testing anything. It's more of an announcement that 'fuck this shit.'
-
the actual needed Javascript will be relatively light compared to this thing
That depends on whether you're using Ember or whether you're sane.
-
Ember is not a thing i will be using.
-
Out of the air, how much would we have to pay Microsoft to handle this sort of load with all those fancy technologies you threw there? And some of those are far from free (as in beer)
OH NOES paying money for software! It the APOCALYPSE!!! CATS AND DOGS LIVING TOGETHER!
Says a guy who probably makes a living writing software..
-
Forgive me if i don't think a nodejs forum is a good idea.
you are forgiven, on the condition that you forgive us for making a forum in javascript anyway.
it's going to suffer from performance issues
i disagree, it needent not if it's properly architected.be very difficult to support back end jobs.
not necessarily. that's what crontab is for.
That said, the actual needed Javascript will be relatively light compared to this thing (which isn't saying that much i suppose)
we agree with that!I'm not testing anything.
we will be. that's how we plan on avoiding getting into this situation that discourse has where we keep getting bugs that just won't die.Ember is not a thing i will be using.
Ember needs to
in a
-
We ended up having to move it from the ~$20 a month plan to one at several hundred dollars per month, all for a database that only has around 200MB of data but sporadically needs to handle hundreds of simultaneous requests.
So don't underestimate the cost of doing this all on a Microsoft stack.
Or you could be smart.
If you only have 200 MB of data, put it in a copy of SQL Express and run it in a VM.
-
If his issue is throughput his hardware cpu/memory/network matters more than hd space, but at that point you're better off getting a low grade physical server than a vm, even if it's running express. Virtual machines tend to take a performance hit at high throughput.
-
You can develop whatever you want, you're the one who has to deal with it, not me =p
Cron tab for scheduled jobs will be messy, you're talking about probably mid hundreds of jobs unless you're going to kick off a job that's a runner app.
The no testing was a response to this thread not being under testing, which i think you were trying to make a joke about, but it's kind of hard to tell.
By all means, come up with something that smokes anything i design, it will just mean i don't have to support anything in the future which will be nice.
Do you plan to support windows, or Linux only?
-
The main concern here is that Microsoft server was NOT designed for the "not a personal use, but not a multibillion dollar industry" use case. The price goes from free for something you and a few friends use, to super expensive super quickly. I'm fine with microsoft making money, but that doesn't mean I can afford professional grade software for a system that may never go above moderately busy.
-
Well maybe, there's tons of alternatives. The point is, if you're only storing 200 MB you can do lots of shits to make things less-dumb.
-
You can develop whatever you want, you're the one who has to deal with it, not me =p
Cron tab for scheduled jobs will be messy, you're talking about probably mid hundreds of jobs unless you're going to kick off a job that's a runner app.
yeah a runner app would be the way to go with that.which i think you were trying to make a joke about, but it's kind of hard to tell.
was i too dry in my delivery? sorry.By all means, come up with something that smokes anything i design, it will just mean i don't have to support anything in the future which will be nice.
i'll aim to!although with what we're designing i doubt WTDWTF will want to move to it. we're doing rather an odd take on forums.
PBP role players on the other hand will probably be excited about it.
Do you plan to support windows, or Linux only?
Linux is the primary target, but we will aim to have it Windows compatible. In theory we'll be perfectly windows compatible so long as we're careful in our package selection as NodeJS has decent windows support, the only failing tends to be in the packages which might not support windows by having custom C code that assumes *nix or by having bash install scripts that they really shouldn't have.
-
True.
-
OH NOES !PANICK PANCKICIK MONEY FOR SOFRQWARE PANICSKG!!
If you don't like it, just don't fucking use it.
-
Load balancing supported via Nginx routing, allowing for true 24/7 up time using 2+ servers (Small print here about stuff)
If you need any help with nginx/load balancing/SSL termination/what-have-you I'm more than happy to pitch in if you send me a message or something.
-
The primary reason i would support postgres is to remove the giant cost of sql server.
Well maybe, there's tons of alternatives. The point is, if you're only storing 200 MB you can do lots of shits to make things less-dumb.
Yes Exactly. I don't exactly see how that was a healthy response to my statement?
-
That's kind of a dumb stance to take, but the problem is in his selected solution, not in the application pricing (in this instance) - but sql server DOES have a sticker shock when moving from express to standard based on your licensing per computer cores.
-
In general I'm good on the load balancing, but i might take you up on that for ssl termination, thus far I've not really seen any way for nginx to blind pass an ssl connection to a server, most examples show nginx as the ssl endpoint and http connect to the target server.
My primary concern is to have end to end encryption support, basically i want to be able to disable port 80 on the server, but have nginx handle the ssl upstream translation so 80/443 are both supported.
Does nginx support a full passthrough connection, or do i need something like squid?
-
The big problem with SQL Server is that the pricing STILL hasn't been sanity checked for cloud computing environments (where "per-core" is meaningless.)
Unless you're huge enough to negotiate a deal that covers all your users, like Amazon is. Cloud users are just fucked by SQL Server licensing.
-
I would imagine the new CEO is planning on fixing that soonish. He is making broad strides to make microsoft appeal to smaller installations as well
-
Nginx should work fine as a reverse proxy for SSL termination; it's one of its designed features. As for port 80 - you wouldn't want to disable that so much as just 301 to HTTPS and enable the HTTP
Strict-Transport-Securityheader with a 1 yearmax-age. We aren't at DEFCON 1 with HTTPS here so having that initial 301 shouldn't be a major risk.
-
You can have nginx run in SSL mode on both ports and there's a fake HTTP error code you can catch and redirect to the HTTPS site.
-
I've been debating having the redirect at code level or the reverse proxy level. To me, it seems to make more sense at the proxy level and add https strict headers. Still not sure what the actual best practice is, just that i've done both and doing it in code is easier, but comes with some quirks related to first load.
-
I would imagine the new CEO is planning on fixing that soonish. He is making broad strides to make microsoft appeal to smaller installations as well
Why don't you send an e-mail to Satya Nadella?
I once e-mailed SteveB while he was still the head honcho and he replied within minutes...
-
-
That's actually fairly impressive. Though I wouldn't be surprised if it was handled by someone else and just sent out with his name attached.
However, I do not have any personal stake in this matter, nor do I represent anyone who does, so until I have a matter that needs solving, I will leave the management to the professionals
-
Best practice is to 301 (and rewrite) to HTTPS so older browsers cache it and to enable STS header so that newer browsers will be protected against SSLStrip attacks.
-
Or you could be smart.
If you only have 200 MB of data, put it in a copy of SQL Express and run it in a VM.
@blakeyrat said:Well maybe, there's tons of alternatives. The point is, if you're only storing 200 MB you can do lots of shits to make things less-dumb.
In an attempt to paint myself out of being TRWTF the app works, it worked in testing and it even worked in-house on a dedicated VM (as you suggested) with all the nodes pointing to it over a VPN. The point of moving it to Azure was to do away with the VPN or us having to manage any of it.
What we had underestimated was the impact of these mysterious "Database Throughput Units" - TBH we just assumed that if it got busy it would slow down; it's just for instructions being farmed out to remote nodes so tasks taking 30 minutes instead of 30 seconds would have been fine. We didn't anticipate "Not enough DTUs" = "requests cancelled / ignored"...
So next time we'll bear that in mind.However (and I hope this is an anti-WTF) we figured spending some money on the database made the original design work so let's swallow ~$6k extra a year instead of spending more valuable human hours on re-engineering the solution.
TDWTF-Forums
