.net Schedulers
-
Anyone ever used a scheduler library in .net? We have an awful homegrown one that I'm thoroughly fed up with.
I'll be embedding it into a Windows service and managing from a web app.
-
-
@accalia Yeah, except that's just a parser. All the "is it time to go yet?" code is the hard part.
I could embed a windows port of cron, but I suspect there are more robust and flexible tools available.
-
A while ago I added Hangfire to a project I'm working on. I'm quite happy about it: set-up was a breeze and it can do exactly what I want. The website heavily advertises its way to integrate with ASP.Net, but I have it running in a Windows service.
It plays nicely with dependency injection, too.
-
I should probably note that we are looking at thousands of persistent schedules, configurable on the fly, occurring dozens to hundreds of times a day each.
-
That is a lot of scheduled jobs.
Hangfire supports multiple workers, as well as assigning jobs to different queues. The latter is an attribute of the job though, it is not something you can configure when scheduling the task.
An alternative: Quartz.NET.
-
This post is deleted!
-
@AlexMedia Was looking at Quartz.net.
Looks plausible, at least.
-
I've used Quartz.net, too. It's got a nice fluent interface.
-
Why don't you just find a web interface to Windows Scheduled Tasks? That seems like it'd make more sense to me.
Other than that, you can just use the
System.Timer
class to implement this. It already does everything correctly in the "non-polling" way. Not sure what you need more than that...The tricky part will be the interface design, of course, and the IPC for the web app to communicate with the Service. The Service itself should be relatively trivial to write.
EDIT:
@skotl said in .net Schedulers:
I've used Quartz.net, too. It's got a nice fluent interface.
I've never used it, but it looks over-engineered to me. Why is it talking directly to databases for example? Instead of letting you persist the data on your own?
-
@blakeyrat Persisting complicated schedules sucks serious shit. It looks like you can have it skip the perisitence if you want to bootstrap every startup, but the integrated persistence looks like a useful thing in my case.
-
@Weng It's not fun, if that's what you mean, but it's also not difficult. Just a few hours of grunt-work.
I just don't like libraries that go WAY out of their purview. The library is for scheduling a bunch of tasks, why does it need a database? That's not your job. Let someone else do that job.
Anyway, I don't have any better recommendations, but personally I'd roll-my-own. .net already contains all the tools you need, you just need to wire them together. And the bulk of the labor by an order of magnitude will be designing the web UI anyway.
-
@blakeyrat As to Windows scheduler, our server admin teams consider it to be their exclusive domain and will add/delete/etc. jobs at will.
I agree with you for most scheduler implelentations - just roll your own. No big deal.
Trust me, however - this use case is too complex for that to be maintainable. We tried. Thrice.
-
@Weng said in .net Schedulers:
Trust me, however - this use case is too complex for that to be maintainable. We tried. Thrice.
What makes you think this third-party has succeeded where you have failed? Have you analyzed why you failed? Does this third-party do a better job of that thing?
-
@blakeyrat Quartz is shit for anything even remotely advanced. We're trying to purge it from our codebase for all new development, we have top entries on our db for tens of thousands of quartz polls since it doesn't do any type of batching, and has no concept of sleep timers / events.
-
@blakeyrat Because Time is apparently hard for the lame developers I'm allowed to hire.
This is also not a core competitive advantage feature of our product and therefore a really wasteful place for us to spend time if we don't have to.
For things in that category, if a third party product works reliably and supports what we need, and it's less work to integrate with than to build ourselves, we should use the third party product.
The only reason we ever did it in house is because some numpty in the past thought the ability to process things on a schedule was our entire competitive advantage.
And then we did it again because another numpty thought it would be cheaper to do in house.
Now I have reliable maintenance cost metrics that will let me buy out of rewriting the whole thing again. Not because it doesn't work, but because we need to add features that break the implementation.
-
@Weng I don't think a third-party library is going to fix your problems.
-
@blakeyrat The name say "Quartz Enterprise Scheduler". It's enterprise quality software, it must be good.
-
@fbmac To be fair, and to bolster it's enterpriseyness even further, quartz originally came from J2EE. That may explain the over-engineering.
Anyways - it's worked for me the handful of times that I've used it, and I have no problem with storing in the database (YMMV).
-
By the way, I'd also disagree those from the "how hard can it be to roll your own?" camp.
Sure - for simple recurring tasks, it probably is easier to make your own but for anything of medium or more complexity then it quick becomes a pain in the butt. Last Friday of the month? 1st and 3rd Thursdays? Every Monday for the weekly reports, except the last Monday in the month, where we run the montlhy report?
20th of each month except where that falls on a Monday?
-
@Matches said in .net Schedulers:
@blakeyrat Quartz is shit for anything even remotely advanced. We're trying to purge it from our codebase for all new development, we have top entries on our db for tens of thousands of quartz polls since it doesn't do any type of batching, and has no concept of sleep timers / events.
Interesting. I'm using Quartz on our 'big' project but I only have about 40 jobs at any time. Seems to work great at this low level though. I liked it because it was pretty much drop it in and it just works, even the persistence.
If you only need a small amount of scheduling I would recommend it based on how its been for me.
-
@skotl said in .net Schedulers:
Sure - for simple recurring tasks, it probably is easier to make your own but for anything of medium or more complexity then it quick becomes a pain in the butt. Last Friday of the month? 1st and 3rd Thursdays? Every Monday for the weekly reports, except the last Monday in the month, where we run the montlhy report?
20th of each month except where that falls on a Monday?Well keep in mind, my first suggestion was to just make/find a web interface to Windows Task Scheduler, which handles all of that complexity easily.
The real deal-killer for Quartz, at least from my perspective, is the polling. If it has to poll that often, it's been designed very badly at least at one level.
-
Fuck Quartz. Those ass-monkey developers think a built-in on by default phone home update check is a good idea for a library.
http://www.quartz-scheduler.org/documentation/best-practices.html#production-system-tips
-
@skotl said in .net Schedulers:
To be fair, and to bolster it's enterpriseyness even further, quartz originally came from J2EE. That may explain the over-engineering.
We use it. It works well enough. I haven't used anything else like that before and I'm frankly befuddled at @blakeyrat's befuddlement about using a DB to persist stuff.
-
@blakeyrat said in .net Schedulers:
Well keep in mind, my first suggestion was to just make/find a web interface to Windows Task Scheduler, which handles all of that complexity easily.
Is that stuff easy to backup / restore / transfer to another server as needed?
-
-
@boomzilla said in .net Schedulers:
Is that stuff easy to backup / restore / transfer to another server as needed?
It's fully API'ed.
-
@blakeyrat You should really request access to the Lounge, where we talk extensively about all my unfixable problems. This gig is all about alleviating symptoms.
-
@boomzilla said in .net Schedulers:
We use it. It works well enough. I haven't used anything else like that before and I'm frankly befuddled at @blakeyrat's befuddlement about using a DB to persist stuff.
The point is it's talking to the DB directly. IMO a scheduling library shouldn't have a dependency on a database. It should simply expose its configuration so the application implementing it can persist the data however it wants.
-
@blakeyrat said in .net Schedulers:
The point is it's talking to the DB directly. IMO a scheduling library shouldn't have a dependency on a database. It should simply expose its configuration so the application implementing it can persist the data however it wants.
It persists the data how you tell it. Telling it to use the application's database is convenient because it's basically guaranteed to be there and the tasks are always there if the database is there. It's simpler than having an additional dependency or thing to backup, etc.
That said, I'm on a relatively ancient version, 1.6, so I assume it works significantly different to the current stuff.
-
@Matches are we talking about 1 poll a second or 1000 polls a second? Because one of those is just noise in my database.
-
@Weng Lounge, you say? :shifty_eyes: How would one go about locating this... Lounge?
-
@error I didn't say anything about any lounge
-
@error said in .net Schedulers:
Lounge, you say? :shifty_eyes: How would one go about locating this... Lounge?
It's in the Private section of the forum.
-
-
@Weng and no more gamification involved
-
@error said in .net Schedulers:
How would one go about locating this... Lounge?
Turn Left and carry on until morning.
-
-
@blakeyrat join us, there'll be cake there
-
@fbmac If someone wants to add me they can.
-
@blakeyrat you can just click join group now, come on
-
@fbmac Uh ok? I don't know where that's located.
-
-
@fbmac I pushed a button which does stuffs.
-
@Weng I always use WMI Win32_ScheduledJob class to schedule jobs.
If the job runs in Windows Service, all you need to do is to run "sc control [service name] value" to send custom command to the service.
-
@asdf it's not that there aren't any, it's just that I'm at a point where I'm too tired/fed up to deal with it twice.
-
@Arantor Oh, okay. Hang in there!
-
@Weng talking about 15000+ polls within 3s
Granted at least part of that is a lot of jobs, but when you're comparing a couple hundred or so jobs the schedule should have at least some minimal knowledge of what it's polling is doing, or an option on how to consolidate/cache polling. There's really no reason for a quartz job that has a timer of a day to poll every second a dozen times.
Most of our jobs are hour/day level, but the polling is constant for all jobs, instead of a single job that polls fast to trigger the jobs that are ready.
In my head it makes more sense to say
- 1 running poll thread
- Job ready
- Delegate execution call to execute job
- Job sleeps when done
You don't need 500 threads polling to see if the job is ready after 1s if the job is scheduled to sleep for a day. Just tie your update method to trigger the job if the schedule changes, but even that should be picked up by the single poll thread.
-
@Matches Siiiigh.
-
@Weng we use an aws 8xlarge ec2 for SQL server (writes for scanning loops, website, forums, etc), and execution takes about 50ms of cpu time. Negligible as a general impact, but still, it's a fucking polling scheduler. That is ultra micro making a number impact.