Platform for data export



  • I'm making a data export tool for my firm's electronic health records platform. The platform runs on Windows, and uses the Advantage Database System (I guess it was originally by Sybase, but whatever). There is a long list of business rules and a vaguely complex format specification. Normally, I'd just use Haskell, but there could be many gotchas with database drivers, Windows integration, and certification by the state, and I want to go as gotcha-free as possible.

    What are my options for a development platform? I'm looking for something that:

    • can provide a pleasant development environment. I like Vim and the command line, etc. I don't have experience with Visual Studio, but it would be okay to learn. But if I go that route, what are my language choices? Do any fit my other constraints?

    • uses a sane language. Not VB.NET. C# is sane, but I don't know it very well, and this seems like more of a "scripting language" task (i.e. C# seems like enterprisey over-kill). Haskell would be okay -- I know it well enough to make use of its strengths to get the task done fast -- except for the Windows/IDE/integration situation. I'm considering Python, but I don't know the state of IDEs on Windows.

    • can do databases by ODBC.

    • "integrates" with version control. Again, I don't mind using a sane command line as my development environment, but I doubt cmd.exe qualifies.

    If you had no constraints except these, which tools would you pick?


  • BINNED

    Given that you're coming from Haskell, F# is about as sane as it's going to get given your other requirements.


  • 🚽 Regular

    @antiquarian said:

    Given that you're coming from Haskell, F# is about as sane as it's going to get given your other requirements.

    That seems like sensible advice given what little I know about both languages.

    @Captain said:

    C# is sane, but I don't know it very well, and this seems like more of a "scripting language" task (i.e. C# seems like enterprisey over-kill)

    Coming from a C/ASM beginning (and loving C#) I'd say C# is only as enterprisey as you want to make it. I like the memory management features a great deal and apart from that you can make it very terse if you're so inclined.
    Personally I'd use C# for this particular task.

    @Captain said:

    pleasant development environment

    @Captain said:
    I don't have experience with Visual Studio, but it would be okay to learn

    I've used a lot of IDEs and VS is my favourite by far...mostly because the design is reasonably sane and intuitive and it doesn't crash every 5 minutes like most of the others.


  • BINNED

    I should mention that constructors in F# aren't curried, which annoyed me to no end.



  • @Captain said:

    which tools would you pick

    The important question is why do you want to build a platform at all. If I had only the information you posted and I made the assumption that the creation of data export jobs was assigned to your team to carry out, then I might say that all you need to do is build some libraries for the things that come up all the time in your world.

    However, I assume you are building a platform so that non-developers can build export jobs. The details of the testing workflow, promotion to production process, approvals, and all of the business-ey stuff really drives what you will need to build.

    For example, it may turn out that due to outsourcing and/or communications issues that it is vital that the work in progress be stored in a file. Or maybe due to the need to track approvals and the need for accountability that job definitions should be stored "with the system" and all interactions should be tightly controlled. So, this could turn into a loosely organized set of command line tools, or a mega-IDE.



  • The important question is why do you want to build a platform at all.

    I don't want to build a platform. I want to use a platform that makes doing my job easy, given that I have certain preferences and constraints. I can't change the database, for example. Good pretty printing libraries, in whatever language I end up using, would make my life much, much easier. But languages tend to be paired to development environments, and I need to work in Windows. Etc.



  • If you're stuck on Windows and are doing a lot of database work, .Net/Visual Studio really is hard to beat.

    Every time one of our network guys gets me to do DevOps type stuff for them, I find it much easier to make a Windows Service in C# rather than make the PowerShell script/Task Scheduler thing they envisioned.



  • @Captain said:

    uses a sane language. Not VB.NET. C# is sane, but I don't know it very well, and this seems like more of a "scripting language" task (i.e. C# seems like enterprisey over-kill).

    C# is perfectly suitable to do "scripting language"-esque tasks. There's very little (if any) overheard/boilerplate that Visual Studio won't just automatically do for you as if by magic.

    If you like the CLI, just create a new Console Application project, grab "Command Line Parser" off Nuget for the arguments, and shove the Main() full of your task. You're golden.



  • Super. Looks like C#.NET + VS (+ Linq, I guess) is what I'm looking for.



  • Or F#, since you seem to be into functional programming.



  • He could even go IronPython if he wants. It all works.



  • A bit off-the-beaten-path but we've been using Camel for over a year now as our integration platform.
    It addresses pretty much all of the EIPs and leaves you to writing transform code rather than getting stuck on file, database, soap, rest, etc wiring.

    The downside if you like C# (as I do) is that it's predominantly Java-based but it's a piece of piss to create SOAP or REST services in it so you still use Camel for all the EIP / transport shit and use C# and Linq (great for querying sets of objects) for the transformation part.


Log in to reply