Throwing ideas at the wall to see what sticks...
-
Many people have noted that PowerShell is cool because it operates on objects, rather than being stringly typed like older, more primitive shells that are stuck in the 1970s.
Many people have also noted that this advantage is of sharply limited utility because stdin and stdout are essentially text streams, so all communication with programs becomes stringly typed anyway.
Speaking purely hypothetically, what if that wasn't the case?
Imagine an OS (and this would have to happen at the OS level) where a program's manifest/metadata/whatever would define its input and output as a stream of type
T, using some sort of shared memory model to allow for efficient communication between processes without having to marshal everything to and from strings.There would, of course, be a standard "serialize" utility that could turn arbitrary objects into text, for printing to consoles or log files.
Good idea? Bad idea? What sorts of interesting things could go horribly
-shaped under a model like this?
-
Isn't this more or less what COM/COM+/DCOM are/were?
-
Needs more endofunctors.
-
@boomzilla You sure? Seems like this is more of an exo-functor thing...
-
@mikehurley said in Throwing ideas at the wall to see what sticks...:
Isn't this more or less what COM/COM+/DCOM are/were?
Kind of, but more less than more. COM is about embedding third-party services in your process (or proxies to them, in the case of out-of-proc COM servers), and DCOM is about communicating with COM servers on the far side of a network connection. This idea is about building a better pipe to chain one program to another.
-
@Mason_Wheeler I'd compare it to Android bundles. Which, by the way, work pretty well. Much better than if everything would have to be passed as text strings, anyway.
-
There have been plenty of systems trying to build a better system for messaging between software than just throwing strings of characters or bits at each other. I think every single one has collapsed under it's own weight eventually.
I am no fan of stringly typed interfaces, but when you structure the interfaces, you also limit them and make them slower, especially of you want some polyglot levels of interaction and libraries dealing with the naughty bits of mangling things for the various platforms to support.It's something that you'd have to build the OS around I think. And while I'd welcome a new and well designed OS instead of the top ones we have now, I don't see it happening.
-
@Carnage said in Throwing ideas at the wall to see what sticks...:
It's something that you'd have to build the OS around I think.
So basically...
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
Imagine an OS (and this would have to happen at the OS level)
-
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
@Carnage said in Throwing ideas at the wall to see what sticks...:
It's something that you'd have to build the OS around I think.
So basically...
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
Imagine an OS (and this would have to happen at the OS level)
Yes, and the OS needs to be designed around it.
It'd be an interesting undertaking, if the
wasn't so strong around this forum, I'd say we have pretty much all the skills necessary to actually do it on these forums.The interresting bit would be to make such an OS play nicely with linux and/or microsoft software to make it have even a snowballs chance in hell of taking off.
-
@Carnage said in Throwing ideas at the wall to see what sticks...:
The interresting bit would be to make such an OS play nicely with linux and/or microsoft software to make it have even a snowballs chance in hell of taking off.
I dunno. iOS managed to become successful without that. (Granted, that was mostly due to Apple's brand image and RDF, but still, this shows it's possible...)
-
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
@Carnage said in Throwing ideas at the wall to see what sticks...:
The interresting bit would be to make such an OS play nicely with linux and/or microsoft software to make it have even a snowballs chance in hell of taking off.
I dunno. iOS managed to become successful without that. (Granted, that was mostly due to Apple's brand image and RDF, but still...)
Yes, there was a new niche to fill with a new OS.
Unfortunately, that's not the case with desktop, or phones, these days.
There have been plenty of OSes as good or better than Windows, Linux and MacOS, none of them are really around these days.
-
@Carnage And for whatever reason, I read that and my mind immediately goes to TempleOS.
-
We can serialize things to stdout and achieve exact the same
-
@sockpuppet7 ...much more slowly.
-
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
Imagine an OS (and this would have to happen at the OS level) where a program's manifest/metadata/whatever would define its input and output as a stream of type
T, using some sort of shared memory model to allow for efficient communication between processes without having to marshal everything to and from strings.Who defines all these
Ts? What do I do when theTthat matches what I'm doing doesn't have enough stuff for me?My honest impression is that it would turn into something like a bureaucratic nightmare. Or everyone would have their own thing and you'd have to do much uglier things than parsing text streams.
-
@mikehurley said in Throwing ideas at the wall to see what sticks...:
COM/COM+/DCOM
Every time I had to use something distributed as com+ it was a nightmare. I hope these ideas stay dead
-
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
@sockpuppet7 ...much more slowly.
Much more slowly than what?
-
@boomzilla said in Throwing ideas at the wall to see what sticks...:
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
Imagine an OS (and this would have to happen at the OS level) where a program's manifest/metadata/whatever would define its input and output as a stream of type
T, using some sort of shared memory model to allow for efficient communication between processes without having to marshal everything to and from strings.Who defines all these
Ts? What do I do when theTthat matches what I'm doing doesn't have enough stuff for me?My honest impression is that it would turn into something like a bureaucratic nightmare. Or everyone would have their own thing and you'd have to do much uglier things than parsing text streams.
In our codebase, all our services take and return objects that are Plain Old Data objects (basically, something easily serializable to JSON). If that's a hard and fast rule, then adapting a Foo to a Bar is pretty straightforward. You might want duck typing here.
-
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
@sockpuppet7 ...much more slowly.
It really doesn't matter for the vast majority of applications, while on the other hand shared memory between completely independent processes (i.e. independently developed, combined and run) is a huge gun pointed at your foot. If you're processing amounts of data where it would matter, a shell script is not the right tool, but it hardly ever does. Parsing a megabyte of JSON takes a few milliseconds on a modern CPU, and that's not even the fastest codec. Something like Protocol Buffers would probably be too rigid; faster formats like Sereal exist but given that everyone and their grandmother can deal with JSON it's probably not even worth using something lesser-known for that last bit of performance.
-
Congrats, you've found the billion dollar question.
Yes, structure is good. Getting programmers to accept that structured data is good is unironically one of my life goals. Not full C++ classes, just a "JSON-equivalent" format. String, number, list of things, maybe a list of key-values.
It's not as simple as doing that, however. The basic protocol should be very simple if you want adoption and efficient implementations. But then you need to build up from there, because people will want to store and pass more than just a list of objects. You'll need to add standards for things like
- File format info
- Version info
- Ways to add or remove stuff without breaking forward compatibility
- Types and validation
- Standardized formats for objects you might want to include in your objects (images, list of string localizations...)
-
@Carnage said in Throwing ideas at the wall to see what sticks...:
I am no fan of stringly typed interfaces, but when you structure the interfaces, you also limit them and make them slower, especially of you want some polyglot levels of interaction and libraries dealing with the naughty bits of mangling things for the various platforms to support.
The biggest single problem is that now every program that interacts with another program has to deal with whatever idiotic internal data model that that other program uses. Because there's no hope in hell of persuading everyone to use a single compatible data model; that's just not going to happen (and everyone who's ever done integration work should know it in their bones).
Strings of plain text don't make things much easier except that they're pretty simply human readable and actually that helps a lot in reality…
-
@dkf said in Throwing ideas at the wall to see what sticks...:
Strings of plain text don't make things much easier except that they're pretty simply human readable and actually that helps a lot in reality…
You clearly haven't seen some of the plain-text output I've seen!
-
@dkf said in Throwing ideas at the wall to see what sticks...:
Strings of plain text don't make things much easier except that they're pretty simply human readable and actually that helps a lot in reality…
Since we're talking about that stuff, can I expound on my theories here?
ASCII is not more human-readable than other formats. If you look at the bytes, you won't understand a thing. So why is it treated as such? Because every text editor knows how to transform them to and from characters.
(In fact, ASCII has File separator, Group separator, Record separator and Unit separator characters. So it's essentially the first binary serialization format for text based data)A well-established structured data format could work the same. A standard binary format could be shown as XML (or similar syntax) text for viewing. Then you edit the text and it edits the file.
OK, I admit there are some differences. ASCII is an "unstructured" format, so any fragment of it will still make sense. Other formats, not so much, and that can pose some problems, but I don't see that a showstopper.
-
@dkf said in Throwing ideas at the wall to see what sticks...:
The biggest single problem is that now every program that interacts with another program has to deal with whatever idiotic internal data model that that other program uses. Because there's no hope in hell of persuading everyone to use a single compatible data model; that's just not going to happen (and everyone who's ever done integration work should know it in their bones).
PowerShell uses the .NET type system and conventions. If we're talking about an OS built on the same concepts, then just use the Apple principle: "our way or the highway."
-
@error said in Throwing ideas at the wall to see what sticks...:
@dkf said in Throwing ideas at the wall to see what sticks...:
The biggest single problem is that now every program that interacts with another program has to deal with whatever idiotic internal data model that that other program uses. Because there's no hope in hell of persuading everyone to use a single compatible data model; that's just not going to happen (and everyone who's ever done integration work should know it in their bones).
PowerShell uses the .NET type system and conventions. If we're talking about an OS built on the same concepts, then just use the Apple principle: "our way or the highway."
There is a reason why the Unix family is the one where combining system tools with a shell language is the encouraged and everyday way to do things, on Windows it's far less common and it was close to nonexistent on MacOS before they brought in the unixish foundations. Nowadays Apple basically has two classes of users: the ones you see at techn conferences who use it like a slick Linux that enjoys talking to their iPhones, and the ones who can't see a command prompt without getting a nervous twitch and use it like they've always used their Apple, clicking their way through the officially blessed applications.
-
@LaoC said in Throwing ideas at the wall to see what sticks...:
@error said in Throwing ideas at the wall to see what sticks...:
@dkf said in Throwing ideas at the wall to see what sticks...:
The biggest single problem is that now every program that interacts with another program has to deal with whatever idiotic internal data model that that other program uses. Because there's no hope in hell of persuading everyone to use a single compatible data model; that's just not going to happen (and everyone who's ever done integration work should know it in their bones).
PowerShell uses the .NET type system and conventions. If we're talking about an OS built on the same concepts, then just use the Apple principle: "our way or the highway."
There is a reason why the Unix family is the one where combining system tools with a shell language is the encouraged and everyday way to do things, on Windows it's far less common and it was close to nonexistent on MacOS before they brought in the unixish foundations.
At least Classic MacOS had AppleScript; it had more application support than COM Automation had back in the day.
-
Also
@dkf said in Throwing ideas at the wall to see what sticks...:Because there's no hope in hell of persuading everyone to use a single compatible data model; that's just not going to happen (and everyone who's ever done integration work should know it in their bones).
You say that, but there's many standards that have gotten close to "100% market penetration". TCP/IP, USB, ASCII, Unicode, C, HTTP/HTML, OpenGL...
-
@Parody said in Throwing ideas at the wall to see what sticks...:
At least Classic MacOS had AppleScript; it had more application support than COM Automation had back in the day.
Oh, hi Blakey!
-
@Mason_Wheeler said in Throwing ideas at the wall to see what sticks...:
@Parody said in Throwing ideas at the wall to see what sticks...:
At least Classic MacOS had AppleScript; it had more application support than COM Automation had back in the day.
Oh, hi Blakey!
Thankfully no, but I was a big fan of AppleScript from my time in the desktop publishing world. I ended up mostly duplicating what our application could do in AppleScript on the Windows side using COM. Also got to break out of the object model once or twice in useful ways, which was a big benefit for the Mac folks when my counterpart duplicated what I'd done.
-
@anonymous234 Isn't this the entire point of Google's Protocol Buffers?
-
@anonymous234 said in Throwing ideas at the wall to see what sticks...:
Also
@dkf said in Throwing ideas at the wall to see what sticks...:Because there's no hope in hell of persuading everyone to use a single compatible data model; that's just not going to happen (and everyone who's ever done integration work should know it in their bones).
You say that, but there's many standards that have gotten close to "100% market penetration". TCP/IP, USB, ASCII, Unicode, C, HTTP/HTML, OpenGL...
And many of those have taken decades to do so. USB is probably the most rapidly adopted one there, and that's because it was so much better than what went before for almost all uses. TCP/IP was quite quick too, because of its true killer app (which is… DNS). HTTP has reached about as far as it can: the systems that don't use it now have specific reasons for not doing so (though I suppose it only took around 10-15 years to reach that extent).
-
@powerlord said in Throwing ideas at the wall to see what sticks...:
Isn't this the entire point of Google's Protocol Buffers?
The further up the application stack you go, the harder it is to standardize. The fundamental problem is that people don't actually understand each other, and use the same words to mean (subtly) different things. That in turn means that the way you encode the shared actual reality you're discussing ends up different, and the various bits of software end up unable to talk to each other even though this is fucking idiotic.
This problem is much more acute once you do inter-organization work, especially across borders. Within a single corporate setting, the use of terms can be standardized (often by executive fiat). That doesn't work once you have totally independent groups with totally independent pieces of software. (The endpoint of that is either war or standardization. Or both.)
JSON wins out over Protocol Buffers on the simple grounds that, in a pinch, someone can just eyeball the text.
-
@LaoC said in Throwing ideas at the wall to see what sticks...:
Nowadays Apple basically has two classes of users: the ones you see at techn conferences who use it like a slick Linux that enjoys talking to their iPhones, and the ones who can't see a command prompt without getting a nervous twitch and use it like they've always used their Apple, clicking their way through the officially blessed applications.
Three: those who sit somewhere in the middle between those extremes. I do most of my computing in what you call “officially blessed applications” on my iMac but also always keep a terminal window open for the odd thing I need to do that’s easier that way. Not that I use the terminal every day, but I do often enough that I don’t bother to close that window — but I don’t think this makes me someone who sees macOS as a slick Linux.
-
@dkf said in Throwing ideas at the wall to see what sticks...:
JSON wins out over Protocol Buffers on the simple grounds that, in a pinch, someone can just eyeball the text.
I like RON — though I've never actually used it — but I think they've lost the @Polygeekery market appeal by not naming it RSON.
-
@Zecc I've never seen the point of it. Why does "struct" need to exist as a independent construct from "map" or "array", especially since it can hold the contents of either?
-
@pie_flavor said in Throwing ideas at the wall to see what sticks...:
Why does "struct" need to exist as a independent construct from "map" or "array", especially since it can hold the contents of either?
Because Ruby distinguishes them.
Did you want some other deeper, actually meaningful reason?
-
@dkf what does Ruby have to do with anything?
-
@pie_flavor Ah, I thought it was a similar thing called RON that was Ruby Object Notation.
Anyway, going through the grammar (because the simpler explanations omit a lot of stuff) I see that there's both lists and tuples and named tuples and options and enums and … well, to cut the story short, there's the whole complex type system in there. And there doesn't seem to be a strong canonicalization scheme either, nor is the grammar actually checked against what they really claim (it doesn't mention
Nonefor example). Other flaws are nasty ones like the lack of syntactic distinguishing between some of the classes. Summary: it's a stinking mess written by a hack who doesn't know the first thing about writing specs, worse even than the pile of inconsistent crap that is Markdown.Stick to JSON. It does very little and is great because of it.
-
@dkf You've missed it slightly -
None, as a bare ident, is covered byunit_struct.
But yeah. Clusterfuck.
