Containers for Windows

blakeyrat

There'd have to be a billion of them. Every video channel * every video format * every audio channel * every audio format * every supported resolution. Then of course there's stuff you input to the hardware that's completely separate from its output, like whether its in TV or FM mode, where does that go? Another new file? One that's write-only?

Linux doesn't lock files, so what happens if two applications send contradictory configuration to the hardware simultaneously? One of them just crashes, or? The hardware can only lock on to one video channel at a time, what if one application tries to view channel 5 and another channel 7? Does one just get static, or the digital equivalent? How does the failing application know it's failing and not just getting video that happens to be black frames?

Gribnit

@blakeyrat said:

a billion

Neh.. probably not a billion. The video card is only sending one mode to a given display at a time. There may be some write-only files involved, indeed. As far as file locks go, dunno, I've been disallowed from deleting in use files, which seems suspiciously lockish to me.

Remaining questions I'll refer to the available identities and transforms well established that allow viewing things as trees. There may be some sort of protocol involved or standard, who knows. Writing your PID in "something/hasthiscard" seems pretty decent, tho it did take more than a second to think of.

blakeyrat

It's not just that "treating hardware as virtual files" is a bad idea (although it is), it's that I literally can't see how it could possibly work with hardware like that TV/FM tuner.

And I'll note that years ago when I stupidly tried to build a digital video recorder machine in Linux, it did not work. Every with several Linux experts helping me type in gibberish at the command line. The best we ever got was a postage-stamp-sized image and no audio. Despite the capture card being on the application's "supported hardware" list. So I'm still bitter about that.

@Gribnit said:

Remaining questions I'll refer to the available identities and transforms well established that allow viewing things as trees.

What are you talking about? Flowcharts?

Look, you don't know how/if it works, just admit it. Instead of referring me to "the available identities and transforms well established that allow viewing things as trees". (Do those identities and transforms have a phone number? Twitter handle? Pinterest?)

Gribnit

I think the mistake was in doing it stupidly.

blakeyrat

The mistake was assuming that because the hardware (a Hauppauge DVR-150 or something like that) was listed as "supported" by both the OS and the DVR software, that it would actually work.

It did not.

Nor could anybody I consulted get it to work.

The best excuse I got was "maybe Hauppauge changed the chipset inside it but didn't change the model number", which I called bullshit on then and I'm calling bullshit on now. How about, "maybe the makers of that Linux distro and DVR program fucking lied about what hardware works with it"?

I wasted days on that shit.

Gribnit

Well, could be that the distro and software makers lied, or that you and your friends couldn't manage to fuck a stump together. Not sure..

Gribnit

@blakeyrat said:

Look, you don't know how/if it works, just admit it. Instead of referring me to "the available identities and transforms well established that allow viewing things as trees". (Do those identities and transforms have a phone number? Twitter handle? Pinterest?)

Oh, sorry. No, I don't know exactly how it works. I know that competent people do get it to work.

I was deliberately being a twat about the fact that you have no fucking idea that yes, there are identities and transforms that allow representing many graphs as one or more trees, similar to those that you blindly use to represent trees as arrays and then think you fucking understand shit about CS. Look this shit up on social media.

boomzilla

@blakeyrat said:

it's that I literally can't see how it could possibly work with hardware like that TV/FM tuner.

Yes, but you get confused by an error message on a web page that uses the word, "We," so you'll excuse us if we're not overly impressed by your ignorance.

dse

@blakeyrat said:

... oookay, but how do you tell it what channel you want? Whether you want it in FM or TV mode? Etc.

You need 2-way communication

Good point, and it is where ioctl comes to rescue (this is also the case for Windows btw).
This is how one can get the capabilities, and this is to set a format. File is just an abstraction, behind the file interface even on Windows one could package the DirectX/DirectShow/WPF/COM/(insert latest technology MS wants you learn as your lord and savior who solves all the problems but you have to throw away your past knowledge) and present a simple file interface.

blakeyrat

@Gribnit said:

I was deliberately being a twat about the fact that you have no fucking idea that yes, there are identities and transforms that allow representing many graphs as one or more trees, similar to those that you blindly use to represent trees as arrays and then think you fucking understand shit about CS.

Ok, first of all, I know that data structures exist to represent data as trees, and I know that you can make a tree of trees. I don't care if you believe me or not, because this is besides the point anyway. The real point is:

What the holy fuck does it have to do with what we were talking about?

@boomzilla said:

Yes, but you get confused by an error message on a web page that uses the word, "We," so you'll excuse us if we're not overly impressed by your ignorance.

You seem to think my questions have been answered on this forum before. Why not look up a link to the answers and give it to me? Of course you can't, because you're wrong, but. Hey whatever.

@dse said:

Good point, and it is where ioctl comes to rescue (this is also the case for Windows btw).

Windows uses DirectShow for devices like this. I've written applications making heavy use of DirectShow and I've never heard of anything called "ioctl". (Given, I was writing them in .net...)

But what does "ioctl" do? Hardware in Linux is just files, right? Is it just a helper that reads/writes a correctly-formatted file to the hardware? Or is everybody whose told me over the years that "everything is a file" in Linux a liar?

@dse said:

File is just an abstraction,

Well fucking duh. I'm asking legitimate questions, don't be condescending.

@dse said:

behind the file interface even on Windows one could package the DirectX/DirectShow/WPF/COM/(insert latest technology MS wants you learn as your lord and savior who solves all the problems but you have to throw away your past knowledge) and present a simple file interface.

I highly doubt you could re-implement all of the features of DirectShow (including filters, encoders, muxers, overlays, etc.) and create an "interface" to it that consists of files. Or if you could, there'd be no point since it'd be 50,000 times easier to just use DirectShow's already-existing data structures to do it.

But thank you for giving an actual concrete answer instead of referring me to the available identities and transforms well established that allow viewing things as trees. Those guys are jerks.

boomzilla

@blakeyrat said:

You seem to think my questions have been answered on this forum before. Why not look up a link to the answers and give it to me? Of course you can't, because you're wrong, but. Hey whatever.

I don't see what my expectations about the answers to your questions have to do with what I said. You're acting like your ignorance proves something profound about the inadequacy of something other than yourself.

No, I don't have any knowledge of the subject of video capture hardware. I've never used or even owned one. But I do lack your arrogance to confuse my ignorance with impossibility.

blakeyrat

@boomzilla said:

No, I don't have any knowledge of the subject of video capture hardware.

Ah. Well, then I'll just refer you to the available identities and transforms well established that allow viewing things as trees.

I have it on good authority that they are the experts.

boomzilla

That's a much better response than yelling about how it's all impossible.

Shoreline

@DogsB said:

I'm waiting for someone to burst my bubble though.

I get the impression this might happen if you start trying to use it with something other than Java. Correct me if I'm wrong.

DogsB

@Shoreline said:

I'm waiting for someone to burst my bubble though.

I get the impression this might happen if you start trying to use it with something other than Java. Correct me if I'm wrong.

Consider my bubble... not quite burst but oddly shaped. Mostly because I work in Java and I think its wonderful that I don't have fuck around with stackato anymore.

dse

@blakeyrat said:

But what does "ioctl" do? Hardware in Linux is just files, right? Is it just a helper that reads/writes a correctly-formatted file to the hardware? Or is everybody whose told me over the years that "everything is a file" in Linux a liar?

If possible, a file interface is the simplest (and therfore the best).
You can read Character Devices from here:

Character devices, the simplest of Linux's devices, are accessed as files,
applications use standard system calls to open them, read from them, write
to them and close them exactly as if the device were a file.
This is true even if the device is a modem being used by the PPP daemon
to connect a Linux system onto a network ...

So Character devices are the simplest form of abstraction, but no not everything is file, only when file interface can sufficiently represent the underlying device (or virtual device), it makes sense to use it.

Next comes Block Devices:

Block devices also support being accessed like files.
The mechanisms used to provide the correct set of file operations for
the opened block special file are very much the same as for character
devices. ...

So even block devices are sort of a file, but not quite.

Now the problem is that any abstraction is leaky, like you like everything about frame buffers being files but only if you could talk to the driver to also set the brightness without having to write it to /sys/driver/video/brightness which could be non-atomic and slow (file IO), so there is ioctl which is a syscal to directly call something in your driver.

This abstraction actually comes from UNIX and not Linux btw.

blakeyrat

So how is locking handled for something like a TV Tuner that can only be tuned into one station at once, but two applications-- one of which wants channel 5 and the other wants channel 7?

dse

That is were drivers matter (the one behind the file interface). A dumb driver of course cannot handle that situation, same as a dumb library that cannot handle re-entrant calls, or concurrent resource management (single hardware is a resource that needs a lock somewhere). A proper driver will manage the context per-application, so applications will not know about this single resource. I remember even Windows did not have a per-application speaker volume.
When Linux was not a big target (no Android or Steam) and companies (not those related to networking or server products of course) did not care writing good drivers that was a problem, X became a singletone that managed resource access and Multimedia in Linux sucked.

blakeyrat

??? The quality of the driver isn't a factor here at all. I'm talking about hardware abstraction. I mean, maybe the driver is shit, but we're talking about what the application is supposed to do, which is entirely a different conversation.

How do you write an application in Linux-world that can talk to all video capture cards now and in the future? How do they react if current capture cards can only tune into one channel at a time, and will they "fix themselves" if future ones do not have this limitation?

I don't give a shit about drivers, I'm just talking about the HAL. You know, what DirectShow is in Windows.

TimeBandit

@dse said:

A dumb driver of course cannot handle that situation, same as a dumb library that cannot handle re-entrant calls

And only a dumb user will run two programs to play two different channels from the same tuner card

dse

Application should not care, you get the fd back and assume you are the sole owner of the thing. The driver should keep track of how many open file handles are in use and by whom then synchronize access to the single resource.

blakeyrat

@dse said:

Application should not care, you get the fd back and assume you are the sole owner of the thing.

What if one application's using the TV tuner and another's trying to use the radio? Would those be the same file descriptor? Even though it's a completely different type of data?

dse

While I agree, if the dumb user pays the bills it is my job to make it happen, now it may not be a satisfactory experience but it should not crash or eat my files, can maybe drop frames.

dse

When you call open it returns a new file descriptor, the only file descriptors that are the same across applications are stdin, stderr and stdout.

TimeBandit

@blakeyrat said:

What if one application's using the TV tuner and another's trying to use the radio?

The video output would be the device /dev/video0 while the FM tuner would be another device like /dev/radio0

TimeBandit

@dse said:

it should not crash or eat my files

I think what will happen is something like this :

App A send request to card to switch to channel 5
Card switch to channel 5
App A open the output stream (/dev/video0)
App B send request to card to switch to channel 7
Card switch to channel 7
App B open the output stream (/dev/video0)

Voilà ! Both app are now watching channel 7

Same as if you sit on the sofa, get the remote and switch the TV to channel 5.
Then your girlfriend sit beside you, grab the remote and switch to channel 7.

It's not like you are gonna argue with her on which channel to watch

FrostCat

Cool! I did miss that.

"Then we bundle it with NSIS as a single exe and sign it."

Nice. This is what I'm considering doing to change our app, instead of using a 1990s-era version of Wise.

Hey @apapadimoulis, do you need a different cert for code signing than you use for HTTPS on websites, or are they the same? I'm wondering if I could get the people in charge of our website to help me/let me sign our installer.

powerlord

@Bulb said:

On the other hand Java community never gave a damn, nor did Ruby community and just took on bundling everything.

Makes me wonder what kind of build system the people whose Java apps you've used actually use.

Maven, Gradle, Ivy, and probably every other build system out there require you to put the version number of the dependency in it. Which ends up at the end of the filename.

Here's the dependency list of a test web application I was setting up in Gradle the other day.

dependencies {
	compile 'org.apache.logging.log4j:log4j-core:2.3'
	compile 'org.slf4j:slf4j-api:1.7.12'
	compile 'org.slf4j:slf4j-log4j12:1.7.12'
	compile 'org.springframework:spring-webmvc:4.2.1.RELEASE'
	compile 'org.springframework.security:spring-security-web:4.0.2.RELEASE'
	compile 'org.thymeleaf:thymeleaf-spring4:2.1.4.RELEASE'
	compile 'org.hibernate:hibernate-entitymanager:5.0.1.Final'
	compile 'mysql:mysql-connector-java:5.1.36'
	provided 'javax:javaee-web-api:7.0'
}

That expands out to a list of 30 jar files for the first 8 items all of which end in things like -2.3.jar and -1.7.12.jar.

bugmenot

@TwelveBaud said:

But it's not context-aware...

I have no idea what you mean by "context-aware". I thought I made the limitations and intended use of SO_REUSEPORT pretty clear in my comment.

@TwelveBaud said:

...so it's useless in this context.

That would be why the first words out of my hands were:

@bugmenot said:

Not that this does what APap seemed to want to do, but...

dse

App A should not talk to card, but should open /dev/video0 and use that to ioctl the channel 5.
App B should not talk to card, but should open /dev/video0 and use that to ioctl the channel 7.
Driver should realize these are 2 file descriptors, so when App A reads /dev/video0 should fill it with channel 5 (similarly fill it with channel 7 for App B).

For all that matters a dumb app can open both channels at once, each a different channel, or maybe it wants to overlay one on top of the other (like a little preview), so it is not that dumb after all.

bugmenot

@TwelveBaud said:

...self-healing...

Isn't this just "Reinstall missing or changed files from the copy of the original install file that we squirrelled away on disk for you."? Every Linux package manager does something effectively like that, too. It's kinda a basic feature.

@TwelveBaud said:

It also supports per-user versus per-machine installs...

It's far from the only installer that does that. ;)

@TwelveBaud said:

... InstallCast (dedicated tab in Add/Remove Programs to grab installers from) ... But no, no one uses any of those things...

I have literally never seen anyone use this InstallCast thing. Is it big in The Enterprise or something?

apapadimoulis

@FrostCat said:

Hey @apapadimoulis, do you need a different cert for code signing than you use for HTTPS on websites, or are they the same?

I'm 82% certain you need a special code signing certificate. They were a bit more expensive... and apparently McAffee and other garbage A/V software still claim that it's a malware every now and then.

powerlord

@FrostCat said:

Hey @apapadimoulis, do you need a different cert for code signing than you use for HTTPS on websites, or are they the same? I'm wondering if I could get the people in charge of our website to help me/let me sign our installer.

Oh, I herpa derped and missed this question.

Sadly, code signing certs and HTTPs certs are different. Especially if you have Extended Validation certs as I don't think there are EV certs for code signing.

Having said that, iirc Code Signing certs tend to be a lot cheaper...

FrostCat

@apapadimoulis said:

I'm 82% certain you need a special code signing certificate.

Ok. Probably not worth bothering with, then.

FrostCat

@powerlord said:

Having said that, iirc Code Signing certs tend to be a lot cheaper...

With that, I did a bit of checking. Thawte wants $300/year for code signing or EV SSL; regular SSL is $200. There's multi-year discounts, and I didn't look at anyone else.

dcon

@apapadimoulis said:

commercial tools (InstallShield, etc) weren't designed for non-interactive use

But they come with tools for doing automated builds. We only used the GUI for development. The build was completely automated.

@apapadimoulis said:

whereas now anyone can maintain it

When dealing with MSI, this is a scary statement (I realize you're speaking about your non-msi installer here). Because (as we agreed earlier) there is a high learning curve for understanding the underlying MSI technology. And without understanding that - well, it's like watching a java-person try to use C-pointers correctly.

Tsaukpaetra

@bugmenot said:

I have literally never seen anyone use this InstallCast thing. Is it big in The Enterprise or something?

I've been trying to implement this for ages, but I don't have the ability to build MSI files that launch setup installers for other programs. I mean, should be easy (in theory), but... problems.

Nice thing is that all my users (if they ever poke around Control Panel) will be able to install TestApp1, TestApp2, etc. without my intervention! Or even, all computers that join the domain automatically get calc.exe installed on their computers (if they're Windows 10)!

blakeyrat

@dse said:

Driver should realize these are 2 file descriptors, so when App A reads /dev/video0 should fill it with channel 5 (similarly fill it with channel 7 for App B).

Right; but if the tuner can only do one channel at a time, how does it communicate to the application that either:

the app isn't seeing the channel it asked for, or
it has no video available because someone else is using the tuner right now?

What's the mechanism for that communication?

Tsaukpaetra

E_ACCESS_DENIED?
E_IO_ERROR?

Filed under: Or perhaps even E_NOT_A_NUMBER

PleegWat

@blakeyrat said:

if the tuner can only do one channel at a time, how does it communicate to the application

The ioctl() to switch channels will fail with an appropriate error code. There's a standardized list, but I don't have it in front of me, so I can't tell you which codes would be likely.

blakeyrat

Without file locking, how does "ioctl" know the first application isn't done with the tuner yet?

PleegWat

It still has the file descriptor open.

Gribnit

@blakeyrat said:

Well fucking duh. ..., don't be condescending.

@blakeyrat said:

The quality of the driver isn't a factor here at all. I'm talking about hardware abstraction.

You are a ~~national~~NATO treasure.

TwelveBaud

It doesn't. Which is why you lock the (fake) file.

ioctl (or, if you insist, DeviceIoControl) is just like read or write or stat, only it's for "other" things that don't fit nicely into those other paradigms. But it works on "files" just as well as read or write does, and it follows all the same rules.

dse

@blakeyrat said:

1) the app isn't seeing the channel it asked for, or

ioctl should fail with an error code

@blakeyrat said:

2) it has no video available because someone else is using the tuner right now?

It is Ok if some one else is using it, driver should provide a transparent way to multiplex between users (with potential quality drop, but with clever zero copy it actually is most of the time Ok). But if there is absolutely no way, it should throw an file access error when opening /dev/video0

blakeyrat

@TwelveBaud said:

It doesn't. Which is why you lock the (fake) file.

Another thing Linux fans always tell me is it doesn't have file locking. That's a lie also?

powerlord

@blakeyrat said:

@TwelveBaud said:
It doesn't. Which is why you lock the (fake) file.

Another thing Linux fans always tell me is it doesn't have file locking. That's a lie also?

Did they tell you that or did you misinterpret when they said that Linux doesn't automatically lock open files?

You have to explicitly ask for file locks in Linux using the flock (easy to use) or fcntl (more difficult to use, but more powerful) system calls.

Salamander

"flock() places advisory locks only; given suitable permissions on a file, a process is free to ignore the use of flock() and perform I/O on the file."

http://linux.die.net/man/2/flock

"The implementation of mandatory locking in all known versions of Linux is subject to race conditions which render it unreliable: a write(2) call that overlaps with a lock may modify data after the mandatory lock is acquired; a read(2) call that overlaps with a lock may detect changes to data"

http://linux.die.net/man/2/fcntl

Looks pretty broken to me.

dse

Yes, there is no mandatory file lock in Linux that I am aware of. How is it done in Windows? Does that mean I can lock a file that cannot be deleted even by root? I guess that is something Malwares would like to use.

For advisory lock I use flock(fileno(pflock), LOCK_EX | LOCK_NB) == 0 in Linux and named mutex in Windows OpenMutex(MUTEX_ALL_ACCESS, FALSE, lpName).

wft

@blakeyrat said:

... oookay, but how do you tell it what channel you want? Whether you want it in FM or TV mode? Etc.

Ever heard of an ioctl(), dickwad?