The WTFish streaming system
-
So I'm new in my job, beginning my 2nd month here. Someone mentioned there's a need for streaming video content and another department in another building is working on a solution (that will be used by another branch of the institution, and is in no way related to us except the fact we share the same room with another, non-tech departament that focus on producing media-related material). I casually mentioned to my (tech, non-management) boss I have been involved in a very small video streaming project, for 2 months, 2 or 3 years ago. He smiles ironically and says something in the lines of "oh, now you are in charge of it".
My
only previous experience in video streaming consisted in operatin
Windows Media Encoder on a win-xp station and occasional small camera
and editing. However, as this is a serious task in a big organization, I
start considering requirements such as open-source (that's an
institution-wide requirement),dedicated server, running RTMP or RTSP or
other streaming protocols, able of running live or on-demand stream,
multiple concurrent streams on the same server, serving the media from a
different machine than the one recording it, serving different stream
qualities based on client connection speeds... You know, proper stuff.I
started breafly researching for Red5 because I heard about it before.
There is barely to none documentation available and when there is some,
it's scattered all over the web, outdated and/or confliting.Then I get a link of what they have now: it's composed of a single webpage, with a Flowplayer window (using the pre-html5 flowplayer system)
and a list of links that activate videos on that window. There are only
four videos, ranging from 5 to 27 minutes in length. They are served as
FLV-contained Sorenson Spark Video and Shockwave ADPCM audio. With a
resolution of 1280x720p. Mind you, the player windows is 480x360px. The
5min video is 497 MB in size. There is also a "download button". When
clicked, it serves VOB files. That are actually much smaller than the FLV files served by the player -- the same video is 280 MB in this case.Bonus
WTF 1: At least, it's better than the 1997-geocities-looking page they
had before (and it's still operational) with a similar setup, but with
Real Player files instead.Bonus WTF 2: The manager of the
non-tech departmen we share phisycal space is excited about the HTML5
capabilities of the player (there are none) and how it adapts to mobile
3G networks (after failing to adapt to my 1mbps home connection, I doubt it could perform on 3G).Bonus WTF 3: Did I mention my team is the Information Security team?
TL;DR
version: the "streaming experts" are serving on-demand video encoded in
HUGE filesizes and that can be downloaded as DVD-format VOB files.
-
@atipico said:
open-source (that's an
institution-wide requirement)Found the WTF.
@atipico said:
They are served as
FLV-contained Sorenson Spark Video and Shockwave ADPCM audio.Oh, so open source ISN'T a requirement. You liar.
@atipico said:
TL;DR
version: the "streaming experts" are serving on-demand video encoded in
HUGE filesizes and that can be downloaded as DVD-format VOB files.So now the thing is, to avoid getting the standard "nobody will see your TL;DR version if you put it at the BOTTOM" spiel is to boldface it. WTF number 2.
-
@blakeyrat said:
Oh, so open source ISN'T a requirement. You liar.
It is something between a not-really-strictly-enforced-requirement and a somewhat-ignored-recomendation. But there is an open-source policy, not only in all departments of this institution, but in all similar institutions. There is a law requiring all public institutions all over the country to move to open source / free software. (I'm not familiar with the exact terms of this law). There are aa few that will adhere, and even defend the benefits of this law. At the same time, there are lots of proprietary software (99% of the time it means Windows desktop PCs). Either it came preinstalled, or is pirated. The reason I consider the use of free software is because proprietary software will require a written request to some department and that will take about 234532645785796 years to travel the gears of the bureaucratic machine, while open source I can just install and use (on the cases where there is enough documentation and that is accurate and can be followed by someone who doesn't know vim and isn't supposed to be an advanced linux geek who doesn't need trivial things such as documentation).
-
@atipico said:
open source I can just install and use (on the cases where there is enough documentation and that is accurate and can be followed by someone who doesn't know vim and isn't supposed to be an advanced linux geek who doesn't need trivial things such as documentation).
In other words: after tinkering with it for 234532645785796 years.
-
@blakeyrat said:
@atipico said:
Oh look everyone, it's our precious little rebel snowflake.open-source (that's an institution-wide requirement)
Found the WTF.
-
@Zylon said:
I'm picking up some awkward sexual tension here...@blakeyrat said:
@atipico said:
Oh look everyone, it's our precious little rebel snowflake.open-source (that's an institution-wide requirement)
Found the WTF.
-
@Zylon said:
@blakeyrat said:
@atipico said:
Oh look everyone, it's our precious little rebel snowflake.open-source (that's an institution-wide requirement)
Found the WTF.
While Blakeyrat is often flat-out wrong on anything open source, you're doing it wrong if you choose your solutions based on whether they are free or not instead of whether they work or not.
(if you are not working for national defence or thing like that)
-
@atipico said:
@blakeyrat said:
Oh, so open source ISN'T a requirement. You liar.
It is something between a not-really-strictly-enforced-requirement and a somewhat-ignored-recomendation. But there is an open-source policy, not only in all departments of this institution, but in all similar institutions. There is a law requiring all public institutions all over the country to move to open source / free software. (I'm not familiar with the exact terms of this law). There are aa few that will adhere, and even defend the benefits of this law. At the same time, there are lots of proprietary software (99% of the time it means Windows desktop PCs). Either it came preinstalled, or is pirated. The reason I consider the use of free software is because proprietary software will require a written request to some department and that will take about 234532645785796 years to travel the gears of the bureaucratic machine, while open source I can just install and use (on the cases where there is enough documentation and that is accurate and can be followed by someone who doesn't know vim and isn't supposed to be an advanced linux geek who doesn't need trivial things such as documentation).
Are you sure you aren't from my country, because...we have that exact law..oh but you mentioned 3G... nevermind.
-
@blakeyrat said:
@atipico said:
open-source (that's an institution-wide requirement)
Found the WTF.
Go play WOW or something you bore.
-
@serguey123 said:
Are you sure you aren't from my country, because...we have that exact law..oh but you mentioned 3G... nevermind.
Yes, but we have a very special kind of 3G. The ADs for all the 3 or 4 big Telecoms on TV say it's unlimited, but it's the kind of unlimited where after you hit a limit of some Mb / Gb your speed will be limited to dialup-like. But it's unlimited. Of course, you can buy credits and use more, or subscribe for a bigger data plan. But it's unlimited. Orwell would be proud of the doublespeak.
@TheLazyHase said:
you're doing it wrong if you choose your solutions based on whether they are free or not instead of whether they work or not.
Good thing then that I choose the solutions based on requirements. The fact there's a policy requiring (or recommending) open source solutions is one of them, one that I have no control over it. (To be honest, there are no clear specs besides "streaming video", and no written specs or anything, everything is verbal).
-
@atipico said:
I casually mentioned to my (tech, non-management) boss I have been involved in a very small video streaming project, for 2 months, 2 or 3 years ago. He smiles ironically and says something in the lines of "oh, now you are in charge of it".
I'm not sure if the WTF here is you mentioning it, or him taking it as "you're experienced enough to tak ownership of it" and you detailing your limited experience here rather than to to him. Do you consider yourself up to the task?
@TheLazyHase said:
While Blakeyrat is often flat-out wrong on anything open source, you're doing it wrong if you choose your solutions based on whether they are free or not instead of whether they work or not.
I read that being open-source was a requirement. I didn't see anything about being free as a requirement.
-
@atipico said:
(To be honest, there are no clear specs besides "streaming video", and no written specs or anything, everything is verbal).
.. hence how your organisation arrived at the current solutions?
-
Honestly, build ffmpeg with libx264 support, re-encode them as h.264/aac videos, and serve them, as any HTML5-capable browser should support x264. If they don't like that idea, use WebM (vp8+vorbis in a MKV container), which ffmpeg should still support. Serve it up with lighttpd and an off-the-shelf HTML5 player.
Done.
-
@atipico said:
@serguey123 said:
Are you sure you aren't from my country, because...we have that exact law..oh but you mentioned 3G... nevermind.
Yes, but we have a very special kind of 3G. The ADs for all the 3 or 4
big Telecoms on TV say it's unlimited, but it's the kind of unlimited
where after you hit a limit of some Mb / Gb your speed will be limited
to dialup-like. But it's unlimited. Of course, you can buy credits and
use more, or subscribe for a bigger data plan. But it's unlimited.
Orwell would be proud of the doublespeak.France, I suppose ? I recognize the "unlimited" scheme.
Using only open source in administrationat least in France is supposed to be so that a foreign company cannot put backdoor in our system. It kind of make sense for army, law enforcement and maybe research, but it's a tad paranoid to put it in everywhere.
@Cassidy said:
I read that being open-source was a requirement. I didn't see anything about being free as a requirement.
Free as in open, not free as in free beer. The reasoning is the same : people should never use shitty open source software if a vendor have something closed that work, and people should not dismiss open source that work because it's open.
Of course, some people like Blakeyrat want so badly open software to be shitty they alway find a way to make them unworkable anyhow :p
-
@Circuitsoft said:
Honestly, build ffmpeg with libx264 support, re-encode them as h.264/aac videos, and serve them, as any HTML5-capable browser should support x264. If they don't like that idea, use WebM (vp8+vorbis in a MKV container), which ffmpeg should still support. Serve it up with lighttpd and an off-the-shelf HTML5 player.
If they really need only on-demand video, that would work fine (why lighttpd or nginx over apache)? If they need live streaming, we'll have to think of something else.
@Cassidy said:
I read that being open-source was a requirement. I didn't see anything about being free as a requirement.
"Free" in English might mean either free of charge (price) or free as in freedom (license, rights, philosophy, etc). In my language there are two different words and the government policy is aimed toward the 2nd meaning. Add that to the fact that I reffered to "open source" and "free software" as if it was the same thing when it is not. You are all free to mock me as much as you want, go on...
@Cassidy said:
and you detailing your limited experience here rather than to to him
I did mention it to him and he is aware. I'm far from being an expert and I'm not sure there are other people in my (small) team much more experienced on this than me. The hiring process of this institution, and the process by which people get assigned to their departments, and the process by which the deparments (don't) get assigned projecst related / relevant to their area... Damn, it's so crazy it would need another post.
@Cassidy said:
Do you consider yourself up to the task?
No exactly, but I'm new here (5 weeks) and my first non-solo job and I used to work on fixing computers, I'm willing to do anything in order to learn new things. Better than sitting around doing nothing, as I did in my first 2-3 weeks. (And many do for years as this is public service). In the worst case, I'll say "sorry, I can't do it". Or at the very least tell the guys from the other branch how to properly encode a video. (I somewhat fear that would be a politically unwise move to mess with work made by others, as I'm really new here and new to public institutions' politics and have no allies here and do not want to make enemies).
@Cassidy said:
how your organisation arrived at the current solutions?
This institution is so huge and fragmented, and many "solutions" here applied just on a branch or department, and are "things that have been this way for quite a long time, it was this way before I arrived and the guy who knew how to do it has left long ago". And no documentation. At least many things I noticed in this short period since I arrived.
-
@atipico said:
"Free" in English might mean either free of charge (price) or free as in freedom (license, rights, philosophy, etc). In my language there are two different words and the government policy is aimed toward the 2nd meaning.
We will (should?) understand "libre" (this is where "liberty" came from) and "gratis" or similar words. Especially since LibreOffice is becoming popular.
@atipico said:
Yes, but we have a very special kind of 3G. The ADs for all the 3 or 4 big Telecoms on TV say it's unlimited, but it's the kind of unlimited where after you hit a limit of some Mb / Gb your speed will be limited to dialup-like. But it's unlimited.
Around here telcos got into trouble for that, for fixed broadband. The number of GB in was in the fine print, but that is not good enough. But for 3G/4G they tend to advertise the included amount of GB, since most of them start charging between 5c and $20 (!) per MB after going over.
"Unlimited" in your sense means no excess charges, which is not common here on the mobile networks. But the 4G (LTE) networks tend to be much faster than most people's home fixed broadband. Hell even the 2nd best 4G can upload at over 15Mbps and my work connection can only manage ~350kbps.
-
@atipico said:
"Free" in English might mean either free of charge (price) or free as in freedom (license, rights, philosophy, etc). In my language there are two different words and the government policy is aimed toward the 2nd meaning.
Okay, something lost in translation (as TheLazyHase also confirmed).
@TheLazyHase said:
Free as in open, not free as in free beer. The reasoning is the same : people should never use shitty open source software if a vendor have something closed that work, and people should not dismiss open source that work because it's open.
Agreed.
@atipico said:
Add that to the fact that I reffered to "open source" and "free software" as if it was the same thing when it is not. You are all free to mock me as much as you want, go on...
I accused TheLazyHase of the same - but now I see that the UK interpretation of the term differs from other countries.
However, it doesn't stop people in UK conflating the same thing: there are many that believe "open-source" simply means "they don't need to pay for it" and/or "Linux products".
@TheLazyHase said:
Of course, some people like Blakeyrat want so badly open software to be shitty they alway find a way to make them unworkable anyhow :p
Open-source will never be accepted by the closed-minded.
@atipico said:
The hiring process of this institution, and the process by which people get assigned to their departments, and the process by which the deparments (don't) get assigned projecst related / relevant to their area... Damn, it's so crazy it would need another post.
See what you did there? You just volunteered again!
Okay, my bad for presuming a bad decision had been based upon a throwaway comment and now you're stuck in a no-win position you can't extract from. Just to stick the boot in further: you actually sound like you've got quite a strong grasp on the current situation and are proceeding in a promising direction.
Only advice I can give (based upon what appears to be missed from your post but you may have considered it anyway) is stakeholder analysis: determine who has an interest in your project and to what degree. You may find there are some domain experts that you could use; you've certainly identified a few that consider themselves experts but seem to have chosen an expensive method. There could be some individuals involved in authorisation and procurement that you don't yet know of - better to get all this info up front rather than discover a blocker much further down the line.
@atipico said:
@Cassidy said:
how your organisation arrived at the current solutions?
This institution is so huge and fragmented, and many "solutions" here
applied just on a branch or department, and are "things that have been
this way for quite a long time, it was this way before I arrived and the
guy who knew how to do it has left long ago". And no documentation. At
least many things I noticed in this short period since I
arrived.
Sorry, my comment was casting aspersions about how things are currently as a result of unclear and vague specifications coupled with a lack of documentation, rather than a question.I've visited many client sites where they appear to be knowledge-driven solely because information has been passed down by word of mouth and very little documentation exists. The net effect is someone leaving takes that knowledge with them, requiring expensive information rediscovery ("Fred used to do that - nobody knows what he actually did") and people are following processes created well in the past with nobody wanting to investigate how modern methods and technology could facilitate the same outcomes faster, cheaper, or with less effort - a culture of "it works, leave it alone, we'll carry on wasting money doing it the old-fashioned way".
A lot of this behaviour could be summarised as living in denial: for any improvement opportunity to be recognised, there needs to be an acceptance that things are not working in the most effective manner - a driver for change should emerge and projects may be commissioned. Unfortuntely most people don't want to be told that they're "working wrong" or that everything they've been doing for the last few years has been a wasted effort - they'd prefer the ostritch approach to bad news and this ain't helped by senior staff not wanting to "upset the applecart". They're only contributing to the problem, and yet are in the ideal position to address it.
-
-
@serguey123 said:
Are you sure you aren't from my country, because...we have that exact law..oh but you mentioned 3G... nevermind.
Atipico is from Brazil, not Russia. Lots of countries have a law similar to that, and most of them ignore it, just like Brazil.
About 3G, it works here, differently from wired internet. But the government will fix that soon.
-
@atipico said:
@Cassidy said:
and you detailing your limited experience here rather than to to him
I did mention it to him and he is aware. I'm far from being an expert and I'm not sure there are other people in my (small) team much more experienced on this than me.
Don't mind. The comment he made is quite tipical. He just acknowledged that it is you in charge of that project now... Next week it may be somebody else.
@atipico said:
The hiring process of this institution, and the process by which people get assigned to their departments, and the process by which the deparments (don't) get assigned projecst related / relevant to their area... Damn, it's so crazy it would need another post.
Wait, the Brazilian Government hires by test. Everybody do a test, the first ones get hired. Where is the WTF in it?
Ok, the process by wich people and work get assigned to departments is a WTF in most institutions. There are a few exceptions, but most of them assign people by head count (IT needs 3 people, here, take those 3 random new hires) and assign work by politics ("No, my team won't do that!" - "Oh well, my team won't do it either!" - "It's your responsibility!" - "Yeah, but it's you that needs the result.", and etc).
-
@Mcoder said:
Wait, the Brazilian Government hires by test. Everybody do a test, the first ones get hired. Where is the WTF in it?
Relies on the efficiency of the test, and knowledge being finite.
Passing a test simply means you've shown a capability of passing a test. I've interviewed many staff proud of their exams and yet can't answer some fundamental questions (I suspect bootcamps being involved). Contrast that to people that have no exams to their name but years of experience.
@Mcoder said:
and assign work by politics ("No, my team won't do that!" - "Oh well, my team won't do it either!" - "It's your responsibility!" - "Yeah, but it's you that needs the result.", and etc).
That sounds like no clear lines of ownership and accountability. It's quite prevalent in UK public sector, too. Private... no so much.
-
@Cassidy said:
@TheLazyHase said:
Of course, some people like Blakeyrat want so badly open software to be shitty they alway find a way to make them unworkable anyhow :p
Open-source will never be accepted by the closed-minded.I only call-out shitty open source software. I've never called-out software purely for being open source. In fact I'm using an open source web browser right now.
It is true however that I don't believe the open source philosophy and culture leads to good software development. Quite the opposite: the philosophy says "ship early, ship often" regardless of quality level. The culture seems dead-set against any QA process or user testing. Meaning, the only really successful open source projects are those run by corporations that already have established software practices. Hobby open source projects are virtually always crap. This is not likely to change in the near future.
-
@Mcoder said:
Atipico is from Brazil
Ok, but when I hear banana republic, Brazil is not the first country that comes to mind.
@Mcoder said:not Russia
Wait... you think I'm from Russia, comrade?
@Mcoder said:Lots of countries have a law similar to that, and most of them ignore it, just like Brazil.
Well I think in our case it was a retarded cost cutting measure or something but nobody realized about the overhead that all this changing would produce so it ended in half assed implementation which is the standard here.
@Mcoder said:About 3G, it works here
Awesome, maybe one day we'll get any G here (the only one so far is that pesky 6.693 × 10^11)@Cassidy said:
the ostritch approach
You do know that the actual animals don't do that, right? They would have become extinct eons ago if their standard response to danger was as stupid as that.
-
@blakeyrat said:
I only call-out shitty open source software. I've never called-out software purely for being open source. In fact I'm using an open source web browser right now.
Mmm.. I got the impression you called-out shitty software, open-source or not, from some prior posts (Oracle's offerings, MS-SQL installer, et al).
@blakeyrat said:
Hobby open source projects are virtually always crap. This is not likely to change in the near future.
The only change that will result in an improvement is if the hobbyist projects get swallowed up by corporations that bestow some proper organisation upon their development lifecycles - which involves throwing money and resources at it. Unfortuntely, that's had the effect of pissing off the hobbyists who have upsticks and vanished, often meaning that the corporation is left holding a corpse with all the specialised medical staff forming a new Peoples Popular Front of Judea elsewhere.
-
@serguey123 said:
@Cassidy said:
the ostritch approach
You do know that the actual animals don't do that, right?I didn't actually.
However, I guessed that people understood the reference.
-
@Cassidy said:
@blakeyrat said:
I only call-out shitty open source software. I've never called-out software purely for being open source. In fact I'm using an open source web browser right now.
Mmm.. I got the impression you called-out shitty software, open-source or not, from some prior posts (Oracle's offerings, MS-SQL installer, et al).Are "calls-out shitty open source software" and "calls-out shitty software" somehow mutually-exclusive?
Why are the "software developers" on this site so goddamned awful at basic logic? Should I start including a Venn diagram on all my posts? Christ.
-
@atipico said:
(I somewhat fear that would be a politically unwise move to mess with work made by others, as I'm really new here and new to public institutions' politics and have no allies here and do not want to make enemies).
This is probably especially true if you're in a government-related business, which might be the case from you mentioning public service.
I've found it helpful, in this position, to find out who's the actual dev, and contact him, leading off with "hey, I'm new here and I don't want to step on any toes, but if we transcode the video, it'll be a lot smaller, which will result in saving some money by needing less disk, less bandwidth, etc." The key here is the part where you make clear you don't want to upset anyone's rice bowl. It helps if you can find out ahead of time if he knows that there's a desire to improve things, so that the first time he hears about it ISN'T from you.
-
@blakeyrat said:
Are "calls-out shitty open source software" and "calls-out shitty software" somehow mutually-exclusive?
No. But you've railed against closed-source, too. You don't want to give the impression you're biased.. do you?
@blakeyrat said:
Why are the "software developers" on this site so goddamned awful at basic logic?
If they weren't, many front-page articles would be pretty empty.
-
@blakeyrat said:
Are "calls-out shitty open source software" and "calls-out shitty software" somehow mutually-exclusive?
They are when you explicitly qualify one as exclusive.
@blakeyrat said:
I only call-out shitty open source software.
We know what you meant, but what you meant was not what you said.
-
@blakeyrat said:
I don't believe the open source philosophy and culture leads to good software development. Quite the opposite: the philosophy says "ship early, ship often" regardless of quality level. The culture seems dead-set against any QA process or user testing.
I think you might genuinely believe this, and I don't know whether to lol or facepalm.
-
@blakeyrat said:
Should I start including a Venn diagram on all my posts?
If it will help you stop posting your shitty rants before they make it to the general public, by all means - go ahead. It won't help you get your post count to 10,000, or whatever target you appear to have set yourself, however.
-
@blakeyrat said:
@atipico said:
open-source (that's an institution-wide requirement)
Found the WTF.
dude the mindless anti-OSS hate is just as stupid as mindless apple love, mindless microsoft love, mindless apple hate, mindless microsoft hate, etc.
OSS and closed source have their ups and downs. if you want to criticize something be specific and accurate. some OSS is shit. so is some closed source.
-
@Kazan said:
@blakeyrat said:
@atipico said:
open-source (that's an
institution-wide requirement)Found the WTF.
dude the mindless anti-OSS hate is just as stupid as mindless apple love, mindless microsoft love, mindless apple hate, mindless microsoft hate, etc.
OSS and closed source have their ups and downs. if you want to criticize something be specific and accurate. some OSS is shit. so is some closed source.
The open-ness of a piece of software is unrelated to its quality in the same way red cars aren't faster than blue cars.
-
@Ben L. said:
The open-ness of a piece of software is unrelated to its quality in the same way red cars aren't faster than blue cars.
Of course not. That's what the stripes are for.
-
@pkmnfrk said:
@Ben L. said:
The open-ness of a piece of software is unrelated to its quality in the same way red cars aren't faster than blue cars.
Of course not. That's what the stripes are for.
and after market spoilers.
-
@Kittemon said:
@blakeyrat said:
You should lol. Reserve your facepalms for his attempts to use open source software.I don't believe the open source philosophy and culture leads to good software development. Quite the opposite: the philosophy says "ship early, ship often" regardless of quality level. The culture seems dead-set against any QA process or user testing.
I think you might genuinely believe this, and I don't know whether to lol or facepalm.
