It's only 9 more days
-
I asked a colleague (who has permissions to and is responsible for doing certain data extractions and transformations) to pull a certain version of our transformation rules and apply them to the data. The job needs to scan our entire database (currently about 20T records scattered across about 100 tables) and takes about 9 days to run at low priority in the background.
After a week, he finally kicks off the process. After 16 days, he finally announces that the results are available - for the wrong version of the rules.
I reply, informing him of his incorrect choice of rules. He says: no problem, I'll just run it again. Erm, you realize this kills my ability to deliver on time and I'm going to need to let people know. He replies that it should be: No problem, it's only 9 more days.
Sadly, when I informed our customer service folks that the response would be delayed by nearly two weeks, they didn't care.
More time to get paid to surf I guess.
-
@snoofle said:
Sadly, when I informed our customer service folks that the response would be delayed by nearly two weeks, they didn't care.
Can I have your customers when you're done with them? Mine are much more grumpy about that kind of thing.
-
I'm building a product for customers who haven't even signed the SOW yet. If that helps you feel better.
-
@blakeyrat said:
I'm building a product for customers who haven't even signed the SOW yet. If that helps you feel better.
In two months, when you're finished, and they're finally ready to sign the SOW, you'll do a demo and they'll say "wait, that's not what we wanted" (even tho it meets the spec) and will drop the contract talks.
-
@blakeyrat said:
I'm building a product for customers who haven't even signed the SOW yet. If that helps you feel better.
Specs? I've heard of those before...
-
@zelmak said:
@blakeyrat said:
I'm building a product for customers who haven't even signed the SOW yet. If that helps you feel better.
In two months, when you're finished, and they're finally ready to sign the SOW, you'll do a demo and they'll say "wait, that's not what we wanted" (even tho it meets the spec) and will drop the contract talks.
I just learned one of our customers thought we were going to do customer support for their customers using the network we built for them. Uh, no. We just build and monitor the network.
Finding one of the Business Analysts to actually Show Us The Contract is ... painful. I think because there's not one, but I try not to be quite that pessimistic.
-
Ok, perhaps this is water in the desert, or pearls for swines, who knows, but ... did you tell your colleague that he should do trial runs for such jobs first? And check the results? On the first 1M records?
Or, better: apply the rules to a 1M randomly selected records. That will give you all the data you need. Ok, it won't be accurate at the 0.0000001% level, but 20T (by the way, shouldn't that be 20G? 20T is really a lot) is not going to give you that much more information.
-
@TGV said:
(by the way, shouldn't that be 20G? 20T is really a lot)
20B, not 20G, but I think you are right. 20T would mean an average table size of 200B and from what Snoofie's posted before, he's only got a few tables over 1B.
-
@morbiuswilters said:
@TGV said:
(by the way, shouldn't that be 20G? 20T is really a lot)
20B, not 20G, but I think you are right. 20T would mean an average table size of 200B and from what Snoofie's posted before, he's only got a few tables over 1B.
B? As in billion rather than giga-?
If it takes 9 days, I'd find 2E13 more plausible than 2E10.
-
@pjt33 said:
B? As in billion rather than giga-?
Yes. You don't say "I have 20 giga-rows" because you'd sound like an asshole.
@pjt33 said:
If it takes 9 days, I'd find 2E13 more plausible than 2E10.
Depends on what's he's doing. For 20B rows, that's 0.00003888s for each row. That doesn't seem unreasonable at all.
-
@snoofle said:
More time to get paid to surf I guess.
I was sourced out for six months in the late nineties, at a company that apparently didn't have much work for me. So that was indeed mostly six months of surfing. As the six months came to an end, all of a sudden there was a frenzy because they wanted to keep me there (they were very happy with my surfing prowess, or so it seems), but sadly they didn't manage to find the budget.That company was, via other companies, owned by Worldcomm.
-
@morbiuswilters said:
@pjt33 said:
B? As in billion rather than giga-?
Yes. You don't say "I have 20 giga-rows" because you'd sound like an asshole.
You must be from one of 'em non-SI countries. So according to your logic, 20T rows means 20 thousand?
@morbiuswilters said:
@pjt33 said:
If it takes 9 days, I'd find 2E13 more plausible than 2E10.
Depends on what's he's doing. For 20B rows, that's 0.00003888s for each row. That doesn't seem unreasonable at all.
0.00000003888s (400ns) on the other hand, that would be one sweet database server. We could use that instead of RAM...
-
@TGV said:
You must be from one of 'em non-SI countries.
Yes, but that's irrelevant. Saying "twenty giga-rows" makes you sound retarded, no matter where you are from.
@TGV said:
So according to your logic, 20T rows means 20 thousand?
Uh, no.. 20T would be 20 trillion. 20 thousand would be 20K, to resolve the ambiguity (in some cases it might also be 20G ("twenty grand") or 20M (used in finance)). This isn't "my logic", this is the common way you shorten quantities in English.
@TGV said:
0.00000003888s (400ns) on the other hand, that would be one sweet database server. We could use that instead of RAM...
No.
(9 * 86400) / (2 * 10^10) = 0.00003888 [38.88us]
-
20T is 2E13, right? So ... 39ns?
By the way, your µ lacks a leg.
-
@TGV said:
20T is 2E13, right? So ... 39ns?
Reread the thread. pjt33 said if it takes 9 days 2E13 is more plausible. I said that 20B (2E10) is 39us per-row, which is very plausible (and 39ns is not-so-plausible).
@TGV said:
By the way, your µ lacks a leg.
No, my keyboard lacks a mu key. So I'm using "u", which is perfectly understandable and valid for cases where the Latin alphabet is the only thing available.
-
@morbiuswilters said:
Reread the thread. pjt33 said if it takes 9 days 2E13 is more plausible. I said that 20B (2E10) is 39us per-row, which is very plausible (and 39ns is not-so-plausible).
Recalibrate your context parser: I was refering to the 20T scenario. 39ns per row (I just copied your number and added three zeroes) would be one sweet db server.
@morbiuswilters said:
No, my keyboard lacks a mu key. So I'm using "u", which is perfectly understandable and valid for cases where the Latin alphabet is the only thing available.
Strange. G was perfectly understandable, even correct, yet you changed it. Mu? Can't be bothered. Is this an example of dickweedery?
@morbiuswilters said:
Can we be friends?
No, probably not.
-
@TGV said:
G was perfectly understandable, even correct, yet you changed it.
It's ambiguous, at best; incorrect, at worst. If you are working with units where SI prefixes are used, G would mean giga. Number of database rows is not a unit where SI prefixes are used. Do you really need me to explain such elementary things? Or are you being dense on purpose because you think it makes you look smart?
@TGV said:
Mu? Can't be bothered.
As I said, "u" is unambiguous and acceptable in cases where the Greek alphabet isn't available.
@TGV said:
@morbiuswilters said:
Can we be friends?
No, probably not.
Hmm.. I don't remember asking to be friends, but there it is, quoted and everything.. I guess my memory is faulty.
-
@morbiuswilters said:
No, my keyboard lacks a mu key.
Get a Mac: µ (Option-m) even on US-English keyboard.
Or Alt-230 on Windows - I remember doing a lot of capacitor values years ago
-
@Zemm said:
Get a Mac
No.
@Zemm said:
(Option-m) even on US-English keyboard.
Or Alt-230 on Windows - I remember doing a lot of capacitor values years ago
Neither of those are keys, they're key combinations. Screw that.
-
@morbiuswilters said:
Neither of those are keys, they're key combinations. Screw that.
So is "N" and "S" in this quote.
-
@Zemm said:
@morbiuswilters said:
Neither of those are keys, they're key combinations. Screw that.
So is "N" and "S" in this quote.
Not necessrily...mixed case can be done completely with key sequences. There is never a need to hold two (or more) keys at the same time.
-
@TheCPUWizard said:
Not necessrily...mixed case can be done completely with key sequences. There is never a need to hold two (or more) keys at the same time.
So safes are opened with 'sequences' and never 'combinations?' Methinks you are being unnecessarily pedantic here...
-
@PJH said:
@TheCPUWizard said:
Not necessrily...mixed case can be done completely with key sequences. There is never a need to hold two (or more) keys at the same time.
So safes are opened with 'sequences' and never 'combinations?' Methinks you are being unnecessarily pedantic here...True, but it is such fun! Cor example a rotary "combination" lock isopened by a sequence, bt the multi-digit ones (such as on many briefcases) are truely opened by a combination.
<ducking but not running....>
-
