The Official Status Thread
-
@dkf said in The Official Status Thread:
@Gąska said in The Official Status Thread:
I'm way more bothered by the wrong order of events than I should of.
I ran the test code several times while hunting the issue, and it was one of these fun issues that had the actual locus of the problem quite a way from where the crash occurred. With many runs through the code that crashed (without crashes) beforehand, when it was processing some other data that was legal. The combination makes figuring things out with a debugger miserable, but I lucked out when I started wondering why the file offsets were crazy when they should have been zero. Yes, the crazy values were copies of uninitialised data from elsewhere…
But it “wasn't very effective” as I found (and fixed) the root cause quickly.
Is the direct memory management solely for all of our amusement or is there a strong performance driver?
-
-
@topspin said in The Official Status Thread:
@Gąska said in The Official Status Thread:
than I should of.
At least you get less have the complement.
-
system: bitch! What should I make for breakfast?!?
Bitch: fuck you I'm sleeping...
Bitch 2: you know what I want...
-
@dkf said in The Official Status Thread:
Status: A wild SEGV appears! It uses Uninitialised Memory. It is not very effective.
My wild SEGV last week exhibited signs of heap corruption. One-off, prod, coredumps VERBOTEN. Put it on "call me back if it happens again".
-
@HardwareGeek said in The Official Status Thread:
@Tsaukpaetra I must say the novel ways in which things break for you never cease to amaze.
@error_bot xkcd server problem
-
-
@error said in The Official Status Thread:
server problem
Speaking of, I'm getting quite annoyed at how my IPv6 address keeps changing, despite my IPv4 address totally not for the past several years.
It's not like I'm swapping out MAC addresses! I just randomly get a subtly-different prefix for no apparent reason!
I should look into having the router be more insistent on what prefix it wants....
-
@Tsaukpaetra said in The Official Status Thread:
@error said in The Official Status Thread:
server problem
Speaking of, I'm getting quite annoyed at how my IPv6 address keeps changing, despite my IPv4 address totally not for the past several years.
It's not like I'm swapping out MAC addresses! I just randomly get a subtly-different prefix for no apparent reason!
I should look into having the router be more insistent on what prefix it wants....
ed. he was never found
-
@Carnage said in The Official Status Thread:
@dkf said in The Official Status Thread:
@sloosecannon said in The Official Status Thread:
How does one.. grab one gummy instead of two? Like... That does not seem like a normal failure mode
Would grabbing three instead of two seem more likely?
Or just have one escape unseen in the mental mists of morning to live out it's days in the land of lost socks while you retrieve two for consumption.
That I could see
-
Status: Today I learned that SQL 2000's BULK INSERT is stupid about the ROWTERMINATOR parameter.
BULK INSERT dbo.PayPalExport2020 FROM 'Download2015.CSV' WITH (CODEPAGE='ACP', DATAFILETYPE='CHAR', FIELDTERMINATOR=',', ROWTERMINATOR='\n', FIRSTROW=2, KEEPNULLS, FIRE_TRIGGERS);
PayPal exports their transactions separated by commas or tabs and UNIX line feeds (LF). SQL Server cannot read this file. If I change the line ending to Windows (CRLF), that same statement works even though it shouldn't. If I change the statement to specify \r\n, which is what's actually in the file, it doesn't. SQL can also read the Macintosh (CR) format if I specify a \r in the statement.
-
@Gribnit said in The Official Status Thread:
Is the direct memory management solely for all of our amusement or is there a strong performance driver?
The library this is in doesn't use garbage collection because GC has a nasty habit of being an all-or-nothing proposition. (I'm going through the code module this came from with a fine-toothed comb, trying to find places where it doesn't handle memory correctly. It's… not written in a style that makes this easy.)
-
@Zenith said in The Official Status Thread:
Today I learned that SQL 2000's BULK INSERT is stupid about the ROWTERMINATOR parameter.
Microsoft + CSV =
-
Left my DeLorian at home.
-
@dkf I'll take some blame for using SQL 2000. The last version I did imported/exported files on was 2012 or 2016. I used to have a script the modify the records by passing through a view when I was doing code review for fuckups. PayPal's CSVs have a WTF in that they don't have an ordering field despite representing financial transactions applied in a specific order so I need to fix that too.
-
@Zenith said in The Official Status Thread:
PayPal's CSVs have a WTF in that they don't have an ordering field despite representing financial transactions applied in a specific order so I need to fix that too.
They're not independent records then, I get that (formally, they're a sequence, not a set), but surely you can put them into a table with an autoincrement column (with that column not specified as being used in the import) and then sort by that by default?
-
@dkf said in The Official Status Thread:
@Zenith said in The Official Status Thread:
PayPal's CSVs have a WTF in that they don't have an ordering field despite representing financial transactions applied in a specific order so I need to fix that too.
They're not independent records then, I get that (formally, they're a sequence, not a set), but surely you can put them into a table with an autoincrement column (with that column not specified as being used in the import) and then sort by that by default?
Assuming I was always importing every year in order and only once, that might work.
This is such a pain in the ass. Turns out, SQL 2000 can't handle CSV quoting. So every place PayPal had a transaction in yen, with a comma, it messed up the fields. The shortest way around that was to use the TSV and hope nobody did anything stupid with tabs. In either case, it won't escape the quotes so every field has its values quoted.
Also I was trying to use a trigger to possibly generate my own key on inserts. Nope, triggers fire per statement. Once those records land in a table, every select appears to jumble the order. The only place it seems like the order is maintained is inside the trigger. That means I'm writing logic like this:
DECLARE TABLE @T (n0, n1, n2, nn); DECLARE d CURSOR FOR inserted; WHILE LOOP() BEGIN FETCHNEXT @n1, n2, nn; INSERT INTO @T some_function_based_on_calculated_uniqueness(), @n1, @n2, @nn); END
-
@Zenith said in The Official Status Thread:
SQL 2000
-
@HardwareGeek said in The Official Status Thread:
Status: Happy. I just spent $35.50 on chocolate.
I have discovered that the world's best chocolate (that I have personally experienced, anyway) is certified gluten-free, so I can eat it. I just need to get a second and third job to afford it. $8.50 / 50g for their regular bars. $9.50 / 50g for their single-origin cru varieties. $18.50 / 50g for their 70% cocoa, 100% Criollo bars.
Status: Even happier. I have chocolate. Currently savoring an intense but still somewhat sweet 70%.
-
status: tracing unknown bacterial infection that has been slowly becoming more noticeable over the past few days. Why my immune system is not taking care of this I'm not sure...
-
This post is deleted!
-
@Tsaukpaetra said in The Official Status Thread:
Why my immune system is not taking care of this I'm not sure...
It, too, has 55.5555555% reliability?
-
@loopback0 said in The Official Status Thread:
@Zenith said in The Official Status Thread:
SQL 2000
Can you confirm it works any better with 2012 or 2016? I didn't want to have to write a custom importer. Not yet anyway. I only need these now because PayPal shortened the window for transaction visible online without exporting and some stuff I have to verify fell out of that window.
-
Status: Being stupid and trying to reduce database calls in a specific instance but realizing that just using the two (simple) database calls is a lot more understandable and readable than one (supremely complicated re-joining-same-table) database calls.
-
Did... did @Tsaukpaetra just do something that's the opposite of a ?!
-
@Zerosquare said in The Official Status Thread:
Did... did @Tsaukpaetra just do something that's the opposite of a ?!
Something something stopped clock something something.
-
STATUS: sigh
RIP friend
-
@Zenith Today's episode of I Guess SQL 2000 Really Is
I'm trying to build a procedure to run bulk inserts. I know I've done this before but I can't be bothered to find what hard drive the script is sitting on. Anyway I'm playing a game of string replace and it's crapping out around 300 characters.
DECLARE @SQL AS VARCHAR(4096); SET @SQL = 'BULK INSERT ''@V'' FROM ''@L'' WITH (CODEPAGE=''ACP'',DATAFILETYPE=''CHAR'',FIRSTROW=@S,KEEPNULLS,FIRE_TRIGGERS,FORMATFILE=''@F'');'; SET @SQL = REPLACE(@SQL, '@V', 'PayPalExport' + RIGHT(REPLICATE('0',8) + CONVERT(VARCHAR(4),ISNULL(@Year,0)) + '2020', 8)); SET @SQL = REPLACE(@SQL, '@L', @Location); SET @SQL = REPLACE(@SQL, '@S', CONVERT(varchar(3),@star)); SET @SQL = REPLACE(@SQL, '@F', @Format); SELECT @SQL;
Of course the documentation for 2000 is PDF only with no table of contents. The reason I wanted to look at that was because I remember newer versions of SQL showing built in functions in the object tree and sometimes the parameter lengths were different than I expected. Although I'd expect this to truncate at something like 128 characters or some other multiple of 2.
Edit: Even without the REPLACE, just plain old concatenation, it's still stopping at a little over 300 characters...
Edit More: Yes, somehow,
SET @SQL = REPLICATE('0', 500) + REPLICATE('0', 500);
returns 1000 just fine...
-
@Zenith Is
@Location
around 198 characters long? If so, maybe@S
needs quoted.
-
Status: Speaking of SQL. Having been tasked with adding some reporting functionality to the stored-value-cards thing, I'm looking at what's in play at the moment.
Seems a NodeJS thing using Sequelize? So I take a look at documentation... Like, say, how to do Where clauses:
That's right, we turned a three-line SQL clause into some dozens of bracket hell. Useful!
-
@Tsaukpaetra speaking of SQL, I'm wishing our database wasn't so normalized. I mean, it's absolutely a good thing. Except when you want to filter by a parameter that's 6 tables away (but is rather critical to this thing working at all). Easy joins (each one has an id for the one next in the chain), and better than 6 extra queries, but...
-
status it's time for a new mouse. I seem to go through them way too fast. The buttons stop reliably clicking and dragging and start to stutter. Of course this one was a cheap walmart store brand one I bought 2-ish years ago and I play MMOs with lots of click and hold for mouse look and strafing, so...
-
@Benjamin-Hall I buy not-cheap ergonomic mice, and the switches and encoders seem to wear out just as fast. The one I'm using now, the buttons are working fine, but the wheel is kaput. You don't realize how much you use it until it doesn't work.
-
-
@Benjamin-Hall said in The Official Status Thread:
status it's time for a new mouse. I seem to go through them way too fast. The buttons stop reliably clicking and dragging and start to stutter. Of course this one was a cheap walmart store brand one I bought 2-ish years ago and I play MMOs with lots of click and hold for mouse look and strafing, so...
My Logitech MX518 dates from 2006. Had to put new feet on it last year (and I cleaned up the insides while I had the chance) but it was and still is working fine otherwise. They put out a new version in recent years that I hope is as good but probably isn't. :(
-
@Benjamin-Hall said in The Official Status Thread:
Easy joins (each one has an id for the one next in the chain), and better than 6 extra queries, but...
I like to put the obvious
JOIN
s in asVIEW
s so that I can write the query simply. (I don't bother setting up triggers; it's just for queries.)
-
@Benjamin-Hall said in The Official Status Thread:
I play MMOs with lots of click and hold for mouse look and strafing, so...
My life improved once I mapped one of my mouse's side buttons to left-drag.
My life further improved once I mapped the other one to Window's Task View.
This of course was only possible because I wasn't otherwise using them.X-Mouse Button Control (suggested by someone in this forum) is nice for having per-application settings.
-
@HardwareGeek said in The Official Status Thread:
The one I'm using now, the buttons are working fine, but the wheel is kaput. You don't realize how much you use it until it doesn't work.
You're @HardwareGeek. Replacing the encoder is a 5-minute job with a soldering iron.
-
@TwelveBaud I also found that the following statement doesn't work.
REPLACE('somethingsomething', '@darkside', NULL)
to check, but I know many string functions in C# treat NULL as a blank string in such operations. Not SQL though! Not even when the operation doesn't happen. Nope, returns NULL.
Status: In other news, US Bank is stupid. Last year my part time job forced everybody into direct deposit and I took the path of least resistance signing up for their debit card. It e-mails (and I think texts) me every week with deposits and every few weeks I log in to their site to download pay stubs and the like. Google has real trouble with their log in page, often deciding to run out of memory even though I'm not actually out of memory.
Well today was dumber than usual, because US Bank decided to make me reset my user ID. Because reasons (I've only had this set up for 6 months so who knows why). Then I had to provide a phone number. Then I went to look up and make sure my alerts were still set ("your settings are right where you left them" and all). It says my phone number was 111-111-1111. Uh oh, something didn't do the needful!
So I'm "downloading" this stuff by printing to PDF. I don't know what's wrong with them or Chrome. I've downloaded three runs of statements. First clocked in anywhere between 164KB and 167KB. Second all clocked in exactly 53KB. These are all 159KB. The contents aren't wildly different, maybe 2-3 characters between them. It's a basic pay stub and I've been working the same hours all along. The weird part is, I downloaded this batch twice and got different file sizes both times...
-
@Zenith said in The Official Status Thread:
Not SQL though!
That depends. If you were using Oracle then it would indeed work the way you think. The rest of us plebeians need to COALESCE…
-
While on the subject of SQL, is this a stupid thing to do in
an AFTER INSERT triggerthe sproc which inserts PingLog entries, or not really?WHILE (SELECT COUNT(*) FROM [PingLog] WHERE [ClientID] = @ClientID) > 10000 BEGIN BEGIN TRANSACTION; WITH D AS ( SELECT TOP 5000 ROW_NUMBER() OVER (ORDER BY [PingDate] DESC) AS [rn] FROM [PingLog] WHERE CLIENTID = @ClientID ) DELETE FROM D WHERE [rn] > 10000; COMMIT TRANSACTION; END;
Just need to make sure there's an index on ClientID and PingDate first.
-
@Zecc said in The Official Status Thread:
Just need to make sure there's an index on ClientID and PingDate first.
There wasn't. I'm not surprised. Some of the tables my predecessors created didn't even have primary keys..
-
@Zecc The answer is "Yes, it's stupid. SELECT TOP 5000 and ROW_NUMBER() > 10000 don't go well together"
-
@Zecc I saw that and wondered if I'd just misunderstood the SQL
-
Let's try this again. Is this a stupid thing to do? Could I do better?
WHILE (SELECT COUNT(*) FROM [PingLog] WHERE [ClientID] = @ClientID) > 10000 BEGIN BEGIN TRANSACTION; WITH D AS ( SELECT *, ROW_NUMBER() OVER (ORDER BY [PingDate] DESC) AS [rn] FROM [PingLog] WHERE CLIENTID = @ClientID ) DELETE TOP(5000) FROM D WHERE [rn] > 10000; COMMIT TRANSACTION; END;
(the TOP 5000 in the DELETE is just to avoid large transactions)
-
@HardwareGeek said in The Official Status Thread:
@Benjamin-Hall I buy not-cheap ergonomic mice, and the switches and encoders seem to wear out just as fast. The one I'm using now, the buttons are working fine, but the wheel is kaput. You don't realize how much you use it until it doesn't work.
My wireless MS ergo mouse is starting to die. When I click on something it doesn't always get focus. Of course, that could just be Ubuntu also...
-
@Tsaukpaetra said in The Official Status Thread:
status: tracing unknown bacterial infection that has been slowly becoming more noticeable over the past few days. Why my immune system is not taking care of this I'm not sure...
It's because you died. That's called rot.
-
In case someone's losing sleep over what I've ended up doing , it was this:
IF (SELECT COUNT(*) FROM [PingLog] WHERE ClientID = @ClientID) > @RemainingCount BEGIN DECLARE curs CURSOR FORWARD_ONLY FOR SELECT * FROM [PingLog] WHERE ClientID = @ClientID ORDER BY [PingDate] DESC, [Id] DESC OFFSET @RemainingCount ROWS; OPEN curs; BEGIN TRY FETCH NEXT FROM curs; WHILE @@FETCH_STATUS = 0 BEGIN DELETE FROM [PingLog] WHERE CURRENT OF curs; FETCH NEXT FROM curs; END END TRY BEGIN CATCH END CATCH; CLOSE curs; DEALLOCATE curs; END;
which is basically amounts to something like
WITH D AS ( SELECT * FROM [PingLog] WHERE ClientID = @ClientID ORDER BY [PingDate] DESC, [Id] DESC OFFSET @RemainingCount ROWS ) DELETE D;
but with a cursor added to the mix.
-
@Zecc The joys of retention code.
In theory it is all the same and I love unifying stuff but somehow retention always ends up being a big pile of special cases.
I try to do it with partition drops whenever possible. But that's oracle experience, I have zero knowledge about mssql apart from overlapping theory.
-
Another alternative. I think I'll keep this one.
ALTER PROCEDURE [dbo].[sp_trim_PingLog] @ClientID INT, @RemainingCount INT = 10000 AS BEGIN SET NOCOUNT ON; DECLARE @deleted INT = 1; WHILE @deleted > 0 BEGIN WITH D AS ( SELECT * FROM [PingLog] WHERE ClientID = @ClientID ORDER BY [PingDate] DESC, [Id] DESC OFFSET @RemainingCount ROWS ) DELETE TOP(5000) D; SET @deleted = @@ROWCOUNT; END; END;
Honestly, a single DELETE would have been enough if not for me wanting to keep the transaction log from exploding. Which won't happen again in the future after I'm done setting up this sproc called periodically.