@ben_lubar tries to parse <del>ServerCooties</del> data
-
@ben_lubar said in Refreshrefreshrefreshrefresh:
@accalia said in Refreshrefreshrefreshrefresh:
@sloosecannon said in Refreshrefreshrefreshrefresh:
Paging @accalia - what does servercooties say about the update at 11:09:22 EST today?
https://servercooties.io/static/servercooties.sql.zip
have fun.
ben@australium:~$ mkdir servercooties ben@australium:~$ cd servercooties/ ben@australium:~/servercooties$ ls ben@australium:~/servercooties$ wget https://servercooties.io/static/servercooties.sql.zip --2016-07-01 21:13:18-- https://servercooties.io/static/servercooties.sql.zip Resolving servercooties.io (servercooties.io)... 45.55.146.244, 2604:a880:800:10::76a:f001 Connecting to servercooties.io (servercooties.io)|45.55.146.244|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 69915409 (67M) [application/zip] Saving to: ‘servercooties.sql.zip’ servercooties.sql.z 100%[===================>] 66.68M 1018KB/s in 53s 2016-07-01 21:14:11 (1.26 MB/s) - ‘servercooties.sql.zip’ saved [69915409/69915409] ben@australium:~/servercooties$ unzip servercooties.sql.zip Archive: servercooties.sql.zip inflating: var/www/SockSite/servercooties.sql ben@australium:~/servercooties$ sqlite3 var/www/SockSite/servercooties.sql SQLite version 3.11.0 2016-02-15 17:29:24 Enter ".help" for usage hints. sqlite> .tables checks2 pages sqlite> select count(*) from checks2; Error: database disk image is malformedWhen you say "fun", do you mean
?Interesting idea from a StackOverflow user: http://stackoverflow.com/a/15874884/2664560
ben@australium:~/servercooties$ cat <( sqlite3 var/www/SockSite/servercooties.sql .dump | grep "^ROLLBACK" -v ) <( echo "COMMIT;" ) | time sqlite3 fixed_servercooties.db Error: near line 7059611: cannot commit - no transaction is active Command exited with non-zero status 1 93.82user 1.00system 1:35.82elapsed 98%CPU (0avgtext+0avgdata 5764maxresident)k 0inputs+314712outputs (0major+806minor)pagefaults 0swaps ben@australium:~/servercooties$ sqlite3 fixed_servercooties.db SQLite version 3.11.0 2016-02-15 17:29:24 Enter ".help" for usage hints. sqlite> select count(*) from checks2; 7059111 sqlite>
-
-
@ben_lubar What's on the vertical axis? (I see that events have been more frequent since mid-May, but not what the significance of that is.)
-
@ben_lubar said in @ben_lubar tries to parse ServerCooties data:
Interesting idea from a StackOverflow user: http://stackoverflow.com/a/15874884/2664560
ooooh. that's what caused the corrupt table.....
brb. automating that cleanup in the dump so you don't have to do it manually.
-
@dkf said in @ben_lubar tries to parse ServerCooties data:
What's on the vertical axis?
Response time of the front page of the forum.
-
@ben_lubar just so I too can experience the joy of telling someone how to do a job they already know how to do, is there any recurring daily pattern?
-
I seem to have accidentally invented Discourse
-
It's here if anyone wants to crash their browser: http://web.archive.org/web/20160703215418/https://dl.dropboxusercontent.com/s/pd2lp5qp9mttdth/servercooties-3f4859e9-26a5-48b9-9446-cebe13d7bf0b.html
-
@ben_lubar said in @ben_lubar tries to parse ServerCooties data:
Error: database disk image is malformed
accalia@isitjustmeorservercooties:~$ cat makebackup.sh #!/bin/bash TEMP="$(mktemp -d)" cd "${TEMP}" sqlite3 /var/www/SockSite/servercooties.sql '.dump' | sqlite3 servercooties.sql zip servercooties.sql.zip servercooties.sql mv servercooties.sql.zip /var/www/SockSite/static/servercooties.sql.zip cd .. rm -r "${TEMP}" accalia@isitjustmeorservercooties:~$ crontab -l 0 0 * * * nice /home/accalia/makebackup.sh accalia@isitjustmeorservercooties:~$nevar again should you have that issue.
of course you may need to tweak your script if you pull the zip file and process automatically as the superfluous path components in the zip file have been removed.
-
@accalia said in @ben_lubar tries to parse ServerCooties data:
you may need to tweak your script
My script looks for any file in the zip, and if there are multiple files or no files, it errors.
-
Excerpt from the output of my NodeBB !!SCIENCE!!-ifier:
2016/07/05 13:02:48 user:1:"ben_lubar" 2016/07/05 13:02:48 user:2:"zogstrip" 2016/07/05 13:02:48 user:3:"Remy" 2016/07/05 13:02:48 user:4:"apapadimoulis" 2016/07/05 13:02:48 user:5:"mark_bowytz" 2016/07/05 13:02:48 user:6:"awinter" 2016/07/05 13:02:48 user:7:"InedoJohn" 2016/07/05 13:02:48 user:8:"kharnagy" 2016/07/05 13:02:48 user:9:"PaulaBean" 2016/07/05 13:02:48 user:10:"prashantP" 2016/07/05 13:02:48 user:11:"morbiuswilters" 2016/07/05 13:02:48 user:12:"dhromed" 2016/07/05 13:02:48 user:13:"blakeyrat" 2016/07/05 13:02:51 user:14:"boomzilla" 2016/07/05 13:02:51 user:15:"moderator" 2016/07/05 13:02:56 user:16:"PJH" 2016/07/05 13:02:56 user:17:"ammoQ" 2016/07/05 13:02:56 user:18:"DaveK" 2016/07/05 13:02:56 user:19:"El_Heffe" 2016/07/05 13:02:56 user:20:"Cassidy" 2016/07/05 13:03:00 user:21:"Lorne Kates" 2016/07/05 13:03:01 user:22:"snoofle" 2016/07/05 13:03:02 user:23:"error" 2016/07/05 13:03:02 user:24:"frits" 2016/07/05 13:03:07 user:25:"Zecc" 2016/07/05 13:03:08 user:26:"asuffield" 2016/07/05 13:03:08 user:27:"derula" 2016/07/05 13:03:23 user:28:"dkf" 2016/07/05 13:03:23 user:29:"da_Doctah" 2016/07/05 13:03:23 user:30:"serguey123" 2016/07/05 13:03:23 user:31:"tster" 2016/07/05 13:03:23 user:32:"Xyro" 2016/07/05 13:03:23 user:33:"Sutherlands" 2016/07/05 13:03:23 user:34:"C_Octothorpe" 2016/07/05 13:03:23 user:35:"Ronald" 2016/07/05 13:03:23 user:36:"Lingerance" 2016/07/05 13:03:24 user:37:"Zemm" 2016/07/05 13:03:28 user:38:"chubertdev" 2016/07/05 13:03:31 user:39:"Weng" 2016/07/05 13:03:31 user:40:"boog" 2016/07/05 13:03:31 user:41:"hoodaticus" 2016/07/05 13:03:31 user:42:"KattMan" 2016/07/05 13:03:31 user:43:"Matt_Westwood" 2016/07/05 13:03:31 user:44:"PSWorx" 2016/07/05 13:03:33 user:45:"flabdablet" 2016/07/05 13:03:33 user:46:"SpectateSwamp" 2016/07/05 13:03:33 user:47:"DOA" 2016/07/05 13:03:33 user:48:"real_aardvark" 2016/07/05 13:03:33 user:49:"Spectre" 2016/07/05 13:03:33 user:50:"MiffTheFox" 2016/07/05 13:03:33 user:51:"Zylon" 2016/07/05 13:03:33 user:52:"AbbydonKrafts" 2016/07/05 13:03:34 user:53:"Code_Dependent" 2016/07/05 13:03:34 user:54:"operagost" 2016/07/05 13:03:34 user:55:"Jaime" 2016/07/05 13:03:34 user:56:"Jeff_S" 2016/07/05 13:03:34 user:57:"TheCPUWizard" 2016/07/05 13:03:34 user:58:"ObiWayneKenobi" 2016/07/05 13:03:34 user:59:"OzPeter" 2016/07/05 13:03:34 user:60:"KenW" 2016/07/05 13:03:34 user:61:"danixdefcon5" 2016/07/05 13:03:34 user:62:"FredSaw" 2016/07/05 13:03:34 user:63:"brazzy" 2016/07/05 13:03:52 user:64:"mott555" 2016/07/05 13:03:52 user:65:"dtech" 2016/07/05 13:03:52 user:66:"TGV" 2016/07/05 13:03:52 user:67:"Mason_Wheeler" 2016/07/05 13:03:52 user:68:"galgorah" 2016/07/05 13:03:52 user:69:"lolwtf" 2016/07/05 13:03:53 user:70:"Carnildo" 2016/07/05 13:03:53 user:71:"tgape" 2016/07/05 13:03:53 user:72:"Severity_One" 2016/07/05 13:03:53 user:73:"bjolling" 2016/07/05 13:03:53 user:74:"pjt33" 2016/07/05 13:03:53 user:75:"CodeSimian" 2016/07/05 13:03:53 user:76:"vt_mruhlin" 2016/07/05 13:03:54 user:77:"amischiefr" 2016/07/05 13:03:59 user:78:"locallunatic" 2016/07/05 13:04:00 user:79:"Scarlet_Manuka" 2016/07/05 13:04:00 user:80:"ParkinT" 2016/07/05 13:04:01 user:81:"mikeTheLiar" 2016/07/05 13:04:01 user:82:"The_Quiet_One" 2016/07/05 13:04:01 user:83:"eViLegion" 2016/07/05 13:04:01 user:84:"Random832" 2016/07/05 13:04:01 user:85:"stratos" 2016/07/05 13:04:01 user:86:"RayS" 2016/07/05 13:04:01 user:87:"Steve_The_Cynic" 2016/07/05 13:04:01 user:88:"bridget99" 2016/07/05 13:04:01 user:89:"Maurits" 2016/07/05 13:04:02 user:90:"dlikhten" 2016/07/05 13:04:02 user:91:"shadowman" 2016/07/05 13:04:02 user:92:"dubwai" 2016/07/05 13:04:02 user:93:"emurphy" 2016/07/05 13:04:02 user:94:"m0ffx" 2016/07/05 13:04:02 user:95:"Coyne" 2016/07/05 13:04:02 user:96:"Welbog" 2016/07/05 13:04:02 user:97:"savar" 2016/07/05 13:04:02 user:98:"Eternal_Density" 2016/07/05 13:04:02 user:99:"ekolis" 2016/07/05 13:04:03 user:100:"djork"At the time of posting, it's up to
2016/07/05 13:08:25 user:624:"Groaner"2016/07/05 13:13:16 user:1661:"varun"All it's doing is reading user data from the MongoDB database. It's not writing the data anywhere or trying to keep it all in memory.
I had to add a call to SetCursorTimeout(0) because on the first test run, the cursor for the "get list of user IDs" query expired while I was iterating through it.
We only have what, 140k users? This shouldn't take more than a few lifetimes.
-
2016/07/05 13:17:41 user:95923:"Gawgrerge" 2016/07/05 13:17:41 Cursor not found, cursor id: 28781399268
MongoDB
-
By the way, here's my current code: https://play.golang.org/p/6wAZlwPoWF
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
@accalia said in @ben_lubar tries to parse ServerCooties data:
you may need to tweak your script
My script looks for any file in the zip, and if there are multiple files or no files, it errors.
you should be good then.
-
@ben_lubar Filed a bug report: https://jira.mongodb.org/browse/SERVER-24899
-
@ben_lubar
http://stackoverflow.com/questions/38211363/mongodb-cursor-not-found-even-with-no-timeout
-
Go home htop you're drunk
-
So apparently the bug that's affecting me might be one that was fixed in the latest version. Time to update mongodb...
-
-
db.objects.find({_key: 'users:joindate'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function(uid) { db.objects.findOne({_key: 'user:' + uid.value}); db.objects.findOne({_key: 'user:' + uid.value + ':settings'}); db.objects.find({_key: 'uid:' + uid.value + ':ip'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'uid:' + uid.value + ':ignored:cids'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'uid:' + uid.value + ':ignored_tids'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'uid:' + uid.value + ':followed_tids'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'uid:' + uid.value + ':favourites'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'following:' + uid.value}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'uid:' + uid.value + ':upvote'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); db.objects.find({_key: 'uid:' + uid.value + ':downvote'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).forEach(function() { }); })Ok, either this has the same error as the Go version or I'm confused as shit.
-
The Go version is sending
"query" : { "find" : "objects", "filter" : { "_key" : "users:joindate" }, "sort" : { "score" : 1, "value" : 1 }, "projection" : { "score" : 1, "_id" : 0, "value" : 1 }, "skip" : 0 },whereas the mongo shell is sending
"query" : { "find" : "objects", "filter" : { "_key" : "users:joindate" }, "sort" : { "score" : 1, "value" : 1 }, "projection" : { "_id" : 0, "value" : 1, "score" : 1 }, "noCursorTimeout" : true },It looks like the field is set here: https://github.com/mongodb/mongo/blob/r3.2.7/src/mongo/shell/query.js#L225-L227
I think mongod is looking at the bson property instead of the wire format flag.
-
@ben_lubar I had this cursor not found problem, and worked around with smaller queries.
-
Fix SetCursorTimeout. See https://jira.mongodb.org/browse/SERVER-24899 by BenLubar · Pull Request #295 · go-mgo/mgo
Tailable, AwaitData, and NoCursorTimeout were not set, and for some reason MongoDB has two copies of each flag. And it was looking at the other set.
-
Not having to wait 48 hours to click a button is obviously
-
@ben_lubar If you just used something other than Go you'd never have had this problem.
-
@boomzilla said in @ben_lubar tries to parse <del>ServerCooties</del> data:
@ben_lubar If you just used something other than Go you'd never have had this problem.
If NodeBB used something other than MongoDB, I would have never had this problem.
-
postgres=# begin; BEGIN postgres=# explain select post_id, count(*) filter (where up) as upvotes, count(*) filter (where not up) as downvotes, (select array_agg(c.user_id) from (select b.user_id from user_post_votes b where b.post_id = a.post_id and b.up order by b.last_changed desc limit 10) c) as recent_upvotes from user_post_votes a group by post_id order by upvotes desc limit 10; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------ Limit (cost=19125014779.85..19125014779.88 rows=10 width=9) -> Sort (cost=19125014779.85..19125015107.92 rows=131226 width=9) Sort Key: (count(*) FILTER (WHERE a.up)) DESC -> GroupAggregate (cost=10000367436.85..19125011944.11 rows=131226 width=9) Group Key: a.post_id -> Sort (cost=10000367436.85..10000373951.23 rows=2605751 width=9) Sort Key: a.post_id -> Seq Scan on user_post_votes a (cost=10000000000.00..10000045217.51 rows=2605751 width=9) SubPlan 1 -> Aggregate (cost=69533.60..69533.61 rows=1 width=8) -> Limit (cost=69533.44..69533.47 rows=10 width=16) -> Sort (cost=69533.44..69533.49 rows=20 width=16) Sort Key: b.last_changed DESC -> Index Scan using user_post_votes_pkey on user_post_votes b (cost=0.43..69533.01 rows=20 width=16) Index Cond: (post_id = a.post_id) Filter: up (16 rows) postgres=# create index user_post_votes_by_post on user_post_votes (post_id); CREATE INDEX postgres=# explain select post_id, count(*) filter (where up) as upvotes, count(*) filter (where not up) as downvotes, (select array_agg(c.user_id) from (select b.user_id from user_post_votes b where b.post_id = a.post_id and b.up order by b.last_changed desc limit 10) c) as recent_upvotes from user_post_votes a group by post_id order by upvotes desc limit 10; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------ Limit (cost=11123914.76..11123914.78 rows=10 width=9) -> Sort (cost=11123914.76..11124242.82 rows=131226 width=9) Sort Key: (count(*) FILTER (WHERE a.up)) DESC -> GroupAggregate (cost=0.43..11121079.01 rows=131226 width=9) Group Key: a.post_id -> Index Scan using user_post_votes_by_post on user_post_votes a (cost=0.43..144009.93 rows=2605751 width=9) SubPlan 1 -> Aggregate (cost=83.48..83.49 rows=1 width=8) -> Limit (cost=83.33..83.35 rows=10 width=16) -> Sort (cost=83.33..83.38 rows=20 width=16) Sort Key: b.last_changed DESC -> Bitmap Heap Scan on user_post_votes b (cost=4.58..82.90 rows=20 width=16) Recheck Cond: (post_id = a.post_id) Filter: up -> Bitmap Index Scan on user_post_votes_by_post (cost=0.00..4.58 rows=20 width=0) Index Cond: (post_id = a.post_id) (16 rows) postgres=# create index user_post_votes_latest_upvotes_per_post on user_post_votes (post_id, last_changed desc) where up; CREATE INDEX postgres=# explain select post_id, count(*) filter (where up) as upvotes, count(*) filter (where not up) as downvotes, (select array_agg(c.user_id) from (select b.user_id from user_post_votes b where b.post_id = a.post_id and b.up order by b.last_changed desc limit 10) c) as recent_upvotes from user_post_votes a group by post_id order by upvotes desc limit 10; QUERY PLAN ---------------------------------------------------------------------------------------------------------------------------------------------------- Limit (cost=5761560.03..5761560.06 rows=10 width=9) -> Sort (cost=5761560.03..5761888.10 rows=131226 width=9) Sort Key: (count(*) FILTER (WHERE a.up)) DESC -> GroupAggregate (cost=0.43..5758724.28 rows=131226 width=9) Group Key: a.post_id -> Index Scan using user_post_votes_by_post on user_post_votes a (cost=0.43..144009.93 rows=2605751 width=9) SubPlan 1 -> Aggregate (cost=42.62..42.63 rows=1 width=8) -> Limit (cost=0.43..42.49 rows=10 width=16) -> Index Scan using user_post_votes_latest_upvotes_per_post on user_post_votes b (cost=0.43..84.55 rows=20 width=16) Index Cond: (post_id = a.post_id) (11 rows) postgres=# rollback;Can I create an index on a GROUP BY?
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
@boomzilla said in @ben_lubar tries to parse <del>ServerCooties</del> data:
@ben_lubar If you just used something other than Go you'd never have had this problem.
If NodeBB used something other than MongoDB, I would have never had this problem.
If TDWTF used something other than NodeBB...
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
Can I create an index on a GROUP BY?
How does that differ from creating an index on the thing that you're grouping by? (
post_id?)
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
Can I create an index on a GROUP BY?
Two responses come to mind:
-
That's gibberish
-
If you put the result in a temp table, you can put whatever indexes on the temp table you want, so knock yourself out
And your pasting of that gibberish really makes me appreciate that SQL Server has a graphical tool to generate those reports in an actually-readable way.
-
-
2016-07-07T22:39:17.423+0000 I COMMAND [conn58] command 0.objects command: find { find: "objects", filter: { _key: "posts:pid" }, sort: { score: 1, value: 1 }, projection: { _id: 0, value: 1, score: 1 }, skip: 0, noCursorTimeout: true } keyUpdates:0 writeConflicts:0 numYields:44002 reslen:194 locks:{ Global: { acquireCount: { r: 88006 } }, MMAPV1Journal: { acquireCount: { r: 44007 } }, Database: { acquireCount: { r: 44003 } }, Collection: { acquireCount: { R: 44003 }, acquireWaitCount: { R: 4 }, timeAcquiringMicros: { R: 16888 } } } protocol:op_query 418920ms 2016/07/07 17:41:26 errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit."This is the equivalent of
SELECT id, created_at FROM posts ORDER BY created_at, idtaking 7 minutes to return an error about memory usage ON A TABLE WITH INDEXES ON BOTH OF THE FIELDS USED IN THE QUERY.
-
> db.objects.find({_key: 'posts:pid'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).explain() 2016-07-07T22:48:10.243+0000 E QUERY [thread1] Error: explain failed: { "ok" : 0, "errmsg" : "errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.\"", "code" : 96 } : _getErrorWithCode@src/mongo/shell/utils.js:25:13 throwOrReturn@src/mongo/shell/explainable.js:31:1 constructor/this.finish@src/mongo/shell/explain_query.js:176:24 DBQuery.prototype.explain@src/mongo/shell/query.js:508:12 @(shell):1:1
-
> db.objects.find({_key: 'posts:pid'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1}).explain() { "queryPlanner" : { "plannerVersion" : 1, "namespace" : "0.objects", "indexFilterSet" : false, "parsedQuery" : { "_key" : { "$eq" : "posts:pid" } }, "winningPlan" : { "stage" : "PROJECTION", "transformBy" : { "_id" : 0, "value" : 1, "score" : 1 }, "inputStage" : { "stage" : "FETCH", "inputStage" : { "stage" : "IXSCAN", "keyPattern" : { "_key" : 1, "score" : -1 }, "indexName" : "_key_1_score_-1", "isMultiKey" : false, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 1, "direction" : "backward", "indexBounds" : { "_key" : [ "[\"posts:pid\", \"posts:pid\"]" ], "score" : [ "[MinKey, MaxKey]" ] } } } }, "rejectedPlans" : [ { "stage" : "PROJECTION", "transformBy" : { "_id" : 0, "value" : 1, "score" : 1 }, "inputStage" : { "stage" : "SORT", "sortPattern" : { "score" : 1 }, "inputStage" : { "stage" : "KEEP_MUTATIONS", "inputStage" : { "stage" : "SORT_KEY_GENERATOR", "inputStage" : { "stage" : "FETCH", "inputStage" : { "stage" : "IXSCAN", "keyPattern" : { "_key" : 1, "value" : -1 }, "indexName" : "_key_1_value_-1", "isMultiKey" : false, "isUnique" : true, "isSparse" : true, "isPartial" : false, "indexVersion" : 1, "direction" : "forward", "indexBounds" : { "_key" : [ "[\"posts:pid\", \"posts:pid\"]" ], "value" : [ "[MaxKey, MinKey]" ] } } } } } } } ] }, "serverInfo" : { "host" : "085323710f1b", "port" : 27017, "version" : "3.2.7", "gitVersion" : "4249c1d2b5999ebbf1fdf3bc0e0e3b3ff5c0aaf2" }, "ok" : 1 }> db.objects.find({_key: 'topics:tid'}, {_id: 0, value: 1, score: 1}).noCursorTimeout().sort({score: 1, value: 1}).explain() { "queryPlanner" : { "plannerVersion" : 1, "namespace" : "0.objects", "indexFilterSet" : false, "parsedQuery" : { "_key" : { "$eq" : "topics:tid" } }, "winningPlan" : { "stage" : "PROJECTION", "transformBy" : { "_id" : 0, "value" : 1, "score" : 1 }, "inputStage" : { "stage" : "SORT", "sortPattern" : { "score" : 1, "value" : 1 }, "inputStage" : { "stage" : "KEEP_MUTATIONS", "inputStage" : { "stage" : "SORT_KEY_GENERATOR", "inputStage" : { "stage" : "FETCH", "inputStage" : { "stage" : "IXSCAN", "keyPattern" : { "_key" : 1, "value" : -1 }, "indexName" : "_key_1_value_-1", "isMultiKey" : false, "isUnique" : true, "isSparse" : true, "isPartial" : false, "indexVersion" : 1, "direction" : "forward", "indexBounds" : { "_key" : [ "[\"topics:tid\", \"topics:tid\"]" ], "value" : [ "[MaxKey, MinKey]" ] } } } } } } }, "rejectedPlans" : [ { "stage" : "PROJECTION", "transformBy" : { "_id" : 0, "value" : 1, "score" : 1 }, "inputStage" : { "stage" : "SORT", "sortPattern" : { "score" : 1, "value" : 1 }, "inputStage" : { "stage" : "KEEP_MUTATIONS", "inputStage" : { "stage" : "SORT_KEY_GENERATOR", "inputStage" : { "stage" : "FETCH", "inputStage" : { "stage" : "IXSCAN", "keyPattern" : { "_key" : 1, "score" : -1 }, "indexName" : "_key_1_score_-1", "isMultiKey" : false, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 1, "direction" : "forward", "indexBounds" : { "_key" : [ "[\"topics:tid\", \"topics:tid\"]" ], "score" : [ "[MaxKey, MinKey]" ] } } } } } } } ] }, "serverInfo" : { "host" : "085323710f1b", "port" : 27017, "version" : "3.2.7", "gitVersion" : "4249c1d2b5999ebbf1fdf3bc0e0e3b3ff5c0aaf2" }, "ok" : 1 }"I want you to sort this list by the first column." "Ok, I'll sort the list by the first column."
"I want you to sort this list by the first column, and then for things that are the same in the first column, compare the second column." "Ok, I'll load the entire database into memory and then sort it."
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
"I want you to sort this list by the first column, and then for things that are the same in the first column, compare the second column." "Ok, I'll load the entire database into memory and then sort it."
So it's webscale only if you have a very small web?
-
@ben_lubar Remember what Blakey said about temp tables?
-
@FrostCat said in @ben_lubar tries to parse <del>ServerCooties</del> data:
@ben_lubar Remember what Blakey said about temp tables?
That doesn't work before you get the data into the database that supports temp tables.
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
That doesn't work before you get the data into the database that supports temp tables.
Geez, Ben, Blakey even told you how to deal with that: read the shit in without sorting. Inserts will build the index. You can filter after that.
I've been doing that for like 20 years.
-
@FrostCat said in @ben_lubar tries to parse <del>ServerCooties</del> data:
Inserts will build the index.
There already is an index on the column it's sorting by.
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
There already is an index on the column it's sorting by.
Well then there's something wrong. But by all means don't try not sorting because there's obviously no way that could work.
-
@Arantor was the only TDWTF forum member to have posted over 10000 posts in 1 year.
fake edit: But then @boomzilla did it too.
fake edit: FUCKING HELL @accalia HIT 1
2345kfake edit: @abarker hit 10k as well
fake edit: goddamnit @accalia stop making so many posts in the past you hit 1
6789k I mean 20kfake edit: a wild @RaceProUK has appeared!
fake edit: @Onyx has entered the battle!
fake edit: hi @FrostCat
fake edit: @accalia hits 22222 posts per year. The top 20 posters each have more than double the top poster from any time on CS
fake edit: @blakeyrat hits 10k. every user that ever had more than 10k posts per year has never dropped below that level.
fake edit: @accalia hits 25k.
@Arantor drops below 10k. 
fake edit:
@Onyx
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
fake edit: hi @FrostCat
What, the inaugural spoon holder doesn't get to know what his final count was?
-
@ben_lubar said in @ben_lubar tries to parse <del>ServerCooties</del> data:
The top 20 posters each have more than double the top poster from any time on CS
I had--I believe--fewer than 640 posts on CS. In like 10 years.
-
@FrostCat said in @ben_lubar tries to parse <del>ServerCooties</del> data:
What, the inaugural spoon holder doesn't get to know what his final count was?
Your welcome.
-
@boomzilla said in @ben_lubar tries to parse <del>ServerCooties</del> data:
Your welcome.
My welcome what?
-
@FrostCat You tell me. Its you'res.
-
@boomzilla said in @ben_lubar tries to parse <del>ServerCooties</del> data:
You tell me. Its you'res.
It may be, but I don't know to what you're referring.
-
@FrostCat Apparently your penchant for giving your password to blakey.
-
@FrostCat said in @ben_lubar tries to parse <del>ServerCooties</del> data:
@boomzilla said in @ben_lubar tries to parse <del>ServerCooties</del> data:
Your welcome.
My welcome what?
-
@boomzilla said in @ben_lubar tries to parse <del>ServerCooties</del> data:
Apparently your penchant for giving your password to blakey.
That implies that you're me.
-
@error mmmm, bacon.
Loading...
Travis CI
