@Rhywden said:
Erm, if the data store is filtered then I don't really see the point in your complaint...
There are two stores here: the online server memory and the flat file. These will eventually be replaced by a database. The purpose of the current system is to stick as close to the semantics the database will use as possible.
@Rhywden said:
Also, my example was just one of the problems. Just imagine a JSON object, where every other entry was undefined and similar issues.
This is easy: [1,,3,,5,,7,,9]... and so on. The point @tarunik made earlier about JS arrays already having a notation for this and JSON for some reason not supporting it is very relevant here.
@Rhywden said:
Done. Simply use my_array.getByKey(1) or something. I mean, even if your original proposal worked you'd still have to deal with the missing entries. Prototype replacement functions for the methods you need once and you're done.
Searching the entire array for an object's inner key is much slower than accessing that object at an element's position. Also, it doesn't reflect the semantics of the database at all. Simply keeping the nulls I'm stuck with and adding a few lines to deal with them is far less work than creating the machinery necessary to support this manual indexing proposal. With plain arrays, I just push; with your proposal, I need to keep track of the next key manually, which is another thing the DB handles that I won't have to.
@ben_lubar said:
Arrays, like everything else in JavaScript, are hashes.
On the bottom level, yes; top-level semantics are relevant here. For instance, you can't just arbitrarily call members of Array on any old Object.
@flabdablet said:
is fine; if JSON allowed its arrays to be sparse and used the same notation as Javascript does, it wouldn't need an extra keyword.
This is true; I should have said "value" instead of "keyword". I can't off the top of my head think of a reason to include the keyword.
@flabdablet said:
In that case, you could tidy things up a bit by just accepting the use of null as the JSON placeholder value for undefined JS array elements, and passing JSON.parse() a reviver function that returns undefined whenever it sees a null.
Indeed, but instead of bothering with these (which would also require me to modify the middleware here, which I have not even looked into doing), I'm just skipping nulls in the client code for array-like data "manually" just as the functions walking those arrays would have skipped undefined.
@Eldelshell said:
eah, I meant unsorted not unordered. Anyway, the point stands on relying on an array's index of an element to base decisions upon.
That index is the unique identifier of the data at that index. Please explain in depth how basing access to that data on its index (sort of the very definition of an array) is stupid. Also, the array is indeed sorted--everything gets pushed onto the back. The order is the order in which items were added. (Sort of like a DB table with a unique, monotonously increasing key... hrm...)
@flabdablet said:
Yes, this is treating a sparse array as if it were mostly a hashmap. But in Javascript, as in many other languages that support sparse arrays, arrays implement hashmap semantics as well as the indexing, ordering and all-at-once processing you typically get with non-sparse arrays, and using them can result in cleaner and more concise code than generalizing to non-numeric keys and using straight-up hashmaps would do.
I could not have said this better.
@flabdablet said:
Inner-platforming Javascript in order to avoid its perfectly usable O(1) array lookup operation in favour of a verbose O(n) EAV-style replacement? Very enterprisey.
The burn is strong with this one.
@Rhywden said:
That yielded the following in Chrome on an i5 3570
In duplicating your test I found that the #1 slow thing was creating those arrays. Interestingly, reading them in from JSON was way faster.
Object size: 10^2 or ~2^7 vs 10^7 or ~2^23 is about 2-3x... looks more like O(log2(n)) to me. I wonder what searching that large object for your inner key would have been...
@Rhywden said:
Yes, your point being? If you can't use the "perfectly usable" JS arrays due to deficiencies in your data source, then I don't quite see why you criticise me for posting an alternative?
You seem to have missed my original point: JSON does not support the full range of JS values, making processing arrays sent via JSON require logic that does not exist for arrays not sent via JSON. This could be fixed with a small adjustment to the spec.
@flabdablet said:
But you can, and @VaelynPhi already is.
Indeed, I am. The magic code looks something like this:
map(sparseObj, function(v){ if(v===null || v===undefined) return; })
@Rhywden said:
Secondly, if the stringify-method does indeed accept such functions, then I don't understand the OP's complaint at all. I mean, we're all familiar with the concept that casting from one data type to another doesn't always happen with 100% accuracy and without any data loss.
As I pointed out earlier in this reply, I do not have (easy) access to the middleware that is doing this translation to and from JSON; in fact, it's only this issue that made me aware of the fact that it is using JSON for this purpose. Also, it is highly unlikely that a developer would have access to this kind of middleware, and often would not be able to change it.
@ben_lubar said:
var foo = function(){};
console.log(JSON.stringify(foo)); // undefined
I wasn't going to bring up JSON's other oversights, since I think it's obvious that many of them would require significant respeccing.
@Jaime said:
A spare array is something that probably shouldn't exist in your data interchange format.
Except that sparseness is a semantic feature of data that one would possibly like to interchange. As a rule, if you want to communicate it between parts of your program, somewhere there is someone who wants to communicate it between machines, probably for about the same reasons.
@Jaime said:
The accepted way to serialize the type of data that JavaScript can handle as a sparse array is a collection of name-value pairs.
This is a common approach, yes; whether it is good is another argument entirely.
@accalia said:
fun fact. in C array index IS an O(1) oepration.
funner fact there are three different speeds that O(1) operation can complete in.
listed in fastest to slowest speeds
Index value is in CPU cache (L1, L2 or L3 (which are different speeds, but they're very close))
Index value is in Core memory (RAM)
Index value is in RAM that has been paged to disk.
I LOLed, and scared my cat, when I originally read this. You forgot one:
- Index value is stored in a database on IIS and isn't available because the server is rebooting for updates.
@ben_lubar said:
It is O(n).
The data do not seem to support this.
@FrostCat said:
Maybe it is, but n is something like .0000001, so who cares?
For Big-O, n is generally an integer representing operations, iterations, or data elements. A fractional value is... novel here.
@dkf said:
Jaime:
Yeah, but that treats the whole data structure as an untyped mess.
Yes, it's an honest representation.
I LMAOed.
@flabdablet said:
In fact any decent hash table implementation is likely to be closer to O(log n) than O(n), and its actual t(n) function will look something like a + b log n with b very small relative to a. The entire point of a good hash table is to achieve a nearly constant lookup time.
I tried finding something reliably showing this benchmark, but I didn't want to have to look through 140 revisions on jsperf to find the one that didn't think [{ key: 1000000, value: "bob" },{ key: 1000001, value: "sue" }] was a valid array to test lookup on.
@flabdablet said:
Personally I would still strongly prefer that internal hashmap implementation details for my client language of choice did not leak into my JSON interchange formats; {"key": "0", "value": "zero"} as a proposed replacement for {"0": "zero"} just smells horrible to me.
Which itself is a poor way to represent ["zero"]. (I know, you said hashmaps... I was just saying.)
@Jaime said:
In order to end up with the problem the OP had, you almost certainly are in a situation where you should model it as a map or a key/value pair.
This is very not true. Given that clients are able to add or remove elements, the arrays are dynamic by nature. And, as I have already said, I am not using a map and everything works fine. The issue is that functions expecting true arrays (which have undefined values) require extra handling when those values are null instead. JSON simply supporting sparse arrays would complete remove this issue.
@Jaime said:
I don't understand how people end up with these JSON blobs that don't have a defined schema. How would you expect to do anything meaningful with the data if you don't even know its structure? How does a property "go missing"?
For objects it would be passing strange for there to be no schema or no idea what to expect. For an array, there are two possible "empty" values: null or undefined. One is, in effect, "this is empty because nothing goes here", while the other is "nothing is what should be here".
@flabdablet said:
Exactly, which is why using the untidy version smells bad.
Unnecessarily prolix data interchange formats give me hives. The entire point of text-based data interchange is easy human readability; obscuring the actual data in redundant boilerplate is an error, to my way of thinking.
And in my case, it completely bypasses the builtin methods for dealing with array objects, making them harder to work with, which was my original complaint about JSON not preserving undefined.
@ben_lubar said:
Well, if you're using Go, you just use someone else's serialization library (the standard library has json and xml support) and your own data and it works without writing any code specific to your types.
Okay, now you're starting to sound like a Jehovah's Witness even to me.
@cartman82 said:
Also NaN, Infinity and FUCKING COMMENTS.
Fuck json.
I'm actually a little surprised these aren't represented... but I suppose portability might be an issue there... but I suppose that brings up all the other arguments in this thread....
I think the JSON spec needs to grow a little.
@ben_lubar said:
Go constants are
Seriously, WTH? I think we need a @lubarbot to automatically translate all his posts into lojban.