JSONx is Sexy LIESx

PleegWat

Does XML support sparse arrays?

Does XML support arrays? Nested arrays, without an object layer inbetween?

Jaime

Can it do sparse arrays?

Nothing can magically do both traditional arrays and spares arrays without binding the implementation tightly to a particular language. For example, in a more traditional language where arrays only have integer keys and cannot be sparse, the typical default serialization for an array is usually something like (using JSON here for this example, but don't focus on that):

{
  "anarray": [ "first value", "second value", "third value" ]
}

However, a sparse array would need to be a different thing in these languages, maybe a specific SparseArray class, or maybe a map of some sort. If we used this SparseArray class, and made an array that happened to have sequential integer keys starting with zero, it wouldn't serialize to the output above, it would serialize something like:

{
  "anarray": {
    "0": "firstvalue",
    "1": "secondvalue",
    "2": "thirdvalue"
  }
}

JavaScript is the weird duck that uses array notation to access properties (or maybe it uses property notation to access array elements, that's just as weird). So, only in JavaScript does this entire conversation exist. This only leads to two possible conclusions:

JSON shouldn't be used for 90% of the things we use it for since it's a serialization format that bakes in oddball JavaScript quirks.
The whole concept of asking JSON to support this one weird JavaScript behavior is detrimental to the entire non-JavaScript world.

The answer really is #2, which is why I keep insisting that a JavaScript-side converter should be used to support what the OP asked for. There are arguments for #1. If those arguments prevail and JSON starts picking up more JavaScript-y things, then it will probably be the end of JSON being used as an interchange format and the beginning of the end of widespread support of JSON in pretty much everything. I don't mean support will end everywhere, I mean JSON will no longer be the preferred "use it everywhere" solution.

Luhmann

@FrostCat said:

What are you using that you still need those?

Bulb

@Jaime said:

@flabdablet said:
Can it do sparse arrays?

Nothing can magically do both traditional arrays and spares arrays without binding the implementation tightly to a particular language.

That said, YAML has these !tags for type-annotation and every parser should support defining appropriate pre/post-processing for data of specific types. So you can tell it to serialize sparse arrays as either

---
!SparseArray
  0: this
  1: that
  3: whatever

or

---
!SparseArray
   - this
   - that
   - !undefined
   - whatever

or even

---
  - this
  - that
  - !undefined
  - whatever

(hm, perhaps you should be using the tag namespace or something; I didn't take time to really understand it) because in JavaScript the object can be just array, so it does not need a type annotation and you just define the !undefined.

dkf

@PleegWat said:

Does XML support arrays? Nested arrays, without an object layer inbetween?

Yes, provided you've got a DTD or Schema (and provided you're not doing an array of strings). It's not used very much for this because it's a PITA, but NMTOKENS is a thing.

flabdablet

@Jaime said:

Nothing can magically do both traditional arrays and spares arrays without binding the implementation tightly to a particular language.

I don't see why distinguishing a sparse arrray from a dense one requires any more deserialization "magic" than distinguishing a POCO from a hashmap.

There's also nothing stopping data serialization syntax like ["zero", "one",,, "four", 55:"fifty-five", "fifty-six"] being useful for dense arrays with a receiver-defined default value. C99 allows the use of array initializers like {"zero", "one", [4]="four", [55]="fifty-five", "fifty-six"} even though the arrays so initialized are not in fact sparse.

The use of successive commas to skip undefined array values is not allowed in C99, but it's by no means unique to Javascript: CSV allows the same convention.

powerlord

@accalia said:

fair ehough, and that is standards compliant.

is anyone using it?

also where did your post come from. when i posted my nota bene you post was no where to be seen! even after i left the thread and returned your post wan't there....

Well, the problem with Java is that you have 12 competing libraries to do any specific thing.

If I had to guess, I'd say that the most widely used Java lib for JSON is Google's GSON, which started as an internal project at Google.

kilroo

@Bulb said:

How should one take that seriously.

(in reference to Lua Object Notation, for those who don't feel like looking back at what Bulb was quoting.)

In my defense, I never said that one should take it seriously.

tar

@flabdablet said:

C99 allows the use of array initializers like {"zero", "one", [4]="four", [55]="fifty-five", "fifty-six"} even though the arrays so initialized are not in fact sparse.

Emphasis mine.

flabdablet

Not quite sure why you'd bother emphasizing that. Point I'm trying to make is that whether array elements not explicitly specified end up set to some default value (as C99 does) or whether they're left out entirely (as Javascript does), a data serialization format allowing some tidy way to avoid explicit specification of every single value in an array - that is, a sparse array representation - is both useful and precedented.

tar

Ah, I see, I wasn't sure why you seemed to be saying "sparse arrays are great, here's an example of something that's not in fact a sparse array".

But I'm still not convinced that you can't just use a map in the data to initialize the sparse array...

dkf

@tar said:

But I'm still not convinced that you can't just use a map in the data to initialize the sparse array...

Because it's got {braces}! That means it is a map, not an array! Can't you see the difference?!?!

VaelynPhi

@tar said:

Isn't the REAL WTF the fact that JavaScript supports both undefined and null, and that they have similar but not identical semantics?

Arguably, but in a dynamic language with functional behaviours, there is a good use for the difference: walking arrays skips undefined elements but processes null elements.

@tar said:

If PHP was JavaScript it would also have real_undefined and real_null and and non-commutative equality relationship between all of them.

Functions to do escaping already have a proliferation issue, IMHO...

@Jaime said:

Nothing can magically do both traditional arrays and spares arrays without binding the implementation tightly to a particular language.

Perhaps not magically, but there's no reason a good spec couldn't support it with the details of parsing left to the implementations in various languages. I don't see why, for instance, Java couldn't refuse to parse a sparse array into a native array rather than an array object that supports sparseness.

@Jaime said:

it wouldn't serialize to the output above, it would serialize something like:

Not if the serializer were reasonably written.

@Jaime said:

The answer really is #2, which is why I keep insisting that a JavaScript-side converter should be used to support what the OP asked for. There are arguments for #1. If those arguments prevail and JSON starts picking up more JavaScript-y things, then it will probably be the end of JSON being used as an interchange format and the beginning of the end of widespread support of JSON in pretty much everything. I don't mean support will end everywhere, I mean JSON will no longer be the preferred "use it everywhere" solution.

This is very Chicken Little. Seriously, a reasonable spec with accommodations in parsers and serializers for converting to and from the language of the spec could easily handle all of this with none of the apocalyptic consequences.

@flabdablet said:

There's also nothing stopping data serialization syntax like ["zero", "one",,, "four", 55:"fifty-five", "fifty-six"] being useful for dense arrays with a receiver-defined default value. C99 allows the use of array initializers like {"zero", "one", [4]="four", [55]="fifty-five", "fifty-six"} even though the arrays so initialized are not in fact sparse.

Interesting...

@tar said:

But I'm still not convinced that you can't just use a map in the data to initialize the sparse array...

Because for most cases, a hash is not an array. The available functions for manipulating them are different, even in JS where arrays are hashes (but with those functions and properties tacked on). The difference in practice is the difference between having a sparse array that behaves like a native array versus having a hash with boilerplate to emulate an array.

tar

@VaelynPhi said:

Because for most cases, a hash is not an array. The available functions for manipulating them are different, even in JS where arrays are hashes (but with those functions and properties tacked on). The difference in practice is the difference between having a sparse array that behaves like a native array versus having a hash with boilerplate to emulate an array.

So use the native sparse array of your choice in your runtime, and when the time comes, dump it into a JSON object. You ought to be able to losslessy roundtrip your data in either direction. Everybody wins! Great success!

flabdablet

Again, the issue is not that what @VaelynPhi needs to do cannot be done with JSON as it stands. The issue is that the JSON spec disallows an array serialization notation that is (a) tidy (b) useful (c) unobtrusive (d) already in Javascript, and that there appears to be no good reason for that omission.

Jaime

@VaelynPhi said:

could easily handle all of this with none of the apocalyptic consequences

It's not about apocalyptic consequences - it's about the fact that the only feature of JSON that gave it the market position it has today was the ease of implementation in JavaScript ten years ago. If it were any more complex than it is (a trivial/naive deserializer is just an eval() call), it wouldn't have "won".

You want a smart deserializer, but JSON was designed around a dumb one. It wasn't designed to be good, not to be flexible, it was designed to work reliably in 2001 - which wasn't easy at the time. If you want something better, choose a different format. JSON has so many implementations that any proposed enhancements will take many years to trickle into the real world.

Also, back to post #1, no language that doesn't have both a "null" and an "undefined" has "invent undefined" on it's roadmap. Undefined is the problem, not JSON.

tar

I think we have three options here:

Write a reference implementation demonstrating the proposed sparse array serialization, and see if we can get anywhere with the JSON standardization committee.
Accept the status quo and try to move on with our lives the best we can.
Build a time machine and go back in time to kill Doug Crockford.

I'm leaning towards #3, but I could be swayed to one of the other options...

flabdablet

@Jaime said:

If it were any more complex than it is (a trivial/naive deserializer is just an eval() call), it wouldn't have "won"

This argument would have more force were it not for the fact that the very feature @VaelynPhi bemoans the lack of in JSON does work in a Javascript eval() - hence the thread title.

flabdablet

@Jaime said:

no language that doesn't have both a "null" and an "undefined" has "invent undefined" on it's roadmap. Undefined is the problem, not JSON.

There are endless ways to implement sparse arrays, in any language you like, that do result in a semantic distinction between elements never defined and elements defined with a value that means "unknown".

ben_lubar

// copyright oracle

var unknown = "";

tar

OK, now I've been motivated to actually fire up node.js, I'm now of the opinion that the "empty element in array" trick is just a weird gimplementation detail of Netscape 2.0 LiveScript which somehow managed to get finessed into canonical behaviour by virtue of having been a bug for so long that it turned into a feature. Supporting evidence:

$ nodejs
> a = eval("[1,,3]");
[ 1, , 3 ]
> a
[ 1, , 3 ]
> a.length
3
> b = eval("{1:1,,3:3}");
... }
... "
... ;
... );
... lol
... wtf?
... ...
Invalid REPL keyword
undefined
> b
ReferenceError: b is not defined
    at repl:1:2
    at [...yakyak...]
>

Prosecution rests.

Just one other thing:

{
    "0": [ "zero", "one" ],
    "4": "four",
    "55": [ "fifty-five", "fifty-six" ]
}

OffByOne

@dkf said:

Animated GIF emoji?? FML...

flabdablet

@tar said:

b = eval("{1:1,,3:3}");

You need parens around the curlies to stop eval trying to treat that as a code block. Also, object property names (aka keys) have to be strings, can't be numbers. Also, missing elements are only allowed in array declarations, not in object declarations: array semantics imply default sequential numeric indexes while general object semantics just don't. Something like

{"foo": "bar",, "baz": "quux"}

makes no sense at all, so I find it completely unsurprising that object literals don't allow multiple adjacent commas. Compare

["bar",, "baz"]

Note that Javascript and JSON array literals don't specify indexes; the indexes are implicitly defined by the positions of the values within the literal. That means that multiple commas with nothing in between do have a reasonable meaning: skipping index values. The Javascript literal above specifies a sparse array, of length 3, with only elements [0] and [2] defined.

Javascript arrays being a special case of Javascript objects is an implementation detail, by no means universal even among languages that do feature native sparse array datatypes.

And again: I see no good reason why array literal syntax that works in Javascript should have been left out of JSON. If an incompletely specified JSON array gets fed to JSON parser that has no sparse array type available to parse it into, I can see no reason why the parser couldn't just fall back to filling in the holes with zero or NULL or whatever default seems reasonable.

PJH

@OffByOne said:

Animated GIF emoji?? FML...

You're welcome. I don't plan on having too many of them though

flabdablet

@tar said:

{

"0": [ "zero", "one" ],

"4": "four",

"55": [ "fifty-five", "fifty-six" ]

}</pre

If I were to assign that to a variable, say x, I'd end up with

x["0"][0] === "zero"
x["0"][1] === "one"
x["4"] === "four"
x["55"][0] === "fifty-five"
x["55"][1] === "fifty-six"
</pre

This seems quite straightforward, and I'm not sure what your point is.

flabdablet

@tar said:

> a
[ 1, , 3 ]
> a.length
3

The length of a Javascript array is defined as being its maximum index plus 1. Again, this is straightforward and well specified and I'm not sure what point you're trying to "prosecute".

VinDuv

@PJH said:

Feature request: animated @codinghorror avatar instead of this one.

PJH

@VinDuv said:

Feature request: animated @codinghorror avatar instead of this one.

I swiped that of a vBulletin board I used to frequent. Feel free to make one yourself if you want...

accalia

~~unless you plan on allowing animated avatars i think his request will be denied~~..... oh wait.... he's asking for @ 's avatar to be used for the emoji...

carry on then. i need more tea.

tar

@flabdablet said:

Something like

{"foo": "bar",, "baz": "quux"}

makes no sense at all,

It should be interpreted as:

{"foo": "bar", undefined: undefined, "baz": "quux"}

To be, you know,consistent with the array case.

@flabdablet said:

<pre
x["0"][0] === "zero"
x["0"][1] === "one"
x["4"] === "four"
x["55"][0] === "fifty-five"
x["55"][1] === "fifty-six"

This seems quite straightforward, and I'm not sure what your point is.

Alright then:

{
   "!serialization_format_for": "GodDamnSparseArrayYouPedant",
    "0": [ "zero", "one" ],
    "4": "four",
    "55": [ "fifty-five", "fifty-six" ]
}

Also, what the hell is a <pre tag, and why do I need to use one to manually reconstruct formatting inside a quote? [spoiler]That question was rhetorical.[/spoiler]

flabdablet

@tar said:

It should be interpreted as:

{"foo": "bar", undefined: undefined, "baz": "quux"}

To be, you know,consistent with the array case.

TDEMSYR.

tar

@flabdablet said:

TDEMSYR

Why doesn't replacing omitted literals with undefined make sense? It makes sense in the array case?

flabdablet

Because successive commas in an array literal neither create an index nor assign an array item. They just indicate that an index value should be skipped.

stephen@kitchen:~$ js
js> test = ['zero',,'two']
["zero", , "two"]
js> for (i in test) print('test[' + i + '] = ' + test[i])
test[0] = zero
test[2] = two
js> test[undefined] = undefined
js> for (i in test) print('test[' + i + '] = ' + test[i])
test[0] = zero
test[2] = two
test[undefined] = undefined
js>

Object literals don't generate an implicit sequence of index values, so the notion of "skipping" makes no sense.

tar

@flabdablet said:

Object literals don't generate an implicit sequence of index values, so the notion of "skipping" makes no sense.

Well, yes, that's fine, but I never said anything about anything skipping anything, just filling syntactic holes with undefined. I should perhaps have used {"foo": "bar",: , "baz": "quux"} as my example though.

tar

If I keep flogging this horse for long enough, it should reanimate, right?

flabdablet

@tar said:

I should perhaps have used {"foo": "bar",: , "baz": "quux"} as my example though.

But the additional : , doesn't do anything. It's pointless syntactic bloat.

accalia

@flabdablet said:

But the additional : , doesn't do anything. It's pointless syntactic bloat.

flabdablet

103608468370811

Keep Calm and Carry On

<p>On closing the Keep Calm-o-Matic.</p>

accalia

Touche.

PleegWat

@accalia said:

Touche.

Shouldn't that be Touché?

accalia

maybe. i'm not a stellar spellar.

flabdablet

Feller tell 'er: speller weller.

dkf

@flabdablet said:

Object literals don't generate an implicit sequence of index values, so the notion of "skipping" makes no sense.

They appear to make sense in lon! Or at least the syntax diagrams allow them. Given that they're the only thing in there that even vaguely smells like a specification (and they're incomplete; comments are missing despite being explicitly called out in the examples) such constructs must have a legal meaning. Right?

Filed under: oh look, another broken piece of on the internet…

antiquarian

2 more avatars for next year. Thanks guys!

HardwareGeek

@antiquarian said:

2 more avatars for next year. Thanks guys!

Sorry, beat you to it. :P Feel free to use the "don't feed the troll" one, though.

ben_lubar

Online YAML Parser

tar

"SCRATCH": {
"GOBLIN": "nutxox",

What am I even reading now?

ben_lubar

I've been looking through the language files for a certain game and they are quite ambiguous. Not only do languages share words with the same spelling and different meanings, but there are also these:

ûd means "CHAOS" in GOBLIN or "TO HARVEST" in DWARF or "TO SCALE" in GOBLIN
ìle means "BEAST" in ELF or "VIGOR" in ELF
iÿe means "BEWILDER" in ELF or "BOLD" in ELF
ûst means "CHEERFUL" in GOBLIN or "KNOT" in GOBLIN
ûn means "COURTESY" in GOBLIN or "TO FLY" in GOBLIN
èna means "DEEP" in ELF or "ELBOW" in ELF
eÿi means "DROOP" in ELF or "GLEAM" in ELF
åm means "BABY" in DWARF or "YAWN" in DWARF
èle means "BALL" in ELF or "STRAY" in ELF
oslê means "BANNER" in GOBLIN or "GLADE" in GOBLIN
âs means "MASTER" in GOBLIN or "TALL" in GOBLIN

tar

I see.

So the 100 meter overview would seem to be that HUMANS are better at languaging than inferior races?

ben_lubar

"eslo oma" means "climate bloat" in elf, "equity untrustworthy" in goblin, and "shore walk" in human.
"or kod" means "everlasting knight" in goblin, "large blot" in dwarf, and "lip stalk" in human.

Maybe this is why they have so many wars.