Help Bites--necro edition

Benjamin Hall

So I have a small question that doesn't deserve its own complete thread and the Help Bites thread has gone the way of all the earth. So I'm restarting it afresh.

Here's the question:

What's the best way of stuffing simple tabular data into a CSV and sending it to a browser for download using PHP? Can I just trust the built-in function (fputcsv)?

pie_flavor

@benjamin-hall said in Help Bites--necro edition:

Can I just trust

@benjamin-hall said in Help Bites--necro edition:

PHP

no

</>

dkf

@benjamin-hall said in Help Bites--necro edition:

CSV

browser

What could possibly go wrong? (Answer: Excel. That's what will be the bane of your life if you do this.)

PHP

Run for the hills!

heterodox

@dkf said in Help Bites--necro edition:

PHP

Run for the hills!

Meh. I remember phpExcel as being reasonably easy to use and has the download functionality built-in while using native spreadsheets and not anything that could be misinterpreted.

Seems to have been succeeded by PhpSpreadsheet.

japonicus

@dkf said in Help Bites--necro edition:

What could possibly go wrong? (Answer: Excel. That's what will be the bane of your life if you do this.)

This.

If you happen to have the fortune of inhabiting a parallel universe where users don't automatically load csv files into Excel then you may be fine with fputcsv

Even then, if there's a risk that fields may contain linefeeds then it may be safer to pre-process the data to remove them (replace with spaces or something). The csv spec. does support line-breaks within fields but too many csv-consumers baulk at them. I would also be inclined to strip nulls and all other control codes.

If you need Excel compatibility then watch out for fields beginning with '='. Whether quoted or not, Excel will try to interpret the contents as a formula. Excel will also mangle anything that looks even vaguely like a date.

In general, if you need to support Excel then avoid CSV like the plague and use phpExcel or PhpSpreadsheet to output in xlsx format. That will give you scope to circumvent many of the gotchas.

EDIT PHP's fputcsv and fgetcsv escape_char parameter worries me. The concept of a csv escape character is fundamentally wrong - csv uses quoted strings and repeats "" to escape quote characters. fgetcsv is definitely broken (a bug that php devs refuse to fix) and I wonder if fputcsv has similar issues.

Benjamin Hall

Here's the thing. First, this is download only. It's basically a bulk reporting tool--the data lives elsewhere and I'm just putting it into a form I can slice and dice offline. I control the whole process start to finish.

A record consists of the following:

A string that I control (an assignment title). No line-breaks or anything funny except apostrophes.
An email address. All of these are guaranteed to be of the form $s.$s@<fixed domain>, where the $s parts are alphabetic strings. The input code will reject anything else.
A time-stamp (generated by the database) as a string.
A float between 0.0 and 100.0 (% completed)

And that's it. I'd rather not have to install any extra libraries onto this server, as it's shared hosting.

ben_lubar

@benjamin-hall If there are no double quotes in your entire data set, you can use fputcsv safely.

dkf

@benjamin-hall said in Help Bites--necro edition:

No line-breaks or anything funny except apostrophes.

If every field has no newlines, commas, tabs, double quotes, leading equalities or anything outside the printable subset of ASCII (plus space), you'll be fine. You'll avoid the multitude of bugs, as they're all in those (honking great) edge cases.

Back when I was doing a lot more with this sort of thing, we had two key problems: we had to deal with names of people and names of things (specifically titles of academic papers published in non-English journals and the names of their authors, some of whom were on the same project and sticklers for having their name right). All the non-ASCII characters were totally the bane of our lives. Also, the data in the database itself was corrupted, with encodings varying between rows depending on what system that particular data entry grunt had been using. (There was no record of what the encoding was, of course, and Ruby had at that point some egregious bugs with string handling that exacerbated the problems.) It felt very much like torturing the strings into submission, and then torturing them again to try to make Excel not choke on the result. Bleah.

heterodox

@benjamin-hall said in Help Bites--necro edition:

And that's it. I'd rather not have to install any extra libraries onto this server, as it's shared hosting.

That's an attitude I don't quite get; it's PHP. You can make a lib folder anywhere and put the libraries there. You can have a local PEAR installation and mass-download libraries. It's not like you're relying on a PECL extension or anything (and even that can be dealt with); libraries are just more PHP files.

Of course if you don't want the complexity/overhead, I get that. But I tend to wire in libraries early because I figure I'll need them eventually. (I don't have a need for this functionality now but will quickly when scope creeps -- plus I can take a little extra time, use some formatting options, and get "bonus points" for the resulting download looking really slick.)

Unperverted Vixen

Dumb JavaScript questions:

First, you're "supposed" to use ===/!==, not ==/!=, right?

Second, if I have a value that can be null or undefined, how would I write that in the below conditional? Just using != works, but I didn't think that's what your'e supposed to do. Using !== doesn't catch undefined.

function Blargh(e) {
        e.preventDefault();

        var a = $('#A').is(":checked");
        var b = $("#B").val();

        if (!a && (b !== null)) {
                /* How do I get in here when b is undefined? */
        }
}

hungrier

@unperverted-vixen ===/!== works for me with undefined, at least in the Chrome console.

> a = 'chair'
"chair"
> b = undefined
undefined
> b === undefined
true
> a !== undefined
true
> a === undefined
false
> b !== undefined
false

Unperverted Vixen

@hungrier So you're saying I have to add a third case to the conditional, then?

if (!a && (b !== null) && (b !== undefined))

That seems overcomplicated and ripe for making mistakes when changing it in the future.

Zecc

@unperverted-vixen said in Help Bites--necro edition:

First, you're "supposed" to use ===/!==, not ==/!=, right?

Most of the time, because type coercion is behind some surprising behavior behind comparisons between 0, "", "0", [] , and false.

However, if you don't want to write b !== null && b!== undefined for reasons, you'll be happy to know that all of these are false:

0 == null
NaN == null
[] == null
( {} ) == null
"" == null
"0" == null

While these ones, however, are true:

null == null // duh!
undefined == null

Use b != null, or alternatively b == undefined. You'll be fine.

hungrier

@unperverted-vixen If you want any truthy value to evaluate to true, I would use !!b instead of comparing it with null or undefined. But if you also need it to allow 0 then you'd either need to check !== for both, or check != like @Zecc's suggestion.

Zecc

I've found this, also:

Object.is() - JavaScript | MDN

The Object.is() static method determines whether two values are the same value.

This is not the same as being equal according to the == operator. The == operator applies various coercions to both sides (if they are not the same Type) before testing for equality (resulting in such behavior as "" == false being true), but Object.is doesn't coerce either value.

Edit: but then Object.is(null, undefined) === false. Forgot what you were asking for for a moment there.

Unperverted Vixen

@Zecc , @hungrier - thanks. One last silly question: you say !!b will work as long as I don't care about 0 (which I don't). In that case, couldn't I just say b?

hungrier

@unperverted-vixen You could, but !!b makes it explicit that you're coercing it into a boolean. If you're looking at it a year later, it might help jog your memory to what you were doing a bit better than just b.

Unperverted Vixen

@hungrier Makes sense. Thanks again!

coderpatsy

If you want to be more explicit about it, there's Boolean(b) which does the same thing.

pie_flavor

I'm trying to convert a direction vector and up vector to a Euler angle. I have a vague idea of what a direction vector is, a slightly better idea of what a Euler angle is, and no idea what an up vector is. The only method I can find is this stackoverflow answer, but despite seeming like they know what they're talking about, the ending equation for roll has a single argument for atan2 and a cryptically named abs function which clearly isn't absolute value because it would need to return a single number for the expression to make sense. Every other answer I can find is just telling the asker that they won't find the roll without another vector.
I'll bet there's answers I'm missing because I don't understand them. But I don't understand them. I've got two Vector3ds and need to come up with a third, and that's pretty much what I know.

dkf

@pie_flavor You should read below the line too. The correction (changing a / to a ,) is in the answer's comments:

angle_B = atan2( Dot(W0,U) , Dot(U0,U) / abs(W0) * abs(U0) )

I'd assume that atan2 and abs have their usual meanings (applied to doubles) from the C math library; masses of languages follow those rules (after allowing for differences in type management).

pie_flavor

@dkf I saw that, but didn't know if it's right; the person seemed unsure. And if you read the answer, W0 and U0 are vectors, not doubles. I know of no function named abs that converts a vector of doubles into a single double.

dkf

@pie_flavor Oh, that'd then be a vector magnitude, and the author was more used to writing mathematics.

angle_B = atan2(dot_prod(W0,U), dot_prod(U0,U) / vec_mag(W0) * vec_mag(U0))

where vec_mag is vector magnitude and dot_prod is dot product.

However, I'm not sure if that is correct. Going back to the original answer, we get this (which does look cromulent assuming the weird definition of abs):

cos(angle_B) = Dot(U0,U) / abs(U0) / abs(U)
sin(angle_B) = Dot(W0,U) / abs(W0) / abs(U)

Now, tan(x) = sin(x) / cos(x) so:

tan(angle_B) = (Dot(W0,U) / abs(W0) / abs(U)) / (Dot(U0,U) / abs(U0) / abs(U))
tan(angle_B) = (Dot(W0,U) / abs(W0)) / (Dot(U0,U) / abs(U0))

and hence (switching to a saner notation):

angle_B = atan2(dot_prod(W0,U) / vec_mag(W0), dot_prod(U0,U) / vec_mag(U0))

That feels much better as it has sane ratios on each side, and it gives a different result to what we started with. I'm guessing someone got really sloppy.

pie_flavor

@dkf Thank you!
Edit: So basically:

fun Vector3d.directionToEuler(up: Vector3d): Vector3d {
    val w = Vector3d(-y, x, 0.0)
    val u = w.cross(this)
    return Vector3d(Math.asin(z), Math.atan2(y, x), Math.atan2(w.dot(up) / w.length(), u.dot(up) / u.length()))
}

?

topspin

@pie_flavor Out of curiosity: is this homework or are you trying to adapt one interface to another, or something else?
Because it seems a bit tricky if you're trying to compute something you don't really understand, and the question seems ill-defined: there's (lots of) different definitions of euler angles as far as I remember. Are you sure you want, e.g., heading-pitch-bank and not bank-pitch-heading?

pie_flavor

@topspin said in Help Bites--necro edition:

@pie_flavor Out of curiosity: is this homework or are you trying to adapt one interface to another, or something else?
Because it seems a bit tricky if you're trying to compute something you don't really understand, and the question seems ill-defined: there's (lots of) different definitions of euler angles as far as I remember. Are you sure you want, e.g., heading-pitch-bank and not bank-pitch-heading?

Minecraft plugin. Trying to port code over from software which gives head-rotation in direction vectors to software which gives head-rotation in Euler angles. I had a peek in the inside of the class I'm using, and I know that it's pitch-yaw-roll in the vector from the variable names in there.
The end goal is being able to constantly teleport an entity three blocks in whatever precise direction the player is looking, while facing the player.

topspin

@pie_flavor said in Help Bites--necro edition:

and I know that it's pitch-yaw-roll in the vector from the variable names in there.

Seems to be the correct order then. The good thing in this case is that you can just plug it in and see if it's correctly doing what you want.

pie_flavor

@topspin That I can.

2018-05-22_18-04-39.mp4

topspin

@pie_flavor I have no idea what I'm looking at, but I'm glad it works. ;)

pie_flavor

@topspin Forgot some radian conversions. Works perfectly now.