Paginate the URI

Eldelshell

You can paginate over a list of resources in different ways:

A header: X-PAGE: 2
A parameter: &page=2
A path parameter: resource/page/2

But today I'm seeing the most stupid way of doing this:

resource/sort_by_year_p1.json

Extra points for adding the order by clause to the name of the resource.

The person that defined this API shouldn't be allowed to work with computers any more.

cartman82

What a rude API

resource/sort_by_year_p1_please.json

Better.

Maciejasjmj

@Eldelshell said:

Extra points for adding the order by clause to the name of the resource.

sort_by_year'; DROP TABLE YEARS; --_p1.json?

Eldelshell

Didn't work, and yes, I tried it.

dkf

Hmm, I know to make a Java REST service engine actually parse something like that into sanity.

@GET
@Path("resource/sort_by_{column:\w+}_p{page:\d+}.json")
@Produces("application/json"}
SomeResultType listContents(@PathParam("column") String column, @PathParam("page") int page)

I need a break.

blakeyrat

I dunno, that doesn't seem that bad to me.

Other than using its own delimiter ('_' instead of '/'), how is that different than your third bullet-point?

Ok, I admit "sort_by" should be "sortby" or something if '_' is actually a delimiter. Which is a minor WTF.

The REAL WTF would be learning that every single sort and item option is a separate file on the back-end, instead of just parsing it out of the URL. Or that the URL is "parsed" by inserting the text directly into a SQL query. But there's not enough info to tell that.

Yamikuronue

@Eldelshell said:

resource/sort_by_year_p1.json

should obviously be

resource/sort/byYear/page/1

Because Rails!

In all seriousness, I've not yet seen an elegant way to handle pagination, sorting, filtering, or complex queries on the backend. What's the RESTful way to say "Give me a list of every chat log in which Person A and Person B were both seen, but exclude any logs marked as X-rated"? I usually end up requesting all logs and querying on the front end, which works better for sorting than filtering.

Eldelshell

I would define an endpoint out of that and call it a day.

Yamikuronue

@Eldelshell said:

define a model

Explain?

Eldelshell

Sure, you can parse the shit out of it instead of using any of the millions of libraries that already parse URL params for you. If you have several ways of doing this, you go for the most PITA way seems like a good idea.

Onyx

I just tack on a regular parameter string on the end. So it's: domain/page/method/?sortby=year&page=1.

Even though everything is AJAX-y by deafult and I can easily hide all that guff, I don't want to.

For one, it makes history management easier, meaning I can just push the state onto the history stack and let the engine re-read the parameters when you hit back.

And anyone who is knowledgeable enough to tinker with the URL will find it familiar and non-confusing. For regular unwashed masses, they most likely won't even see it or it will be faded out, since most modern browsers clean up anything after the ? from the visible URL.

Eldelshell

Sorry, I meant an endpoint, not a model.

logs/yamikuronue?user=A&user=B&nsfw=false

blakeyrat

@Eldelshell said:

Sure, you can parse the shit out of it instead of using any of the millions of libraries that already parse URL params for you.

It's like 5 lines of code.

Look, I'm a software developer. I'm not a "Lego-together 46,432 different buggy open source libraries to get something that might work but probably won't". There's a difference.

Yamikuronue

@blakeyrat said:

"Lego-together 46,432 different buggy open source libraries to get something that might work but probably won't"

You can shorten that to "webdev" or "js ninja" if you like.

Eldelshell

Then I'm thankful I don't have to work with you and learn every wheel you reinvent.

Onyx

@Yamikuronue said:

You can shorten that to "webdev" or "js ninja" if you like.

Who the hell upgraded randomGallery.js? $DEITY damn it, it clashed with fancyLightboxThing-0.0.3.js since both bastards use the global namespace and half the site blew up!

blakeyrat

There's a balance.

Using a bloated library to replace 5 lines of URL-parsing: stupid.

Using a bloated library to communicate with a SOAP server which changes their WSDL daily: life-saver.

EDIT: here's what I use instead of MVC API, for example:

``` using System.Collections; using System.Net; using System.Web; using System.Web.SessionState; using System.Collections.Generic; using System.Text.RegularExpressions; using System.Linq;

using SteamLibrary;

namespace SteamLibraryQuery.api
{
public class APIRequest : IHttpHandler, IRequiresSessionState
{
protected bool isJsonPRequest = false;
protected HttpContext context = null;
protected HttpRequest request = null;
protected HttpResponse response = null;

	public void ProcessRequest(HttpContext httpContext)
	{
		context = httpContext;
		request = context.Request;
		response = context.Response;

		if (context.Session["loggedIn"] != null && (bool)context.Session["loggedIn"])
		{
			string jsonpCallback = null;

			if (!string.IsNullOrEmpty(request.QueryString["callback"]))
			{
				isJsonPRequest = true;
				jsonpCallback = request.QueryString["callback"];

				if (!ValidateCallbackFunction(jsonpCallback))
				{
					response.StatusCode = (int)HttpStatusCode.BadRequest;
					response.Write("{\"error\": \"Invalid JSONP callback function name\"");
					return;
				}
			}

			Hashtable responseData = CreateResponseData(request, response);
			string jsonStr = JSON.Encode(responseData);

			if (isJsonPRequest) // JSONP response
			{
				response.ContentType = "text/javascript";
				string jsonPStr = jsonpCallback + "(" + jsonStr + ");";
				response.Write(jsonPStr);
			}
			else // Normal JSON response
			{
				response.ContentType = "application/json";
				response.Write(jsonStr);
			}
		}
		else
		{
			response.StatusCode = (int)HttpStatusCode.Unauthorized;
			response.Write("{\"error\": \"Not authorized\"");
			return;
		}
	}

	protected string GetParam(string param)
	{
		// JSONP request params must be in the query string
		if (!string.IsNullOrEmpty(request.QueryString[param]))
		{
			return request.QueryString[param];
		}

		if (!isJsonPRequest)
		{
			if (!string.IsNullOrEmpty(request.Params[param]))
			{
				return request.Params[param];
			}
		}

		return null;
	}

	protected object GetSession(string name)
	{
		return context.Session[name];
	}

	public virtual Hashtable CreateResponseData(HttpRequest request, HttpResponse response)
	{
		Hashtable responseData = new Hashtable();

		return responseData;
	}

	public void SetFourOhFour()
	{
		response.StatusCode = (int)HttpStatusCode.NotFound;
	}

	public void SetNotAuth()
	{
		response.StatusCode = (int)HttpStatusCode.Unauthorized;
	}

	public void SetServerError()
	{
		response.StatusCode = (int)HttpStatusCode.InternalServerError;
	}

	public bool IsReusable
	{
		get
		{
			return false;
		}
	}

	// Javascript function name parser adapted from: https://gist.github.com/Daniel15/3074365

	/// <summary>
	/// Reserved words in JavaScript - These can't be used in callback function names
	/// </summary>
	private static readonly HashSet<string> _reservedWords = new HashSet<string>
	{
		"break", "do", "instanceof", "typeof", "case", "else", "new", "var", "catch", "finally", 
		"return", "void", "continue", "for", "switch", "while", "debugger", "function", "this", 
		"with", "default", "if", "throw", "delete", "in", "try", "class", "enum", "extends", 
		"super", "const", "export", "import", "implements", "let", "private", "public", "yield",
		"interface", "package", "protected", "static", "null", "true", "false"
	};

	/// <summary>
	/// Regular expression that all callback function names are validated against
	/// </summary>
	private static readonly Regex _validationRegex = new Regex(@"^[a-zA-Z_$][0-9a-zA-Z_$]*(?:\[(?:"".+""|\'.+\'|\d+)\])*?$", RegexOptions.Compiled);

	/// <summary>
	/// Validates that the callback function name is valid
	/// </summary>
	/// <param name="functionName">Name of the function.</param>
	/// <returns></returns>
	public static bool ValidateCallbackFunction(string functionName)
	{
		// Ensure each segment matches the regex, and isn't in the reserved words list.
		return functionName.Split('.').All(segment => _validationRegex.IsMatch(segment) && !_reservedWords.Contains(segment));
	}
}

}

</details>

Every feature in MVC API I care about, 1/100th the code.

loopback0

@dkf said:

Java ... sanity.

Does not compute.

accalia

@Onyx said:

Who the hell upgraded randomGallery.js? $DEITY damn it, it clashed with fancyLightboxThing-0.0.3.js since both bastards use the global namespace and half the site blew up!

if i see your library pollute the global namespace then we're going to have trouble.

this is why ALL my web JS reads:

/*eslint opions */
(function(window, $){
    //My craptacular JS code here
}(window, jQuery));

Thanks to @Yamikuronue for introducing me to ESlint

you can replace jQuery with your DOM library of choice

Yamikuronue

Ew. How do you unit test?

loopback0

Testing is for suckers. Much better to just release changes to your JS app to the public and let them test it there, Discostyle.

Yamikuronue

I need to be more drunk, because this thread is 100% my daily life at work XD

Well, minus the weekly tea party. Bring in some black tea and milano cookies and we'd have my workweek.

accalia

pretty well actually,

My main JS will do this:

/*eslint opions */
(function(window, $){
    //My craptacular JS code here
    window.MyCraptacularLibrary = SomeObjectIveBeenBuildingAbove;
}(window, jQuery));

and any suplimentary JS will hand objects/functions off that main reference. I only expose one object to global namespace, and the self executing function creates a closure that can't be escaped by accident.

Yamikuronue

Ok, you're excused. My devs love to shove things into closures to provide scope so they don't have to make objects.

You can read the above in the same tone you might hear a mother say "My boys love to eat their own boogers".

accalia

@Yamikuronue said:

"My boys love to eat their own boogers".

:eyeroll.png: all our webdevs do that. and only one of them is male.

I'll admit that i don't like to make prototype style objects if i can avoid it, I'm a backend developer and prototypical inheritance blows my mind.... but i also care about having testable/clean code.

Yamikuronue

With my move into SQA we only have one female dev remaining. She's pretty cool; I showed her a pie chart of failing unit tests and she frowned, then jumped right on tracking down who and why so they'd be green again two days later. There are a couple guys who seem to get it too, but there are a few more that don't, and have admitted to me that if they could get away with it they'd never do unit testing.

Onyx

What is that collapsible sorcery? Why the fuck does the Google search for "html collapse content" return only JavaScript shenanigans? Why didn't anybody tell me this exists?

It's the fucking Illuminati, I bet ya!

accalia

unit testing is painful, but necessary. in my prvious job i maintained a library ot almost 30K unit/functional/integration/UI tests (about equal numbers of each) and in the time i was there those tests caught at least two regressions that if they had been released and the auditors caught them would incur fines that could easily top 1bn$

boomzilla

@Onyx said:

What is that collapsible sorcery?

Pretty sure it only works in chrome (and I guess the version that opera pushes out now).

blakeyrat

@Onyx said:

What is that collapsible sorcery?

I think Riking clued-me-in on it, when I bitched that Discourse didn't have a collapsible version of a spoiler tag.

Turns out, there's a HTML5 tag named "DETAILS" that creates a collapsible, and Discourse doesn't (yet) filter it out. Apologies to anybody reading this 4 years from now when Discourse does, and it's all spammy.

Onyx

@boomzilla said:

Pretty sure it only works in chrome (and I guess the version that opera pushes out now).

Well, when you google for "html5 details" specifically, it pops up. So it's in the standard. Meaning FFX and/or IE should support it sometime within a decade.

And yes, works in current Opera.

locallunatic

Not all browsers support it yet. Firefox and IE don't but I think Safari does.

boomzilla

@Onyx said:

Meaning FFX and/or IE should support it sometime within a decade.

Yeah, though I just tried in FF again and it still doesn't work. I remember we played with those tags for a while around here before we got the spoiler plugin.

loopback0

@blakeyrat said:

Apologies to anybody reading this 4 years from now when Discourse does, and it's all spammy.

Apologies to anyone still using Discourse 4 years from now in general.

Onyx

@boomzilla said:

I remember we played with those tags for a while around here before we got the spoiler plugin.

I think that was <spoiler>, not <details>. This is the first time I saw it myself.

blakeyrat

Well whatever. Somewhere in this mass of unsearchable, unorganized forum bullshit is a thread where I complain that there's no expando in Discourse (even though Discourse claims to support BBCode, and BBCode has a expando standard, what a shocker that Discourse half-asses something it claims to support!), and one of the Discodevs, I think it was Riking but I no longer remember, clued-me-in on DETAILS.

boomzilla

@Onyx said:

I think that was <spoiler>, not <details>. This is the first time I saw it myself.

Here is some early-ish talk about details (NB: some thread participation by @Onyx in both, though only the first one after details tags were mentioned):

http://what.thedailywtf.com/t/poll-shouldnt-the-spoilers-plugin-be-disabled/1254/20

http://what.thedailywtf.com/t/poll-smoking-cigarette-in-this-century/939/138

Discosearch didn't show any obvious places where it was used, but there you go...

aliceif

@system said:

~~Q. Why can't the questions in the FAQ expand and collapse?~~
Apparently we can in Chrome (but not as of IE11 or FF29) if you feel like mucking about with HTML instead of Markdown (who wouldn't).
A: Because this isn't actually an FAQ. It's the bastard child of a wiki page and a thread topic. Around here that pretty much means this thing will be entirely useless and probably end up full of dick jokes.
Also this means that custom jQuery can't be applied so there's no way to make appearing and disappearing paragraphs/divs or whatever crazy thing markdown does here to screw up post formatting (don't worry we'll come up with an XSS attack to fix it soon).

This contains a <details> tag. I hope, at least. Quoting ~~maybe~~ probably broke it.

Onyx

@boomzilla said:

Discosearch

Ah. So it doesn't work when you remember exactly what you want to find, but it works great to show people what they forgot about. Got it.

boomzilla

@Onyx said:

Ah. So it doesn't work when you remember exactly what you want to find, but it works great to show people what they forgot about. Got it.

I find stuff using it all the time. Just...the search is on cooked text. Though this comment shows up from the markdown tutorial (possibly the clue that blakey got from @riking):

http://what.thedailywtf.com/t/markdown-tutorial/382/16?u=boomzilla

blakeyrat

And of course, none of this fixes the problem that Discourse claims to support BBCode, but BBCode has an expando tag that Discourse just fucking ignores. Because Discourse is a giant flaming dog-turd.

dcon

@accalia said:

/*eslint opions */

So does eslint have opinions or options? Maybe just onions. Yeah, definitely onions.

Magus

@blakeyrat said:

Discourse claims to support BBCode

I wonder if that includes newline...[br]Testing...

Nope. [br]1[/br] Double nope.

Yamikuronue

We had a 20-minute outage this morning that we could estimate cost $220k. Turns out, one of our new clients had a different workflow than the old ones, which has been broken for years but nobody used it. I would rather maintain 30K tests. My coworkers see it differently.

accalia

@Yamikuronue said:

My coworkers see it differently.

they see it wrongly!

chubertdev

"What do you know about this?"
"I'm a published author on WTFs!"

Eldelshell

"I'm a spy"

Ragnax

@Yamikuronue said:

In all seriousness, I've not yet seen an elegant way to handle pagination, sorting, filtering, or complex queries on the backend. What's the RESTful way to say "Give me a list of every chat log in which Person A and Person B were both seen, but exclude any logs marked as X-rated"? I usually end up requesting all logs and querying on the front end, which works better for sorting than filtering.

JSON Format (OData Version 2.0) · OData - the Best Way to REST

OData is a standard interchange format that tailors very well to a LINQ-like delayed read / pull semantics model with over the wire partial data sets. Though the original Atompub XML format was way too verbose for comfort, the JSON format that came with v2 of the standard is actually pretty good.

anonymous234

Oh come on, when was the last time we actually had a major bug that rendered the site unusable?

...today? Well, shit.

accalia

@discoursebot