Oh, the fun we shall have, the games we shall play!
-
[sigh]
So, a large portion of the codebase I inherited was written by people who liked nothing more than slapping together some third party libraries, and calling it a day (hey, I get it, coding is hard, and the TV calls to you, but you know, they ARE paying you to do this stuff ... just saying).Lots of our MVC web-apps use JSON+AJAX to submit forms to the MVC Controller backends, for ... raisins (seriously, has our entire industry forgotten that the FORM ITSELF is capable of submitting ITSELF to the backend?).
Anyway, things have been humming along fine for one of the apps since I have taken ownership of it ... until last week. I am just now getting to the bug report on it, so I pulled the logs (one of the first things I did when I inherited everything, was replace all of the:
catch{/*LOL NOPE*/} blocks with actual logging [sigh ... kids]And the error I saw just makes me want to slap someone. Here it is, in all its glory:
"Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property"So, to boil it down, there is an MVC view with a form on it that is wrapping up all of the data from the form into a JSON object and sending that to the Controller via AJAX (bcuz frontend!). Only problem is, the JSON deserializer they are using (native .NET System.Web.Script.Serialization.JavaScriptSerializer) is choking on the amount of data the JSON object contains.
You know what WOULDN'T have choked? (sing along, kids, if you know the words)
A basic HTTP Post of the FORM ITSELF to the controller. The form even HAS a submit button (it calls the JS functions that wrap everything up in JSON, then submits via jQuery, because JS!)!And MVC.NET has LONG had the FormCollection to EASILY catch the contents of a submitted form and do stuff with it. I mean ... WHY?! Why are these fucking hipsters spending SO MUCH TIME re-inventing the wheel? You can still submit the form via JS after doing your hipster shit with JS. I know jQuery can call the form submission trigger!
Now I have to FIX this shit (and no, I am not allowed to just rip out the hipster bullshit, because that would introduce "risk"). I will never forgive Brendan Eich for inflicting this abomination on the universe =_=
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
So, a large portion of the codebase I inherited was written by people who liked nothing more than slapping together some third party libraries, and calling it a day
You worked on Discourse?!
-
@blakeyrat said in Oh, the fun we shall have, the games we shall play!:
@Vaire said in Oh, the fun we shall have, the games we shall play!:
So, a large portion of the codebase I inherited was written by people who liked nothing more than slapping together some third party libraries, and calling it a day
You worked on Discourse?!
Careful now, them's fightin' words =_=
-
@Vaire To make this even more of a WTF, you can use AJAX to POST form data too (I do it myself). Why on earth you'd need JSON for that at all is beyond me.
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
@Vaire To make this even more of a WTF, you can use AJAX to POST form data too (I do it myself). Why on earth you'd need JSON for that at all is beyond me.
Please don't pick at the Jenga tower of my sanity. I am hanging on by a thread here =_=
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
catch{/LOL NOPE} blocks
Now I want to put a
catch { //LOL NOPE }
block in some code.
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
So, to boil it down, there is an MVC view with a form on it that is wrapping up all of the data from the form into a JSON object and sending that to the Controller via AJAX (bcuz frontend!). Only problem is, the JSON deserializer they are using (native .NET System.Web.Script.Serialization.JavaScriptSerializer) is choking on the amount of data the JSON object contains.
You know what WOULDN'T have choked? (sing along, kids, if you know the words)
A basic HTTP Post of the FORM ITSELF to the controller. The form even HAS a submit button (it calls the JS functions that wrap everything up in JSON, then submits via jQuery, because JS!)!I have literally run into exactly the same problem (well, the error message was different) with an AJAXy form that hijacked Submit. The level of when you find something like that is pretty extreme.
-
@loopback0 said in Oh, the fun we shall have, the games we shall play!:
@Vaire said in Oh, the fun we shall have, the games we shall play!:
catch{/LOL NOPE} blocks
Now I want to put a
catch { //LOL NOPE }
block in some code.Well, I actually wrote: catch {/*LOL NOPE*/} (without the backslashes), but fucking markdown decided to stab me instead. [sigh]
-
@Vaire I'm sorry... perhaps you should visit the Emergency Cute Things thread?
-
@anotherusername said in Oh, the fun we shall have, the games we shall play!:
@Vaire said in Oh, the fun we shall have, the games we shall play!:
So, to boil it down, there is an MVC view with a form on it that is wrapping up all of the data from the form into a JSON object and sending that to the Controller via AJAX (bcuz frontend!). Only problem is, the JSON deserializer they are using (native .NET System.Web.Script.Serialization.JavaScriptSerializer) is choking on the amount of data the JSON object contains.
You know what WOULDN'T have choked? (sing along, kids, if you know the words)
A basic HTTP Post of the FORM ITSELF to the controller. The form even HAS a submit button (it calls the JS functions that wrap everything up in JSON, then submits via jQuery, because JS!)!I have literally run into exactly the same problem (well, the error message was different) with an AJAXy form that hijacked Submit. The level of when you find something like that is pretty extreme.
We should form a club :D
-
@Vaire that's why you put ` characters around code that you don't want markdown'd.
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
@loopback0 said in Oh, the fun we shall have, the games we shall play!:
@Vaire said in Oh, the fun we shall have, the games we shall play!:
catch{/LOL NOPE} blocks
Now I want to put a
catch { //LOL NOPE }
block in some code.Well, I actually wrote: catch {/*LOL NOPE*/} (without the backslashes), but fucking markdown decided to stab me instead. [sigh]
Should've used a
code block
edit:
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
@Vaire I'm sorry... perhaps you should visit the Emergency Cute Things thread?
Can't, have to go fix this urgent-production-is-broken(for this one order that has been in process for months)-fix-it-NOOOOOOW "issue"
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
We should form a club
Mahogany would give you a good solid heft.
-
There is one legitimate reason to use AJAX to do a form submission - it doesn't result in reloading the page.
Being able to decide whether or not that's worth the added complexity is what distinguishes good software engineers from bad ones.
-
@Dragnslcr not even. You can set the form's target to an iframe and avoid reloading the page.
-
@Dragnslcr said in Oh, the fun we shall have, the games we shall play!:
There is one legitimate reason to use AJAX to do a form submission - it doesn't result in reloading the page.
Being able to decide whether or not that's worth the added complexity is what distinguishes good software engineers from bad ones.
Yeah, that's why I use it.
(Well that and it's kinda bad UX to redirect the user to an API endpoint... and POST submissions cause that stupid "this will resubmit your data" popup on refresh)
But fuuuuuuuuuuq why would you convert everything into JSON for submit? That's insanely stupid!
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
POST submissions cause that stupid "this will resubmit your data" popup on refresh
That's why you do a redirect.
-
@anotherusername Yeah but then I can't use an API endpoint as my submit target. And it feels hacky to me personally. I'm a fan of the AJAX submit stuff cause it feels like it's a better UX overall for very little work (jQuery Ajax form plugin makes this very simple to implement).
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
Can't, have to go this this urgent-production-is-broken(for this one order that has been in process for months)-fix-it-NOOOOOOW "issue"
pausing only to report your problem here, and then continue to participate in the conversation, right?
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
Yeah but then I can't use an API endpoint as my submit target
What? Why not?
The submit will be a POST. The redirect performs a GET. The URLs of the two don't even need to be different.
-
@anotherusername said in Oh, the fun we shall have, the games we shall play!:
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
Yeah but then I can't use an API endpoint as my submit target
What? Why not?
The submit will be a POST. The redirect performs a GET. The URLs of the two don't even need to be different.
Because then I'll be returning a 302 to every client of my API.
Or, I'll need seperate endpoints or some other way to indicate it's a web client.
-
@sloosecannon can't you just do it when the request was a POST?
-
@anotherusername Sure, but if I'm sending a request from a non-web client, it would be a POST too, yes?
-
@sloosecannon and the 302 will confuse it?
I hesitate to say it, but... user-agent sniffing?
-
@anotherusername It wouldn't confuse it per se (this is all hypothetical really, since the only client currently in use is the web version), but it feels icky and hacky.
I feel like the 5 lines of code required to do the ajax form submit instead are much cleaner.
But idk, I guess I'm doing something hacky there too, just on a different side of things.
So idk.
Everything is hacky. The web sucks. This is why we can't have nice things, etc.
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
@Dragnslcr said in Oh, the fun we shall have, the games we shall play!:
There is one legitimate reason to use AJAX to do a form submission - it doesn't result in reloading the page.
Being able to decide whether or not that's worth the added complexity is what distinguishes good software engineers from bad ones.
Yeah, that's why I use it.
(Well that and it's kinda bad UX to redirect the user to an API endpoint... and POST submissions cause that stupid "this will resubmit your data" popup on refresh)
But fuuuuuuuuuuq why would you convert everything into JSON for submit? That's insanely stupid!
Preaching to the choir, my friend, preaching to the choir ;)
-
@anotherusername said in Oh, the fun we shall have, the games we shall play!:
@sloosecannon and the 302 will confuse it?
I hesitate to say it, but... user-agent sniffing?
No! Bad @anotherusername , go sit in the corner.
-
It wasn't too bad of a fix. The MaxJsonLength property for the System.Web.Script.Serialization.JavaScriptSerializer object was set/get (unlike a lot of built-in library classes, that usually make properties get-only, for ... raisins). So I was able to just immediately set it to int.MaxValue-1 after instantiation. (knocking off 1, because of occasional issues with memory management and signed vs. unsigned int-32s sometimes causing problems, and that is just a habit for me now)
Protip for those of you who may ever deal with this later (you have my sympathies). Out of curiosity, I searched for the error message and looked at the top StackOverflow answer for the issue. They say to use a config file setting to change it. I tried it out (config file change to prod would be less obnoxious than code change to prod), and no, that doesn't work. The value stayed the same as the default 2097152 value.
If anyone was wondering, when I traced it and pulled the dataset that was choking it, it was 2169325 characters of creamy goodness, about 2.1MB of data. Now the stupid thing has 2,147,483,646 of space to work with. [sigh]
-
@Dragnslcr said in Oh, the fun we shall have, the games we shall play!:
There is one legitimate reason to use AJAX to do a form submission - it doesn't result in reloading the page.
You can easily submit a form without loading a page. There's about 36 different ways of accomplishing that.
-
@Vaire is that being sent as a POST, or is it a GET?
-
@anotherusername said in Oh, the fun we shall have, the games we shall play!:
@Vaire is that being sent as a POST, or is it a GET?
POST. That was one of the first things I checked ;)
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
It wasn't too bad of a fix. The MaxJsonLength property for the System.Web.Script.Serialization.JavaScriptSerializer object was set/get (unlike a lot of built-in library classes, that usually make properties get-only, for ... raisins). So I was able to just immediately set it to int.MaxValue-1 after instantiation. (knocking off 1, because of occasional issues with memory management and signed vs. unsigned int-32s sometimes causing problems, and that is just a habit for me now)
Protip for those of you who may ever deal with this later (you have my sympathies). Out of curiosity, I searched for the error message and looked at the top StackOverflow answer for the issue. They say to use a config file setting to change it. I tried it out (config file change to prod would be less obnoxious than code change to prod), and no, that doesn't work. The value stayed the same as the default 2097152 value.
If anyone was wondering, when I traced it and pulled the dataset that was choking it, it was 2169325 characters of creamy goodness, about 2.1MB of data. Now the stupid thing has 2,147,483,646 of space to work with. [sigh]
Just... Out of morbid curiosity.... How much of that was object metadata?
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
@Vaire said in Oh, the fun we shall have, the games we shall play!:
It wasn't too bad of a fix. The MaxJsonLength property for the System.Web.Script.Serialization.JavaScriptSerializer object was set/get (unlike a lot of built-in library classes, that usually make properties get-only, for ... raisins). So I was able to just immediately set it to int.MaxValue-1 after instantiation. (knocking off 1, because of occasional issues with memory management and signed vs. unsigned int-32s sometimes causing problems, and that is just a habit for me now)
Protip for those of you who may ever deal with this later (you have my sympathies). Out of curiosity, I searched for the error message and looked at the top StackOverflow answer for the issue. They say to use a config file setting to change it. I tried it out (config file change to prod would be less obnoxious than code change to prod), and no, that doesn't work. The value stayed the same as the default 2097152 value.
If anyone was wondering, when I traced it and pulled the dataset that was choking it, it was 2169325 characters of creamy goodness, about 2.1MB of data. Now the stupid thing has 2,147,483,646 of space to work with. [sigh]
Just... Out of morbid curiosity.... How much of that was object metadata?
Roughly eyeballing it, I would say about 10-15%
-
@Vaire FANTASTIC
-
@sloosecannon said in Oh, the fun we shall have, the games we shall play!:
@Vaire FANTASTIC
-
@Vaire well, there's that at least.
The one that I found, which I've posted about previously and was going to link to and couldn't, was using GET. As a result, the maximum size was only a few K.
-
@anotherusername said in Oh, the fun we shall have, the games we shall play!:
@Vaire well, there's that at least.
The one that I found, which I've posted about previously and was going to link to and couldn't, was using GET. As a result, the maximum size was only a few K.
Yeah, GET is restricted by the standards-defined length of URLs. NOT a good way to submit large sets of data. And yes, I have seen people try o_O
-
This reminds me so much of current "let's AngularJS everything!" trend. If you don't have a single page application, don't use angular. Because - what the hell for? And if your SPA is actually a normal multipage web application with some pages trying to pretend to be SPA - it's not a SPA and don't use angular FFS. I sometimes think web development is ran by some secret hipster cabal.
-
@Vaire Never bothered checking myself.
But the JSON that I submitted would get serialized into the post method as the arguments.
So, I could write a controller method
dosomething(int a, int b)
And as long as my json had a and b in it, those values would be placed into the individual arguments.
Do named form fields work this way if you just post the form?
-
@blakeyrat Can you submit a form and get the result and append it to the current page, without use AJAX?
-
@xaade you can submit the form to an iframe and grab the frame's contents after it loads. Although, I'm personally torn on whether that's better or worse than just using AJAX to submit the form.
-
@xaade I think you'd have to use an iframe as an intermediary. It's been a long time since I spent 8 hours a day in DOM and JavaScript.
-
@NeighborhoodButcher said in Oh, the fun we shall have, the games we shall play!:
This reminds me so much of current "let's AngularJS everything!" trend. If you don't have a single page application, don't use angular. Because - what the hell for? And if your SPA is actually a normal multipage web application with some pages trying to pretend to be SPA - it's not a SPA and don't use angular FFS. I sometimes think web development is ran by some secret hipster cabal.
This. So much this! I will fight to my dying breath to hold the hipster hoard back. I may let them promote me to management at some point just so I can refuse to hire them =_=
-
@xaade said in Oh, the fun we shall have, the games we shall play!:
@Vaire Never bothered checking myself.
But the JSON that I submitted would get serialized into the post method as the arguments.
So, I could write a controller method
dosomething(int a, int b)
And as long as my json had a and b in it, those values would be placed into the individual arguments.
Do named form fields work this way if you just post the form?
Um, yes and no. You can do that with the controller, but the JSON in question is an object with literally thousands of lines of data in it. Writing a method signature with that amount of data in it, I am pretty sure would break the space-time-continuum :D
-
@Vaire pretty sure you'd be okay dumping it into an
<input type="hidden">
before you submit.
-
@anotherusername said in Oh, the fun we shall have, the games we shall play!:
@Vaire pretty sure you'd be okay dumping it into an
<input type="hidden">
before you submit.If I am going to do that, I am going to just rip all of the nonsense out and submit the form directly.
-
@Vaire said in Oh, the fun we shall have, the games we shall play!:
Um, yes and no. You can do that with the controller, but the JSON in question is an object with literally thousands of lines of data in it. Writing a method signature with that amount of data in it, I am pretty sure would break the space-time-continuum
So...
Yes, but we're using MVC wrong.
-
@xaade I am pretty sure EVERYONE uses MVC wrong. It is a design-pattern, people. It isn't a classification of software. It isn't a type of software. And it CERTAINLY isn't the ONLY way to write software
-
Ok.
Let's look at the equivalent.
I have too many variables to send to this method, so I wrote this god-class to store them all.
It also says that you aren't treating your controller methods as atomic, so you're probably submitting the entire application state to one method to write all the updates, or something like that. Just speculating on why you'd need to send that much data.