I need to throw some XML around


  • Discourse touched me in a no-no place

    I've mentioned this bullshit project I'm working on a few times before. The normal way I would do this has, unfortunately, been closed off to me, and I've hit upon some ridiculous Rube Goldberg solution that I am having trouble implementing, because it's full of stuff I haven't worked with before, so I'm looking for some suggestions, if I'm Doing It Wrong, how to do it more right, and/or making my current code work if my current idea isn't utterly unreasonable.

    Here's what I gotta do: Webserver #2 gets XML posted at it. This isn't a form: I'm getting a POST with a Content-Type of text/xml or maybe application/xml or something, and the request body is straight XML. I need to take that and get it back to webserver #1. My idea was to write a Java servlet on WS#2, read in the XML, open a URLConnection to WS#1, and essentially replicate input to output (that is, POST request with an XML body). I am also writing the program on WS#1 that will ultimately receive this XML, and I have a lot more flexibility there, and am not worried about that piece.

    Does what I've described so far sound unduly horrible, horrible but workable, or not too bad? If "you should throw that idea out and start over" I'm open to suggestions for an alternate method, but I'll probably have to ask more questions about specifics.

    The problem I'm having at the moment is I'm not sure what's wrong. I am actually--as far as I can tell--streaming the XML back out to WS#1, but that program is simply sending back "sorry, I didn't get XML". (Essentially, I have a black-box IsXML method I can call).

    An alternative that occurred to me is to, in the servlet, strip out a subset of the XML and pass that on. The reasoning behind that involves stupidity in my language's design that will just make anyone who hears it angry, but it gives me some options.

    If my idea's not horrible I can post the Java I've written and maybe someone can say something like "you fool, you've got your method calls in the wrong order" or "you forgot to call this method" or something.

    This project's got a deathmarch feel to it and I just want to get it over with.


  • sockdevs

    Is there a reason the XML has to go via WS#2? Why can't it go straight to WS#1?



  • How are you reading the data on WS#1? It might be better to just send the XML from WS#2 instead of the entire post request. However setting up a proxy servlet in java is not that difficult.



  • I moved 4 deliberately unhelping posts to a new topic: Coding harassment



  • This post is deleted!

  • Discourse touched me in a no-no place

    @xaade said:

    Can you copy the entire post and move it over changing the header?

    Well, I thought that was what I was doing, but apparently not.


  • Discourse touched me in a no-no place

    @RaceProUK said:

    Is there a reason the XML has to go via WS#2? Why can't it go straight to WS#1?

    The answer boils down to "CORS, which will not be enabled in WS#1."


  • Discourse touched me in a no-no place

    @jaming said:

    How are you reading the data on WS#1?

    In a way that works. It's a bit much to explain because of the languge I'm using.

    I wrote a standalone test program that uses MSXML to read in (previously saved) XML from a file on disk and posts it to the program on WS#1, and that works. I'm essentially trying to replace that functionality with my servlet.



  • @FrostCat said:

    Well, I thought that was what I was doing, but apparently not.

    Have you used something to see what you are generating? That may point you to what is going wrong with your attempt to slap a different header on it.



  • Ok, that post is magically gone for me. Neither here or there.

    Maybe it's because you are reposting.

    Are you sure your header isn't being duplicated by the final post?



  • @xaade said:

    Ok, that post is magically gone for me.

    Oops, I think I may have Jeff'd it. Sorry.


  • Discourse touched me in a no-no place

    @boomzilla said:

    I moved 4 deliberately unhelping posts

    Just so you know, it didn't bother me; I just didn't want this topic to get entirely derailed. I've been stuck on this stupid project for far too long because I was forced to do it with technology I'm not very familiar with, and I'm not primarily a web dev anyway, and the people who are familiar with the application I'm trying to interface with are essentially ignoring me, but the big boss doesn't seem to care about any of that, but keeps nagging my boss on a daily basis looking for progress.

    It's seriously damaging my calm because I've mostly enjoyed working here for 6+ years and these people threw a task at me that's I'm not the best person for the job, and are basically refusing to help.


  • Discourse touched me in a no-no place

    @locallunatic said:

    Have you used something to see what you are generating? That may point you to what is going wrong with your attempt to slap a different header on it.

    I would love to, if I knew of such a thing.



  • tcpdump's my favorite, though it takes some fiddling to get at the TCP payload (or just load it into wireshark)

    EDIT: That's a linux tool though. Wireshark itself can also capture, and has a windows version. And if you're on windows you probably have a GUI on that server anyway.


  • Discourse touched me in a no-no place

    @PleegWat said:

    tcpdump's my favorite, though it takes some fiddling to get at the TCP payload (or just load it into wireshark)

    My knowledge of those tools consists entirely of "I have heard of wireshark". :/



  • https://www.wireshark.org/. Just hit the record button (and the stop button once you've got your data)


  • Discourse touched me in a no-no place

    Here's some Java, maybe this will help.

    I am mostly using--at the moment, because it's faster--the standalone program I mentioned above: it reads a file off the disk that contains representative XML and POSTs it (using MSXML) at the servlet. The intention of this doPost method is "read the incoming XML, open a URLConnection, pass the XML to an endpoint on WS#1". The endpoint is responding with "I am not receving XML".

    I don't know what is wrong with the code below.

    public void doPost(HttpServletRequest request, HttpServletResponse response)
            /*throws ServletException, IOException*/
    {
        ServletInputStream sis;
        ServletOutputStream sos;
        DocumentBuilderFactory docFactory;
        DocumentBuilder docBuilder;
        Document doc;
        URL url;
        HttpURLConnection huc;
        OutputStream ros;
        InputStream ris;
        try
        {
            int cl = request.getContentLength();
            sos = response.getOutputStream();
            sis = request.getInputStream();
            byte[] b = new byte[cl];
            sis.read(b);
    
            url = new URL("http://blahblah");
            huc = (HttpURLConnection) url.openConnection();
            huc.setUseCaches(false);
            huc.setDoInput(true);
            huc.setDoOutput(true);
            huc.setRequestMethod("POST");
            huc.setAllowUserInteraction(false);
            huc.setRequestProperty("Content-Type", "text/xml");
            huc.setRequestProperty("Content-Length", Integer.toString(cl));
            ros = huc.getOutputStream();
            ris = huc.getInputStream();
            ros.write(b);
            byte[] x = new byte[4096];
            
            int i = ris.read(x);
            sos.println("got back " + i + " bytes");
            sos.println(new String(x));
        }


  • @PleegWat said:

    Just hit the record button (and the stop button once you've got your data)

    If you can't put it on server #2 you should be able to point it at your box and watch the incoming.


  • Discourse touched me in a no-no place

    Maybe some other approach makes sense? Let me see if I can explain this. Sorry if it sounds a bit disjointed; I'm somewhat frazzled.

    Machine #1 has an application that runs under IIS, and is the main thing. Machine #2 has a standalone application that runs under Tomcat.

    Tomcat application can be put inside an iframe in your own application. You simply start it with a blob of XML, and it does it's thing, and when the user is done, he clicks Submit. Application #2 then fires XML back at a URL of your choice. Under the environment stated, this seems to blow up because IIS needs CORS enabled. That will not happen, asserted The Guardian Of Application #1. App #2 and Tomcat cannot be put on the same machine, for the same reason.

    My thought was, have App #2 post to a servlet on Tomcat, which will just forward the request to the receiving endpoint on App #1/IIS, but I can't quite figure out how to make that work.


  • Discourse touched me in a no-no place

    @PleegWat said:

    https://www.wireshark.org/. Just hit the record button (and the stop button once you've got your data)

    Zarquon, I feel a blakeyrant coming on.

    I grabbed the PortableApp and ran it. "Sorry, I can't capture packets because you don't have WinPcap installed." :headdesk:

    "Each Windows package comes with the latest stable release of WinPcap, which is required for live packet capture."

    Apparently a portableapp isn't a package, so I guess I'll try deleting this and grabbing the regular installer.

    I need that "i'm five nested problems deep" XKCD.



  • Dumb question:

    Are you forgetting to call huc.connect() ? url.openConnection doesn't do it on its own.

    I mean I guess it could be doing it automagically when you try writing to the outputstream...



  • @FrostCat said:

    The endpoint is responding with "I am not receving XML".

    Does the same thing happen if you use a mime type of application/xml instead of text/xml?



  • @FrostCat said:

    In a way that works. It's a bit much to explain because of the languge I'm using.

    That doesn't exactly answer my question, but okay.

    [code]
    ros.write(b);
    byte[] x = new byte[4096];
    int i = ris.read(x);
    [/code]
    I think you need to flush your output stream before you read here.

    [code]int cl = request.getContentLength();[/code]
    Also you might want to make sure this is correct because the client could be sending the data chunked.


  • mod

    I didn't read all the intervening posts, but a quick sanity check: Are you 100% sure there's no whitespace before the XML declaration? That is 80% of the time what makes something I'm parsing that I know is xml come up as "not xml". Try having your endpoint trim whitespace.


  • Discourse touched me in a no-no place

    Hey, check out that Capture column. You know what's NOT a link? the WinPCap URL. :headdesk:



  • @FrostCat, Looks like you're looking for a reverse proxy. How about this?


  • mod

    I remember this. It's bullshit. You have to install winpcap separate.



  • Uh, it is bundled with the WIN installer.


  • mod

    I remember the bundled install not working for whatever reason. Don't recall why, but I recall being very frustrated.



  • Buh? The base installer did it just fine for me, wonder what the difference is.


  • Discourse touched me in a no-no place

    @powerlord said:

    Are you forgetting to call huc.connect() ? url.openConnection doesn't do it on its own.

    I was--I may have accidentally removed it.

    @powerlord said:

    I mean I guess it could be doing it automagically when you try writing to the outputstream...

    I think that's happening, because you can see I'm reading the response from the URLConnection and sending it back to the response, and my test program gets data I would expect.

    Side annoyance: Do you think this automagic "some things--not just writing to the outputstream" is actually in the JavaDoc? 'Cos it's not. I saw that as an aside on some sample code.


  • Discourse touched me in a no-no place

    @Salamander said:

    Does the same thing happen if you use a mime type of application/xml instead of text/xml?

    Yes, that's one thing I knew to try.


  • Discourse touched me in a no-no place

    @jaming said:

    That doesn't exactly answer my question, but okay.

    Sorry. It's a proprietary language thing. Essentially I have a handle to an object when my program starts. I have a couple of methods I can call, like IsXML, and a property I can use to get the text as a string. One problem: For raisins, that property will only return ~32000 bytes, and will silently truncate the remainder, so I was using the XML handle because it seems to be the only way to get the data. (I suppose I could split up the XML in the servlet, but that seems baroque.)

    @jaming said:

    I think you need to flush your output stream before you read here.

    I've seen some sample code that does that and at one point I was, but it didn't seem to help.

    @jaming said:

    Also you might want to make sure this is correct because the client could be sending the data chunked.

    My testing indicated it wasn't but I can change that to a "read 4K or whatever bytes at a time" kind of loop if need be, and probably will if I can ever get the basics working.


  • Discourse touched me in a no-no place

    @Yamikuronue said:

    Try having your endpoint trim whitespace.

    I can't :frowning: It's basically a black box.


  • Discourse touched me in a no-no place

    @jaming said:

    Uh, it is bundled with the WIN installer.

    I'll give that a try in the morning, but I'll also consider just installing it separately, based on @Yamikuronue's experience.


  • Discourse touched me in a no-no place

    @locallunatic said:

    Buh? The base installer did it just fine for me, wonder what the difference is.

    FWIW the PortableApp download simply doesn't include the wpcap.dll--I looked.


  • mod

    Trim on the way out?


  • Discourse touched me in a no-no place

    @Yamikuronue said:

    Trim on the way out?

    I could do that, although I honestly don't think that's the problem.


  • Discourse touched me in a no-no place

    @IngenieurLogiciel said:

    Looks like you're looking for a reverse proxy. How about this?

    I will look at this. I'll have to look at the license, of course, to see if it's suitable.


  • Discourse touched me in a no-no place

    @FrostCat said:

    the license

    Apache 2.0.

    Well, shit, I don't have the mental energy tonight to figure that out. Could I use this as a drop-in component fitting between two closed-source commercial apps?


  • sockdevs

    Yes


  • Discourse touched me in a no-no place

    @RaceProUK said:

    Yes

    Cool.

    Oh, look, it depends on other shit, namely Apache HttpClient. Hopefully that's something I can drop into Tomcat easily, heh.



  • @FrostCat said:

    Just so you know, it didn't bother me; I just didn't want this topic to get entirely derailed.

    OK. I was kind of bored, and it's been kind of a thing lately to not derail stuff in here right off the bat. Plus I was a little bored.



  • i've used fiddler to intercept http traffic. it's like wireshark but easier to use. wireshark captures too much things.


  • Discourse touched me in a no-no place

    @Jarry said:

    i've used fiddler to intercept http traffic.

    Yeah, fiddler's great, I've used it before. I thought you used it for browsers, though. Can I capture server-to-server traffic with it?



  • I shoved 7 posts to a new topic that probably won't be able to contain @FrostCat's rage either: When Coding Goes Wrong: Frustrated coder considers writing Java GUI



  • Just give up already. I moved a post to an existing topic: When Coding Goes Wrong: Frustrated coder considers writing Java GUI



  • i undserstand that you can set it as a proxy.
    you have access to server #1?

    let me see if i can get it to work in my local tomcat




  • Discourse touched me in a no-no place

    @Jarry said:

    you have access to server #1?

    Server #1, the IIS server, I can do the equivalent of putting web pages on, and that's it. In particular, I may not install anything on it or change any kind of configuration because who knows what mayhem may break loose.

    #2, the Tomcat server, I have much more control over.

    My goal, remember, is: a page on the Tomcat server wants to feed me some XML. I need to parse data out of it and use it to update a few records in a database on Server #1. I do not care in the slightest how I do that, as long as it doesn't involve something that might be considered too exotic by people who are still holding together a web app written in the 90s and have only started using jQuery in the last couple years (or might have some possible risk of damaging Server #1 in some unforeseen and probably unlikely way). IOW, I can get away with writing a servlet, especially since I already have Tomcat on Server #2.

    Server #1 may not have enough disk space or CPU power to even safely add more software. The production equivalent of Server #1 is much more powerful, but the high priest is afraid of installing new software on it, too, because it might have cooties or the plague.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.