... because file handling is so yawn



  • public static String readFileToString(String p_filePath) throws IOException { requireNonNullAndNonEmpty("p_filePath", p_filePath); File inputFile = new File(p_filePath);
    	if (! inputFile.exists())
    		throw new IOException("File: " + p_filePath + " does not exist");
    
    	if (inputFile.isDirectory())
    		throw new IOException("Path: " + p_filePath + " refers to a directory, not a file");
    
    	FileInputStream ins = null;
    
    	try {
    		ins = new FileInputStream(inputFile);
    		byte[] buffer = new byte[1024];
    		int numBytesRead = 0;
    		ByteArrayOutputStream bos = new ByteArrayOutputStream(1024);
    
    		while(-1 != (numBytesRead = ins.read(buffer)))
    			bos.write(buffer, 0, numBytesRead);
    
    		return bos.toString();
    	}
    
    	finally {
    		tryClose(ins);
    	}
    }
    

    called by:

    public static String readFileToStringOrAbort(String p_filePath) { requireNonNullAndNonEmpty("p_filePath", p_filePath); String result = null;
    	try {
    		return readFileToString(p_filePath);
    	}
    
    	catch (IOException ioe) {
    		String eMsg = ioe.getMessage();
    		String msgTrailer = (eMsg == null) ? "" : " - " + eMsg;
    		abort("Error reading: " + p_filePath + msgTrailer);
    	}
    
    	return result;
    }
    

    invoking:

    public static void tryClose(InputStream p_inputStream) { if (p_inputStream != null) try { p_inputStream.close(); }
    	catch (IOException ioe) {
    
    	}
    }
    
    (maybe not quite such a WTF as what we are about to receive, but I'm never too sanguine about exception swallowing)

    ... and then we have this:

    public static void abort(String p_msg) { syserr(p_msg); System.exit(1); }

    So if the file's not there, or is a directory, or if there's anything else going wrong with reading the file, the application just shuts down.

    (cough commons.io cough)



  • Discourse seems to have screwed up your code formatting.

    The best way to format code here is using triple backticks (```) instead of <code> tags. You can include a language name directly after the opening tag to enable syntax highlighting.

    Like this (view raw):

    class FooBar extends Bar
    {
    }
    

  • Discourse touched me in a no-no place

    @Choonster said:

    Discourse seems to have screwed up your code formatting.

    Yeah - they've used <code>/<code> instead of the ``` fences. Discourse appears to be ignoring the fact that it should be ignoring the indentation within them.

    @quite - have a read of https://what.thedailywtf.com/t/escaping-and-posting-code/779. but the below is how it should probably look like:


    	public static String readFileToString(String p_filePath) throws IOException {
    		requireNonNullAndNonEmpty("p_filePath", p_filePath);
    		File inputFile = new File(p_filePath);
    
    		if (! inputFile.exists())
    			throw new IOException("File: " + p_filePath + " does not exist");
    
    		if (inputFile.isDirectory())
    			throw new IOException("Path: " + p_filePath + " refers to a directory, not a file");
    
    		FileInputStream ins = null;
    
    		try {
    			ins = new FileInputStream(inputFile);
    			byte[] buffer = new byte[1024];
    			int numBytesRead = 0;
    			ByteArrayOutputStream bos = new ByteArrayOutputStream(1024);
    
    			while(-1 != (numBytesRead = ins.read(buffer)))
    				bos.write(buffer, 0, numBytesRead);
    
    			return bos.toString();
    		}
    
    		finally {
    			tryClose(ins);
    		}
    	}
    

    called by:

    	public static String readFileToStringOrAbort(String p_filePath) {
    		requireNonNullAndNonEmpty("p_filePath", p_filePath);
    		String result = null;
    
    		try {
    			return readFileToString(p_filePath);
    		}
    
    		catch (IOException ioe) {
    			String eMsg = ioe.getMessage();
    			String msgTrailer = (eMsg == null) ? "" : " - " + eMsg;
    			abort("Error reading: " + p_filePath + msgTrailer);
    		}
    
    		return result;
    	}
    

    invoking:

    	public static void tryClose(InputStream p_inputStream) {
    		if (p_inputStream != null) try {
    			p_inputStream.close();
    		}
    
    		catch (IOException ioe) {
    
    		}
    	}
    

    (maybe not quite such a WTF as what we are about to receive, but I'm never too sanguine about exception swallowing)

    ... and then we have this:

    	public static void abort(String p_msg) {
    		syserr(p_msg);
    		System.exit(1);
    	}
    


  • Life's too short. But thanks for the heads up.



  • The appropriateness of this depends on type of the project.

    Is this a one-off import utility? Whatever.

    Is this a server or desktop app? Now we're talking WTF.



  • Or just a blank line after <code>:

    public static void tryClose(InputStream p_inputStream) {
    	if (p_inputStream != null) try {
    		p_inputStream.close();
    	}
    
    	catch (IOException ioe) {
    
    	}
    }
    


  • Assuming that's C# there's already a one-liner to read a UTF8 file into a string, so all of that is hilariously unnecessary:

    var fileContents = File.ReadAllText(filename);

    Also checking that a file exists before you read from it is technically a race condition, since it could potentially disappear between you checking its existence and you doing the read. (Pre-checking makes sense if and only if you give the user a chance to rectify the path.) So you have to handle the "file not found" exception in any case. So in code like this, you might as well kibosh the pre-check and just handle the relevant exceptions.


  • Discourse touched me in a no-no place

    @blakeyrat said:

    checking that a file exists before you read from it is technically a race condition

    This. Ten thousand times this. It's not merely a “technically” either; it's a real one, and has been the origin of a number of vulnerabilities in badly written software. If you want to open a file, open it and handle the error cases like a pro…



  • Or in short, write your software to be correct to the spec not, "oh, meh, it works most of the time I guess..."



  • @NedFodder said:

    Or just a blank line after <code>

    Or since it's indented, just post it like that.

    public static void tryClose(InputStream p_inputStream) {
        if (p_inputStream != null) try {
            p_inputStream.close();
        }
    
        catch (IOException ioe) {
    
        }
    }


  • @blakeyrat said:

    Or in short, write your software to be correct to the spec

    Wait... there's a spec?!



  • @tar said:

    Wait... there's a spec?!

    Well yes and no.

    In this case, it's more like: "be aware of the existence of pre-emptive multitasking, which has been standard in literally every OS for decades you dumb idiots, what are you, a moron? Who allowed you behind a keyboard? Jesus!"

    Not a spec per se. The spec does say you have to be prepared to handle file not found exceptions, however.



  • There are cases when you might be waiting on a file to arrive, especially when it comes to backend batch integration processes with older technologies. Should I just keep trying to read the file in a wait loop, and catching exceptions? This is a big problem when dealing with FTP systems that don't lock the file from both writers and readers while writing it - I might end up reading an incomplete file. Better to do some "magic" like checking existence, then the size and comparing it to a user-defined time that we'll let FTP be done. Especially when the AS400 people won't give you a "signal" file or some other way to know that "hay we're done!"

    Yes, I probably should use a real integration technology like message queues or any number of other possibilities. But client requirements != good practice.



  • Use this



  • @NTW said:

    There are cases when you might be waiting on a file to arrive, especially when it comes to backend batch integration processes with older technologies.

    Put a watcher on it, so the OS will tell you.

    Polling sucks. Do not poll. Ever. Yes, I know it's a server app and it's not like you have battery concerns, but still: do not poll. Ever. Get into the habit.

    If you're polling in C#, you're definitely doing something wrong. I think it's safe to say that there's literally nothing you could possibly be waiting on in C# that requires polling.



  • Korn shell can do watchers? I know the tech above is C# and it's great there's a class to help. But I find a lot of places still are running Unix and oracle with archaic rules, like "no executables!!"



  • @NTW said:

    Korn shell can do watchers?

    It's a kernel/filesystem feature. I've been assured Linux has it by various Linux fans. I wouldn't expect it to run at the shell's level.

    @NTW said:

    I know the tech above is C#

    Nope; it's in the NT Kernel. You can (and should) easily install file system watchers in any language. Polling is generally speaking the wrong thing to do no matter what language or libraries you're using.

    @NTW said:

    But I find a lot of places still are running Unix

    Unix I don't know.



  • @NTW said:

    Korn shell can do watchers?

    Linux variant here.



  • Agreed that its wrong in principle. In practice, if the file is the daily sales file and the clients budget doesn't allow for a app dev's time to build an exe to do one tiny piece of the analytics data load, I don't think it can be terribly wrong to just poll for the file in a shell script and call it a day.

    Again I said there are lots of better ways, but when you are stuck with 1990s tech and clients that won't change, you do what usually works.


  • Fake News

    @Quite said:

    return bos.toString();

    Ah, that's just nice: it interprets the bytes using the JVM's default encoding which means you might get mojibake when using another JVM or tweaked startup settings.

    @Quite said:

    System.exit(1);

    This reminds me of the time some guy converted a standalone tool to a library, it was "fun" the first few times to see an application server shutdown for no clear reason.



  • @JBert said:

    it was "fun" the first few times to see an application server shutdown for no clear reason

    I see you also play Dwarf Fortress.



  • @cartman82 said:

    Is this a one-off import utility? Whatever.

    In my experience, it's never a once-off utility. It usually starts with that intent, and even "I'll clean it and make it better so that I can reuse it later". But that never happens. And you end up reusing it often enough to lament not fixing it, but not often enough to justify the time to fix it up.

    That's what's happening with another one of our supposedly last contact import runs today...



  • @AgentDenton said:

    In my experience, it's never a once-off utility. It usually starts with that intent, and even "I'll clean it and make it better so that I can reuse it later". But that never happens. And you end up reusing it often enough to lament not fixing it, but not often enough to justify the time to fix it up.

    That's what's happening with another one of our supposedly last contact import runs today...

    Also, if you need a quick one-off utility, JAVA is probably not the best language to pick.


  • Discourse touched me in a no-no place

    @cartman82 said:

    Also, if you need a quick one-off utility, JAVA is probably not the best language to pick.

    That depends. Is the purpose of the utility to consume all your system memory and give astonishingly verbose error messages?



  • @dkf said:

    Is the purpose of the utility to consume all your system memory and give astonishingly verbose error messages?

    Isn't that the purpose of pretty much all utilies, though?



  • @Choonster said:

    The best way to format code here is using triple backticks (```) instead of <code> tags.

    TRWTF is that these two things are treated differently, when they're conceptually identical.



  • If you didn't know TRWTF around here is Discourse... hi, you must be new!



  • No, it's too easy to state that Discourse as a whole is a WTF. To truly appreciate the absurdity, you've got to savor each WTF for it's own WTFiness. That way it's like a whole nativity calendar of WTF, with each day bringing a new surprise and new gift from St. Atwood.



  • @BaconBits said:

    That way it's like a whole nativity calendar of WTF, with each day bringing a new surprise and new gift from St. Atwood.

    I'm liking this forum. It's nice to know that there are many folks that don't think Coding Horror is just TEH BEST EVAR.



  • Actually, <code> is inline. Why Discourse has three or more markup languages being processed simultaneously with regular expressions, though, I will never understand.


  • I survived the hour long Uno hand

    @BaconBits said:

    they're conceptually identical.

    Triple backticks is a pre and a code. If you put both, they should be the same. Single backticks are equal to a code tag.


  • FoxDev

    @NTW said:

    @BaconBits said:
    That way it's like a whole nativity calendar of WTF, with each day bringing a new surprise and new gift from St. Atwood.

    I'm liking this forum. It's nice to know that there are many folks that don't think Coding Horror is just TEH BEST EVAR.

    That's because we've peeked behind the green curtain and seen the reality of the person



  • @NTW said:

    @BaconBits said:
    That way it's like a whole nativity calendar of WTF, with each day bringing a new surprise and new gift from St. Atwood.

    I'm liking this forum. It's nice to know that there are many folks that don't think Coding Horror is just TEH BEST EVAR.

    Atwood himself doesn't bother me. His blog is occasionally good, and I like StackOverflow. But Discourse is terrible software and Atwood is the face of that project, like it or not. I'm not going to be nice about it just because he's done some nice things before. I'm not nice to Microsoft about their crappy software even though some of their products (Excel, Outlook, SQL Server) are absolutely fantastic.



  • @Yamikuronue said:

    Triple backticks is a pre and a code. If you put both, they should be the same.

    Test code:

    ```
    test1 (triple backticks)
    ```
    

    Result:

    test1 (triple backticks)
    

    MD5 of "test1 (triple backticks)":
    a993478f423620c2c462f994702afcd3


    Test code:

    <pre><code>test2 (pre and code)</code></pre>
    

    Result:

    test2 (pre and code)

    MD5 of "test2 (pre and code)":
    209427228909cf0b7635db423866ebd8


    Test code:

    <code><pre>test3 (code and pre)</pre></code>
    

    Result:

    test3 (code and pre)

    MD5 of "test3 (code and pre)":
    724d12b067ec082f4a29a6788e772d00


    Looks like they're :disco:same.


  • FoxDev

    I wonder…

    <pre><code class="lang-auto hljs xml">test2 (pre and code)</code></pre>
    
    test2 (pre and code)

    Edit: OK, that didn't work


  • Discourse touched me in a no-no place

    @BaconBits said:

    I like StackOverflow.

    The best thing about it is that Jeff doesn't work there any more.



  • No, the best thing was that they blocked a former coworker from asking questions because he was too retarded to write intelligent questions.


  • Discourse touched me in a no-no place

    @Arantor said:

    they blocked a former coworker from asking questions because he was too retarded to write intelligent questions.

    So...

    @dkf said:

    Jeff doesn't work there any more.

    ❓



  • This was after Jeff had left, meaning the damage lives on.



  • @cartman82 said:

    The appropriateness of this depends on type of the project.

    Is this a one-off import utility? Whatever.

    Is this a server or desktop app? Now we're talking WTF.

    It's a desktop app. WTF indeed.



  • @blakeyrat said:

    Assuming that's C# there's already a one-liner to read a UTF8 file into a string, so all of that is hilariously unnecessary:

    var fileContents = File.ReadAllText(filename);

    Also checking that a file exists before you read from it is technically a race condition, since it could potentially disappear between you checking its existence and you doing the read. (Pre-checking makes sense if and only if you give the user a chance to rectify the path.) So you have to handle the "file not found" exception in any case. So in code like this, you might as well kibosh the pre-check and just handle the relevant exceptions.

    Java actually (I have no direct influence over that), but indeed, the configuration of the project includes the de rigueur 3rd-party library (commons.io) which allows exactly that capability. Replaced it with that, blew the rest away.



  • @NTW said:

    There are cases when you might be waiting on a file to arrive, especially when it comes to backend batch integration processes with older technologies. Should I just keep trying to read the file in a wait loop, and catching exceptions? This is a big problem when dealing with FTP systems that don't lock the file from both writers and readers while writing it - I might end up reading an incomplete file. Better to do some "magic" like checking existence, then the size and comparing it to a user-defined time that we'll let FTP be done. Especially when the AS400 people won't give you a "signal" file or some other way to know that "hay we're done!"

    Yes, I probably should use a real integration technology like message queues or any number of other possibilities. But client requirements != good practice.

    Well that's as maybe, but in this case the test for whether the file is there calls system exit.



  • @cartman82 said:

    In my experience, it's never a once-off utility. It usually starts with that intent, and even "I'll clean it and make it better so that I can reuse it later". But that never happens. And you end up reusing it often enough to lament not fixing it, but not often enough to justify the time to fix it up.
    That's what's happening with another one of our supposedly last contact import runs today...

    Also, if you need a quick one-off utility, JAVA is probably not the best language to pick.

    It is if that's the environment you're familiar with and the infrastructure to support your code is already there, and you can grab some code that does something similar. But in general you're right, I tend to use Perl when I can for these things.



  • @Yamikuronue said:

    Single backticks are equal to a code tag.

    Not quite. Any matched set of ` inline is a code tag.



  • I was going to point out that it was Java, but I didn't want to provoke a Blakeyrant.

    Anyway, Java 7+ has this static method in the java.nio.file.Files class:

    List<String> Files.readAllLines(Path, Charset)

    but as you can see from the return type, it splits the input at line breaks. Java 8 also adds a version that just takes the Path and assumes UTF-8 (or more specifically, StandardCharsets.UTF_8) for the character set.

    Note that, unlike File, Path is an interface. The easiest way to get one is to convert a File into one using the toPath method or get it from a FileSystem object (i.e. FileSystems.getDefault().getPath("path", "to", "file") )

    Side note: You may want to glance at the FileSystem class as it has some things that were previously unavailable to Java, such as file system watching.



  • @powerlord said:

    I was going to point out that it was Java, but I didn't want to provoke a Blakeyrant.

    Anyway, Java 7+ has this static method in the java.nio.file.Files class:

    List<String> Files.readAllLines(Path, Charset)

    but as you can see from the return type, it splits the input at line breaks. Java 8 also adds a version that just takes the Path and assumes UTF-8 (or more specifically, StandardCharsets.UTF_8) for the character set.

    Note that, unlike File, Path is an interface. The easiest way to get one is to convert a File into one using the toPath method or get it from a FileSystem object (i.e. FileSystems.getDefault().getPath("path", "to", "file") )

    Side note: You may want to glance at the FileSystem class as it has some things that were previously unavailable to Java, such as file system watching.

    Good call, but the code being maintained here is java 6 standard -- for some reason we can't move up to 7 because "something breaks", which needs to be fixed by someone in our team but not me.


  • Discourse touched me in a no-no place

    @Quite said:

    for some reason we can't move up to 7 because "something breaks"

    That'll either be in the security (stuff changed internally in the SSL handling there, which was a PITA for us) or in runtime bytecode generation (upgrade asm if this is the case). Almost anything else would be :wtf:

    Java 8 is more of a pain for us, as that tinkered with the fine details of how the ClassLoaders work. It wouldn't matter, except we have our own (very deeply WTFy) class loading scheme, and it just can't cope. We've a plan to fix it by burning that part of the code to the ground and switching to OSGi, but effort is really limited as we've got loads of other funded things to do at the same time.



  • @powerlord said:

    The easiest way to get one is to convert a File into one using the toPath method or get it from a FileSystem object (i.e. FileSystems.getDefault().getPath("path", "to", "file") )

    Alternatively, Paths.get("path/to/file") also works



  • @ben_lubar said:

    Any matched set of ` inline is a code tag.

    Eventually results in a code tag: Yes. Takes the same path to get there: No, not even close.



  • @NTW said:

    Korn shell can do watchers? I know the tech above is C# and it's great there's a class to help. But I find a lot of places still are running Unix and oracle with archaic rules, like "no executables!!"

    Sure. Any shell can invoke inotifywait.


Log in to reply