Why is Everybody so clueless on the importance of Desktop Search to the Masses?



  • @SpectateSwamp said:

    I have been checking for words in thedailyWTF.txt file. so far they seem to be able to find everything I look for. A lot slower than SSDS but still found. I have looked for "security is nuts" and found that string. What gets lost when creating an index. Something must. I know Swamp search looks at all the data, so it has to find the results. See!!! I don't have all the answers. Answer my question O smart ones.

    The index stores information about the different files on your system.  It does not store the complete contents of each file.  You can use the index to determine which files contain a given word, but you cannot reconstruct the original files just by looking at the index.  This is by design: including the full contents of every file would make searching the index much slower.  The idea is that you use the index to find the file you're looking for, then use the appropriate application to open the file.  This is what a desktop search utility is supposed to do.

    SSDS lets you view the contents of text files.  It does nothing to help you find them.  In fact, from what I've seen, the user has to manually type in the full path of the file that they want to search.  This is the precise opposite of what a desktop search utility is supposed to do.

    Also, I'd appreciate it if you would at least acknowledge my video request from earlier (show SSDS searching for a word at the bottom of an 800 MB file). 



  • Crazy thread

    1500 replies

    1500 replies? This is getting ridiculous!

    Best. Troll thread. Ever. 



  • @spenk said:

    Secondly you are not showing us the steps required by SSDS to make this search work, if you want to be transparent and beyond reproach show us the clean copy of search.exe being installed and then any configuration or setup you are performing before searching with SSDS. If you are not showing these steps it proves nothing as you are in the habit of pre-configuring SSDS for each of your videos, this is not therefore a typical end user situation.

    Yeah, mr. Swamp, allow me to clarify the "tough questions" challenge above: I'm not interested in necessarily correct results in fastest time, I'm just interested in how you get the results and how average users will react when faced with your application. Therefore, to simulate "average user experience", you have to seek the answers to the questions without particularly rehearsing them or otherwise making it too fast to follow, and also by using the "normal" default values of the application, and starting the app at the beginning. If you make this a video response, and your previously oh-so-au-naturel approach to video making shows any signs of using strategic, varied defaults etc, the video is obviously bogus. It is perfectly acceptable, however, to "think aloud" and narrating comprehensively what you are doing in the video ("I'm searching for date in this post, so I have to enter 'blqpfdd' in prompt #52", etc. etc.): You will not be penalised for taking a long time as such. Unlike your obsessions, we're not counting seconds (Murphy's Law corollary #312323: The Operating System Slows It Down Anyway; corollary #312324: Moore's Law is countered by Increasing Bloat, therefore no matter what computer you buy, it's Always Too Slow). We're just interested of seeing how many steps it takes, if the information is presented clearly in the application and is it feasible for "ordinary users" to use the application.



  • Swampie, please... consider this.

    You advocate that SSDS is better because it can zoom through your source.txt (or whatever you call it) and display a line of results fast. I think we'll all agree with this... pretty much any tool can zoom through a text file and show any lines that match. Grep does this. Google does this. Windows Search does this... etc...  We're not arguing that SSDS is better or worse for this. Interfaces are interfaces, and they all differ. You happen to like SSDS's interface, we happen to dislike it.

    But here's the real crux of the problem... why do you absolutely INSIST that everything has to be in a single text file? You keep talking about how easy it is to back up your data by just dragging your C:\search to an external drive and boom, done. But since you're dragging the folder, you're also dragging all the files within that folder. If you had 5000 seperate files in there and had to drag each one seperately, then yes... a single file will be faster, as it's a single thing to drag.

    But you're not dragging 5000 files, you're dragging a folder. One drag, 5000 files copied. Or one drag, 10 files copied. Doesn't matter how many files are in that folder, they're all going to get copied. So why do you keep insisting one single file is better? Is it perhaps that you don't know how to make SSDS search more than one file? Here, let me show you some pseudo-code. You can let your Mad VB Skillz loose on it and make an improvement to SSDS:

     for (every file in a directory) do {
        open file
        search file
        if (results are found) {
            display results
        }
        close file
        move on to next file
    }

    As it stands right now, SSDS works as follows:

    open search.txt
    search through search.txt
    display results

    Why should we spent days and hours convering OUR data into a format only you find useful? We happen to like using file systems as they're designed - storing files. We like having timestamps on our files. We like being able to open a file instantly, in its entirety, in notepad, make a change, and save it again. We're not going to change how we save OUR data on OUR computer the way WE like, just because YOUR tool is incapable of dealing with REAL storage strategies that EVERYONE except YOU use.

    Honestly, do you really think you're saving any time by keeping everything in a single file? Try it sometime. You've said you've made some 'big' files to show how bad the other search engines are. You must still have one or two of them around. Do you have a 500megabyte text file? Try opening it in notepad by double-clicking the file. How long did it take your computer to open that file? How long did it take you to find the spot in the file that you wanted to make your change? How long did it take notepad to save the file and quit afterwards?

    By comparison, with our hundreds and thousands of small files, notepad opens instantly, we make our changes instantly, we save and quite instantly - we're only loading/saving a few kilobytes, after all.

    Of course, you'll claim that you can do the changes instantly, and then just merge the changes. But you only have a few small files to merge. What if you have to build that 500 megabyte file EVERY TIME? You want to add some more descriptive text to a video entry? That's 500 megabytes of file copying/merging. Change someone's email address? 500 megabytes of file copying/merging. Made a typo on that last edit? Yet another 500 megabytes of copying/merging.

    All this, because you either refuse (likely), or are totally incapable (highly likely) of making SSDS handle multiple files, or are utterly incapable of grasping the concept of multiple files (the truth).

    Tell me, do you have any books in your house? Do you glue all the covers together so they're really just one really big long book? When you cook spaghetti, do you tie the ends of the noodles together so it's one big long noodle? Do you do the same with your clothes? All your shirts and pants and socks are tied together into one big long shirtpantsock? Do you then just randomly cut out the part(s) you want to read/eat/wear? How about your hair? Your profile photo shows that you have a lot of it. Must be horrible to have all those seperate individual strands of hair. It'd be much more efficient if they were all joined together into a single long hair.  That way a haircut would be a single snip at the length you want, and not all those hundreds of snips the rest of us have to undergo.

    What about your digital photos and videos? You said previously in this thread that you have 5000 or so of them. Don't think you've mentioned how many video files there are, but there's probably a lot, since you video everything. Do you combine all those 5000 photos into one single large image? Do you merge the videos so those 5000 10-minute clips are one very long 50,000 minute clip?

    Why can't you accept that many small files are better and more efficient than 1 large file? I keep my list of usernames/passwords in one file. If I'm going on a trip and need to take it along, that's just a few kilobytes of data to copy. Same with my bookmarks file (which Firefox 2 keeps as HTML plain-text, by the way). That's about 10 kilobytes. It's a few second, at mosts, to copy those to a USB memory stick and go.

    By comparison, with your humongeous file, first you have to run SSDS, do your search, find where in the file that information is. And then what... what if your passwords are spread over more than the 2 or 3 lines that SSDS can show around a result? You'll have to open the whole file in notepad, cut 'n paste the parts you need, save to a new file, etc... And as we told you earlier, notepad is SLOW for opening files.

    So it took me around 15 seconds to copy my passwords and bookmarks and I'm on the go. In those 15 seconds, you've just barely gotten notepad to show up and display the hourglass, let alone actually show you the start of your massive text file. 

     

    Oh, and another thing. You keep dissing the other search engines because they use indexes. But you know what? Your search.txt file is an index too. Yes, it's true... It's a list of files and descriptions and time offsets and all kinds of other data which exists in other files already. That's an index, you known. I point you at dictionary.com's definition, particularly 7(b): "a reference table that contains the keys or references needed to address data items.". Your source.txt is a reference table, which addresses your data (items): Your videos, your emails, your digital photos, your web clippings, etc...

    Dance around this however you want, but SSDS does use an index - one you have to build manually, and rebuild manually every time anything changes on your system. So explain to us why it is more efficient to have to spend hours and days building that source.txt for our own data, when we can just have Windows/Google/Copernic/Spotlight/Whatever do it automatically for us? We're not going to hit the 10,000 word limit frequently, because we don't generally have files that large. And for files larger than the indexing limits, most of the words will be redundant anyways and show up within those first 10,000 words as is.

     



  • @MarcB said:

    <AWholeLotOfFreakingText/>

     

    Wow man, this bothers you too much. Don't let him get the best of you. Take a deep breath.

    I know ever since my proposed challenge, when he flat out ignored it, I have no plans on even pretending to take SS seriously.

    Certainly any time you spend on this thread is a waste, so I am a little worried you are wasting a lot of your time on this.

    Take it for what it is worth...



  • @SpectateSwamp said:

    The CamCorder doesn't lie.

    ...yet Visual Basic's time functions are, as pointed out in this thread, not very accurate, yet you are apparently relying on them for your benchmarks.

    Did anyone load up Swampy's new video in an external video player and look at the freeze frames? Any analysis on how long it really takes between the keypress and screen update? Any video forensics experts around? Will you post original unmodified DV streams for verification, or do we need to take your word for the validity of the videos? (Why do you trust Google with such important evidence? Don't you fear that Google would tamper with the videos? They have a vested interest in promoting their search app, obviously...)



  • I just found this thread, spent an hour reading some of it and of the links to the other forums, and must admit I'm literally ROFLMAO. 

    Sooo many posts trying to argue with an obviously mentally retarded person stuck several years in the past with the thought that he's revolutionised the world with what's actually nothing more than a useless POS, and who isn't even capable of writing meaningful sentences is a WTF in itself... but sure goes a long way regarding entertainment!

    If SSDS had even the slightest bit of interest, there would at least have been one person noticing it. Havent found that one yet...

    Oh and about not editing videos... I'm sure you'll love the 10 hours of raw footage I'll clean up a bit to keep only 30 minutes for common mortals.




  • Fast fast SSDS

    @stolen_username said:

    Also, I'd appreciate it if you would at least acknowledge my video request from earlier (show SSDS searching for a word at the bottom of an 800 MB file).

    I wasn't lying to you. To find a match near but not on the last page requires a search. If it was on the last page then I would do a "y" option that reads the file once and on the second pass starts the display 1 page before the end of the file. If you want to find something on the 2nd to last page. Then use the "s" search first It reads through finding matching lines only. (40 seconds to read and search the whole file) then the "p" option to display the previous match. It reads to a point 8 or 10 lines before the match and starts from there. Just as if searching from the beginning. The context search "c" is slower because it keeps changing the context lines on each read Adding character counts etc is done on "c" context searchs but not the 's' single line matching display. Create a program that opens a large text file and read to the end doing nothing to the data. I'm sure you will get speeds close to or faster than what I have.

     

    @stolen_username said:

    The index stores information about the different files on your system.

     Thanks for the info. I didn't want to know any of this before doing this search. I might have gotten off on the wrong path. So every word in the file is in the index. I'll maybe do a little more checking into Copernic.

    @stolen_username said:

    SSDS lets you view the contents of text files.  It does nothing to help you find them. 
    With my files they are all in the c:\search\ folder or I don't care about them. But I can search any text or jpg or mpg anywhere on any drive. and have them flashing up on the screen in an instant. A quick merge of all text on c drive doesn't take long. But I have no call to do it. The search has a directory command and I sometimes search that file just to see what files got added today. (ie files that I didn't create) But not much need to look beyond my own folder. The path name defaults to the current folder so I seldom have to enter long path names. Once entered the file name stays on a list of 20 current files (select 1-20 to open them up) I seldom enter the file name, just select from the list at prompt #1. It is very fast to start up and shut down with defaults on the way in. Once the data is displayed the defaults change to back to the previous prompt then previous to that until it closes out. Very quick enter enter enter (SEE) enter enter enter (SSDS is gone) It was designed to be started and stopped lots. if you were running random video display and didn't want to see the one you were on. Kill it and start it again. Enter enter enter will have you back to seeing random video. Start and stop this program lots. That is how I use it.



  • SSDS more than just a search

    @WWWWolf said:

    ...yet Visual Basic's time functions are, as pointed out in this thread, not very accurate, yet you are apparently relying on them for your benchmarks.

    Sometimes showing off is OK. If I hadn't demoed the scrolling text and wanted it faster. I would have never have noticed the problem. That fix will make the slow motion video work better. SEE. Even Great programs have small bugs.

     



  • SSDS hasn't given up on You

    @Kilrah said:

    If SSDS had even the slightest bit of interest, there would at least have been one person noticing it. Havent found that one yet...

    And I haven't given up on you all yet. I have been doing my data like this for years. It works. I'm a greppler, I'm a searcher. From what I've seen by the INDEXERS they have a long long ways to go to catch up. If you want to keep slightly bigger files because you are cutting and pasting lots of info from the net. Try this one.



  • @SpectateSwamp said:

    I have been doing my data like this for years.
     

    Yeah, that's the point. I've never ever seen someone else doing it that way. So, your program might be useful for you - but it stops there. Nobody's crazy enough to use such a broken data "sorting" system.



  • @SpectateSwamp said:

    With my files they are all in the c:\search\ folder or I don't care about them.
     

    So you know where your files are.

    So you're NOT searching for them.

    So your application is NOT a desktop search application.

     

    You say SSDS lets you find things. That's only because you know where things ARE already. You say it's easy to merge all your files together into one big text file. How exactly are you meant to FIND the files in order to merge them together???



  • @MasterPlanSoftware said:

    Wow man, this bothers you too much. Don't let him get the best of you. Take a deep breath.
     

    Naw, he doesn't bother me in the slightest. I wrote him off as a crank long ago. We can't win, because he's got those good-ole' self-refential rules defined for his winning conditions.

    1. SSDS is the ONLY and BEST desktop search
    2. GOTO LINE_000001

    Mostly I'm testing to see if he'll respond to longer posts rather than shorter ones, because we all know Swampies hate short stuff (must be the digital equivalent of a shiny red sports car - compensating for ... *ahem*  short... comings in other areas).

     



  • @rc_pinchey said:

    @SpectateSwamp said:

    With my files they are all in the c:\search\ folder or I don't care about them.
     

    So you know where your files are.

     

    I bet c:\search\ doesn't even have subdirectories. That would be too complicated and useless. There's a search "utility" that will look through the thousands of files anyway. Too bad it only works with text files... when 95% of one's files aren't text.



  • @SpectateSwamp said:

     

    @stolen_username said:

    The index stores information about the different files on your system.

     Thanks for the info. I didn't want to know any of this before doing this search. I might have gotten off on the wrong path. So every word in the file is in the index. I'll maybe do a little more checking into Copernic.

     

     

    NO. reread it - the index stores data ABOUT YOUR FILES. it does not copy each of them fully and read through them each search - that would take forever.... hang on! doesnt that method sound awfully familiar??!!!

     

    admit it, spectate. youre an idiot 



  • Fear no File - no matter how BIG

    @jakkle said:

    read through them each search - that would take forever
    Nobody has a clue about how much text people accumulate. There is absolutely no need to index info, for speed of access. The huge data file it took to break Copernic was over 10 times all the textual data, collected since 1996. The other main problem with indexers is that they don't show you the data in context. SSDS does. They will tell you what file that it is in. What good is that. The same match could be in 10 other files. A little context would help. But indexers don't have that. When I search for information. It is usually for the context.  

     



  • @MarcB said:

    We can't win, because he's got those good-ole' self-refential rules defined for his winning conditions.

    1. SSDS is the ONLY and BEST desktop search
    2. GOTO LINE_000001

    Reminds me of the Time Cube guy. 



  • @Othersider said:

    Reminds me of the Time Cube guy. 

     

     

     Thanks for the link - Is he psychotic or what? 



  • @SpectateSwamp said:

    Nobody has a clue about how much text people accumulate.

    Pure text? Seldom...  I manage small notes and to-do's in Outlook, together with my email and calendar (which is synced over laptops and my smartphone), and in My Documents I keep text in PDF's and Word Documents to preserve or enforce a layout.  These files are joined in a project-based subfolder structure with Excel workbooks, Powerpoint-presentations and much more.  Anything that's beyond my personal responsibility goes on our sharepoint server, which is again filled with a bunch of documents which are not just plain-text files, or even calendars, notes, lists, ...

    Anyway, my point is: people gather text, but seldom this is just "text" and most of the time this text is specifically laid out in different documents, for different uses.  You just cannot remove lay out or specific functions out of a 'text' and expect to be able to do the same thing as you would be able to do in a word processor, spreadsheet or presentation program.

    An example: what would happen if you would remove all layout from your driver's license, and only have the text printed on a piece of paper, and an officer stops you.  Would that piece of white paper with plain text be enough for the officer to prove that you're permitted to drive a vehicle?

    Yes, in the '70's there was not much more than green or amber letters on a black screen.  We moved on from there, to develop and use quite spectacular software to make our lives easier.  My dad never touches plain text files as a common man, and the only text files I touch are scripts to run on our servers, which is something the common man certainly not does on a daily basis.



  • @method1 said:

    Thanks for the link - Is he psychotic or what? 

    Of course not.  We feeble-minded ignoramuses just can't see the truth that is Time Cube.

    Here is an interview with the man himself.



  • Search and a lot lot more

    Just finished merging all the Text files on C 

    The output file was 522MB. Why merge every *.txt file on C drive? Because Swamp Search can. Like that's half a Gig. Next to nothing really.

    I think my next video will try to cover a recent request and demonstrate how I change the control.txt file to hilite other fields, change the string scroll etc. All that can be intimidating until you are in there and making some changes. Mess with the font size and colors. mess with everything. This forum is just getting too long for other searches to bring up quick results. Here is what you need to do to get it all in ONE big file with SSDS:

    On page 1 of this forum, click anywhere on the main screen. enter Ctrl/A to select all the data. And then Ctrl/C to put it in the clipboard.

    Alt tab to the other process running SSDS. At prompt #1 you will have entered thedailyWTF.txt so you are now at prompt #2 enter "z" to paste the contents of the clipboard at the end of thedailyWTF.txt file

    Alt tab to the daily WTF forum and move to page 2 Ctrl/A then Ctrl/C and back to the other to enter 'z' at prompt #2 and back and forth for 30 pages. This forum is like 2 Books long. If you want to find all the search facts. You need a good search. SSDS is it



  • @Kilrah said:

    If SSDS had even the slightest bit of interest, there would at least have been one person noticing it. Havent found that one yet...
     

    Maybe Microsoft will buy it? To paraphrase a quote from The Simpsons:

    @Bill Gates said:

     

    Your application was brought to my attention, but I can't figure out what, if anything, SSDS does, so rather than risk competing with you, I've decided simply to buy you out.




  • @SpectateSwamp said:

    Then give this a shot. the merge takes no time at all. So everybody quit complaining. It has to be faster than indexing. Just has to be.

     


    i can imagine him in a few years locked in a mental hospital screaming:

    <font size="6">IT HAS TO BE FASTER! IT MUST BE FASTER! JUST HAS TO BE!</font>



  • @SpectateSwamp said:

    On page 1 of this forum, click anywhere on the main screen. enter Ctrl/A to select all the data. And then Ctrl/C to put it in the clipboard.

    Alt tab to the other process running SSDS. At prompt #1 you will have entered thedailyWTF.txt so you are now at prompt #2 enter "z" to paste the contents of the clipboard at the end of thedailyWTF.txt file

    Alt tab to the daily WTF forum and move to page 2 Ctrl/A then Ctrl/C and back to the other to enter 'z' at prompt #2 and back and forth for 30 pages. This forum is like 2 Books long. If you want to find all the search facts. You need a good search. SSDS is it

    @SpectateSwamp said:


    Video is by far the most exciting part of Search. I can video
    in a book and go directly to a given page and play that back
    in slow motion. I can video in forum screens.

    @SpectateSwamp said:

     

     It may not be the best at everything. But it does do EVERYTHING


    After lurking in this thread since the beginning, I just had to post.   I'm a bit confused here.  To use computers properly, am I supposed to merge all my data into one huge text file, am I supposed to take screen captures, or am I supposed to use a camcorder to "video in" all the content I'm interested in?  Seems like those are completely different concepts.  If I were to video in a cookbook like you suggested, how would I search for the term "apple pie"?

    Also, aren't computers supposed to be labour-saving devices?  Having to manually copy and paste every page of a forum thread seems like a lot of work.   If I want to save something from the web, like an interesting news article, I usually just click on the "printer-friendly" or "single-page" link and use my browser to save it as HTML, thus preserving something that is more readable than plain text; in many cases, the printer-friendly version even includes pictures from the original.  In the case of this thread, if I really wanted to archive it (which I don't), I would probably write a quick and dirty batch file to use wget to download each of the 30+ pages.  Admittedly,the masses would not do this, but the masses would probably just read the forum online, just like 99.999% of people.

    Here's my question: if SSDS does everything, why can't it automatically save this forum thread in a big text file for you?

    Another question: What do you hope to gain from all of this?  Seriously.  Do you think anyone in this forum has the power to help SSDS achieve worldwide dominance? 



  • @SpectateSwamp said:

    @MarcB said:

    I get a laugh out of "I was YellowHead" bit, given that his avatar has a distinctly yellow cast to it.

    A small part of the 93 election forum. Even more laughs.

    http://www.youtube.com/watch?v=vKwxLB6jO54

    Yellowhead was the last riding in Canada. I came in LAST. If you are going to be last be VERY last.

     

     

     In the video, you said that nobody needs more than a Grade 9 education.  Do you really believe that?  I hate to say this, but that could explain a lot.

     You may think nobody needs a high-falutin' high school diploma or university degree, but who do you think designed your laptop, the camcorder you love so much, and the Internet you use to disseminate your "interesting" views on video and "desktop search".  Who discovered the concepts that make such things possible?  I'll give you a hint: they probably weren't high school dropouts.

     I find it hilarious that you come here and call technical people "nerds", yet without nerds, you wouldn't even be having this conversation with everyone here.



  • @SpectateSwamp said:

    Just finished merging all the Text files on C 

    The output file was 522MB. Why merge every *.txt file on C drive? Because Swamp Search can. Like that's half a Gig. Next to nothing really.

     

     You know, there are applications that will let you store text notes in a centralized place.  Take EverNote: it lets you clip Web pages (without converting to plain text), clip images, take text notes, even capture handwritten notes.  It even supports search.  Wow, it even supports highlighting!

     The difference between apps like EverNote and SSDS?  People actually use EverNote.

     I actually keep a few notes in small text files.  If I want to search them, I'll just use Windows (non-indexed) search and open in JEdit, Notepad++ or (shudder) Notepad.  If I need more advanced (non-indexed) search, I can use JEdit's hypersearch or grep.

     Explain to me how SSDS makes anyone's life or work easier.  You have to understand that nobody wants to merge everything into one big text file.  No matter how many times you tell people how easy it is, they will never see things your way.



  • @SpectateSwamp said:

    Just finished merging all the Text files on C 

    The output file was 522MB. Why merge every *.txt file on C drive? Because Swamp Search can. Like that's half a Gig. Next to nothing really.

     

    How long did this take?  And how did you use SSDS to make this index?

    @SpectateSwamp said:

    On page 1 of this forum, click anywhere on the main screen. enter Ctrl/A to select all the data. And then Ctrl/C to put it in the clipboard.

    Alt tab to the other process running SSDS. At prompt #1 you will have entered thedailyWTF.txt so you are now at prompt #2 enter "z" to paste the contents of the clipboard at the end of thedailyWTF.txt file

    Alt tab to the daily WTF forum and move to page 2 Ctrl/A then Ctrl/C and back to the other to enter 'z' at prompt #2 and back and forth for 30 pages. This forum is like 2 Books long. If you want to find all the search facts. You need a good search. SSDS is it

    And how long would this take?  Using the site's search box takes about 10 seconds between starting to fill in a search term, and getting results.  I don't think I'll be able to copy and paste 31 pages into a textfile in 10 seconds.

     



  • @joemck said:

    Edit: Put random. Periods. In post. So SpectateSwamp can. Read it.

    Yes. That's what I'm trying to preach. All the time. But no one does it. That's why the swampling doesn't seem to listen. Because he doesn't get long sentences.



  • @SpectateSwamp said:

    Just finished merging all the Text files on C 

    The output file was 522MB.

     

    Now that's pretty cool. Just for a laugh I ran a search for *.txt on my C drive. That brought 7606 results, totalling FILE_NOT_FOUND MBytes as apparently trying to select all those files and choosing properties just breaks Explorer without ever bringing the properties page...

    http://i18.photobucket.com/albums/b105/Kilrah/filemenu.jpg 

    The other WTF is the names of the text files... besides the EULA.txt, licence.txt, readme.txt, whatsnew.txt, changelog.txt, the various program log files and random config files you get to see some interesting stuff, like "All content © 2005 DigiPen (USA) Corporation, all rights reserved.txt", "DO_NOT_MANUALLY_DELETE_ANY_SUBFOLDERS.txt" with "DO_NOT_MANUALLY_DELETE_THIS_FOLDER.txt" in each of them,...

    But TRWTF is that 99.99% of those files are totally useless to me, and there is only ONE, that totals 379 bytes that I'd ever be interested in. Of course, a search tool is essential to be able to find what I'm looking for inside it. 

    EDIT: Damn forum not parsing links automatically. 



  • Resistance is futile - In most cases

    @wooter said:

    How long did this take?  And how did you use SSDS to make this index?

    3Min 15 Secs Half of which was doing directory stuff. - Merged all txt on c:\ drive (522MB)

    I sleep better at night knowing, If aliens attack Earth. We will put the word out, that they are bringing a Greppler type search. Then you'll see RESISTANCE.



  • Merge seldom used - But needed

    @CodeSimian said:

    In the case of this thread, if I really wanted to archive it (which I don't), I would probably write a quick and dirty batch file to use wget to download each of the 30+ pages

     What use would that be. Your search finds the context so slowly it's almost useless (19 Seconds). Indexers can only deal with itsy bitsy files. Bigsy wigsy ones make them fall.

    @CodeSimian said:

    am I supposed to merge all my data into one huge text file

     No never. I have about 20 main files. I don't have to merge them to find what I want. My inmail.txt is inmain, my outmail.txt is sent mail. NetStuff.txt and a few others. I seldom do merges. Unless I need a big file to Take Down Copernic or to satisfy some dumb forum questions. Merge is usefull in cleaning up directories and thus destroying FileSystems.


  • @SpectateSwamp said:

    My inmail.txt is inmain, my outmail.txt is sent mail.

     

    So that means each time you receive or send a mail you copy/paste it into the corresponding text file?  How efficient.



  • SSDS less Education could be better

    @CodeSimian said:

     In the video, you said that nobody needs more than a Grade 9 education.  Do you really believe that?

     To do all the jobs in IT, that I had done to that point. I could have had an apprentice with Grade 9 and taught them everything. Seems TOO high of an education can be a detriment to learning. Knowing it all, makes it hard to learn. I taught educated people how to do Cable Billing conversions. They knew nothing about cable or billing. Like my old boss said "It's not Rocket Science" It is mostly accounting and the old extract sort and report steps. The BEST programmer I came across in all my years had Grade 12 and no formal computer education. (That's not me)

    That's why I say even a non programmer can understand this program. Once they get a glimmer of what's going on. Then the unknown and fear dissipate.

    I just gotta show you all how I use SSDS as a spell checker. Just checked "dissipate" WOW what a wonderful use. And I always hated spelling. (no good at it)

     



  • @Kilrah said:

    "All content © 2005 DigiPen (USA) Corporation, all rights reserved.txt"

    W00t, [url="http://www.nuclearmonkeysoftware.com/"]Narbacular Drop[/url]!



  • SSDS shouldn't have to export E-Mails

    @Kilrah said:

    @SpectateSwamp said:

    My inmail.txt is inmain, my outmail.txt is sent mail.

     

    So that means each time you receive or send a mail you copy/paste it into the corresponding text file?  How efficient.

    I started out to include an email dump in SSDS but quit and moved on to other more productive stuff. There must be a good email to text dump program somewhere that I could use? If there isn't. Would you consider it your email or the applications email. If your applications can't do exports to text then there is a problem with that application. All Telco and Cable System billing records came as PLAIN TEXT files. From all types of machines to the VAX.


  • Keep more than just the facts

    @Spectre said:

    @Kilrah said:
    "All content © 2005 DigiPen (USA) Corporation, all rights reserved.txt"
    W00t, Narbacular Drop!

     

    Yeah I got some email with warnings "not to reproduce or resent" etc wow scary stuff. I hope I didn't cut and paste anything that was illegal. Not sure that little box I checked agreeing to Copernics download didn't exclude mistreating the Copernic search like I did. Trouble trouble everywhere.



  • @Spectre said:

    @Kilrah said:
    "All content © 2005 DigiPen (USA) Corporation, all rights reserved.txt"

    W00t, Narbacular Drop!

     

     Yep, congrats ;)

     

    @SpectateSwamp said:

    To do all the jobs in IT, that I had done to that point.

    OK, if you've made them all the same way as SSDS, I can understand. I can also understand the "old boss" thing.



  • SSDS forever Simple

    @Kilrah said:

    OK, if you've made them all the same way as SSDS, I can understand. I can also understand the "old boss" thing.

    I had many other successes in the past. I could tell you about what I did for Rural mapping in Alberta. The utilities dept needed their township maps digitized.

    Another was at Pacific Tel and Telegraph in SF just ask me what the solutions were. I'm trying to stay on topic. Sometimes they close threads for straying too far. It has happend to me. I was innocent

     



  • SSDS internals on Video - Not for the squeamish

    So That is where the next set of video is going. SSDS internals explained. I'll go through the code at the important points. Where and how changes can be made. I might even go over the changes required to make SSDS do a LIVE search of text files. (fiddle with the merge option to redirect the merge info to the logic where SSDS reads from the input file) It will be slower but that is what people here are screaming for. All on shaky shaky video. Sit close to your monitor for these lessons. When you got spaghetti code. You'll need a little help



  • @SpectateSwamp said:

    Your search finds the context so slowly it's almost useless (19 Seconds).
     

    You don't get it, do you?

    <font size="6">READ THIS:</font>

    I'll type slowly, maybe you'll get it if I adjust my typing speed to your reading capabilities.

    When. You. Have. Many. Small. Files: Copernic quick find (due to index), fast display (small files quicker to read), SSDS no find, no display (doesn't work with folders...)

    When. You. Have. ONE. Large. File: Copernic quick find (due to index), longer display (larger files take more time to load), SSDS do find, long read, quick display, which adds up to: SLOW display

    Actually Copernic is more than <font size="3"><font color="#ff0000">6 times faster than SSDS</font></font> even on the large file (according to your numbers)! Why? You did not count in the 1:20 minutes it takes SSDS to even READ the file. 

    So, just for your understanding, a little comparison:

    • Copernic searches the index (no information about how long that took, but I bet under 1 second). Then copernic tries to open the file to display the match. This may take some time for large files, but has nothing to do with Copernic, but the overall system's performance (e.g. faster harddisk -> shorter time). As it has an index, it finds all the files that were indexed!!
    • SSDS opens the file and reads it TWICE!! That takes 1:20 as you said. Then it finds the line in under one second (if you can trust the VB timing functions). SSDS doesn't have an index, so you have to already know the file in advance. Boooo!

    So if you want to do a real comparison: reboot your machine, search with Copernic and write down the time it took. Then reboot your machine again, write down start time, start SSDS, open file, search term, write down end time. THAT gives us the real numbers.

    What you did is like saying: "Oh, it took you ages to buy bread! Well, yeah, you had to look up a bakery in the phonebook first and then you had to go there, but I was quicker anyway. You know, I got out of the car parked right in front of the bakery and just bought the bread! You lose!" 

    Get it into your head once and for all: COPERNIC BEAT YOU! If you don't get why, re-read this post or do proper timing.



  • @tdittmar said:

    So if you want to do a real comparison: reboot your machine, search with Copernic and write down the time it took. Then reboot your machine again, write down start time, start SSDS, open file, search term, write down end time. THAT gives us the real numbers.
     

    Please do! I'd quite like to see the REAL speed comparison.



  • @SpectateSwamp said:

    @CodeSimian said:

    In the case of this thread, if I really wanted to archive it (which I don't), I would probably write a quick and dirty batch file to use wget to download each of the 30+ pages

     What use would that be. Your search finds the context so slowly it's almost useless (19 Seconds). Indexers can only deal with itsy bitsy files. Bigsy wigsy ones make them fall.

     I already said that I don't use indexed search and I don't merge all my (non-plain text data) into large plain text files.  In case you hadn't noticed, HTML is still a form of text, and can be searched by indexed and non-indexed search alike.

     

    @SpectateSwamp said:


    @CodeSimian said:

    am I supposed to merge all my data into one huge text file

     No never. I have about 20 main files. I don't have to merge them to find what I want. My inmail.txt is inmain, my outmail.txt is sent mail. NetStuff.txt and a few others. I seldom do merges. Unless I need a big file to Take Down Copernic or to satisfy some dumb forum questions. Merge is usefull in cleaning up directories and thus destroying FileSystems.

     

     Oh, so instead of merging my data into 1 big text file, I should merge it into 20 big text files.  That sounds much better.



  • @SpectateSwamp said:

    @CodeSimian said:

     In the video, you said that nobody needs more than a Grade 9 education.  Do you really believe that?

     To do all the jobs in IT, that I had done to that point. I could have had an apprentice with Grade 9 and taught them everything. Seems TOO high of an education can be a detriment to learning. Knowing it all, makes it hard to learn. I taught educated people how to do Cable Billing conversions. They knew nothing about cable or billing. Like my old boss said "It's not Rocket Science" It is mostly accounting and the old extract sort and report steps. The BEST programmer I came across in all my years had Grade 12 and no formal computer education. (That's not me)

    That's why I say even a non programmer can understand this program. Once they get a glimmer of what's going on. Then the unknown and fear dissipate.

     

    I didn't ask whether you need a Grade 9 education.  I asked whether you think anybody needs a Grade 9 education, in reference to your statement that "nobody needs more than Grade 9".  Just because someone is educated, doesn't mean they can't learn new things (e.g. cable and billing).  Just because someone is an expert at his job (e.g. cable and billing), doesn't mean he can't benefit from a greater education.

    Do you think your family doctor could have gotten by with only a Grade 9 education?  How about the people who designed your laptop or your camcorder.  Or the people who designed your truck?  Would you feel safe flying in an airplane designed by a high-school dropout? 



  • @SpectateSwamp said:

    I started out to include an email dump in SSDS but quit and moved on to other more productive stuff. There must be a good email to text dump program somewhere that I could use? If there isn't. Would you consider it your email or the applications email. If your applications can't do exports to text then there is a problem with that application. All Telco and Cable System billing records came as PLAIN TEXT files. From all types of machines to the VAX.

     

     You know, you almost have a good point about proprietary file formats.  The real problem is your "solution".  Nobody wants to export their data into plain text files - nobody.  

    I use the Thunderbird mail client which already stores its data in a well-known structured text format called mbox.  So, I am not worried about being unable to access my mail in 10 or 20 years, not that I have anything important to save.  When I want to quickly search my mail (for subject or sender), I simply type in the little search box in the upper-right hand corner of the app.  If I want a more powerful search, I press CTRL-F for a full-text boolean search.  

     Explain to me how manually copying and pasting my emails is more productive and easier.  If I recall correctly, you said that you forward your own received emails to yourself, just to make it easier to copy and paste the sender's name and email address.   Explain to me how this is smarter than keeping all your mail in the original format.

    Do you even understand why computer programs usually use a binary or structured text format as opposed to your vanilla plain text format?  Completely plain text with no structure, is NOT computer-parseable.  Once you dump all your email into "inmail.txt" and "outmail.txt", can you import it back into a mail client?  I didn't think so.  Once everything is plain text with no structure, you are losing many of the benefits of using a computer.  For example, if I keep all my emails in their original format, I can use Thunderbird to search by sender, recipient, subject, or body.  I can sort by any of those fields as well.  I can view my mail in flat mode or threaded mode.  If I were to dump everything into 1 or 2 plain text files, I would only be able to search for keywords.  Do you see how the original mail format is more powerful than unstructured plain text?

     And explain to me why I wouldn't just use a program like EverNote to organize and search my plain text and web notes.   Explain why people pay money to use EverNote but nobody cares about SSDS?

     So all your cable and telco billing records were in plain text.  So I guess none of your IT jobs ever involved a database?  You know, databases are NOT made of huge plain text files.



  • @tdittmar said:

    So if you want to do a real comparison: reboot your machine, search with Copernic and write down the time it took. Then reboot your machine again, write down start time, start SSDS, open file, search term, write down end time. THAT gives us the real numbers.

    I second (or third, or whatever we're up to) this motion.

    Swamp, please make a video of the above instructions. We might actually take the video seriously if you do a proper clean test like the above.



  • @SpectateSwamp said:

    I had many other successes in the past. I could tell you about what I did for Rural mapping in Alberta. The utilities dept needed their township maps digitized.

     

    Let me guess: you used your camcorder to "video in" the maps, created a plain text index, then gave them a copy of SSDS to search the index.  Right? 



  • @SpectateSwamp said:

    That's why I say even a non programmer can understand this program. Once they get a glimmer of what's going on. Then the unknown and fear dissipate.

     

     Nobody needs to understand how their search app works.  They just to need to be able to use it.  You don't need to be a programmer to use Google Desktop Search, just like:

    • You don't need to have a degree in medicine to visit your family doctor
    • You don't need to be an aerospace engineer to fly in a plane
    • You don't need to be a mechanic to drive a car

    On the other hand, to be a good doctor you need a degree in medicine, to design a plane you need a formal education in engineering and to build a desktop search application you just might need an education in Computer Science.

    The fact that no one else is using SSDS should have given you a clue.



  • @SpectateSwamp said:

    @stolen_username said:

    Also, I'd appreciate it if you would at least acknowledge my video request from earlier (show SSDS searching for a word at the bottom of an 800 MB file).

    I wasn't lying to you. To find a match near but not on the last page requires a search. If it was on the last page then I would do a "y" option that reads the file once and on the second pass starts the display 1 page before the end of the file. If you want to find something on the 2nd to last page. Then use the "s" search first It reads through finding matching lines only. (40 seconds to read and search the whole file) then the "p" option to display the previous match. It reads to a point 8 or 10 lines before the match and starts from there. Just as if searching from the beginning. The context search "c" is slower because it keeps changing the context lines on each read Adding character counts etc is done on "c" context searchs but not the 's' single line matching display. Create a program that opens a large text file and read to the end doing nothing to the data. I'm sure you will get speeds close to or faster than what I have.

    I actually tested this.  It took grep about 20 seconds, on average, to find a string at the bottom of an 800 MB file.  I wasn't able to get Meta Tracker to index that far into the file, so I guess you have some kind of victory there.  However, when I had it index my entire home directory (several gigabytes of data), I was able to search for files in less than a second.  If I had to search my home directory with SSDS, it would take at least 20 seconds to find everything.  Sequential search is slow when you have a lot of data.  If you're searching for something within a single file, it's not so bad, but when you're searching an entire desktop, it takes a long time.

    @SpectateSwamp said:

    @stolen_username said:

    SSDS lets you view the contents of text files.  It does nothing to help you find them. 
    With my files they are all in the c:\search\ folder or I don't care about them. But I can search any text or jpg or mpg anywhere on any drive. and have them flashing up on the screen in an instant. A quick merge of all text on c drive doesn't take long. But I have no call to do it. The search has a directory command and I sometimes search that file just to see what files got added today. (ie files that I didn't create) But not much need to look beyond my own folder.

    This is very different from how the people I know use their computers.  Most people have a lot of small files instead of a few big ones.  The reason you're seeing so much resistance here is that SSDS is only useful for you.



  • GDS Example

    For your amusement (animated GIF).

    Image Hosted by ImageShack.us

    That's out of 292,454 items it indexed (15,435 e-mails, 120 web history, and 276,899 files).

    And lookie! It has context!



  • @SpectateSwamp said:

    Just finished merging all the Text files on C 

    The output file was 522MB. Why merge every *.txt file on C drive? Because Swamp Search can. Like that's half a Gig. Next to nothing really.

     

    What steps did you use to deal with files in various sub folders? Equally what is the point of merging all those files when you already have the data in the original files? How about all the many non pure .txt files on the system? Other search tools manage without creating such a large pointless file and they manage without me actually having to do it. On my system I currently have just over 36,000 items indexed (various document formats including mp3 and jpeg) with a total index of 238M - what on earth is the point of your 522MB file?

    @SpectateSwamp said:

    On page 1 of this forum, click anywhere on the main screen. enter Ctrl/A to select all the data. And then Ctrl/C to put it in the clipboard.

    Alt tab to the other process running SSDS. At prompt #1 you will have entered thedailyWTF.txt so you are now at prompt #2 enter "z" to paste the contents of the clipboard at the end of thedailyWTF.txt file

    Alt tab to the daily WTF forum and move to page 2 Ctrl/A then Ctrl/C and back to the other to enter 'z' at prompt #2 and back and forth for 30 pages. This forum is like 2 Books long. If you want to find all the search facts. You need a good search. SSDS is it

    How about this alternate way...

    On page 1 of this thread go to the search box in the top right hand corner and type your search terms.

    Err, that is it really because that does this whole searching thing without the tedium of copy and pasting...

     


     


Log in to reply