@tster said:Because with perl you can do what you need to do for free in about 3 minutes. Honestly, at this point if the OP could post some example data somewhere I would write a perl script that takes two files and produces a diff. Just let me know what the key is. Time and space will be O(n) where n is the number of records. As far as worrying about data getting larger in the future. I'm assuming this 900,000 records is a lot of times worth of data. Assuming they add perhaps 1000 records per day, that means it will take almost 3 years to double the size of the data. At that rate I think you can just buy more memory to keep up with the demand. And you don't need a dedicated machine. Hell, run it on your dev machine at night.I agree that writing your own app would be quick and much cheaper and easier than buying some 3rd party program. If it is available, perl is definitely a better choice for the OP as it is faster than PHP at large-scale text file processing. If the unique ID is an auto-incrementing int, be sure you leave a blank line whenever you encounter a missing row. Otherwise diff isn't going to be able to handle large stretches of deleted files because it only reads ahead so far for a matching row. It sounds like you're pretty much on-target for what the OP needs, though.