Bug in FileSystemWatcher - .NET 2
-
Hi all.
Please excuse my english, as we speak spanish here.
I have a problem with FileSystemWatcher.
I had to monitor some directories, check for file changes... not big deal to figure out. I was about to build something using windows APIs, but my boss keeps on telling me to use NET and its controls.
I hate NET. I feel i have to trust in it, never knowing what the hell it's doing. I'm just suppose to implement a class wich miracously do the job, and i don't have to worry about anything; NET does all the dirty work for me. That sounds just so irresponsable to me... but... well... i used the damn filesystemwatcher... "maybe it's just me" i said... "this thing looks great"... and happened what i was fearing: one of those damn microsoft bugs.
My problem seems to be exactly the same as the one mentioned in this other forum:
http://www.thescripts.com/forum/thread460919.html
Also, here's another angry developer who had the same problem:
http://www.msnewsgroups.net/group/microsoft.public.dotnet.languages.csharp/topic7622.aspx
FileSystemWatcher fails to detect directory activity when, well, it feels like it's time to fail. Some times, just fails, even when you're not doing anything strange. Same thing somethimes works, sometimes just don't, without any apreciable enviromental change. It has feelings, you know.
As i can read in that first forum, FileSystemWatcher fails (just sometimes) when multiple files are dropped (confirmed here) to a directory, and a good idea seems to use it as a trigger for a robot to browse the whole directory.
That solution is crap in my case: i'm talking about thousands of files, and filesystemwatcher fails just sometimes, not everytime. That means: triggering anything on a directory activity detection is useful when failing; otherwise, i end up running multiple instances of, lets say, a browser robot.
Big directories, slow multiple robots for real-time requested information, or a robot running every X time (eliminating any real-time chance); FileSystemWatcher is not reliable.
I had problems to play with Active Directory; maybe it's time to take it more seriously. I'm also starting to think about an application to put files in those directories, wich sounds pathetic but effective.
Anyway, i choose to ask for some help here first. Maybe i'm missing something, i'm doing something wrong. Nothing weird on the code: just event triggers on a FileSystemWatcher control. Had anyone solved this problem? Does anybody knows an alternative to FileSystemWatcher?
I have already running this crap in production, and now have to fix it. I hate myself for ever using that disgusting filesystemshit. Any comment will be appreciated. Thanks in advance.CODE:
private void Form1_Load(object sender, EventArgs e)
{
this.Resize += Form1_Resized;
notifyIcon1.MouseClick += notifyIcon1_click;
watcher = new System.IO.FileSystemWatcher();
watcher.NotifyFilter = NotifyFilters.DirectoryName;
watcher.NotifyFilter = watcher.NotifyFilter | NotifyFilters.FileName;
watcher.NotifyFilter = watcher.NotifyFilter | NotifyFilters.Attributes;
watcher.Changed += log;
watcher.Deleted += log;
watcher.Created += log;
watcher.Renamed += new RenamedEventHandler(renombrado);
watcher.IncludeSubdirectories = true;
}private void log(Object source, System.IO.FileSystemEventArgs e)
{
//crap
String texty = e.ChangeType.ToString() + ":" + System.Environment.NewLine + e.FullPath + System.Environment.NewLine;
String Qry = "";
String temp = String.Empty;
SqlParameter[] arrParametro;
switch (e.ChangeType)
{
case WatcherChangeTypes.Created:
//Se creó un archivo. Guardo un registro.
Qry = "spAltaOriginal";
arrParametro = new SqlParameter[2];
arrParametro[0] = new SqlParameter("@Path", SqlDbType.VarChar);
arrParametro[0].Value = e.FullPath;
arrParametro[1] = new SqlParameter("@Propiedades", SqlDbType.VarChar);
while (String.IsNullOrEmpty(temp))
{
try
{
temp = Canta.Tools.getFileProperties(e.FullPath);
}
catch (ApplicationException p) { }
}
arrParametro[1].Value = temp;
SqlHelper.ExecuteDataset(cnn, CommandType.StoredProcedure, Qry, arrParametro);
break;
case WatcherChangeTypes.Changed:
//Se modificó un archivo.
Qry = "spModificacionOriginal";
arrParametro = new SqlParameter[2];
arrParametro[0] = new SqlParameter("@Path", SqlDbType.VarChar);
arrParametro[0].Value = e.FullPath;
arrParametro[1] = new SqlParameter("@Propiedades", SqlDbType.VarChar);
arrParametro[1].Value = Canta.Tools.getFileProperties(e.FullPath);
SqlHelper.ExecuteDataset(cnn, CommandType.StoredProcedure, Qry, arrParametro);
break;
case WatcherChangeTypes.Deleted:
//Se borró un archivo. Eliminarlo de la base.
Qry = "spBajaOriginal";
arrParametro = new SqlParameter[1];
arrParametro[0] = new SqlParameter("@Path", SqlDbType.VarChar);
arrParametro[0].Value = e.FullPath;
SqlHelper.ExecuteDataset(cnn, CommandType.StoredProcedure, Qry, arrParametro);
break;
case WatcherChangeTypes.Renamed:
break;
}
SetText(texty);
}private void renombrado(object source, RenamedEventArgs e)
{
//Se renombró un archivo. Modificarlo en la base.
String Qry = "spRenombrarOriginal";
String texty = e.ChangeType.ToString() + ":" + System.Environment.NewLine + e.OldFullPath + e.FullPath + System.Environment.NewLine;
SqlParameter[] arrParametro = new SqlParameter[3];
arrParametro[0] = new SqlParameter("@Path", SqlDbType.VarChar);
arrParametro[0].Value = e.OldFullPath;
arrParametro[1] = new SqlParameter("@Propiedades", SqlDbType.VarChar );
arrParametro[1].Value = Canta.Tools.getFileProperties(e.OldFullPath);
arrParametro[2] = new SqlParameter("@NuevoPath", SqlDbType.VarChar);
arrParametro[2].Value = e.FullPath;
SqlHelper.ExecuteDataset(cnn, CommandType.StoredProcedure, Qry, arrParametro);
SetText(texty);
}private void SetText(string texty)
{
//a text1.text lo tengo que establecer en un "thread-safe way"
if (this.textBox1.InvokeRequired)
{
SetTextCallback d = new SetTextCallback(SetText);
this.Invoke(d, new object[] { texty });
}
else
{
this.textBox1.Text += texty;
}
}
-
Seems weird i read one of the links you posted and they seemed to think it was a bug in a win32 API used by the file system watcher.
My suggestion would be mess around with the polling interval(if you can) or write your own file system watcher. I know writing your own defeats the whole using .net stuff but it would only take a day or so to write(assuming you just want basic functionality) so it might be the best option. Just have a separate thread that every n seconds checks the directory for changes and then fires an event. Not sure what the best way of checking changes would be without investigating it but maybe something like checking the datemodified's or sizes as well as ones that didn't exist in the last poll.
Just my 2 cents, you should post up how it turned out, i'd be interested to know.
-
Hi there, element[0].
Thanks for the reply! Beautiful horns, by the way!
Yap, the bug comes apparently from the Win32 API. I'm now figuring out how to implement something like an own filesystemwatcher. The problems is, indeed, the functionality.
Here's the scenario:
Some guys in production need some files to work with. The files are in a common shared repository; just a shared folder, nothing complex. They find, copy, and process the files every time they need it. They also make searches to confirm availability of files. They did everything with windows default tools.
Also, depending on some file properties, it affects a series of workflow states, and this information is shared with other deparments, like Administration.
I made a little robot to read the whole repository, get file properties, and save all that info in a database. I also register every file change and access into an activity table.
This way, it was easy to gave them new tools: a file search tool (this time, in a database), a lot of reports about the files, precise (computed) workflow states and real-time availability (also computed) of production documents (wich were, before this system, shared excel files), etc.
Filesystemwatcher was a key item in this advance. I was able to build a lot of things. Everyone was happy, and given the sucess of the implementation, there are now a lot of inmediate plans to extend the system.
The system is not working thanks to filesystemwatcher and what i can call my irresponsability; i SHOULD had learned something about the right win32 apis before using that #&#%#$ library. (sight)
However, i let this issue on hold because we had a more serious problem a few days ago with a server computer. That server problem also kept me away from reading in depth about the apis failing; just didn´t had the time. So i still don't have a solution for this.
I need to get real-time knowledge of the files in that repository. I could run scripts with timers, but that's not a true solution; eventually, as the repo is already big, there will be erroneous information in production and administration. A full scan takes about 20 minutes now, and it will get slower. There´s also not an option to use filesystemwatcher to trigger any script on the target directory (the repo has subdirectories), as when it fails it ignores entire dirs.
So, i face 3 options, as long as i can see, to get that functionality.
A - Disk activity monitoring. No idea how. Sounds complex and creepy.
B - Some other MS tool like ActiveDirectory. Making queries to ActiveDirectory should be the same as my SQL table. But i had trouble with AD in the past. I just ignored it that time. Looks like a nice try now, if i'm the mood to use MS tools again.
C - Making a tool to put files in the repo. Production will hate it, and so i will. But sounds like the most robust and inmediate solution right now. I already have the tools for getting the files... is just another step in production's everyday's work... an OK thing if it brings happiness... right?... I'm not liying to myself... right fellas?
Please, tell me making that kind of lame tool is ok; i'm about to cry.
I'll let you know, element[0], as soon as i get this fixed (if i make it, of course). I have to read about those APIs for file activity, disk activity, and about Active Directory, so it will take a few days. If i can't figure it out, or i fail to implement it, there will be a new program in production called something like "the uploader" and this history will end.
I'll also start to look for an icon for that program. Please, anybody, any comment, information, or icon will be nice. Thanks in advance.
-
@Xelort said:
So, i face 3 options, as long as i can see, to get that functionality.
A - Disk activity monitoring. No idea how. Sounds complex and creepy.
B - Some other MS tool like ActiveDirectory. Making queries to ActiveDirectory should be the same as my SQL table. But i had trouble with AD in the past. I just ignored it that time. Looks like a nice try now, if i'm the mood to use MS tools again.
C - Making a tool to put files in the repo. Production will hate it, and so i will. But sounds like the most robust and inmediate solution right now. I already have the tools for getting the files... is just another step in production's everyday's work... an OK thing if it brings happiness... right?... I'm not liying to myself... right fellas?
Please, tell me making that kind of lame tool is ok; i'm about to cry.
Unfortunately i totally agree with your assessment here,
A. does seem creepy and scary and i'm guessing most of the win api's you would use would be the same ones as the file system watcher uses so i wouldn't be surprised if you ended up hitting the same bug that messes with the file system watcher.
B. Yep active directory, promises the world but never quite seems to get there, i've had to do a few things with it from .net before and i always found it unnecessarily complicated, i find i can usually figure things out just by looking at method name and return types but i found activedirectory a bit of a stuff around. It might work, i'm assuming your compnay/client already has a licence/server running it.
C. The quickest and easiest but unfortunate. I'm assuming you don't want to do this because it will interupt the flow of how users already interact with the files meaning training and supporting an extra app(even if it is tiny and simple there will still be support requests). It would certainly be the easiest though(if not the neatest) because then you have total control over whats happening and don't need to rely on some on, half-assed win32API.D. There is another somewhat hacky option but it is one of those things that is slightly lame but might get you round having to do C. This is a hack and i'm kind of ashamed of metioning it but it might work. How long does a dir take on the big directory? i'm assuming it would be faster than gathering the information using the .net libraries(in linux an ls -a + some grepping might be ok but a similar thing in windows just feels wrong for some reason), you could dir every n seconds and do a comparison with the data from the previous poll, it seems like a kind of lame solution but i think it would work.
E. You could look at using something like webDav(i think that's what its called) it's what sharepoint uses, it basically is a file system stored in SQL server which can be accessed as a file system thorugh some sort of network share, one advantage of this is that it already stores a bunch of additional info about the file, ie. checkins, check outs etc. but i'm guessing you're too far in now to make a change of that magnitude.
I think i get your position though, you've said it will take x amount of time, which it did but you've been held up by a bug in a third party product and now you've spent way too much time on the project and want to be moving onto something else but this bug you have no control over is holding everything up and making you look bad.
C. is the quick and dirty if that's what you need, the other solutions would take a bit of time and research which i'm guessing you don't have, so unfortunately it might be the best option. If you wanted to get fancy with it check out Visual Studio + Tools for Office, you could write a custom save dialogue and toolbar button to make it a little neater for the end users and then it wouldn't actually be too bad a solution
-
Before writing a file system scanner yourself, as a last attempt you can try this class, http://www.codeproject.com/file/directorychangewatcher.asp which is referenced from the win32 API documentation. If this has the same problems, then you know that there is no working API and you'll have to do expensive disk scanning. If this does work as expected, then the bug is only within the .NET class and you can work around it by calling the API directly.
Good luck.
-
I don't see in the code you provided that you are increasing the buffer size on the watcher object. The default is 4K and if you dump 1,000 of files into a directory you are going to lose events. If you are watching network drives the watcher may disconnect due to network issues. My understanding is that you can trap that by wiring up the OnError event and reconnect your watchers (recreate them and then start them).
I've tested our watcher by dropping 10,000 files into a directory and never lost one event (with 128K buffer).
Also you are performing a database call in your event handler which will slow down performance of the watcher event mechanism. You should queue up these notifications and then run your database call off of the queue and not the event handler.
The FileSystemWatcher class works when configured correctly.
-
bmoneil, thank you very much for your post.
You are right: i never configured filesystemwatcher's buffer size. And, actually, the behaviour of the error matchs that possible mistake: some guy drop files in the directory, FSW detects a few file operations, it stops detecting at some "random" point, and fails detecting the rest of the files.
I will try the OnError practice tomorrow. However, i do not understand how is it that recreating the watchers will detect the missing events. Aren't those events already lost? I mean, between the error and any FSW restart, the files are still flying away... you think i can detect that? Sorry if i'm missing something.
I do understand that restarting a FSW is not time consuming, but i mean: losing a file is really a pain in the ass, as everyone's trying to use it. So, it brings me back to scan the directory if i miss any single file.
Yes, i do know about the database calls. To be honest, i just didn't cared at the time, as it was experimental stuff. When i did let the thing in production, i simply forgot about that. However, i did test the issue without the database calls, and it showed no improvement. Also, it is important to note that this error is not happening always, but just sometimes: i can drop X ammount of files on my directory, and FSW may or may not work. That's the behaviour. Nasty.
I also figured out it could be some network issue, so i tested the program on the server machine, without success. So, i guess network issues are out of the problem.
FYI, i did patch the system with a lame tool as i described in my previous posts, so i do know now when someone puts a file in the repo, in real time, wich was the main problem. I still has the issue of detecting when someone deletes a file, in order to the database do not save erroneous information. I was about to solve this issue with a brute force schedulled scaner (another lame tool), so i could, even when pathetically implemented, forget about fixing this problem (wich has proven to be time consuming).
For the sake of elegance, i tried another clever solution: to mark the repo as a shell folder, so i could alter its behaviour, and therefore detect anything everybody do in there. I could also, eventually, implement some restrictions, extra functionalities... it was a nice alternative to a FSW.
But, again, i had to fix the problem quickly, and the shell folder idea finished when i saw the ammount of data i had to understand before implementing any serious shell extension, and i was advancing really slow in that field.
On the other hand, i really liked this solution and will implement it as soon as i can.About your phrase, "The FileSystemWatcher class works when configured correctly"... well... to be fair, i know shit about how does FSW works, so i can't technically blame it, and i was really pissed off about the thing failing, so i never was objective when analizing it. Therefore, i think you have a point.
But there are still hard feelings between FSW and i. She broke my heart, you know. That's just not nice.I'll do experiments as you described here, and will post the results. Thanks again man.
-
it would seem like this is a fairly common activity... You should be able to "sniff" disk reads and writes (task manager does it if you ask it to). Is the issue file locking? if someone is accessing a file as read/write you don't want anyone else to access it, but if someone accesses it read only other people can access it at the same time?
if something like this is the case (i dunno why you'd want to bring a database into this), there are many filesystems out there, including database file systems, journaling file systems, etc. I'm just wondering if a simple filesystem switch (from NTFS to JFS for example) would alleviate a lot of your problems. Windows really really sucks for file system management by default, assuming you have 2003 enterprise with everything set correctly this shouldn't be an issue, but in all honesty, there's free solutions out there that you can use for a single drive on a system that will give you the monitoring and flexibility you need in this circumstance.
i'm trying to think if there's third party stuff you could install on all the workstations that would be able to signal when they opened a file as read/write, created a new file, or deleted an existing one. This seems like it would be a better solution than trying to monitor a directory and update a database. Do the clients access the database to determine what files are available? if so, that seems enterprisey and wasteful. even if they access your software which monitors the directory to ask IT what files are available and your software queries the database... still, it's a middleman, and is unnecessary.
i don't think anything as severe as disk scanning is needed. you just need a way to hook into filesystem calls, worst case scenario. All things considered, you're really better off switching filesystems, or switching the way you access the data. Concurrent updates to a file are likely to cause a collision, which is why a database-based filesystem would be best (databases are designed to handle concurrent/simultaneous updates).I'm not an it professional, more of a sysadmin, so hence my approach is one of "tweak system to do what you want" as opposed to "tweak applications to behave"
so just my two pesos.
-
there's a reason these NAS storage devices cost 27 thousand american dollars for 2TB of storage, i guess. I dunno if a dinky call to "FSW" is gunna replace the raw power and flexibility of a real storage solution. Best of luck to you!
<blockquote>
- All writes are communicated in time order from source to target, guaranteeing a consistent view of the state of user data at the remote site, at any point in time. (Note: Asynchronous mirroring does not guarantee that the state of the local and remote sites is identical or changed simultaneously. It provides only that each site is consistent to allow for target file system promotion at any time, and it updates in the same sequential order.)
- When promoted to source (or master), the remote system always provides access to every in-order transaction it received from the original source, thereby providing the optimal architecture for data preservation in an asynchronous mirroring model.
- Preserves write ordering, ensuring that files remain consistent on the mirror target, and ensuring that up-to-the-second updates are available should mirror promotion occur.
- Individual Sun StorageTek 5000 NAS volumes can be mirrored from any system to any other Sun StorageTek 5000 NAS family system, giving you lots of flexible and cost-effective options for assuring data availability.
- Fast reversion from source to target for maximum data availability and protection.
- Network (Ethernet) based mirroring lowers costs and lets your users stick with their familiar and favorite technologies, without the need for expensive gateways.</blockquote>
-
Hi there GeneWitch. Thanks for your post.
"i'm trying to think if there's third party stuff you could install on all the workstations that would be able to signal when they opened a file as read/write, created a new file, or deleted an existing one. This seems like it would be a better solution than trying to monitor a directory and update a database. Do the clients access the database to determine what files are available? if so, that seems enterprisey and wasteful. even if they access your software which monitors the directory to ask IT what files are available and your software queries the database... still, it's a middleman, and is unnecessary."
As i understand it, a File System is, in fact, a Database of files. If seen that way, using another database to miror that data is far from being a good idea. However, i do not have (or know, maybe, if possible) the tools to make queries to the file system without expensively looping. That's where another database engine comes in.
As it was discussed in previous posts, there was the idea of using Active Directory, wich seems to be something like "the database of the file system". I could never make it work.
There was also the idea of implementing the repository as a shell forlder in the workstations. That way, i can change its behaviour and, therefore, know its activity. I failed on that too.
FileSystemWatcher is a control included with Visual Studio. Another good idea: just tell it your directory, and it will do all the dirty work. Cool. But didn't worked. It's the same aproach as hooking the file operations; that's what FSW does.
You bring another good idea: change to a another File System. Is there any other File System wich let me make real time queries to it? That's what i'm looking for.
An alternative to FSW (that is, what you're wondering about in the quote) was the question that originated the entire thread.
I implemented the tips bmoneil gave me, and i'm now testing it. If there's any error, even with the FSW parameters changed, i will let you all know. Otherwise, if it was just a FSW buffer issue and it's already fixed, there's no need for any other workaround.Thanks for your time man.
-
Hi Xelort!
I considered similar problem as you have mentioned.
The FSW event happens sometimes, but sometimes not. Everything seemed fine from code but the result was "random".
I had to check certain xml files within some directories, but if the files were copied too fast to the dir., the created event happend only for some files. I've tried to extend the size of InternalBuffer, and register the error event, but the same happened.
But now something came to my mind that I had to try out.. Inside the created event there were 3 deep method call, with Regex, XMLDocument handling, and IO operation.
However I tried to aspire to the most efficient way of solving the problem, it seems it wasn't enough. So I comment everything from created event, and declared a static int counter variable to see, whether the created event happens correctly, or not - with one atomic operation. And it does. I've tried with more the 300 files (total size less then 0.5Mb) and I got the correct number of copied to the directory.So I assume the correct function of the FSW depends on, how complex is your operations inside the events. /I'm not using threads, and don't know how it effects the problem/
Maybe the solution is that, the event only drops the underlying file path into a queue, and other process this queue. It's an asynchronous solution, but it's ok for me..
That's my next step.
Best Regards, Jaccso
-
Hi there people.
Long time since this issue. I wanna be a good boy, so i must post how this problem was solved.
My tests with FSW were all eventually failed. Months passed and erros were detected when everything was supossed to work well. Tried different methods i don't even remember now.
It all ended when implemented Index Service + IXSSO. Works fine, works fast, and requires minimal (if any) maintenace. I've sacrificed a few funcionalities (can't trigger events), but my file database is solid. Case closed.
FileSystem Watcher control is more complex than it says to be. It requires an advanced understanding of its behaviour, that i have not. As long as we're talking about key processes, i can't recommend FSW for anyone without that previous knowledge or a lot of time to experiment. If you're looking for a way of fast search files, go take a look at Index Server. Also, use it with IXSSO instead of OLEDB; the research is minimal and worth it.
That's it, i'm a good boy now. I demand my cookie.
