|
|
Question : Monitor recursive a directory for new files
|
|
Hi, I need to monitor recursively a directory for new created files. I am using the File:Monitor package. (http://search.cpan.org/~andya/File-Monitor-0.10/lib/File/Monitor.pm) which works quite good. The problem is the performance issues. The directory and its subdirectories contain around 600'000 files. One recursive scan by file::monitor takes around 20minutes. I need this to be done within max. 5minutes. Is there any better "magical" way to monitor a directory for the creation of new files ? Many thanks
|
Answer : Monitor recursive a directory for new files
|
|
So each directory gets a new file every time? If not, the above method would still help, as you wouldn't have to readdir for those that didn't get any files. But if most or all directories do get a file, I don't know of a way to speed up the directory reading.
It may be time to look for another approach.
One possibility: have your application put the files in some other directory. Have the perl application look to that directory for it's files, process them and move them to the final directory. eg: you want the files in /final/dir, have the application put them in /tmp/dir, have perl monitor /tmp/dir for new files. For each file it find, process it to the database, then move it to /final/dir.
Another possibility: Have the other application update the database directly.
Another possibility: Have the other application create a list of files it creates everytime it creates a file. Then have the perl program just check this list.
|
|
|
|
|