Why Does Python's Os.walk Fail Silently?

I have the following: def crawl(rootdir): for dir, subdir, files in os.walk(rootdir): for file in files: print 'file found: %s' % file` In my understanding

Solution 1:

Have you read the documentation? Quoting:

"By default, errors from the listdir() call are ignored. If optional argument onerror is specified, it should be a function; it will be called with one argument, an OSError instance. It can report the error to continue with the walk, or raise the exception to abort the walk. Note that the filename is available as the filename attribute of the exception object."

It is trivial to understand the rationale behind it as well: it's much more flexible to choose what to do in case of errors, since listdir errors may not be treated as such by all applications.

Solution 2:

As stated in the os.walk documentation, you need to specify an onerror function that takes an OSError as an argument to get the error. Then you can choose what you want to do with it.

Solution 3:

The other answers here are literal, but I will add what I think the motivation is for this behavior.

Because os.walk returns a generator, it's reasonable for a generator to be empty. An empty generator is not necessarily cause for an exception (and if it is, then one can raise an exception by passing a function to the onerror parameter).

Yet, this reason doesn't answer the question completely because one could say that the default should be to throw an exception if the initial generator is empty. But, doing so would mean that the generator couldn't be created ahead of time from when the directory actually exists. Additionally, if the tree changes, the generator will simply find the new directories as it runs, even if they didn't exist when the generator was created.

Even with these reasons, I think an initial exception would make a lot of sense here since people likely want to walk a directory that already exists before the generator is created. This type of change would probably make the implementation less streamlined since the onerror parameter applies to any error along the walk and can all be treated identically.