Repository navigation
Improve instantiate error message in case where _target_ cannot be loaded - #1864
Conversation
| obj = import_module(mod) | ||
| except Exception as e: | ||
| if n == 0: | ||
| if n == 1: |
There was a problem hiding this comment.
What do you think of:
for n in range(len(parts), 0, -1):
mod = ".".join(parts[:n])
try:
obj = import_module(mod)
except Exception as e:
continue
break
else:
raise ImportError(f"Error loading module '{path}'") from e- The
reversedstatement can be incorporated inrange - The real
trystatement is aroundimport_module; it's not intended for".".join(parts[:n])to raise an exception (that would be a bug). So you can move themod = ...before thetry. - You can replace
if n == 1:with the use of thefor/elsestatement https://book.pythontips.com/en/latest/for_-_else.html#else-clause - If you replace the
if n == 1:withfor/elseyou can even replacen in ... mod = ...with:
for mod in (
".".join(parts[:n]) for n in range(len(parts), 0, -1)
):Just commenting because I like the code puzzle :), feel free to ignore it if you want.
Edit:
Ah apparently the exception e gets cleared when exiting the except: https://docs.python.org/3.3/reference/compound_stmts.html#the-try-statement, so then in the else you have to do something like:
except Exception as e:
_e = e
...
else:
raise ImportError(f"Error loading module '{path}'") from _eBut I think that's a bit ugly 😅
There was a problem hiding this comment.
Thanks for your review @robogast!
Edit: Ah apparently the exception
egets cleared when exiting the except
I ran into this same issue when I was working on the PR 😆
There was a problem hiding this comment.
1 & 4. Yes, but I think that reversed is slightly easier to understand for future readers -- There is already a lot going on with the indices here :)
2. Agreed! Updated in 5de3754.
3. I wonder why python clears the exception e after the except clause... Too bad :P
Previously (before this PR), an ImportError was raised if a submodule mod_a.mod_b could not be found and module and also mod_a has no attribute named `mod_b`. An earlier commit in this PR changed this ImportError behavior to raise an AttributeError instead. This commit reverts that behavior -- now once again an ImportError is raised.
| if n == 0: | ||
| raise ImportError(f"Error loading module '{path}'") from e | ||
| obj = import_module(mod) | ||
| except Exception as exc_import: |
There was a problem hiding this comment.
Also; as far as I can find, import_module should only throw an ImportError, so maybe change the generic Exception to ImportError?
There was a problem hiding this comment.
This except clause also catches errors raised by the module that's being imported (e.g. if the imported module contains a line that says assert False).
Another option here would be to just catch ImportError and let the exceptions from mod bubble up to the user, but that would be potentially breaking change to Hydra's API.
| import_module(mod) | ||
| except ModuleNotFoundError: | ||
| pass | ||
| except Exception as exc_import: |
There was a problem hiding this comment.
Same comment as above, and as ModuleNotFoundError is already handled above, catching the more generic ImportError here should be OK, as import_module should only throw an ImportError.
There was a problem hiding this comment.
Same comment as above -- exceptions raised by mod itself (rather than by import_module) are caught here.
jieru-hu
left a comment
There was a problem hiding this comment.
lgtm! thanks for improving this. only one small nit.
| param( | ||
| "datetime", | ||
| raises( | ||
| ValueError, | ||
| match=re.escape("Invalid type (<class 'module'>) found for datetime"), | ||
| ), | ||
| id="top_level_module", | ||
| ), |
There was a problem hiding this comment.
nice - we can probably remove/update the comment in line 600 in hydra/_internal/utils.py
(github won't let me comment on that line)
This PR improves the error handling of
hydra.utils.instantiatefor the casewhere
_target_cannot be properly resolved.Closes #1863
Below is a summary of the changed error handling (with tracebacks condensed).
The case where a module cannot be found:
Before:
Above, note the confusing
ValueError: Empty module name.After:
The case where an attribute cannot be found on an object:
Before:
Note above the misleading
ModuleNotFoundError: No module named 'pickle.dumps'; 'pickle' is not a package.After:
The case where an attribute cannot be found on a module (and no submodule can be found with the correct name):
Before:
After:
The case where a module contains a syntax error or other error (as in #1863)
Before:
After: