Commits


pengwa authored and GitHub committed 65b316a138e
Consolidate ORTModule logging (#16078) ### Consolidate ORTModule logging There are few improvements for ORTModule loggings: - All ORTModule logging are used logger that is initialized in `ortmodule.py`. - Manage all export logs same way, e.g. use ` _logger.suppress_os_stream_output(log_level=self._debug_options.logging.log_level)` to control exporting related logs suppressing or not. If any warning or errors suppressed, `self._warning_log_detected_during_export` will be set to True, then when we log ORTModule feature matrix, we will also told users there are logs suppressed. - Downgrade some warnings. We had some warnings for years, and looks many models have them by default, no action we actually can take, so downgrade them to make user logging cleaner. - PyTorch export requires update of custom export function signature changes, otherwise, _symbolic_context_handler complains with warnings, so update custom export function adaption for version >=1.13 PyTorch. - Add ORTModule feature matrix summary, **this is supposed to be only places users see our logs by default** (unless they use INFO or VERBOSE). Features ON/OFF states are shown clearly to them in case they want to try some features in OFF states. This logs only shows up in rank 0 (if there are multiple rank), the intention is we want user to see a useful and clean output from ORTModule by default. The outputs shown as below:   - `reinitialize_ortmodule` in util.py is only used by ortmodule.py, moving it into ortmodule.py, then utils takes no dependency on `orttraining/orttraining/python/training/ortmodule/_custom_op_symbolic_registry.py`, then `_custom_op_symbolic_registry.py` can call functions defined in utils.py (without recursively include).