Add %%lprun_all magic function (#383)

* Add %%lprun_all for easier profiling in IPython

* Add additional IPython tests

* Add -z option to lprun for compatibility

* Ensure the temporary file is always cleaned up

* Fix linting error

* ENH: `LineStats` aggregation (#380)

* Stat aggregation

line_profiler/__init__.py
    Now also importing `line_profiler.line_profiler.LineStats`

line_profiler/line_profiler.py[i]
    pathlib
        Removed unused import in stub file
    LineStats
        New `line_profiler._line_profiler.LineStats` subclass helping
        with utilization and aggregation, with the following added
        methods:
        - `__repr__()`
        - `__add__()`, `__iadd__()`
        - `print()`
        - `from_files()`, `to_file()`
        - `from_stats_objects()`
    LineProfiler
        get_stats()
            Now returning a `line_profiler.line_profiler.LineStats`
            object instead of a `line_profiler._line_profiler.LineStats`
        dump_stats(), print_stats()
            Now deferring to the respective `LineStats` methods
    load_stats()
        Now an alias to `LineStats.from_files()`
    main()
        Updated to be able to take multiple positional arguments

* Tests

tests/test_cli.py::test_multiple_lprof_files()
    New test for using `python -m line_profiler` with multiple `.lprof`
    files

tests/test_line_profiler.py::test_load_stats_files()
    New test for loading stats files (old and new, single and multiple)

* Speed up test

tests/test_cli.py::test_multiple_lprof_files()
    Sped up by avoiding spawning new Python processes

* CHANGELOG entry

* Fix lint

* Address review comments and add test

* Potentially fix Windows string parsing?

* `LineProfilerMagics` refactoring, coverage boost, other fixes (#6)

* Stub

* Consolidate common functionalities

- `line_profiler/ipython_extension.py[i]`
  - General:
    Added type comments for sanity checks
  - `_ParseParamResult`:
    New private class summarizing the result of argument-parsing
  - `_RunAndProfileResult`:
    New private class summarizing the result of calling a function with
    profiling
  - `_PatchProfilerIntoBuiltins`:
    New private helper context-manager class for monkey-patching a
    profiler into `builtins`
  - `LineProfilerMagics`:
    - `_parse_parameters()`:
      - Refactored from `parse_parameters()`, taking over more of the
        postprocessing boilerplate
      - Now returning a `_ParseParamResult`
    - `_run_and_profile()`:
      New private method to call a callable, profile and time the call,
      and handle exception-related boilerplate;
      returns a `_RunAndProfileResult`
    - `_lprun_get_top_level_profiled_code()`,
      `_lprun_all_get_rewritten_profiled_code()`:
      New private helper methods for handling the compilation of code in
      the `%%lprun_all -p` and `%%lprun_all` cases
    - `_handle_end()`:
      New private method to handle end-of-magic boilerplate (IO, return
      values)
    - `lprun()`:
      - Simplified implementation to use the above helper methods
      - Fixed erroneous error message when a function to be profiled is
        not found ("Could not find module [...]"
        -> "Could not find function [...]")
    - `lprun_all()`:
      - Simplified implementation to use the above helper methods
      - Fixed misuse of cell magic in docstring ("%lprun_all [...]"
        -> "%%lprun_all [...]")
      - Fixed edge case where the cell is blank
      - Now ensuring that the profiling output is formatted before
        deleting tempfiles

* More Sphinx-friendly docs

* Prettier output and scoping fix

- `line_profiler/ipython_extension.py`:
  - `_RunAndProfileResult`:
    - `tempfile`:
      New attribute
    - `output`:
      Updated implementation to allow for module- (i.e. top-) level
      profiling data (from the code object `.tempfile` compiles into) to
      be formatted correctly
    - `_make_show_func_wrapper()`:
      New private helper method for creating a monkey-patched clone of
      `line_profiler.line_profiler.show_func()` which implements the
      above
  - `_PatchProfilerIntoBuiltins`:
    Simplified implementation to use a `_PatchDict` internally
  - `_PatchDict`:
    New private comtext-manager helper class for monkey-patching
    arbitrary namespaces
  - `LineProfilerMagics`:
    - `_run_and_profile()`:
      Added argument `tempfile`, which is passed to
      `_RunAndProfileResult`
    - `_lprun_all_get_rewritten_profiled_code()`:
      No longer uses `SkipWrapper`
    - `lprun_all()`:
      - Removed nesting of the `cell` content into a mock function;
        the content is now directly written to the tempfile, compiled,
        and executed, allowing for prettier output
      - Fixed scoping of names: local names in `cell` are now correctly
        reflected in `IPython.core.getipython.get_ipython().user_ns`
- `tests/test_ipython.py::TestIPython`
  - `loops`, `lprun_all_cell_body`:
    Extracted common assets into class attributes
  - `test_lprun_all_*()`:
    - Removed line-number offsets induced by the nesting of the cell
      content into a mock function
    - Added checks that the cell content is executed in the correct
      scope

* Boost coverage

- `line_profiler/ipython_extension.py`
  - General:
    Added `# pragma: no cover` to type-checking only stuff
  - Removed the following unused code:
    - `SkipWrapper`
    - `_ParseParamResult.__getitem__()`
    - `_RunAndProfileResult.return_value`

- `tests/test_ipython.py`
  - `TestIPython`:
    Refactored into the classes `TestLPRun` and `TestLPRunAll`, which
    are no longer `unittest.TestCase` subclasses
  - `TestIPython.test_init()`:
    Refactored and parametrized into the following tests:
    - `test_lprun_profiling_targets()`:
      Test `-m`
    - `test_lprun_file_io()`:
      Test `-D` and `-T`
    - `test_lprun_timer_unit()`:
      Test `-u`
    - `test_lprun_skip_zero()`:
      Test `-s` and `-z`
    - `test_lprun_exception_handling()`:
      Test exception handling in the magics
  - `TestLPRunAll.test_lprun_all_*()`:
    Updated implementations

* Fix grammar in documentation

One space after a period is generally considered correct. Also, add a clarification that the -s option is meant to eventually be deprecated, and that -z is used for consistency with the CLI.

---------

Co-authored-by: Theelx <[email protected]>

* Hotfixes for failing tests on #383 (#7)

* Yet another hotfix (#8)

---------

Co-authored-by: Terence Tsang <[email protected]>
Co-authored-by: Jon Crall <[email protected]>

Author: 3 people

.gitignore

@@ -20,6 +20,7 @@ MANIFEST
 pypi-site-docs.zip
 index.html
 line_profiler/_line_profiler.html
+*.lprof
 
 .coverage
 tests/coverage.xml

CHANGELOG.rst

@@ -3,10 +3,12 @@ Changes
 
 5.0.1
 ~~~~~
-* FIX: ref-count leaks #372
+* ENH: Add %%lprun_all for more beginner-friendly profiling in IPython/Jupyter #383
 * FIX: mitigate speed regressions introduced in 5.0.0
 * ENH: Added capability to combine profiling data both programmatically (``LineStats.__add__()``) and via the CLI (``python -m line_profiler``) (#380, originally proposed in #219)
 * FIX: search function in online documentation
+* FIX: ref-count leaks #372
+* FIX: mitigate speed regressions introduced in 5.0.0 #376
 * FIX: Use import system to locate module file run by ``kernprof -m`` #389
 
 5.0.0
@@ -56,6 +58,7 @@ Changes
   * ``kernprof`` and ``python -m line_profiler`` CLI options
   * ``GlobalProfiler`` configurations, and
   * profiler output (e.g. ``LineProfiler.print_stats()``) formatting
+* ENH: Added capability to combine profiling data both programmatically (``LineStats.__add__()``) and via the CLI (``python -m line_profiler``) (#380, originally proposed in #219)
 
 4.2.0
 ~~~~~