Skip to content

changelog.md

Latest commit

 

History

History
600 lines (364 loc) · 17.8 KB

changelog.md

File metadata and controls

600 lines (364 loc) · 17.8 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

[2.9.1] - 31-July-2026

Fixed

[2.9.0] - 19-April-2026

Highlights

  • Added --check-max-tb-line=<int> to control one-line traceback output for additional failures.
  • Traceback output now supports two tiers:
    • full pseudo-tracebacks for early failures (--check-max-tb)
    • one-line traceback output for later failures (--check-max-tb-line)
  • Improved one-line traceback output for context-manager checks:
    • points to the failing line inside with check: blocks
    • includes exception type and message when available

Migration Notes

  • If you already use --check-max-tb, behavior is unchanged unless you also set --check-max-tb-line.
  • To keep output compact while preserving useful location info, consider:
    • --check-max-tb=1 --check-max-tb-line=10

Added

  • added type testing with ty
    • now testing with both mypy and ty, may reduce that to just ty in the future
  • --check-max-tb-line=<int> option to enable one-line traceback formatting for additional check failures.

Changed

  • traceback reporting can now be split by threshold:
    • failures up to --check-max-tb use full pseudo-tracebacks.
    • failures above that and up to --check-max-tb-line use one-line tracebacks.

[2.8.0] - 2026-Mar-22

Added

  • Support Request #122: Feature proposal: Capability to mark checks as xfail
  • Allow check helper functions, including raises(), to accept an xfail reason. A failing check can now produce an xfailed test result, while a passing check does not produce xpass unless the test is already marked with @pytest.mark.xfail.
  • (docs) Document xfail support for check helper functions in the README.

[2.7.7] - 2026-Mar-20

Fixed

  • Allow CheckContextManager to catch any exception, not just AssertionError (#198). Non-assertion exceptions (IndexError, KeyError, AttributeError, etc.) are now properly caught and reported as check failures.

[2.7.6] - 2026-Mar-04

Fixed

  • (typing) Set comparison function types such that None is not allowed

[2.7.5] - 2026-Mar-04

Changed

  • Passing None to comparison functions like greater(), less(), etc. used to raise an exception, which caused a test failure and stopped the test.
  • That's unexpected with a check function.
  • So now the check fails with a meaningful error message and continues with the rest of the test.
  • (typing) Allow numpy floats with comparison functions to pass mypy.

[2.7.4] - 2026-Mar-03

Fixed

  • (typing) Allow mixed int/float arguments in strict mypy for comparison helpers by adding float numeric overloads for greater(), greater_equal(), less(), and less_equal() in check_functions.py.

[2.7.3] - 2026-Mar-02

Added

  • Add __init__.pyi stub file for IDE intellisense (e.g. VSCode) and static type checkers.
  • Add PEP 561 py.typed marker so the package is recognized as typed.
  • (examples) Add test_example_mypy.py to validate import pytest_check as check with mypy.

Fixed

  • Ignore dist-packages in pseudo stack trace - Fixes #196 - Thanks @siferati
  • (typing) Resolve remaining mypy errors for import pytest_check as check usage; internal adjustments in check_functions.py.

Changed

  • (ci) Rename "quality" job to "static-analysis"; split into separate jobs for static-analysis, pytest_earliest, and coverage; use Python 3.14 for quality jobs.
  • (tox) Add mypy env coverage for import-as style in examples.

[2.7.2] - 2026-Feb-27

Fixed

  • #184: Fix issues with mypy. The example in this issue now passes mypy. Also mypy is now being tested as part of tox.
  • (typing) Allow dynamic helper access on CheckContextManager for static analysis, so patterns like check.equal(...) type-check in fixture-annotated tests.

Changed

  • (typing) Update internal _Comparable* protocol comparison signatures to satisfy strict mypy checks.
  • (tox) Enable and expand mypy env checks to include examples/.
  • (tox) Install httpx in the mypy env so example type checking works without import ignores.
  • (examples) Switch decorator usage in test_example_check_func_decorator.py to import check_func directly for static type checking.
  • (examples) Add test_example_context_manager_type.py to document and validate context-manager fixture typing behavior.
  • (ci) Add a GitHub Actions quality job to run tox -e lint,mypy,pytest_earliest,coverage.

[2.7.1] - 2026-Feb-27

Added

  • (docs) Document custom message support (msg) in check.raises().
  • (tests) Add tests for custom messages in check.raises().
  • (docs) Document that built-in check methods return bool values.
  • (docs) Update pseudo-traceback examples to reflect the default single pseudo-traceback and show how to increase it with --check-max-tb=5.
  • (docs) Add check.call_on_fail() section with usage guidance.
  • (examples) Add runnable logging example in examples/logging_to_a_file/.

Fixed

  • (tests) Make test_raises_custom_msg compatible with pytest < 7.3 short-summary output.

[2.7.0] - 2026-Feb-24

Added

  • Support for raises parameter in @pytest.mark.xfail when using check context managers - Fixes #145 - Thanks @shairoth12

Fixed

  • Fix #190: pyright errors
  • Fix #145: Is the 'raises' parameter in xfail ignored? - not anymore

Changed

  • Internal: Add fixture run_example_test in tests - Thanks @even-even
  • Update all to include 'check' in pytest_check module - Fixes #190 - Thanks @taylermulligan

[2.6.2] - 2025-Nov-28

Fixed

  • Greater/less_equal support numpy scalars #189 - Thanks @dpinol

Changed

  • (internal) Updated publish pipeline: repository_url -> repository-url

[2.6.0] - 2025-Oct-6

Added

  • raises returns the exception value, so the expected error message can be verified. - Thanks @shairoth12

[2.5.4] - 2025-Sept-14

Changed

  • raises & any_failures exported from top init.py module so that mypy does not complain that they are not explicitly exported. - Thanks @dpinol
  • Make _Comparable* protocols compatible with builtins - Thanks @SebastianGrans
  • Turn on testing for Python 3.14 in CI

[2.5.3] - 2025-Apr-3

Changed

  • Relative paths in tracebacks
    • PR #179, thanks @dpinol
  • Add testing of Python 3.14

[2.5.2] - 2025-Mar-29

Fixed

  • Fixes a weird corner case where an internal error could be generated by (1) having a session scope fixture fail in teardown and (2) having the last test in the session be skipped. Thanks @asyash26 for reporting and creating reproduction steps. Resolves #170

[2.5.1] - 2025-Mar-18

Fixed

  • Set a minimum pluggy version during type checking. Resolves #176, thanks @d-ryzhykau

[2.5.0] - 2025-Feb-12

Changed

  • Add type hints, remove support for Python 3.8, resolves #163, thanks @mfulgo

Fixed

  • Fixes #171, check doesn't respect runxfail option, thanks @taylermulligan

[2.4.2] - 2025-Feb-8

Changed

  • Add testing for Python 3.13 to GH actions
  • Add testing for Python 3.13 and 3.14 to tox.ini
  • Switch packaging from flit to hatchling

[2.4.1] - 2024-Aug-27

Added

  • is_nan()
  • is_not_nan()

[2.4.0] - 2024-Aug-27

Changed

  • Drop Python 3.7
  • Add testing for Python 3.12
  • Add tox-uv to tox.ini and main.yml

[2.3.1] - 2024-Jan-18

Changed

  • Documentation Update, README.md
    • Turn help function list into a table with param lists
    • Show an alternative method of creating a helper function using check.fail()

[2.3.0] - 2024-Jan-17

Added

  • between_equal(b, a, c)
    • as a shortcut for between(b, a, c, ge=True, le=True)
  • fail(msg) - indicate test failure, but don't stop testing

[2.2.5] - 2024-Jan-17

  • Fix 155
    • Summaries from 2.2.3 are cool, but don't work with xdist
    • Turn off summaries while xdist is running
    • I hope I'm not seeing a pattern here

[2.2.4] - 2024-Jan-8

  • Fix 153
    • Summaries from 2.2.3 are cool, but don't work for pytest < 7.3
    • Remove them for earlier pytest
    • Add tox test run for pytest 7.0.0
    • Change dependencies to require 7.0.0 pytest

[2.2.3] - 2023-Dec-31

[2.2.2] - 2023-Sept-22

Fixed

[2.2.1] - 2023-Aug-11

Changed

  • Increase Python range to include 3.7.0. Thanks EliahKagan

[2.2.0] - 2023-July-14

Added

  • Pseudo traceback additions
  • -l or --showlocals shows locals
  • __tracebackhide__ = True is honored.
  • if assert or other exception is involved,
    • the exception is included as part of the traceback.

Changed

  • pseudo traceback changes
  • The red color is used more selectively.
    • this is intended to help readability
  • Other minor formatting changes.
    • Please let me know if you have any issues with the changes

[2.1.5] - 2023-June-6

Fixed

  • Fix 127 IndexError when running a check in a thread - Thanks fperrin

[2.1.4] - 2023-Feb-13

Added

  • Include tests and examples in sdist - PR 121

[2.1.3] - 2023-Feb-9

Added

  • Publish-to-pypi.yml workflow

[2.1.2] - 2023-Jan-13

Changed

  • README.md - clean up documentation for --check-max-tb. Thanks alblasco.
  • Minor internal cleanup - removed some debug code.

[2.1.1] - 2023-Jan-12

Changed

  • check.call_on_fail(func) - Experimental
    • Experimental feature - may change with any release, even a minor release
    • Name changed from check.set_failure_callback(func).
    • Also, I warned you I could change that at any time.
      • No tomatoes thrown, please.
      • It's better, right? Thanks to Hynek for the suggestion.

[2.1.0] - 2023-Jan-10

Added

  • check.set_failure_callback(func) - Experimental
    • Experimental feature - may change with any release, even a minor release
    • Function to allow a callback function to be registered.
    • This "func" function will be called for each failed check, and needs to accept a string parameter.
    • Examples:
      • print : check.set_failure_callback(print)
        • allows stdout printing while test is running (provided -s is used).
      • logging.error : check.set_failure_callback(logging.error)
        • failure reports to logging
    • See examples/test_example_logging.py for runnable examples

[2.0.0] - 2023-Jan-8

Added

  • With the following change, the default is now pretty darn fast, and most people will not need to modify any settings to get most of the speed improvements.
  • --check-max-tb=<int> - sets the max number of pseudo-traceback reports per test function.
    • Default is 1.
    • After this, error is still reported
      • The error reports continue, they just won't have a traceback.
      • If you set --check-max-report, then the reports stop at that number, with or without tracebacks.a
  • check.set_max_tb(int) - overrides --check-max-tb for the test function it's called from. Value is reset after one test function.

Deprecated

  • check.set_no_tb and --set-no-tb will be removed in a future release. (probably soon)
  • check.set_no_tb is deprecated.
    • For now, it internally calls set_max_tb(0). See discussion below.
  • --check-no-tb is deprecated.
    • It's also short lived.
    • Since --check-max-tb is more useful, the default for --check-max-tb is 1, which is already pretty fast. And --check-max-tb=0 has the same effect as --check-no-tb.

Changed

  • PR 109. Update README.md with conda install instructions. Thanks Abel Aoun.

Reason for major version bump

The default behavior has changed. Since --check-max-tb=1 is the default, the default behavior is now:

  • Show traceback for the first failure per test. (controlled by --check-max-tb, default is 1)
  • Show error report for the remaining failures in a test. (controlled by --check-max-report, default is None)

Old default behavior was the same except all tracebacks would be reported.

My logic here:

  • The actual error message, like check 1 == 3, is more useful than the traceback.
  • The first traceback is still useful to point you to the test file, line number, of first failure.
  • Other failures are probably close by.

If this logic doesn't hold for you, you can always set --check-max-tb=1000 or some other large number.

[1.3.0] - 2022-Dec-2

  • Most changes are related to speedup improvements.
    • use --check-no-tb --check-max-report=10 to get super zippy.
  • check.between() added just for fun

Changed

  • Rewrote check.equal() and other helper functions to not use assert.
  • Check functions report things like FAILURE: check 1 < 2 now.
    • It used to say FAILURE: assert 1 < 2, but that's not really true anymore.

Added

  • --check-no-tb - turn off tracebacks
  • --check-max-report - limit reported failures per test function
  • --check-max-fail - limit failures per test function
  • check.set_no_tb() - turn off tracebacks for one test function
  • check.set_max_report() - limit reports for one test function
  • check.set_max_fail() - limit failures for one test function
  • check.between(b, a, c) - check that a < b < c

[1.2.1] - 2022-Nov-30

Changed

  • Remove colorama as a dependency, and simplify how we do "red".

  • Change the formatting of context manager checks such that if a msg is included, we see both the message and the assert failure. Previously, the message would replace the assert failure.

[1.2.0] - 2022-Nov-25

Added

[1.1.3] - 2022-Nov-22

Fixed

  • While using the check fixture, allow check.check as a context manager.
    • this is unnecessary, the check fixture now works as a context manager.
    • but apparently some people have been using with check.check:
    • so this fix makes that work.

[1.1.2] - 2022-Nov-21

Added

  • Allow raises from check object.
    • Now with check.raises(Exception): works.

[1.1.1] - 2022-Nov-21

Changed

  • README update

[1.1.0] - 2022-Nov-21

Added

  • Red output in pseudo-traceback

Changed

  • Refactored code into more modules
  • All functionality now possible through check fixture or from pytest_check import check.
    • The check object from either of these methods can be used as either a context manager or to access the check helper functions.
  • Lots of rewrite in README

Deprecated

  • API change.
    • Access to check helper functions via import pytest_check as check is deprecated.
    • It still works, but will be removed in a future version.
    • However, there is no deprecation warning for this.

[1.0.10] - 2022-09-29

Fixed

  • issue 80 --tb=no should remove tracebacks from xml output

[1.0.9] - 2022-08-24

Fixed

  • issue 55 a problem with multiple drive letters.
    • code is using os.path.relpath(), but sometimes it can fail.
    • when it does fail, fall back to os.path.abspath().
    • thanks rloutrel and kr1zo for your patience and persistence, and for the solution.

[1.0.8] - 2022-08-24

Fixed

  • Support check failures not blowing up if called outside of a test.
    • should also fix 85
    • thanks zerocen for the excellent toy example that helped me to get this fixed quickly.

Changed

  • changelog.md format. :)
  • tox.ini cleanup
  • refactoring tests to use examples directory instead of embedding example tests in test code.
    • this is easier to understand and debug issues.
    • also provides extra documentation through examples directory

[1.0.7] - 2022-08-21

Fixed

  • Handle cases when context is None

[1.0.6] - 2022-08-19

Changed

  • Update pyproject.toml to use newer flit
  • Changed plugin name from pytest_check to pytest-check.

[1.0.5] - 2022-03-29

Added

  • Add raises

[1.0.4] - 2021-09-12

Changed

  • Require Python >= 3.6
  • Remove old Manifest.in file.

[1.0.3] - 2021-09-12

Fixed

  • Fix #64, modifications to maxfail behavior.

[1.0.2] - 2021-08-12

Added

  • Add excinfo to call object in pytest_runtest_makereport().
    • Intended to help with some report systems.

[1.0.1] - 2020-12-27

Changed

  • Remove Beta Classifier
  • Status is now "Development Status :: 5 - Production/Stable"

[1.0.0] - 2020-12-27

Changed

  • Jump to 1.0 version, API is fairly stable.

[0.4.1] - 2020-12-27

Fixed

  • Fix #43/#44 tests with failing checks and failing asserts now report all issues.

[0.4.0] - 2020-12-14

Added

  • Add is_() and is_not()

Changed

  • Requires pytest 6.0 or above. (Removed some cruft to support pytest 5.x)