TS005: TestDiscoverability

Overview

Property Value
ID TS005
Name TestDiscoverability
Group tests
Severity NOTE

Description

Verifies that all test files are discoverable by pytest.

pytest discovers tests by looking for files matching specific patterns (test_*.py or *_test.py). If your test files don’t follow these conventions, they will be silently skipped during test runs, potentially leaving bugs undetected.

What it checks

The check scans all Python files in tests/ or test/ directories and verifies they are discoverable:

  • PASSED: All Python files match test_*.py or *_test.py pattern
  • FAILED: One or more files don’t match the expected patterns
  • NOT_APPLICABLE: No tests directory found

Discoverable patterns

  • test_*.py (recommended, e.g., test_module.py)
  • *_test.py (alternative, e.g., module_test.py)

Allowed files (not required to match patterns)

  • __init__.py (package marker)
  • conftest.py (pytest fixtures)

How to fix

Rename non-discoverable files

# Before - pytest won't discover these
tests/
    my_tests.py          # Wrong: not "test_*" or "*_test"
    tests_for_module.py  # Wrong: not "test_*" or "*_test"
    checks.py            # Wrong: not "test_*" or "*_test"

# After - pytest will discover these
tests/
    test_my_module.py    # Correct: starts with "test_"
    test_module.py       # Correct: starts with "test_"
    test_checks.py       # Correct: starts with "test_"

Use suffix pattern if preferred

# Also valid pytest patterns
tests/
    module_test.py       # Correct: ends with "_test"
    integration_test.py  # Correct: ends with "_test"

Move helper modules to conftest or use private naming

If you have utility files that shouldn’t contain tests:

# Before - checks.py may confuse tools
tests/
    checks.py            # Contains utilities, not tests
    test_module.py

# After - clear separation
tests/
    conftest.py          # Move fixtures here (allowed by default)
    test_module.py

Or use private module naming:

tests/
    _helpers.py          # Private module (starts with _)
    _fixtures.py         # Private module
    test_module.py

Configuration

Skip this check

[tool.pycmdcheck]
skip = ["TS005"]

CLI

pycmdcheck --skip TS005

Why discoverability matters

Silent test skipping

When test files don’t match pytest’s discovery patterns:

  • Tests are silently not run
  • CI/CD reports success when tests weren’t executed
  • Coverage reports show incomplete data
  • Bugs slip through undetected

Common mistakes

# File: tests/my_tests.py  (WRONG - not discoverable)
def test_something():
    assert True

# File: tests/test_my_module.py  (CORRECT - discoverable)
def test_something():
    assert True

IDE integration

Most IDEs recognize test_*.py files for:

  • Test runner integration
  • Coverage highlighting
  • Quick navigation to tests

Difference from TS004

TS004 (TestNamingConvention) and TS005 (TestDiscoverability) check similar things but have different purposes:

  • TS004: Checks that test files follow naming conventions (style/consistency)
  • TS005: Checks that test files will actually be discovered by pytest (functionality)

Both checks help ensure your tests are properly organized and will run.

References