Performance & Watch Mode

Speed up checks with parallel execution, caching, and watch mode

Performance & Watch Mode

pycmdcheck provides several features to speed up your development workflow:

  • Parallel execution: Run checks concurrently using multiple threads
  • Caching: Skip checks when source files haven’t changed
  • Watch mode: Automatically re-run checks when files change

Parallel Execution

By default, pycmdcheck runs checks sequentially. For larger projects with many checks, you can enable parallel execution to speed things up.

CLI Usage

# Enable parallel execution with default 4 workers
pycmdcheck --parallel

# Enable parallel execution with custom worker count
pycmdcheck --parallel --workers 8

# Disable parallel execution (default)
pycmdcheck --no-parallel

Python API

from pycmdcheck import check

# Sequential execution (default)
results = check("./my-package")

# Parallel execution with 4 workers
results = check("./my-package", parallel=True)

# Parallel execution with custom worker count
results = check("./my-package", parallel=True, workers=8)

How It Works

Parallel execution uses Python’s ThreadPoolExecutor to run multiple checks concurrently. Each check runs in its own thread, but they all share the same resolved fixtures. This means:

  • Fixture resolution happens once, before parallel execution begins
  • Check execution happens concurrently across multiple threads
  • Result collection waits for all checks to complete
Note

Parallel execution is most beneficial when you have many independent checks that don’t heavily share resources. For CPU-bound checks, the speedup depends on your system’s thread handling.

Caching

pycmdcheck can cache check results to skip re-running checks when source files haven’t changed.

CLI Usage

# Enable caching (default)
pycmdcheck

# Disable caching
pycmdcheck --no-cache

Cache Location

The cache is stored in a .pycmdcheck_cache directory in your project root. You should add this to your .gitignore:

.pycmdcheck_cache/

Cache Invalidation

The cache is automatically invalidated when:

  • Source files change (based on SHA-256 hashes)
  • New files are added
  • Files are deleted
  • pycmdcheck is upgraded

Clearing the Cache

You can manually clear the cache by deleting the .pycmdcheck_cache directory:

rm -rf .pycmdcheck_cache

Python API

from pycmdcheck.core.cache import CheckCache
from pathlib import Path

# Create a cache instance
cache = CheckCache(Path("./my-package"))

# Compute file hashes for a directory
hashes = cache.compute_directory_hashes(
    Path("./my-package/src"),
    patterns=["*.py"]
)

# Get a cached result
result = cache.get("ST001", file_hashes=hashes)

# Cache a result
cache.set("ST001", result, file_hashes=hashes)

# Clear all cached results
cache.clear()

Watch Mode

Watch mode automatically re-runs checks when files in your project change. This is useful during development when you want immediate feedback.

CLI Usage

# Start watch mode
pycmdcheck --watch

# Watch mode with parallel execution
pycmdcheck --watch --parallel

# Watch mode with custom output format
pycmdcheck --watch --format json

Press Ctrl+C to stop watch mode.

What Files Are Watched

By default, pycmdcheck watches for changes to:

  • *.py - Python source files
  • *.toml - Configuration files (including pyproject.toml)
  • *.md - Markdown documentation

And ignores:

  • *.pyc - Compiled Python files
  • __pycache__/* - Python cache directories
  • .git/* - Git metadata

Debouncing

Watch mode includes debouncing to avoid running checks multiple times for rapid successive changes. By default, it waits 0.5 seconds after the last change before running checks.

Python API

from pycmdcheck.watch import FileWatcher, WatchConfig
from pathlib import Path

# Create a callback function
def on_change():
    print("Files changed!")
    # Run your checks here

# Configure what to watch
config = WatchConfig(
    debounce_seconds=0.5,
    include_patterns=["*.py", "*.toml"],
    exclude_patterns=["*.pyc", "__pycache__/*"],
)

# Create and start the watcher
watcher = FileWatcher(
    Path("./my-package"),
    on_change,
    config=config,
)

# Start watching (blocks until stopped)
# Run this in a separate thread if you need non-blocking behavior
watcher.start()

# To stop watching
watcher.stop()

Combining Features

All performance features can be combined:

# Watch mode with parallel execution and caching
pycmdcheck --watch --parallel --workers 8

# Watch mode with verbose output
pycmdcheck --watch --verbose

# Watch mode checking only specific groups
pycmdcheck --watch --only-group structure

Performance Tips

  1. Use parallel execution for projects with many checks
  2. Enable caching (default) to skip unchanged files
  3. Use watch mode during active development
  4. Limit check scope with --only-group or --only when focusing on specific areas
  5. Consider worker count: More workers isn’t always faster; 4-8 is typically optimal

Troubleshooting

Watch mode not detecting changes

If watch mode isn’t detecting changes:

  1. Check that your files match the include patterns (*.py, *.toml, *.md)
  2. Make sure you’re not editing in a directory that’s excluded
  3. On some systems, certain file operations may not trigger events

Parallel execution is slower

Parallel execution may be slower for:

  • Very few checks (thread overhead dominates)
  • I/O-bound checks that contend for disk access
  • Checks that share mutable state

Consider using --no-parallel if you encounter issues.

Cache not working

If caching seems ineffective:

  1. Ensure the .pycmdcheck_cache directory is writable
  2. Check that you’re running from the same directory
  3. Verify files aren’t changing between runs