Pure native python snap7 (#569)

* add CLAUDE generated native snap7 code

* cleanup tests

* Refactor partner module to match client/server pattern

- Create snap7/partner/__init__.py as base class with factory pattern
- Move existing ctypes partner to snap7/clib/partner.py (ClibPartner)
- Create snap7/native/partner.py pure Python implementation
- Create snap7/native/wire_partner.py for low-level wire protocol
- Update snap7/__init__.py to export ClibPartner and PurePartner
- Add mainloop wrapper to snap7/server/__init__.py to avoid circular imports

The Partner class now works like Client and Server:
- Partner() returns ClibPartner (ctypes, default)
- Partner(pure_python=True) returns PurePartner

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove clib dependency, use pure Python implementation only

This commit completes the migration to a pure Python S7 protocol
implementation, removing the dependency on the native Snap7 C library.

Changes:
- Remove snap7/clib/ folder (ctypes bindings)
- Remove snap7/native/ folder (move contents to snap7/)
- Remove snap7/common.py, snap7/protocol.py, snap7/protocol.pyi
- Flatten structure: client.py, server.py, partner.py at top level
- Add connection.py, datatypes.py, s7protocol.py for protocol handling
- Simplify CI/CD workflows (no native library builds needed)
- Update README.rst and CLAUDE.md for pure Python architecture
- Update pyproject.toml (remove native lib package-data)
- Update all tests to work with native implementation

The package is now a pure Python wheel that works on all platforms
without architecture-specific builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add feature completeness verification and API compatibility tests

- Add delete() and full_upload() methods to Client (was missing vs master)
- Create test_api_compatibility.py: verifies all public exports and method signatures
- Create test_feature_matrix.py: maps all 113 Snap7 C functions to Python methods
- Create test_behavioral_compatibility.py: roundtrip, multi-area, concurrent tests
- Fix 5 tests in test_client.py that referenced clib-specific _lib attribute

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix CI/CD issues: mypy errors, partner port, sphinx docs

- Change partner default port from 102 to 1102 (non-privileged)
- Add missing type annotations across all snap7 modules
- Fix client.py read_multi_vars and write_multi_vars type handling
- Use cast() for proper type narrowing in union types
- Change encode_s7_data parameter type from List to Sequence
- Add missing return type annotations to test methods
- Fix callback type annotations (use SrvEvent instead of str)
- Update example files to use correct API signatures
- Update server.rst documentation for pure Python implementation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix test port conflicts: use unique ports for each test class

- test_server.py: use port 12102
- test_partner.py: use port 12103

This prevents "Address already in use" errors when tests run sequentially.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add port release delays in test teardown to prevent race conditions

The tests were failing on CI due to ports remaining in TIME_WAIT state.
Adding a 0.2 second delay after stopping servers/partners allows the
OS to fully release the port before the next test starts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add SO_REUSEPORT socket option for faster port reuse in tests

On Linux/macOS, SO_REUSEPORT allows multiple sockets to bind to the
same port, which helps prevent "Address already in use" errors when
tests run in quick succession and ports are still in TIME_WAIT state.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove redundant test files

Delete 5 test files that duplicated coverage from other tests:
- test_simple_memory_access.py (2 tests) - covered by test_behavioral_compatibility
- test_write_operations.py (1 test) - covered by multiple integration tests
- test_address_parsing.py (4 tests) - covered by test_native_all_methods
- test_native_server_client.py (8 tests) - covered by test_native_integration_full
- test_integration.py (7 tests) - covered by test_api_compatibility

Reduces test files from 18 to 13 while maintaining full coverage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Consolidate API and integration test files

- Merge test_api_compatibility.py + test_feature_matrix.py → test_api_surface.py
  Combines public export tests, C function mapping, and method signature tests
- Delete test_server_compatibility.py (covered by test_native_all_methods.py
  and test_behavioral_compatibility.py)

Test suite reduced from 574 to 424 tests while maintaining full coverage.
Files reduced from 13 to 11 test files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Rename CLAUDE.md to AGENTS.md

Use the more universal agents.md format for AI guidance files.
See https://agents.md/ for the specification.

Addresses PR review comment from @nikteliy.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add missing API for feature completeness

- Add set_rw_area_callback() stub to server.py for API parity with C library
- Fix get_cpu_state() return format to use S7CpuStatus strings for backwards
  compatibility with master branch (S7CpuStatusRun, S7CpuStatusStop, etc.)
- Add Srv_SetRWAreaCallback to test_api_surface.py function mapping

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove dead _LibMock code from client.py

This was leftover from the C library wrapper transition - no tests use it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove redundant test files, consolidate test suite

- Delete test_native_all_methods.py (32 tests) - duplicated test_client.py
- Delete test_native_integration_full.py (14 tests) - duplicated test_client.py
- Move unique test_context_manager to test_client.py
- Move unique server robustness tests to test_server.py

Test count: 425 → 387 (38 redundant tests removed)
All remaining tests provide unique coverage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Restore CLI interface: convert server to package

- Convert snap7/server.py to snap7/server/ package with __init__.py
- Add snap7/server/__main__.py for CLI: python -m snap7.server
- Rename test_native_datatypes.py to test_datatypes.py (nothing "native" anymore)

This restores the command-line interface that was in master branch:
  python -m snap7.server --help
  python -m snap7.server -p 1102 -v

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Implement real S7 protocol for USER_DATA operations

- Add USER_DATA PDU (0x07) infrastructure for block info and SZL operations
- Implement server handlers for grBlocksInfo (list_blocks, list_blocks_of_type)
- Implement server SZL handler with data for common SZL IDs (0x001C, 0x0011, 0x0131, 0x0232, 0x0000)
- Fix _parse_data_section in both client and server to handle transport_size=0x00 for USERDATA requests (was incorrectly dividing by 8)
- Update client SZL functions to use real protocol: read_szl, get_cpu_info, get_cp_info, get_order_code, get_protection
- Fix get_cp_info to handle signed c_byte values properly
- Update tests to verify real protocol behavior

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Implement real S7 protocol for clock operations

- Add build_get_clock_request and build_set_clock_request to s7protocol.py
- Add parse_get_clock_response for BCD time format parsing
- Implement server _handle_get_clock and _handle_set_clock handlers
- Update client get_plc_datetime and set_plc_datetime to use real protocol
- Server returns actual system time, accepts set requests (logs but doesn't persist)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add TODO.md documenting remaining protocol implementations

Documents what's needed for:
- Control operations (compress, copy_ram_to_rom)
- Authentication (set_session_password, clear_session_password)
- Block transfer (upload, download, delete)

Includes protocol details, implementation notes, and priority order.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Implement real S7 protocol for block transfer operations

- Add upload/download/delete handlers to server
- Client upload() sends real START_UPLOAD, UPLOAD, END_UPLOAD sequence
- Client full_upload() sends real protocol and wraps with MC7 header
- Client download() sends real REQUEST_DOWNLOAD, DOWNLOAD_BLOCK, DOWNLOAD_ENDED sequence
- Client delete() sends real PLC_CONTROL with PI service "_DELE"
- Update tests to use real protocol instead of skipping
- All 390 tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix ruff formatting

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Speed up test suite by 3.7x (67s → 18s)

Changes:
- Reduce server accept timeout from 1.0s to 0.1s for responsive shutdown
- Switch tests from subprocess to thread-based server (no startup delay)
- Remove unnecessary time.sleep() calls in test fixtures

Before: 67.62s for 390 tests
After: 18.21s for 390 tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix S7 protocol write operations for real PLC compatibility

- Fix S7PDUType enum: add ACK (0x02) for write responses, rename
  RESPONSE to ACK_DATA (0x03) for read responses
- Update parse_response to accept both ACK and ACK_DATA response types
- Fix transport size in write request data section: use proper S7
  transport size codes (0x03=BIT, 0x04=BYTE, 0x05=INT, etc.) instead
  of incorrectly using word_len values
- Update server code to use new ACK_DATA enum name

The main issue was that write requests used incorrect transport size
codes in the data section, causing PLCs to reject them with error
class 0x81 (application relationship error).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Consolidate CI workflows into single test.yml

Merge linux-osx-test.yml and windows-test.yml into a unified test.yml
that tests across all platforms (Linux, macOS, Windows) in a single
workflow with a combined matrix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix S7-1200/1500 write response handling and remove TODO.md

- Fix check_write_response to check header error codes first before data
- S7-1200/1500 PLCs return ACK (type 2) with error codes for failed writes
- Update server _build_error_response to use ACK type for errors
- Remove obsolete TODO.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix USERDATA PDU header size (10 bytes, not 12)

USERDATA PDUs use a 10-byte header without error_class/error_code,
while ACK/ACK_DATA use 12-byte headers. This was causing "Data section
extends beyond PDU" errors when parsing USERDATA responses from real PLCs.

Changes:
- s7protocol.py: parse_response() now detects PDU type and uses correct
  header size (10 bytes for USERDATA, 12 bytes for ACK/ACK_DATA)
- server/__init__.py: Build USERDATA responses with 10-byte header
- client.py: Check for errors in data section return_code for USERDATA
  responses (errors are in data section, not header)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add human-readable S7 return code descriptions

Error messages now include descriptive text for S7 return codes:
- 0x0a: "Object does not exist"
- 0x05: "Invalid address"
- 0x03: "Accessing the object not allowed"
- etc.

Example: "Read SZL failed: Object does not exist (0x0a)"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add end-to-end test suite for real PLC testing

Adds test_client_e2e.py with comprehensive tests against a real Siemens
S7 PLC. Tests are marked with @pytest.mark.e2e and require:
- A real PLC connection (configure IP, rack, slot at top of file)
- Two data blocks: DB1 (read-only) and DB2 (read-write)

Run with: pytest tests/test_client_e2e.py -m e2e

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Skip e2e tests by default, require --e2e flag

E2e tests require a real PLC connection and should not run in CI or
by default. Use --e2e flag to enable them:

  pytest tests/test_client_e2e.py --e2e

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add command-line options for PLC connection parameters

E2e tests can now be configured via command line:

  pytest tests/test_client_e2e.py --e2e \
    --plc-ip=192.168.1.10 \
    --plc-rack=0 \
    --plc-slot=1 \
    --plc-port=102 \
    --plc-db-read=1 \
    --plc-db-write=2

Also supports environment variables: PLC_IP, PLC_RACK, PLC_SLOT, etc.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix S7 protocol encoding for real PLC compatibility

Fix COTP TPDU size encoding to use ISO 8073 code (1-byte) instead of
raw 2-byte value, which caused connection failures on S7-400 PLCs like
the CPU 414-5H PN/DP.

Fix USERDATA request formats (read_szl, list_blocks_of_type,
get_block_info) to match Snap7 C library encoding: correct RetVal/TS
bytes (0xFF/0x09), proper block type prefixes (0x30), and ASCII block
number encoding.

Fix USERDATA response parsing: list_blocks_of_type items are 4 bytes
(not 2), and get_block_info field offsets now match TResDataBlockInfo.

Update server to produce matching response formats (78-byte block info,
4-byte block list items, 12-byte USERDATA parameter sections).

Add data section return_code checks to list_blocks, list_blocks_of_type,
and get_block_info. Fix db_get to query actual DB size via get_block_info
instead of hardcoded 1024.

Fix CLI options propagation for e2e tests via pytest_sessionstart hook.
Add pytest-html to test deps. Add SZL skip-on-missing resilience to
e2e tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix mypy type error in server block info response

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Apply ruff formatting to s7protocol.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix USERDATA protocol encoding for real PLC compatibility

- Set DataRef byte to 0x00 (not sequence number) in all USERDATA requests
- Fix data section headers (RetVal=0x0A, TransSize=0x00) in list_blocks_of_type,
  get_block_info, and read_szl requests to match Snap7 C library
- Extend list_blocks_of_type payload from 2 to 4 bytes per Snap7 C format
- Fix db_get to propagate get_block_info errors instead of silently falling
  back to 1024 bytes
- Fix hardcoded DB number in test_db_write_int
- Fix CLI args not propagating to e2e tests (use sys.modules lookup)
- Add graceful pytest.skip for block operation tests on unsupported PLCs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Support multi-packet USERDATA responses for SZL and block listing

SZL responses and block-of-type listings can span multiple packets when
the data doesn't fit in one PDU. Parse the "last data unit" byte from
USERDATA response parameters to detect fragmentation, then loop sending
follow-up requests to accumulate all fragments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address review comments from nikteliy

- Replace magic numbers in COTP CR builder with named constants
  (COTP_PARAM_CALLING_TSAP, COTP_PARAM_CALLED_TSAP, COTP_PARAM_PDU_SIZE)
- Log unsupported COTP parameters instead of silently ignoring them
- Add exhaustive assert_never() fallback in encode_s7_data()
- Validate bit addresses (0-7) in parse_address() for DB, M, I, Q areas
- Validate non-negative start address in encode_address()
- Rename AGENTS.md to agents.md per universal convention

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rename agents.md to CLAUDE.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix USER_DATA protocol encoding for real PLC compatibility

The data section header in USER_DATA requests with non-zero payloads
was using 0x0A/0x00 (object does not exist / null) instead of
0xFF/0x09 (data OK / octet string) as specified by the Snap7 C library.
This caused get_block_info() to fail on real PLCs (tested on CPU416-2)
with "Object does not exist" errors.

Fixed in: build_get_block_info_request, build_list_blocks_of_type_request,
build_read_szl_request, build_set_clock_request.

Also adds optional size parameter to db_get() and db_fill() so users
can bypass get_block_info() on PLCs that don't support it, and fixes
db_fill() which was using a hardcoded size of 100.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix signed byte overflow in S7SZL and improve e2e test resilience

S7SZL.Data used c_byte (signed, -128..127) instead of c_ubyte (unsigned,
0..255). When real PLCs return data with high-bit-set bytes (>127), ctypes
stores them as negative values, causing `bytes(szl.Data[:n])` to fail with
ValueError. This broke get_order_code, read_szl_list, get_cpu_info, and
get_protection on real hardware.

Also fix S7OrderCode.V1/V2/V3 fields (version numbers should be unsigned),
add debug logging to get_block_info for protocol diagnostics, and make
test_db_get skip gracefully on PLCs that don't support get_block_info.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add TIA Portal SCL source for e2e test data blocks

Importable SCL file that creates DB1 "Read_only" and DB2 "Data_block_2"
with the exact byte layout and start values expected by test_client_e2e.py.
Includes notes for S7-1200/1500 (disable optimized access) and S7-300/400.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Author: gijzelaerr