From d06a770aa4529ae156160d5a3d05fa9ed9770fcb Mon Sep 17 00:00:00 2001 From: yair Date: Sat, 15 Nov 2025 14:10:27 +0200 Subject: [PATCH] docs: merge UDP control protocol and add script documentation to network guide - Added documentation for launch-ids.py (Python-based camera control) - Added documentation for test_exposure_control.py (UDP control testing) - Added documentation for visualize_line_realtime.py (real-time visualization) - Merged UDP_CONTROL_PROTOCOL.md content into network_guide.md - Includes architecture diagrams, command reference, client examples - Complete end-to-end guide for camera control and monitoring --- network_guide.md | 413 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 411 insertions(+), 2 deletions(-) diff --git a/network_guide.md b/network_guide.md index eb0a5d0..bc0105a 100644 --- a/network_guide.md +++ b/network_guide.md @@ -6,7 +6,7 @@ The camera captures 2456x4 pixels, but we extract and transmit only **one line ( ### Daytime Configuration (200fps) ```powershell -gst-launch-1.0 idsueyesrc config-file=ini/200fps-2456x4pix-cw.ini exposure=5 framerate=200 ` +gst-launch-1.0 idsueyesrc config-file=ini/100fps-10exp-2456x4pix-500top-cw-extragain.ini exposure=0.05 framerate=200 ` ! videocrop bottom=3 ` ! queue ` ! udpsink host=127.0.0.1 port=5000 @@ -28,6 +28,355 @@ gst-launch-1.0 idsueyesrc config-file=ini/100fps-10exp-2456x4pix-500top-cw-extra **Alternative:** To extract the bottom line instead, use `videocrop top=3` +--- + +## Python-Based Control with Dynamic Parameter Adjustment + +For more advanced control including runtime exposure and framerate adjustments, use the Python-based launcher with UDP control interface. + +### Launch Camera with Control Interface + +See [`scripts/launch-ids.py`](scripts/launch-ids.py) for the complete implementation. + +```pwsh +# Setup GStreamer environment and run +. .\scripts\setup_gstreamer_env.ps1 +uv run .\scripts\launch-ids.py +``` + +**Features:** +- Video streaming on UDP port 5000 (127.0.0.1) +- Control interface on UDP port 5001 (0.0.0.0) +- Dynamic exposure control (0.001-1.0 seconds) +- Dynamic framerate control (1-500 fps) +- Real-time parameter updates without restarting + +**Control Commands:** + +| Command | Description | Example | +|---------|-------------|---------| +| `SET_EXPOSURE ` | Set exposure in seconds | `SET_EXPOSURE 0.016` | +| `GET_EXPOSURE` | Get current exposure value | `GET_EXPOSURE` | +| `SET_FRAMERATE ` | Set framerate in Hz | `SET_FRAMERATE 30` | +| `GET_FRAMERATE` | Get current framerate | `GET_FRAMERATE` | +| `STATUS` | Get pipeline status and settings | `STATUS` | + +**Example Usage:** +```pwsh +# Using netcat (nc) or PowerShell +echo "SET_EXPOSURE 0.010" | nc -u 127.0.0.1 5001 +echo "GET_EXPOSURE" | nc -u 127.0.0.1 5001 +echo "STATUS" | nc -u 127.0.0.1 5001 +``` + +### Testing the Control Interface + +See [`scripts/test_exposure_control.py`](scripts/test_exposure_control.py) for automated testing. + +```pwsh +# Run comprehensive test suite (15 tests) +uv run .\scripts\test_exposure_control.py +``` + +**Test Coverage:** +- Get/Set exposure and framerate +- Verify parameter changes take effect +- Test input validation and error handling +- Verify range limits (0.001-1.0s for exposure, 1-500 fps) +- Test invalid commands and syntax + +**Example Test Output:** +``` +Test 1: Get current exposure + Command: GET_EXPOSURE + Response: OK 0.016 + ✓ PASS + +Test 2: Set exposure to 10ms + Command: SET_EXPOSURE 0.010 + Response: OK 0.01 + ✓ PASS +``` + +--- + +## UDP Control Protocol Specification + +Complete technical details for the UDP-based control interface. + +### Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ launch-ids.py Process │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────────┐ ┌──────────────────────────┐ │ +│ │ Main Thread │ │ Control Server Thread │ │ +│ │ │ │ │ │ +│ │ GStreamer │◄────────┤ UDP Socket (Port 5001) │ │ +│ │ Pipeline │ Thread- │ Command Parser │ │ +│ │ - idsueyesrc │ Safe │ Property Setter │ │ +│ │ - videocrop │ Updates │ Response Handler │ │ +│ │ - queue │ │ │ │ +│ │ - udpsink:5000 │ └──────────────────────────┘ │ +│ └──────────────────┘ ▲ │ +│ │ │ +└───────────────────────────────────────────┼───────────────────┘ + │ + │ UDP Commands + │ + ┌────────┴────────┐ + │ Control Client │ + │ (Any UDP tool) │ + └─────────────────┘ +``` + +### Connection Details + +- **Control Port**: 5001 (UDP) +- **Bind Address**: 0.0.0.0 (accepts from any interface) +- **Video Port**: 5000 (UDP) - existing video stream, unchanged +- **Protocol**: UDP (connectionless, stateless) +- **Encoding**: ASCII text +- **Delimiter**: Newline (`\n`) + +### Command Format + +Commands follow this general structure: +``` +COMMAND [PARAMETERS]\n +``` + +Commands are case-insensitive, but UPPERCASE is recommended for clarity. + +### Detailed Command Reference + +#### 1. SET_EXPOSURE + +**Description**: Sets the camera exposure time. + +**Syntax**: +``` +SET_EXPOSURE +``` + +**Parameters**: +- ``: Exposure time in seconds (float) + - Range: 0.001 to 1.0 seconds (1ms to 1000ms) + - Examples: `0.016` (16ms), `0.001` (1ms), `0.100` (100ms) + +**Response**: +``` +OK +``` +or +``` +ERROR +``` + +**Examples**: +``` +Client: SET_EXPOSURE 0.016\n +Server: OK 0.016\n + +Client: SET_EXPOSURE 2.0\n +Server: ERROR OUT_OF_RANGE: Exposure must be 0.001-1.0 seconds\n +``` + +#### 2. GET_EXPOSURE + +**Description**: Retrieves the current exposure time. + +**Syntax**: +``` +GET_EXPOSURE +``` + +**Parameters**: None + +**Response**: +``` +OK +``` + +**Example**: +``` +Client: GET_EXPOSURE\n +Server: OK 0.016\n +``` + +#### 3. SET_FRAMERATE + +**Description**: Sets the camera frame rate. + +**Syntax**: +``` +SET_FRAMERATE +``` + +**Parameters**: +- ``: Frame rate in Hz (float) + - Range: 1.0 to 500.0 fps + - Examples: `22`, `30.5`, `100` + +**Response**: +``` +OK +``` +or +``` +ERROR +``` + +**Example**: +``` +Client: SET_FRAMERATE 30\n +Server: OK 30.0\n +``` + +#### 4. GET_FRAMERATE + +**Description**: Retrieves the current frame rate. + +**Syntax**: +``` +GET_FRAMERATE +``` + +**Parameters**: None + +**Response**: +``` +OK +``` + +**Example**: +``` +Client: GET_FRAMERATE\n +Server: OK 22.0\n +``` + +#### 5. STATUS + +**Description**: Get overall pipeline status and current settings. + +**Syntax**: +``` +STATUS +``` + +**Parameters**: None + +**Response**: +``` +OK exposure= framerate= state= +``` + +**Example**: +``` +Client: STATUS\n +Server: OK exposure=0.016 framerate=22.0 state=PLAYING\n +``` + +### Error Handling + +**Error Response Format**: +``` +ERROR : +``` + +**Common Error Codes**: + +| Code | Description | Example | +|------|-------------|---------| +| `INVALID_COMMAND` | Unknown command | `ERROR INVALID_COMMAND: Unknown command 'FOO'` | +| `INVALID_SYNTAX` | Malformed command | `ERROR INVALID_SYNTAX: Missing parameter` | +| `OUT_OF_RANGE` | Value out of valid range | `ERROR OUT_OF_RANGE: Exposure must be 0.001-1.0` | +| `PROCESSING` | Internal processing error | `ERROR PROCESSING: Failed to set property` | + +### Client Implementation Examples + +#### Python Client +```python +import socket + +def send_command(command, host="127.0.0.1", port=5001): + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + sock.sendto(command.encode() + b'\n', (host, port)) + sock.settimeout(1.0) + response, _ = sock.recvfrom(1024) + sock.close() + return response.decode().strip() + +# Set exposure to 10ms +print(send_command("SET_EXPOSURE 0.010")) + +# Get current exposure +print(send_command("GET_EXPOSURE")) + +# Set framerate to 30fps +print(send_command("SET_FRAMERATE 30")) +``` + +#### Command Line (netcat/nc) +```bash +# Set exposure +echo "SET_EXPOSURE 0.020" | nc -u 127.0.0.1 5001 + +# Get exposure +echo "GET_EXPOSURE" | nc -u 127.0.0.1 5001 + +# Get status +echo "STATUS" | nc -u 127.0.0.1 5001 +``` + +#### PowerShell Client +```powershell +$udpClient = New-Object System.Net.Sockets.UdpClient +$endpoint = New-Object System.Net.IPEndPoint([System.Net.IPAddress]::Parse("127.0.0.1"), 5001) + +# Send command +$bytes = [System.Text.Encoding]::ASCII.GetBytes("SET_EXPOSURE 0.015`n") +$udpClient.Send($bytes, $bytes.Length, $endpoint) + +# Receive response +$udpClient.Client.ReceiveTimeout = 1000 +$receiveBytes = $udpClient.Receive([ref]$endpoint) +$response = [System.Text.Encoding]::ASCII.GetString($receiveBytes) +Write-Host $response + +$udpClient.Close() +``` + +### Implementation Notes + +#### Thread Safety +- The control server runs in a separate daemon thread +- GStreamer properties are inherently thread-safe (GObject properties) +- The `src.set_property()` method can be safely called from the control thread + +#### Non-Blocking Operation +- Control server uses non-blocking socket with timeout +- Does not interfere with GStreamer pipeline operation +- Minimal latency for command processing + +#### Response Timing +- Responses are sent immediately after processing +- Property changes take effect on the next frame capture +- No guaranteed synchronization with video stream + +### Future Enhancements + +Possible extensions to the protocol: +- Add `SET_GAIN` / `GET_GAIN` commands +- Add `SAVE_CONFIG` to save current settings to INI file +- Add `RESET` to restore default settings +- Support batch commands (multiple commands in one packet) +- Add authentication/security for production use + +--- + ### Python/OpenCV Receiver ```pwsh # Basic rolling display @@ -98,9 +447,69 @@ The analyzer automatically detects the format, shows pixel statistics per color uv run .\scripts\udp_sniffer_raw.py ``` +### Real-time Channel Visualization + +For live visualization of RGB/BGR channel values across the line sensor, use the real-time plotter. + +See [`scripts/visualize_line_realtime.py`](scripts/visualize_line_realtime.py) for the implementation. + +```pwsh +# Basic usage (assumes BGR format, 2456 width, port 5000) +uv run .\scripts\visualize_line_realtime.py +``` + +```pwsh +# With custom parameters +uv run .\scripts\visualize_line_realtime.py --format BGR --port 5000 --width 2456 --fps-limit 30 +``` + +**Features:** +- **Dual Plot Display:** + - Top plot: Grayscale luminance (weighted: BGR→0.114B+0.587G+0.299R) + - Bottom plot: Individual RGB/BGR color channels +- **Real-time Statistics:** + - Per-channel min/max/mean/std deviation + - Display FPS counter + - Frame counter +- **Optimized Performance:** + - Packet draining (always displays latest frame) + - Configurable display rate limiting + - Minimal buffer size to reduce latency + +**Command-line Options:** + +| Option | Default | Description | +|--------|---------|-------------| +| `--format` | `BGR` | Input format (`BGR` or `RGB`) | +| `--port` | `5000` | UDP port to receive on | +| `--width` | `2456` | Line width in pixels | +| `--fps-limit` | `30` | Maximum display update rate | + +**Example Output:** + +The visualization shows live channel data with statistics overlay: +``` +Frame: 1523 FPS: 29.8 +Blue : min= 12 max=203 mean= 45.32 std= 28.45 +Green: min= 28 max=241 mean= 68.91 std= 35.12 +Red : min= 35 max=255 mean=102.76 std= 48.23 +Gray : min= 41.23 max=241.67 mean= 72.45 std= 36.89 +``` + +**Use Cases:** +- Verify camera line sensor is working correctly +- Monitor real-time brightness and color balance +- Debug exposure and gain settings +- Analyze scene illumination changes +- Quality control and calibration + +Close the plot window to exit. + +--- + ## Configuration Notes -Both INI files are configured with: + INI file is configured with: - Start Y = 500 (captures from row 500 of the sensor) - Height = 4 pixels - Width = 2456 pixels