Compare commits

...

10 Commits

Author SHA1 Message Date
yair
e58e2319a3 dot stuff 2025-11-14 17:11:18 +02:00
yair
4d35d16c72 udp 2025-11-14 17:00:20 +02:00
yair
2699913e92 rolling 2025-11-14 16:29:31 +02:00
yair
72e7091c61 rolling 2025-11-14 16:18:27 +02:00
yair
94f7c04dc6 Fix recv_raw_column.py height mismatch and update script paths in docs
- Fixed HEIGHT from 480 to 640 to match actual videotestsrc output
- Added DEBUG flag to control debug output visibility
- Added cv2.namedWindow() for proper window initialization
- Updated all Python script references in markdown files to scripts/ folder
- Updated network_guide.md with correct frame dimensions and Python receiver option
2025-11-14 15:33:17 +02:00
yair
ee4fd76e13 rem unused lic 2025-11-14 15:01:03 +02:00
yair
1c7ca124b1 del 2025-11-14 15:00:05 +02:00
yair
cb1e5c7607 docs: convert bash commands to PowerShell and merge ROLLINGSUM docs
- Updated all commands in README.md and ROLLINGSUM_GUIDE.md to use PowerShell syntax
- Changed line continuation from backslash (\) to backtick ()
- Updated environment variable syntax to PowerShell format ()
- Merged DESIGN_ROLLINGSUM.md into ROLLINGSUM_GUIDE.md for comprehensive documentation
- Combined user guide with technical design details in single document
- Added table of contents and improved organization
2025-11-14 14:58:39 +02:00
yair
d0467aaf65 converted cli to pwsh 2025-11-14 14:55:05 +02:00
yair
44083222ee refactor: Output analysis results to results/debug directory
- Update analyze_sma.py to save files to results/debug/
- Add timestamp to archived CSV and plot files
- Update .gitignore to exclude results/ directory
- Update ROLLINGSUM_GUIDE.md with new output locations
2025-11-14 14:34:58 +02:00
11 changed files with 884 additions and 356 deletions

3
.gitignore vendored
View File

@ -38,3 +38,6 @@ ipch/
*.mkv
*.raw
*.dot
gst_plugs/
results/

View File

@ -1,290 +0,0 @@
# Rolling Sum Filter - Design Document
## Overview
A GStreamer element that drops frames based on rolling mean analysis of a single column of pixels. Inspired by the detection algorithm in `cli.py`, simplified for real-time streaming.
## Element Name
**`rollingsum`** - Transform element that analyzes pixel values and selectively drops frames
## Purpose
Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
## Architecture
### Base Class
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
- In-place transform (analysis only, no frame modification)
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
- Returns `GST_FLOW_OK` to pass frames
### Element Structure
```c
struct _GstRollingSum
{
GstBaseTransform element;
/* Properties */
gint window_size; // Number of frames in rolling window (default: 1000)
gint column_index; // Which column to analyze (default: 1, second column)
gint stride; // Row sampling stride (default: 1, every row)
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
/* State */
gdouble *ring_buffer; // Circular buffer of column means
gint ring_index; // Current position in ring buffer
gint frame_count; // Total frames processed
gdouble rolling_mean; // Current rolling mean
};
```
## Algorithm (Simplified from cli.py)
### Per Frame Processing
```
1. Extract column data:
- Select column at column_index
- Sample every stride rows
- Calculate mean of sampled pixels: frame_mean
2. Update ring buffer:
- Store frame_mean in ring_buffer[ring_index]
- Increment ring_index (wrap around)
3. Calculate rolling mean:
- Sum values in ring buffer (up to window_size or frame_count)
- Divide by actual window size
4. Calculate deviation:
- deviation = abs(frame_mean - rolling_mean)
5. Decision:
- If deviation > threshold: DROP frame
- Else: PASS frame
```
### Key Simplifications from cli.py
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
- **No recording logic**: Just drop/pass, no buffering for output segments
- **No patience mechanism**: Immediate decision per frame
## Properties
| Property | Type | Default | Range | Description |
|----------|------|---------|-------|-------------|
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
| `column-index` | int | 1 | 0-width | Which vertical column to analyze |
| `stride` | int | 1 | 1-height | Row sampling stride (1=all rows) |
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping |
## Data Structures
### Ring Buffer
- Array of doubles sized to `window_size`
- Stores column mean for each processed frame
- Circular wrapping via modulo: `ring_index % window_size`
### Frame Analysis
```
For column_index=1, stride=1, frame height=H:
- Extract pixels: frame[:, column_index]
- Sample: frame[::stride, column_index]
- Count: H / stride samples
- Mean: sum(samples) / count
```
## Implementation Files
### Directory Structure
```
gst/rollingsum/
├── CMakeLists.txt
├── gstrollingsum.c
└── gstrollingsum.h
```
### gstrollingsum.h
- Element type definitions
- Structure declarations
- Property enums
- Function prototypes
### gstrollingsum.c
- GObject methods (init, dispose, get/set properties)
- GstBaseTransform methods (transform_ip)
- Helper functions (extract_column_mean, update_rolling_mean)
- Plugin registration
### CMakeLists.txt
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
- Link GStreamer base and video libraries
## Video Format Support
### Initial Implementation
- **Primary target**: Grayscale (GRAY8, GRAY16)
- **Secondary**: Bayer formats (common in machine vision)
### Caps Filter
```c
static GstStaticPadTemplate sink_template =
GST_STATIC_PAD_TEMPLATE ("sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS (
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
)
);
```
## Pipeline Usage Examples
### Basic Usage
```bash
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
rollingsum window-size=1000 column-index=1 threshold=0.5 ! \
autovideosink
```
### Custom Configuration
```bash
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! \
queue ! \
autovideosink
```
### With Format Conversion
```bash
gst-launch-1.0 idsueyesrc ! \
videoconvert ! \
video/x-raw,format=GRAY8 ! \
rollingsum ! \
autovideosink
```
## Performance Considerations
### Memory Usage
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
- Minimal per-frame allocation
### CPU Usage
- Column extraction: O(height/stride)
- Rolling mean update: O(1) using incremental sum
- Very lightweight compared to full-frame processing
### Optimization Opportunities
1. **Incremental mean**: Track sum instead of recalculating
2. **SIMD**: Vectorize column summation
3. **Skip calculation**: Only recalc every N frames if baseline is stable
## Testing Strategy
### Unit Tests
- Ring buffer wrapping
- Mean calculation accuracy
- Threshold comparison logic
### Integration Tests
- Pipeline with videotestsrc
- Pipeline with idsueyesrc
- Frame drop verification
- Property changes during playback
### Test Cases
1. Static video (all frames similar) → all pass
2. Single bright frame → that frame drops
3. Gradual change → frames pass
4. Periodic pattern → pattern frames drop
## Future Enhancements
### Phase 2 (If Needed)
- Add EMA baseline tracking (like cli.py)
- Add variance-based thresholds
- Support multiple columns or regions
- Add metadata output (tag frames with deviation values)
- RGB format support (analyze specific channel)
### Phase 3 (Advanced)
- Full cli.py recording logic
- Buffer and output segments
- Integration with probe detection systems
## Integration with Existing Project
### Build System
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
```cmake
add_subdirectory (rollingsum)
```
### Documentation
Update [`README.md`](README.md):
- Add rollingsum to "Other elements" section
- Add pipeline example
## Mermaid Diagram: Data Flow
```mermaid
graph TD
A[Video Frame] --> B[Extract Column]
B --> C[Calculate Column Mean]
C --> D[Store in Ring Buffer]
D --> E[Update Rolling Mean]
E --> F{Deviation > Threshold?}
F -->|Yes| G[DROP Frame]
F -->|No| H[PASS Frame]
C --> E
style G fill:#ff6b6b
style H fill:#51cf66
```
## Mermaid Diagram: Ring Buffer Operation
```mermaid
graph LR
subgraph Ring Buffer
A[0] --> B[1]
B --> C[2]
C --> D[...]
D --> E[N-1]
E ---|wrap| A
end
F[New Frame Mean] --> G[ring_index]
G --> A
H[Rolling Mean] --> I[Sum all values]
I --> J[Divide by count]
style G fill:#ffd43b
```
## Implementation Checklist
- [ ] Create gst/rollingsum directory
- [ ] Implement gstrollingsum.h
- [ ] Implement gstrollingsum.c
- [ ] Create CMakeLists.txt
- [ ] Update gst/CMakeLists.txt
- [ ] Build and test basic functionality
- [ ] Test with idsueyesrc
- [ ] Update README.md
- [ ] Create feature branch
- [ ] Commit and document
## References
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)

17
LICENSE
View File

@ -1,17 +0,0 @@
Copyright (C) <2003> David Schleef <ds@schleef.org>
Copyright (C) 2016-2019 Tim-Philipp Müller <tim@centricular.com>
Copyright (C) 2016-2017 PlayGineering Ltd.
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

View File

@ -19,13 +19,13 @@ GStreamer plugins for IDS uEye cameras with frame analysis capabilities.
## Usage Examples
### Basic capture from IDS uEye camera
```bash
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! queue ! autovideosink
```
### Frame filtering based on column analysis
Drop frames when column mean deviates from rolling baseline by more than 0.5:
```bash
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! queue ! autovideosink
```
@ -34,17 +34,17 @@ gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.in
### Additional rollingsum examples
Analyze column 320 with larger window:
```bash
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=5000 column-index=320 threshold=0.3 ! queue ! autovideosink
```
Use stride for faster processing (sample every 2 rows):
```bash
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 stride=2 threshold=0.5 ! queue ! autovideosink
```
Lower threshold for more sensitive detection:
```bash
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum threshold=0.2 ! queue ! autovideosink
```
@ -81,7 +81,7 @@ If you prefer to build manually:
3. Install [IDS uEye SDK](https://www.ids-imaging.com) (default path: `C:\Program Files\IDS\uEye\Develop`)
4. Run:
```bash
```powershell
git clone https://github.com/joshdoe/gst-plugins-vision.git
cd gst-plugins-vision
mkdir build
@ -101,7 +101,7 @@ The `build.ps1` script automatically copies plugins to `$env:GST_PLUGIN_PATH` if
### Verify Installation
```bash
```powershell
gst-inspect-1.0 idsueyesrc
gst-inspect-1.0 rollingsum
```
@ -109,3 +109,13 @@ gst-inspect-1.0 rollingsum
## Documentation
- [Rolling Sum Filter Design](DESIGN_ROLLINGSUM.md)
## Debugging
add `$env:GST_DEBUG_DUMP_DOT_DIR='.'`
![](dot_pause-play.svg)
to get dotfile, and view using https://dreampuf.github.io/GraphvizOnline/, or
```pwsh
dot -Tsvg C:\dev\gst-plugins-vision\0.00.02.922833100-gst-launch.PAUSED_PLAYING.dot -o same.svg
```

View File

@ -1,9 +1,28 @@
# GStreamer Rolling Sum Plugin Guide
# GStreamer Rolling Sum Plugin - Complete Documentation
## Table of Contents
- [Overview](#overview)
- [How It Works](#how-it-works)
- [Architecture & Design](#architecture--design)
- [Plugin Properties](#plugin-properties)
- [Basic Usage](#basic-usage)
- [Debugging](#debugging)
- [CSV Analysis](#csv-analysis)
- [Recommended Thresholds](#recommended-thresholds)
- [Troubleshooting](#troubleshooting)
- [Performance](#performance)
- [Integration Examples](#integration-examples)
- [Developer Guide](#developer-guide)
- [References](#references)
## Overview
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames
**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
## How It Works
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
@ -12,15 +31,134 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
5. **CSV Logging**: Records all frame statistics for analysis
### Data Flow
```mermaid
graph TD
A[Video Frame] --> B[Extract Column]
B --> C[Calculate Column Mean]
C --> D[Store in Ring Buffer]
D --> E[Update Rolling Mean]
E --> F{Deviation > Threshold?}
F -->|Yes| G[DROP Frame]
F -->|No| H[PASS Frame]
C --> E
style G fill:#ff6b6b
style H fill:#51cf66
```
### Ring Buffer Operation
```mermaid
graph LR
subgraph Ring Buffer
A[0] --> B[1]
B --> C[2]
C --> D[...]
D --> E[N-1]
E ---|wrap| A
end
F[New Frame Mean] --> G[ring_index]
G --> A
H[Rolling Mean] --> I[Sum all values]
I --> J[Divide by count]
style G fill:#ffd43b
```
## Architecture & Design
### Base Class
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
- In-place transform (analysis only, no frame modification)
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
- Returns `GST_FLOW_OK` to pass frames
### Element Structure
```c
struct _GstRollingSum
{
GstBaseTransform element;
/* Properties */
gint window_size; // Number of frames in rolling window (default: 1000)
gint column_index; // Which column to analyze (default: 1, second column)
gint stride; // Row sampling stride (default: 1, every row)
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
gchar *csv_file; // CSV output file path (default: NULL)
/* State */
gdouble *ring_buffer; // Circular buffer of column means
gint ring_index; // Current position in ring buffer
gint frame_count; // Total frames processed
gdouble rolling_mean; // Current rolling mean
FILE *csv_fp; // CSV file pointer
};
```
### Algorithm (Simplified from cli.py)
**Per Frame Processing:**
1. **Extract column data:**
- Select column at column_index
- Sample every stride rows
- Calculate mean of sampled pixels: frame_mean
2. **Update ring buffer:**
- Store frame_mean in ring_buffer[ring_index]
- Increment ring_index (wrap around)
3. **Calculate rolling mean:**
- Sum values in ring buffer (up to window_size or frame_count)
- Divide by actual window size
4. **Calculate deviation:**
- deviation = abs(frame_mean - rolling_mean)
- normalized_deviation = deviation / 255.0 (for 8-bit video)
5. **Decision:**
- If normalized_deviation > threshold: DROP frame
- Else: PASS frame
**Key Simplifications from cli.py:**
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
- **No recording logic**: Just drop/pass, no buffering for output segments
- **No patience mechanism**: Immediate decision per frame
### Video Format Support
**Initial Implementation:**
- **Primary target**: Grayscale (GRAY8, GRAY16)
- **Secondary**: Bayer formats (common in machine vision)
**Caps Filter:**
```c
static GstStaticPadTemplate sink_template =
GST_STATIC_PAD_TEMPLATE ("sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS (
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
)
);
```
## Plugin Properties
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `window-size` | int | 1000 | Number of frames in rolling window (1-100000) |
| `column-index` | int | 1 | Which vertical column to analyze (0-based) |
| `stride` | int | 1 | Row sampling stride (1 = every row) |
| `threshold` | double | 0.5 | Normalized deviation threshold for dropping frames (0.0-1.0) |
| `csv-file` | string | NULL | Path to CSV file for logging (NULL = no logging) |
| Property | Type | Default | Range | Description |
|----------|------|---------|-------|-------------|
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
| `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) |
| `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) |
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames |
| `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) |
### Understanding Normalized Deviation
@ -35,24 +173,43 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
### Simple Pipeline
```bash
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
videoconvert ! \
video/x-raw,format=GRAY8 ! \
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! \
```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
videoconvert ! `
video/x-raw,format=GRAY8 ! `
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! `
autovideosink
```
### With CSV Logging
```bash
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! \
videoconvert ! \
video/x-raw,format=GRAY8 ! \
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! \
```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! `
videoconvert ! `
video/x-raw,format=GRAY8 ! `
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! `
fakesink
```
### Custom Configuration
```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! `
queue ! `
autovideosink
```
### With Format Conversion
```powershell
gst-launch-1.0 idsueyesrc ! `
videoconvert ! `
video/x-raw,format=GRAY8 ! `
rollingsum ! `
autovideosink
```
## Debugging
### Enable Debug Output
@ -105,7 +262,7 @@ GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...]
#### 1. Verify Plugin Loaded
```bash
```powershell
gst-inspect-1.0 rollingsum
```
@ -149,15 +306,22 @@ frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped
Use the included analysis script:
```bash
uv run analyze_sma.py output.csv
```powershell
uv run scripts/analyze_sma.py output.csv
```
**Output includes:**
- Statistical summary (min/max/mean/std)
- Threshold recommendations based on percentiles
- Standard deviation-based suggestions
- Visualization plots (`output_analysis.png`)
- Visualization plots saved to `results/debug/`
- Archived CSV with timestamp in `results/debug/`
**Output files are automatically organized:**
- `results/debug/output_YYYYMMDD_HHMMSS.csv` - Archived CSV
- `results/debug/output_analysis_YYYYMMDD_HHMMSS.png` - Analysis plots
The `results/` directory is gitignored to keep your repository clean.
### Interpreting Results
@ -176,7 +340,7 @@ Based on analysis of stable camera footage:
### For General Use
```bash
```powershell
# Conservative (1-2% frame drop)
threshold=0.0003
@ -190,17 +354,17 @@ threshold=0.0001
### For Specific Scenarios
**High-speed acquisition** (minimal processing):
```bash
```powershell
window-size=100 threshold=0.0005
```
**Quality-focused** (stable scenes):
```bash
```powershell
window-size=1000 threshold=0.0001
```
**Real-time monitoring** (fast response):
```bash
```powershell
window-size=50 threshold=0.0002
```
@ -212,7 +376,7 @@ window-size=50 threshold=0.0002
**Solution**:
1. Run with CSV logging
2. Analyze with `uv run analyze_sma.py output.csv`
2. Analyze with `uv run scripts/analyze_sma.py output.csv`
3. Use recommended threshold from 90th-99th percentile
### Too many frames dropped (threshold too low)
@ -252,13 +416,32 @@ WARNING rollingsum: Column index 1000 >= width 1224, using column 0
- Choose different `column-index` (avoid edges)
- Use `stride=2` or higher for faster processing
## Performance Tips
## Performance
### Performance Tips
1. **Larger window = more stable** but slower to adapt to scene changes
2. **Stride > 1** reduces computation but less accurate column mean
3. **CSV logging** has minimal performance impact
4. **Debug level 5** can produce massive logs, use only when needed
### Memory Usage
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
- Minimal per-frame allocation
### CPU Usage
- Column extraction: O(height/stride)
- Rolling mean update: O(1) using incremental sum
- Very lightweight compared to full-frame processing
### Optimization Opportunities
1. **Incremental mean**: Track sum instead of recalculating
2. **SIMD**: Vectorize column summation
3. **Skip calculation**: Only recalc every N frames if baseline is stable
## Integration Examples
### Python Script Control
@ -281,7 +464,7 @@ subprocess.run([
])
# Analyze results
subprocess.run(['uv', 'run', 'analyze_sma.py', 'output.csv'])
subprocess.run(['uv', 'run', 'scripts/analyze_sma.py', 'output.csv'])
```
### Adaptive Threshold
@ -298,7 +481,33 @@ recommended_threshold = df['normalized_deviation'].quantile(0.95)
print(f"Recommended threshold: {recommended_threshold:.6f}")
```
## Developer Notes
## Developer Guide
### Implementation Files
**Directory Structure:**
```
gst/rollingsum/
├── CMakeLists.txt
├── gstrollingsum.c
└── gstrollingsum.h
```
**gstrollingsum.h:**
- Element type definitions
- Structure declarations
- Property enums
- Function prototypes
**gstrollingsum.c:**
- GObject methods (init, dispose, get/set properties)
- GstBaseTransform methods (transform_ip)
- Helper functions (extract_column_mean, update_rolling_mean)
- Plugin registration
**CMakeLists.txt:**
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
- Link GStreamer base and video libraries
### Adding New Features
@ -309,14 +518,17 @@ Key files:
### Rebuild After Changes
```bash
```powershell
.\build.ps1 # Windows
```
```bash
./build.sh # Linux
```
### Testing
```bash
```powershell
# Quick test
gst-inspect-1.0 rollingsum
@ -325,16 +537,77 @@ $env:GST_DEBUG="rollingsum:5"
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
```
### Testing Strategy
**Unit Tests:**
- Ring buffer wrapping
- Mean calculation accuracy
- Threshold comparison logic
**Integration Tests:**
- Pipeline with videotestsrc
- Pipeline with idsueyesrc
- Frame drop verification
- Property changes during playback
**Test Cases:**
1. Static video (all frames similar) → all pass
2. Single bright frame → that frame drops
3. Gradual change → frames pass
4. Periodic pattern → pattern frames drop
### Integration with Existing Project
**Build System:**
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
```cmake
add_subdirectory (rollingsum)
```
**Documentation:**
Update [`README.md`](README.md):
- Add rollingsum to "Other elements" section
- Add pipeline example
### Future Enhancements
**Phase 2 (If Needed):**
- Add EMA baseline tracking (like cli.py)
- Add variance-based thresholds
- Support multiple columns or regions
- Add metadata output (tag frames with deviation values)
- RGB format support (analyze specific channel)
**Phase 3 (Advanced):**
- Full cli.py recording logic
- Buffer and output segments
- Integration with probe detection systems
### Implementation Checklist
- [x] Create gst/rollingsum directory
- [x] Implement gstrollingsum.h
- [x] Implement gstrollingsum.c
- [x] Create CMakeLists.txt
- [x] Update gst/CMakeLists.txt
- [x] Build and test basic functionality
- [x] Test with idsueyesrc
- [x] Update README.md
- [x] Create feature branch
- [x] Commit and document
## References
- [DESIGN_ROLLINGSUM.md](DESIGN_ROLLINGSUM.md) - Design document
- [analyze_sma.py](analyze_sma.py) - Analysis tool
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
- [scripts/analyze_sma.py](scripts/analyze_sma.py) - Analysis tool
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
## Support
For issues or questions:
1. Enable debug output (`GST_DEBUG=rollingsum:5`)
1. Enable debug output (`$env:GST_DEBUG="rollingsum:5"` in PowerShell)
2. Generate CSV log and analyze
3. Check this guide's troubleshooting section
4. Review debug output for errors/warnings

125
dot_pause-play.svg Normal file
View File

@ -0,0 +1,125 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 14.0.2 (20251019.1705)
-->
<!-- Title: pipeline Pages: 1 -->
<svg width="1103pt" height="290pt"
viewBox="0.00 0.00 1103.00 290.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 285.85)">
<title>pipeline</title>
<polygon fill="white" stroke="none" points="-4,4 -4,-285.85 1099.2,-285.85 1099.2,4 -4,4"/>
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-268.35" font-family="sans" font-size="10.00">&lt;GstPipeline&gt;</text>
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-255.6" font-family="sans" font-size="10.00">pipeline0</text>
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-242.85" font-family="sans" font-size="10.00">[&gt;]</text>
<g id="clust1" class="cluster">
<title>cluster_node_udpsink0_000001FC4DD89DE0</title>
<path fill="#aaaaff" stroke="black" d="M886.45,-75.6C886.45,-75.6 1083.2,-75.6 1083.2,-75.6 1089.2,-75.6 1095.2,-81.6 1095.2,-87.6 1095.2,-87.6 1095.2,-215.6 1095.2,-215.6 1095.2,-221.6 1089.2,-227.6 1083.2,-227.6 1083.2,-227.6 886.45,-227.6 886.45,-227.6 880.45,-227.6 874.45,-221.6 874.45,-215.6 874.45,-215.6 874.45,-87.6 874.45,-87.6 874.45,-81.6 880.45,-75.6 886.45,-75.6"/>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-216" font-family="Bitstream Vera Sans" font-size="8.00">GstUDPSink</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-206.25" font-family="Bitstream Vera Sans" font-size="8.00">udpsink0</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-196.5" font-family="Bitstream Vera Sans" font-size="8.00">[&gt;]</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-186.75" font-family="Bitstream Vera Sans" font-size="8.00">last&#45;sample=((GstSample*) 000001FC4DE728F0)</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-177" font-family="Bitstream Vera Sans" font-size="8.00">used&#45;socket=((GSocket*) 000001FC4DD85E90)</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-167.25" font-family="Bitstream Vera Sans" font-size="8.00">used&#45;socket&#45;v6=((GSocket*) 000001FC4DD87A30)</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-157.5" font-family="Bitstream Vera Sans" font-size="8.00">clients=&quot;127.0.0.1:5000&quot;</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-147.75" font-family="Bitstream Vera Sans" font-size="8.00">host=&quot;127.0.0.1&quot;</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-138" font-family="Bitstream Vera Sans" font-size="8.00">port=5000</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_node_udpsink0_000001FC4DD89DE0_sink</title>
</g>
<g id="clust3" class="cluster">
<title>cluster_node_queue0_000001FC4BB96850</title>
<path fill="#aaffaa" stroke="black" d="M524.2,-75.6C524.2,-75.6 673.2,-75.6 673.2,-75.6 679.2,-75.6 685.2,-81.6 685.2,-87.6 685.2,-87.6 685.2,-156.6 685.2,-156.6 685.2,-162.6 679.2,-168.6 673.2,-168.6 673.2,-168.6 524.2,-168.6 524.2,-168.6 518.2,-168.6 512.2,-162.6 512.2,-156.6 512.2,-156.6 512.2,-87.6 512.2,-87.6 512.2,-81.6 518.2,-75.6 524.2,-75.6"/>
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-157" font-family="Bitstream Vera Sans" font-size="8.00">GstQueue</text>
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-147.25" font-family="Bitstream Vera Sans" font-size="8.00">queue0</text>
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-137.5" font-family="Bitstream Vera Sans" font-size="8.00">[&gt;]</text>
</g>
<g id="clust4" class="cluster">
<title>cluster_node_queue0_000001FC4BB96850_sink</title>
</g>
<g id="clust5" class="cluster">
<title>cluster_node_queue0_000001FC4BB96850_src</title>
</g>
<g id="clust6" class="cluster">
<title>cluster_node_idsueyesrc0_000001FC4DD4B2C0</title>
<path fill="#ffaaaa" stroke="black" d="M81.98,-75.6C81.98,-75.6 241.98,-75.6 241.98,-75.6 247.98,-75.6 253.98,-81.6 253.98,-87.6 253.98,-87.6 253.98,-186.6 253.98,-186.6 253.98,-192.6 247.98,-198.6 241.98,-198.6 241.98,-198.6 81.98,-198.6 81.98,-198.6 75.98,-198.6 69.98,-192.6 69.98,-186.6 69.98,-186.6 69.98,-87.6 69.98,-87.6 69.98,-81.6 75.98,-75.6 81.98,-75.6"/>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-187" font-family="Bitstream Vera Sans" font-size="8.00">GstIdsueyeSrc</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-177.25" font-family="Bitstream Vera Sans" font-size="8.00">idsueyesrc0</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-167.5" font-family="Bitstream Vera Sans" font-size="8.00">[&gt;]</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-157.75" font-family="Bitstream Vera Sans" font-size="8.00">config&#45;file=&quot;ini/200fps&#45;2456x4pix&#45;cw.ini&quot;</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-148" font-family="Bitstream Vera Sans" font-size="8.00">exposure=3.237784</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-138.25" font-family="Bitstream Vera Sans" font-size="8.00">framerate=300.000000</text>
</g>
<g id="clust7" class="cluster">
<title>cluster_node_idsueyesrc0_000001FC4DD4B2C0_src</title>
</g>
<!-- legend -->
<g id="node1" class="node">
<title>legend</title>
<polygon fill="lightgrey" stroke="black" points="322.95,-67.2 0,-67.2 0,0 322.95,0 322.95,-67.2"/>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-55.05" font-family="sans" font-size="9.00">Legend</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-43.05" font-family="sans" font-size="9.00">Element&#45;States: [~] void&#45;pending, [0] null, [&#45;] ready, [=] paused, [&gt;] playing</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-31.05" font-family="sans" font-size="9.00">Pad&#45;Activation: [&#45;] none, [&gt;] push, [&lt;] pull</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-19.05" font-family="sans" font-size="9.00">Pad&#45;Flags: [b]locked, [f]lushing, [b]locking, [E]OS; upper&#45;case is set</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-7.05" font-family="sans" font-size="9.00">Pad&#45;Task: [T] has started task, [t] has paused task</text>
</g>
<!-- node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0 -->
<g id="node2" class="node">
<title>node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0</title>
<polygon fill="#aaaaff" stroke="black" points="1011.33,-115.6 957.33,-115.6 957.33,-91.6 1011.33,-91.6 1011.33,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="984.33" y="-107.05" font-family="sans" font-size="9.00">sink</text>
<text xml:space="preserve" text-anchor="middle" x="984.33" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb]</text>
</g>
<!-- node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0 -->
<g id="node3" class="node">
<title>node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0</title>
<polygon fill="#aaaaff" stroke="black" points="582.2,-115.6 528.2,-115.6 528.2,-91.6 582.2,-91.6 582.2,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="555.2" y="-107.05" font-family="sans" font-size="9.00">sink</text>
<text xml:space="preserve" text-anchor="middle" x="555.2" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb]</text>
</g>
<!-- node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70 -->
<g id="node4" class="node">
<title>node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70</title>
<polygon fill="#ffaaaa" stroke="black" points="669.2,-115.6 615.2,-115.6 615.2,-91.6 669.2,-91.6 669.2,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="642.2" y="-107.05" font-family="sans" font-size="9.00">src</text>
<text xml:space="preserve" text-anchor="middle" x="642.2" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb][T]</text>
</g>
<!-- node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0&#45;&gt;node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70 -->
<!-- node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70&#45;&gt;node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0 -->
<g id="edge2" class="edge">
<title>node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70&#45;&gt;node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0</title>
<path fill="none" stroke="black" d="M669.61,-103.6C729.66,-103.6 876.61,-103.6 945.95,-103.6"/>
<polygon fill="black" stroke="black" points="945.66,-107.1 955.66,-103.6 945.66,-100.1 945.66,-107.1"/>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-179.05" font-family="monospace" font-size="9.00">video/x&#45;raw</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-168.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;format: BGR</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-158.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;width: 2456</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-147.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;height: 4</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-137.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;interlace&#45;mode: progressive</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-126.55" font-family="monospace" font-size="9.00"> &#160;pixel&#45;aspect&#45;ratio: 1/1</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-116.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;colorimetry: sRGB</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-105.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;framerate: 0/1</text>
</g>
<!-- node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0 -->
<g id="node5" class="node">
<title>node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0</title>
<polygon fill="#ffaaaa" stroke="black" points="188.48,-115.6 134.48,-115.6 134.48,-91.6 188.48,-91.6 188.48,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="161.48" y="-107.05" font-family="sans" font-size="9.00">src</text>
<text xml:space="preserve" text-anchor="middle" x="161.48" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb][T]</text>
</g>
<!-- node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0&#45;&gt;node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0 -->
<g id="edge3" class="edge">
<title>node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0&#45;&gt;node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0</title>
<path fill="none" stroke="black" d="M188.79,-103.6C256.58,-103.6 437.44,-103.6 516.42,-103.6"/>
<polygon fill="black" stroke="black" points="516.23,-107.1 526.23,-103.6 516.23,-100.1 516.23,-107.1"/>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-179.05" font-family="monospace" font-size="9.00">video/x&#45;raw</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-168.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;format: BGR</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-158.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;width: 2456</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-147.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;height: 4</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-137.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;interlace&#45;mode: progressive</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-126.55" font-family="monospace" font-size="9.00"> &#160;pixel&#45;aspect&#45;ratio: 1/1</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-116.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;colorimetry: sRGB</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-105.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;framerate: 0/1</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 12 KiB

222
ini/200fps-2456x4pix-cw.ini Normal file
View File

@ -0,0 +1,222 @@
[Versions]
ueye_api_64.dll=4.93.1730
ueye_usb_64.sys=4.93.1314
ueye_boot_64.sys=4.93.1314
[Sensor]
Sensor=UI308xCP-C
Sensor bit depth=0
Sensor source gain=24
FPN correction mode=0
Black reference mode=0
Sensor digital gain=0
[Image size]
Start X=0
Start Y=0
Start X absolute=0
Start Y absolute=0
Width=2456
Height=4
Binning=0
Subsampling=0
[Scaler]
Mode=0
Factor=0.000000
[Multi AOI]
Enabled=0
Mode=0
x1=0
x2=0
x3=0
x4=0
y1=0
y2=0
y3=0
y4=0
[Shutter]
Mode=0
Linescan number=0
[Log Mode]
Mode=3
Manual value=0
Manual gain=0
[Timing]
Pixelclock=237
Extended pixelclock range=0
Framerate=200.151466
Exposure=4.903189
Long exposure=0
Dual exposure ratio=0
[Selected Converter]
IS_SET_CM_RGB32=2
IS_SET_CM_RGB24=2
IS_SET_CM_RGB16=2
IS_SET_CM_RGB15=2
IS_SET_CM_Y8=2
IS_SET_CM_RGB8=2
IS_SET_CM_BAYER=8
IS_SET_CM_UYVY=2
IS_SET_CM_UYVY_MONO=2
IS_SET_CM_UYVY_BAYER=2
IS_CM_CBYCRY_PACKED=0
IS_SET_CM_RGBY=8
IS_SET_CM_RGB30=2
IS_SET_CM_Y12=2
IS_SET_CM_BAYER12=8
IS_SET_CM_Y16=2
IS_SET_CM_BAYER16=8
IS_CM_BGR12_UNPACKED=2
IS_CM_BGRA12_UNPACKED=2
IS_CM_JPEG=0
IS_CM_SENSOR_RAW10=8
IS_CM_MONO10=2
IS_CM_BGR10_UNPACKED=2
IS_CM_RGBA8_PACKED=2
IS_CM_RGB8_PACKED=2
IS_CM_RGBY8_PACKED=8
IS_CM_RGB10V2_PACKED=8
IS_CM_RGB12_UNPACKED=2
IS_CM_RGBA12_UNPACKED=2
IS_CM_RGB10_UNPACKED=2
IS_CM_RGB8_PLANAR=2
[Parameters]
Colormode=1
Gamma=1.000000
Hardware Gamma=0
Blacklevel Mode=0
Blacklevel Offset=4
Hotpixel Mode=2
Hotpixel Threshold=0
Sensor Hotpixel=0
Adaptive hotpixel correction enable=0
Adaptive hotpixel correction mode=0
Adaptive hotpixel correction sensitivity=3
GlobalShutter=0
AllowRawWithLut=0
[Gain]
Master=0
Red=19
Green=0
Blue=33
GainBoost=1
[Processing]
EdgeEnhancementFactor=0
RopEffect=0
Whitebalance=0
Whitebalance Red=1.000000
Whitebalance Green=1.000000
Whitebalance Blue=1.000000
Color correction=4
Color_correction_factor=1.000000
Color_correction_satU=100
Color_correction_satV=100
Bayer Conversion=1
JpegCompression=0
NoiseMode=0
ImageEffect=0
LscModel=0
WideDynamicRange=0
[Auto features]
Auto Framerate control=0
Brightness exposure control=0
Brightness gain control=0
Auto Framerate Sensor control=0
Brightness exposure Sensor control=0
Brightness gain Sensor control=0
Brightness exposure Sensor control photometry=0
Brightness gain Sensor control photometry=0
Brightness control once=0
Brightness reference=128
Brightness speed=50
Brightness max gain=100
Brightness max exposure=4.903189
Brightness Aoi Left=0
Brightness Aoi Top=0
Brightness Aoi Width=2456
Brightness Aoi Height=4
Brightness Hysteresis=2
AutoImageControlMode=2
AutoImageControlPeakWhiteChannel=0
AutoImageControlExposureMinimum=0.000000
AutoImageControlPeakWhiteChannelMode=0
AutoImageControlPeakWhiteGranularity=0
Auto WB control=0
Auto WB type=2
Auto WB RGB color model=1
Auto WB RGB color temperature=5000
Auto WB offsetR=0
Auto WB offsetB=0
Auto WB gainMin=0
Auto WB gainMax=100
Auto WB speed=50
Auto WB Aoi Left=0
Auto WB Aoi Top=0
Auto WB Aoi Width=2456
Auto WB Aoi Height=4
Auto WB Once=0
Auto WB Hysteresis=2
Brightness Skip Frames Trigger Mode=4
Brightness Skip Frames Freerun Mode=4
Auto WB Skip Frames Trigger Mode=4
Auto WB Skip Frames Freerun Mode=4
[Trigger and Flash]
Trigger mode=0
Trigger timeout=200
Trigger delay=0
Trigger debounce mode=0
Trigger debounce delay time=1
Trigger burst size=1
Trigger prescaler frame=1
Trigger prescaler line=1
Trigger input=1
Flash strobe=0
Flash delay=0
Flash duration=0
Flash auto freerun=0
PWM mode=0
PWM frequency=20000000
PWM dutycycle=20000000
GPIO state=3
GPIO direction=0
GPIO1 Config=1
GPIO2 Config=1
[Vertical AOI Merge Mode]
Mode=0
Position=0
Additional Position=0
Height=2
[Level Controlled Trigger Mode]
Mode=0
[Memory]
Camera memory mode=1

39
network_guide.md Normal file
View File

@ -0,0 +1,39 @@
# how to send a line
real data
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/200fps-2456x4pix-cw.ini exposure=5 framerate=300 `
! queue `
! udpsink host=127.0.0.1 port=5000
```
note: 5ms is bit too fast for us
### Python/OpenCV Receiver
```pwsh
uv run scripts/recv_raw_column.py
```
or rolling like
```pwsh
uv run .\scripts\recv_raw_rolling.py
```
See [`scripts/recv_raw_column.py`](scripts/recv_raw_column.py) for the Python implementation with debug options.
# demo data
## Sender (crop to first column, send raw over UDP)
```pwsh
gst-launch-1.0 -v `
videotestsrc pattern=smpte ! `
videocrop left=0 right=639 top=0 bottom=0 ! `
video/x-raw,format=RGB,width=1,height=640,framerate=30/1 ! `
udpsink host=127.0.0.1 port=5000
```
### GStreamer Receiver (raw UDP → display)
```pwsh
gst-launch-1.0 -v `
udpsrc port=5000 caps="video/x-raw,format=RGB,width=1,height=640,framerate=30/1" ! `
videoconvert ! `
autovideosink
```

View File

@ -18,11 +18,17 @@ import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from pathlib import Path
from datetime import datetime
import shutil
def analyze_csv(csv_file: str = "output.csv"):
"""Analyze the rolling sum CSV data and generate insights."""
# Create output directory
output_dir = Path("results/debug")
output_dir.mkdir(parents=True, exist_ok=True)
# Read the CSV
try:
df = pd.read_csv(csv_file)
@ -30,6 +36,12 @@ def analyze_csv(csv_file: str = "output.csv"):
print(f"Error: CSV file '{csv_file}' not found.")
sys.exit(1)
# Copy input CSV to results directory with timestamp
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
csv_name = Path(csv_file).stem
archived_csv = output_dir / f"{csv_name}_{timestamp}.csv"
shutil.copy(csv_file, archived_csv)
print("=" * 80)
print(f"ROLLING SUM ANALYSIS - {csv_file}")
print("=" * 80)
@ -98,16 +110,17 @@ def analyze_csv(csv_file: str = "output.csv"):
print()
# Create visualizations
create_plots(df, csv_file)
plot_file = create_plots(df, csv_file, output_dir, timestamp)
print("=" * 80)
print("PLOTS SAVED:")
print(f" - {csv_file.replace('.csv', '_analysis.png')}")
print("OUTPUT FILES:")
print(f" CSV Archive: {archived_csv}")
print(f" Analysis Plot: {plot_file}")
print("=" * 80)
def create_plots(df: pd.DataFrame, csv_file: str):
"""Create analysis plots."""
def create_plots(df: pd.DataFrame, csv_file: str, output_dir: Path, timestamp: str) -> Path:
"""Create analysis plots and return the output file path."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
fig.suptitle(f'Rolling Sum Analysis - {csv_file}', fontsize=16, fontweight='bold')
@ -167,11 +180,14 @@ def create_plots(df: pd.DataFrame, csv_file: str):
plt.tight_layout()
# Save the plot
output_file = csv_file.replace('.csv', '_analysis.png')
# Save the plot to results/debug
csv_name = Path(csv_file).stem
output_file = output_dir / f"{csv_name}_analysis_{timestamp}.png"
plt.savefig(output_file, dpi=150, bbox_inches='tight')
print(f"\n✓ Saved analysis plot to: {output_file}\n")
return output_file
if __name__ == "__main__":
csv_file = sys.argv[1] if len(sys.argv) > 1 else "output.csv"

View File

@ -0,0 +1,61 @@
#!/usr/bin/env python3
# /// script
# requires-python = "<=3.10"
# dependencies = [
# "opencv-python",
# "numpy",
# ]
# ///
import socket
import numpy as np
import cv2
# Debug flag - set to True to see frame reception details
DEBUG = False
# Stream parameters (match your GStreamer sender)
WIDTH = 1
HEIGHT = 640 # Default videotestsrc height
CHANNELS = 3
FRAME_SIZE = WIDTH * HEIGHT * CHANNELS # bytes
UDP_IP = "0.0.0.0"
UDP_PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
print(f"Receiving raw {WIDTH}x{HEIGHT} RGB frames on UDP port {UDP_PORT}")
if DEBUG:
print(f"Expected frame size: {FRAME_SIZE} bytes")
cv2.namedWindow("Raw Column Stream", cv2.WINDOW_NORMAL)
frame_count = 0
while True:
data, addr = sock.recvfrom(65536)
if len(data) != FRAME_SIZE:
if DEBUG:
print(f"Received {len(data)} bytes (expected {FRAME_SIZE}), skipping...")
continue
if DEBUG:
frame_count += 1
if frame_count % 30 == 0:
print(f"Received {frame_count} frames")
frame = np.frombuffer(data, dtype=np.uint8).reshape((HEIGHT, WIDTH, CHANNELS))
# scale for visibility
frame_large = cv2.resize(
frame, (200, HEIGHT), interpolation=cv2.INTER_NEAREST
)
cv2.imshow("Raw Column Stream", frame_large)
if cv2.waitKey(1) == 27: # ESC to quit
break
cv2.destroyAllWindows()

View File

@ -0,0 +1,86 @@
#!/usr/bin/env python3
# /// script
# requires-python = "<=3.10"
# dependencies = [
# "opencv-python",
# "numpy",
# ]
# ///
import socket
import numpy as np
import cv2
# Debug flag - set to True to see frame reception details
DEBUG = False
# Rotation mode - set to rotate incoming data before display
# Options: None, cv2.ROTATE_90_CLOCKWISE, cv2.ROTATE_90_COUNTERCLOCKWISE, cv2.ROTATE_180
ROTATION = cv2.ROTATE_90_COUNTERCLOCKWISE # Rotate rows to columns
# Stream parameters (match your GStreamer sender)
COLUMN_WIDTH = 4 # Width from 200fps-2456x4pix-cw.ini
COLUMN_HEIGHT = 2456 # Height from 200fps-2456x4pix-cw.ini
CHANNELS = 3
FRAME_SIZE = COLUMN_WIDTH * COLUMN_HEIGHT * CHANNELS # bytes (29472)
# Display parameters
DISPLAY_WIDTH = 800 # Width of rolling display in pixels
DISPLAY_HEIGHT = COLUMN_HEIGHT
UDP_IP = "0.0.0.0"
UDP_PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
print(f"Receiving raw {COLUMN_WIDTH}x{COLUMN_HEIGHT} RGB columns on UDP port {UDP_PORT}")
print(f"Display width: {DISPLAY_WIDTH} pixels (rolling)")
if DEBUG:
print(f"Expected frame size: {FRAME_SIZE} bytes")
cv2.namedWindow("Rolling Column Stream", cv2.WINDOW_NORMAL)
# Initialize the rolling buffer
rolling_buffer = np.zeros((DISPLAY_HEIGHT, DISPLAY_WIDTH, CHANNELS), dtype=np.uint8)
current_column = 0
frame_count = 0
while True:
data, addr = sock.recvfrom(65536)
if len(data) != FRAME_SIZE:
if DEBUG:
print(f"Received {len(data)} bytes (expected {FRAME_SIZE}), skipping...")
continue
if DEBUG:
frame_count += 1
if frame_count % 30 == 0:
print(f"Received {frame_count} frames, current column: {current_column}")
# Parse the incoming data
frame = np.frombuffer(data, dtype=np.uint8).reshape((COLUMN_WIDTH, COLUMN_HEIGHT, CHANNELS))
# Apply rotation if configured
if ROTATION is not None:
rotated = cv2.rotate(frame, ROTATION)
else:
rotated = frame
# Extract only the first column for smoother rolling (1 pixel/frame)
column = rotated[:, 0:1, :]
# Insert the single column into the rolling buffer at the current position
rolling_buffer[:, current_column:current_column+1, :] = column
# Move to the next column position, wrapping around when reaching the end
current_column = (current_column + 1) % DISPLAY_WIDTH
# Display the rolling buffer
cv2.imshow("Rolling Column Stream", rolling_buffer)
if cv2.waitKey(1) == 27: # ESC to quit
break
cv2.destroyAllWindows()