Compare commits

...

10 Commits

Author SHA1 Message Date
yair
e58e2319a3 dot stuff 2025-11-14 17:11:18 +02:00
yair
4d35d16c72 udp 2025-11-14 17:00:20 +02:00
yair
2699913e92 rolling 2025-11-14 16:29:31 +02:00
yair
72e7091c61 rolling 2025-11-14 16:18:27 +02:00
yair
94f7c04dc6 Fix recv_raw_column.py height mismatch and update script paths in docs
- Fixed HEIGHT from 480 to 640 to match actual videotestsrc output
- Added DEBUG flag to control debug output visibility
- Added cv2.namedWindow() for proper window initialization
- Updated all Python script references in markdown files to scripts/ folder
- Updated network_guide.md with correct frame dimensions and Python receiver option
2025-11-14 15:33:17 +02:00
yair
ee4fd76e13 rem unused lic 2025-11-14 15:01:03 +02:00
yair
1c7ca124b1 del 2025-11-14 15:00:05 +02:00
yair
cb1e5c7607 docs: convert bash commands to PowerShell and merge ROLLINGSUM docs
- Updated all commands in README.md and ROLLINGSUM_GUIDE.md to use PowerShell syntax
- Changed line continuation from backslash (\) to backtick ()
- Updated environment variable syntax to PowerShell format ()
- Merged DESIGN_ROLLINGSUM.md into ROLLINGSUM_GUIDE.md for comprehensive documentation
- Combined user guide with technical design details in single document
- Added table of contents and improved organization
2025-11-14 14:58:39 +02:00
yair
d0467aaf65 converted cli to pwsh 2025-11-14 14:55:05 +02:00
yair
44083222ee refactor: Output analysis results to results/debug directory
- Update analyze_sma.py to save files to results/debug/
- Add timestamp to archived CSV and plot files
- Update .gitignore to exclude results/ directory
- Update ROLLINGSUM_GUIDE.md with new output locations
2025-11-14 14:34:58 +02:00
11 changed files with 884 additions and 356 deletions

3
.gitignore vendored
View File

@ -38,3 +38,6 @@ ipch/
*.mkv *.mkv
*.raw *.raw
*.dot
gst_plugs/
results/

View File

@ -1,290 +0,0 @@
# Rolling Sum Filter - Design Document
## Overview
A GStreamer element that drops frames based on rolling mean analysis of a single column of pixels. Inspired by the detection algorithm in `cli.py`, simplified for real-time streaming.
## Element Name
**`rollingsum`** - Transform element that analyzes pixel values and selectively drops frames
## Purpose
Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
## Architecture
### Base Class
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
- In-place transform (analysis only, no frame modification)
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
- Returns `GST_FLOW_OK` to pass frames
### Element Structure
```c
struct _GstRollingSum
{
GstBaseTransform element;
/* Properties */
gint window_size; // Number of frames in rolling window (default: 1000)
gint column_index; // Which column to analyze (default: 1, second column)
gint stride; // Row sampling stride (default: 1, every row)
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
/* State */
gdouble *ring_buffer; // Circular buffer of column means
gint ring_index; // Current position in ring buffer
gint frame_count; // Total frames processed
gdouble rolling_mean; // Current rolling mean
};
```
## Algorithm (Simplified from cli.py)
### Per Frame Processing
```
1. Extract column data:
- Select column at column_index
- Sample every stride rows
- Calculate mean of sampled pixels: frame_mean
2. Update ring buffer:
- Store frame_mean in ring_buffer[ring_index]
- Increment ring_index (wrap around)
3. Calculate rolling mean:
- Sum values in ring buffer (up to window_size or frame_count)
- Divide by actual window size
4. Calculate deviation:
- deviation = abs(frame_mean - rolling_mean)
5. Decision:
- If deviation > threshold: DROP frame
- Else: PASS frame
```
### Key Simplifications from cli.py
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
- **No recording logic**: Just drop/pass, no buffering for output segments
- **No patience mechanism**: Immediate decision per frame
## Properties
| Property | Type | Default | Range | Description |
|----------|------|---------|-------|-------------|
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
| `column-index` | int | 1 | 0-width | Which vertical column to analyze |
| `stride` | int | 1 | 1-height | Row sampling stride (1=all rows) |
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping |
## Data Structures
### Ring Buffer
- Array of doubles sized to `window_size`
- Stores column mean for each processed frame
- Circular wrapping via modulo: `ring_index % window_size`
### Frame Analysis
```
For column_index=1, stride=1, frame height=H:
- Extract pixels: frame[:, column_index]
- Sample: frame[::stride, column_index]
- Count: H / stride samples
- Mean: sum(samples) / count
```
## Implementation Files
### Directory Structure
```
gst/rollingsum/
├── CMakeLists.txt
├── gstrollingsum.c
└── gstrollingsum.h
```
### gstrollingsum.h
- Element type definitions
- Structure declarations
- Property enums
- Function prototypes
### gstrollingsum.c
- GObject methods (init, dispose, get/set properties)
- GstBaseTransform methods (transform_ip)
- Helper functions (extract_column_mean, update_rolling_mean)
- Plugin registration
### CMakeLists.txt
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
- Link GStreamer base and video libraries
## Video Format Support
### Initial Implementation
- **Primary target**: Grayscale (GRAY8, GRAY16)
- **Secondary**: Bayer formats (common in machine vision)
### Caps Filter
```c
static GstStaticPadTemplate sink_template =
GST_STATIC_PAD_TEMPLATE ("sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS (
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
)
);
```
## Pipeline Usage Examples
### Basic Usage
```bash
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
rollingsum window-size=1000 column-index=1 threshold=0.5 ! \
autovideosink
```
### Custom Configuration
```bash
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! \
queue ! \
autovideosink
```
### With Format Conversion
```bash
gst-launch-1.0 idsueyesrc ! \
videoconvert ! \
video/x-raw,format=GRAY8 ! \
rollingsum ! \
autovideosink
```
## Performance Considerations
### Memory Usage
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
- Minimal per-frame allocation
### CPU Usage
- Column extraction: O(height/stride)
- Rolling mean update: O(1) using incremental sum
- Very lightweight compared to full-frame processing
### Optimization Opportunities
1. **Incremental mean**: Track sum instead of recalculating
2. **SIMD**: Vectorize column summation
3. **Skip calculation**: Only recalc every N frames if baseline is stable
## Testing Strategy
### Unit Tests
- Ring buffer wrapping
- Mean calculation accuracy
- Threshold comparison logic
### Integration Tests
- Pipeline with videotestsrc
- Pipeline with idsueyesrc
- Frame drop verification
- Property changes during playback
### Test Cases
1. Static video (all frames similar) → all pass
2. Single bright frame → that frame drops
3. Gradual change → frames pass
4. Periodic pattern → pattern frames drop
## Future Enhancements
### Phase 2 (If Needed)
- Add EMA baseline tracking (like cli.py)
- Add variance-based thresholds
- Support multiple columns or regions
- Add metadata output (tag frames with deviation values)
- RGB format support (analyze specific channel)
### Phase 3 (Advanced)
- Full cli.py recording logic
- Buffer and output segments
- Integration with probe detection systems
## Integration with Existing Project
### Build System
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
```cmake
add_subdirectory (rollingsum)
```
### Documentation
Update [`README.md`](README.md):
- Add rollingsum to "Other elements" section
- Add pipeline example
## Mermaid Diagram: Data Flow
```mermaid
graph TD
A[Video Frame] --> B[Extract Column]
B --> C[Calculate Column Mean]
C --> D[Store in Ring Buffer]
D --> E[Update Rolling Mean]
E --> F{Deviation > Threshold?}
F -->|Yes| G[DROP Frame]
F -->|No| H[PASS Frame]
C --> E
style G fill:#ff6b6b
style H fill:#51cf66
```
## Mermaid Diagram: Ring Buffer Operation
```mermaid
graph LR
subgraph Ring Buffer
A[0] --> B[1]
B --> C[2]
C --> D[...]
D --> E[N-1]
E ---|wrap| A
end
F[New Frame Mean] --> G[ring_index]
G --> A
H[Rolling Mean] --> I[Sum all values]
I --> J[Divide by count]
style G fill:#ffd43b
```
## Implementation Checklist
- [ ] Create gst/rollingsum directory
- [ ] Implement gstrollingsum.h
- [ ] Implement gstrollingsum.c
- [ ] Create CMakeLists.txt
- [ ] Update gst/CMakeLists.txt
- [ ] Build and test basic functionality
- [ ] Test with idsueyesrc
- [ ] Update README.md
- [ ] Create feature branch
- [ ] Commit and document
## References
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)

17
LICENSE
View File

@ -1,17 +0,0 @@
Copyright (C) <2003> David Schleef <ds@schleef.org>
Copyright (C) 2016-2019 Tim-Philipp Müller <tim@centricular.com>
Copyright (C) 2016-2017 PlayGineering Ltd.
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

View File

@ -19,13 +19,13 @@ GStreamer plugins for IDS uEye cameras with frame analysis capabilities.
## Usage Examples ## Usage Examples
### Basic capture from IDS uEye camera ### Basic capture from IDS uEye camera
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! queue ! autovideosink gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! queue ! autovideosink
``` ```
### Frame filtering based on column analysis ### Frame filtering based on column analysis
Drop frames when column mean deviates from rolling baseline by more than 0.5: Drop frames when column mean deviates from rolling baseline by more than 0.5:
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! queue ! autovideosink gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! queue ! autovideosink
``` ```
@ -34,17 +34,17 @@ gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.in
### Additional rollingsum examples ### Additional rollingsum examples
Analyze column 320 with larger window: Analyze column 320 with larger window:
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=5000 column-index=320 threshold=0.3 ! queue ! autovideosink gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=5000 column-index=320 threshold=0.3 ! queue ! autovideosink
``` ```
Use stride for faster processing (sample every 2 rows): Use stride for faster processing (sample every 2 rows):
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 stride=2 threshold=0.5 ! queue ! autovideosink gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 stride=2 threshold=0.5 ! queue ! autovideosink
``` ```
Lower threshold for more sensitive detection: Lower threshold for more sensitive detection:
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum threshold=0.2 ! queue ! autovideosink gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum threshold=0.2 ! queue ! autovideosink
``` ```
@ -81,7 +81,7 @@ If you prefer to build manually:
3. Install [IDS uEye SDK](https://www.ids-imaging.com) (default path: `C:\Program Files\IDS\uEye\Develop`) 3. Install [IDS uEye SDK](https://www.ids-imaging.com) (default path: `C:\Program Files\IDS\uEye\Develop`)
4. Run: 4. Run:
```bash ```powershell
git clone https://github.com/joshdoe/gst-plugins-vision.git git clone https://github.com/joshdoe/gst-plugins-vision.git
cd gst-plugins-vision cd gst-plugins-vision
mkdir build mkdir build
@ -101,7 +101,7 @@ The `build.ps1` script automatically copies plugins to `$env:GST_PLUGIN_PATH` if
### Verify Installation ### Verify Installation
```bash ```powershell
gst-inspect-1.0 idsueyesrc gst-inspect-1.0 idsueyesrc
gst-inspect-1.0 rollingsum gst-inspect-1.0 rollingsum
``` ```
@ -109,3 +109,13 @@ gst-inspect-1.0 rollingsum
## Documentation ## Documentation
- [Rolling Sum Filter Design](DESIGN_ROLLINGSUM.md) - [Rolling Sum Filter Design](DESIGN_ROLLINGSUM.md)
## Debugging
add `$env:GST_DEBUG_DUMP_DOT_DIR='.'`
![](dot_pause-play.svg)
to get dotfile, and view using https://dreampuf.github.io/GraphvizOnline/, or
```pwsh
dot -Tsvg C:\dev\gst-plugins-vision\0.00.02.922833100-gst-launch.PAUSED_PLAYING.dot -o same.svg
```

View File

@ -1,9 +1,28 @@
# GStreamer Rolling Sum Plugin Guide # GStreamer Rolling Sum Plugin - Complete Documentation
## Table of Contents
- [Overview](#overview)
- [How It Works](#how-it-works)
- [Architecture & Design](#architecture--design)
- [Plugin Properties](#plugin-properties)
- [Basic Usage](#basic-usage)
- [Debugging](#debugging)
- [CSV Analysis](#csv-analysis)
- [Recommended Thresholds](#recommended-thresholds)
- [Troubleshooting](#troubleshooting)
- [Performance](#performance)
- [Integration Examples](#integration-examples)
- [Developer Guide](#developer-guide)
- [References](#references)
## Overview ## Overview
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames. The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames
**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
## How It Works ## How It Works
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column 1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
@ -12,15 +31,134 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold 4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
5. **CSV Logging**: Records all frame statistics for analysis 5. **CSV Logging**: Records all frame statistics for analysis
### Data Flow
```mermaid
graph TD
A[Video Frame] --> B[Extract Column]
B --> C[Calculate Column Mean]
C --> D[Store in Ring Buffer]
D --> E[Update Rolling Mean]
E --> F{Deviation > Threshold?}
F -->|Yes| G[DROP Frame]
F -->|No| H[PASS Frame]
C --> E
style G fill:#ff6b6b
style H fill:#51cf66
```
### Ring Buffer Operation
```mermaid
graph LR
subgraph Ring Buffer
A[0] --> B[1]
B --> C[2]
C --> D[...]
D --> E[N-1]
E ---|wrap| A
end
F[New Frame Mean] --> G[ring_index]
G --> A
H[Rolling Mean] --> I[Sum all values]
I --> J[Divide by count]
style G fill:#ffd43b
```
## Architecture & Design
### Base Class
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
- In-place transform (analysis only, no frame modification)
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
- Returns `GST_FLOW_OK` to pass frames
### Element Structure
```c
struct _GstRollingSum
{
GstBaseTransform element;
/* Properties */
gint window_size; // Number of frames in rolling window (default: 1000)
gint column_index; // Which column to analyze (default: 1, second column)
gint stride; // Row sampling stride (default: 1, every row)
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
gchar *csv_file; // CSV output file path (default: NULL)
/* State */
gdouble *ring_buffer; // Circular buffer of column means
gint ring_index; // Current position in ring buffer
gint frame_count; // Total frames processed
gdouble rolling_mean; // Current rolling mean
FILE *csv_fp; // CSV file pointer
};
```
### Algorithm (Simplified from cli.py)
**Per Frame Processing:**
1. **Extract column data:**
- Select column at column_index
- Sample every stride rows
- Calculate mean of sampled pixels: frame_mean
2. **Update ring buffer:**
- Store frame_mean in ring_buffer[ring_index]
- Increment ring_index (wrap around)
3. **Calculate rolling mean:**
- Sum values in ring buffer (up to window_size or frame_count)
- Divide by actual window size
4. **Calculate deviation:**
- deviation = abs(frame_mean - rolling_mean)
- normalized_deviation = deviation / 255.0 (for 8-bit video)
5. **Decision:**
- If normalized_deviation > threshold: DROP frame
- Else: PASS frame
**Key Simplifications from cli.py:**
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
- **No recording logic**: Just drop/pass, no buffering for output segments
- **No patience mechanism**: Immediate decision per frame
### Video Format Support
**Initial Implementation:**
- **Primary target**: Grayscale (GRAY8, GRAY16)
- **Secondary**: Bayer formats (common in machine vision)
**Caps Filter:**
```c
static GstStaticPadTemplate sink_template =
GST_STATIC_PAD_TEMPLATE ("sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS (
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
)
);
```
## Plugin Properties ## Plugin Properties
| Property | Type | Default | Description | | Property | Type | Default | Range | Description |
|----------|------|---------|-------------| |----------|------|---------|-------|-------------|
| `window-size` | int | 1000 | Number of frames in rolling window (1-100000) | | `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
| `column-index` | int | 1 | Which vertical column to analyze (0-based) | | `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) |
| `stride` | int | 1 | Row sampling stride (1 = every row) | | `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) |
| `threshold` | double | 0.5 | Normalized deviation threshold for dropping frames (0.0-1.0) | | `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames |
| `csv-file` | string | NULL | Path to CSV file for logging (NULL = no logging) | | `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) |
### Understanding Normalized Deviation ### Understanding Normalized Deviation
@ -35,24 +173,43 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
### Simple Pipeline ### Simple Pipeline
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini ! \ gst-launch-1.0 idsueyesrc config-file=config.ini ! `
videoconvert ! \ videoconvert ! `
video/x-raw,format=GRAY8 ! \ video/x-raw,format=GRAY8 ! `
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! \ rollingsum window-size=1000 column-index=1 threshold=0.0002 ! `
autovideosink autovideosink
``` ```
### With CSV Logging ### With CSV Logging
```bash ```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! \ gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! `
videoconvert ! \ videoconvert ! `
video/x-raw,format=GRAY8 ! \ video/x-raw,format=GRAY8 ! `
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! \ rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! `
fakesink fakesink
``` ```
### Custom Configuration
```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! `
queue ! `
autovideosink
```
### With Format Conversion
```powershell
gst-launch-1.0 idsueyesrc ! `
videoconvert ! `
video/x-raw,format=GRAY8 ! `
rollingsum ! `
autovideosink
```
## Debugging ## Debugging
### Enable Debug Output ### Enable Debug Output
@ -105,7 +262,7 @@ GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...]
#### 1. Verify Plugin Loaded #### 1. Verify Plugin Loaded
```bash ```powershell
gst-inspect-1.0 rollingsum gst-inspect-1.0 rollingsum
``` ```
@ -149,15 +306,22 @@ frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped
Use the included analysis script: Use the included analysis script:
```bash ```powershell
uv run analyze_sma.py output.csv uv run scripts/analyze_sma.py output.csv
``` ```
**Output includes:** **Output includes:**
- Statistical summary (min/max/mean/std) - Statistical summary (min/max/mean/std)
- Threshold recommendations based on percentiles - Threshold recommendations based on percentiles
- Standard deviation-based suggestions - Standard deviation-based suggestions
- Visualization plots (`output_analysis.png`) - Visualization plots saved to `results/debug/`
- Archived CSV with timestamp in `results/debug/`
**Output files are automatically organized:**
- `results/debug/output_YYYYMMDD_HHMMSS.csv` - Archived CSV
- `results/debug/output_analysis_YYYYMMDD_HHMMSS.png` - Analysis plots
The `results/` directory is gitignored to keep your repository clean.
### Interpreting Results ### Interpreting Results
@ -176,7 +340,7 @@ Based on analysis of stable camera footage:
### For General Use ### For General Use
```bash ```powershell
# Conservative (1-2% frame drop) # Conservative (1-2% frame drop)
threshold=0.0003 threshold=0.0003
@ -190,17 +354,17 @@ threshold=0.0001
### For Specific Scenarios ### For Specific Scenarios
**High-speed acquisition** (minimal processing): **High-speed acquisition** (minimal processing):
```bash ```powershell
window-size=100 threshold=0.0005 window-size=100 threshold=0.0005
``` ```
**Quality-focused** (stable scenes): **Quality-focused** (stable scenes):
```bash ```powershell
window-size=1000 threshold=0.0001 window-size=1000 threshold=0.0001
``` ```
**Real-time monitoring** (fast response): **Real-time monitoring** (fast response):
```bash ```powershell
window-size=50 threshold=0.0002 window-size=50 threshold=0.0002
``` ```
@ -212,7 +376,7 @@ window-size=50 threshold=0.0002
**Solution**: **Solution**:
1. Run with CSV logging 1. Run with CSV logging
2. Analyze with `uv run analyze_sma.py output.csv` 2. Analyze with `uv run scripts/analyze_sma.py output.csv`
3. Use recommended threshold from 90th-99th percentile 3. Use recommended threshold from 90th-99th percentile
### Too many frames dropped (threshold too low) ### Too many frames dropped (threshold too low)
@ -252,13 +416,32 @@ WARNING rollingsum: Column index 1000 >= width 1224, using column 0
- Choose different `column-index` (avoid edges) - Choose different `column-index` (avoid edges)
- Use `stride=2` or higher for faster processing - Use `stride=2` or higher for faster processing
## Performance Tips ## Performance
### Performance Tips
1. **Larger window = more stable** but slower to adapt to scene changes 1. **Larger window = more stable** but slower to adapt to scene changes
2. **Stride > 1** reduces computation but less accurate column mean 2. **Stride > 1** reduces computation but less accurate column mean
3. **CSV logging** has minimal performance impact 3. **CSV logging** has minimal performance impact
4. **Debug level 5** can produce massive logs, use only when needed 4. **Debug level 5** can produce massive logs, use only when needed
### Memory Usage
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
- Minimal per-frame allocation
### CPU Usage
- Column extraction: O(height/stride)
- Rolling mean update: O(1) using incremental sum
- Very lightweight compared to full-frame processing
### Optimization Opportunities
1. **Incremental mean**: Track sum instead of recalculating
2. **SIMD**: Vectorize column summation
3. **Skip calculation**: Only recalc every N frames if baseline is stable
## Integration Examples ## Integration Examples
### Python Script Control ### Python Script Control
@ -281,7 +464,7 @@ subprocess.run([
]) ])
# Analyze results # Analyze results
subprocess.run(['uv', 'run', 'analyze_sma.py', 'output.csv']) subprocess.run(['uv', 'run', 'scripts/analyze_sma.py', 'output.csv'])
``` ```
### Adaptive Threshold ### Adaptive Threshold
@ -298,7 +481,33 @@ recommended_threshold = df['normalized_deviation'].quantile(0.95)
print(f"Recommended threshold: {recommended_threshold:.6f}") print(f"Recommended threshold: {recommended_threshold:.6f}")
``` ```
## Developer Notes ## Developer Guide
### Implementation Files
**Directory Structure:**
```
gst/rollingsum/
├── CMakeLists.txt
├── gstrollingsum.c
└── gstrollingsum.h
```
**gstrollingsum.h:**
- Element type definitions
- Structure declarations
- Property enums
- Function prototypes
**gstrollingsum.c:**
- GObject methods (init, dispose, get/set properties)
- GstBaseTransform methods (transform_ip)
- Helper functions (extract_column_mean, update_rolling_mean)
- Plugin registration
**CMakeLists.txt:**
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
- Link GStreamer base and video libraries
### Adding New Features ### Adding New Features
@ -309,14 +518,17 @@ Key files:
### Rebuild After Changes ### Rebuild After Changes
```bash ```powershell
.\build.ps1 # Windows .\build.ps1 # Windows
```
```bash
./build.sh # Linux ./build.sh # Linux
``` ```
### Testing ### Testing
```bash ```powershell
# Quick test # Quick test
gst-inspect-1.0 rollingsum gst-inspect-1.0 rollingsum
@ -325,16 +537,77 @@ $env:GST_DEBUG="rollingsum:5"
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
``` ```
### Testing Strategy
**Unit Tests:**
- Ring buffer wrapping
- Mean calculation accuracy
- Threshold comparison logic
**Integration Tests:**
- Pipeline with videotestsrc
- Pipeline with idsueyesrc
- Frame drop verification
- Property changes during playback
**Test Cases:**
1. Static video (all frames similar) → all pass
2. Single bright frame → that frame drops
3. Gradual change → frames pass
4. Periodic pattern → pattern frames drop
### Integration with Existing Project
**Build System:**
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
```cmake
add_subdirectory (rollingsum)
```
**Documentation:**
Update [`README.md`](README.md):
- Add rollingsum to "Other elements" section
- Add pipeline example
### Future Enhancements
**Phase 2 (If Needed):**
- Add EMA baseline tracking (like cli.py)
- Add variance-based thresholds
- Support multiple columns or regions
- Add metadata output (tag frames with deviation values)
- RGB format support (analyze specific channel)
**Phase 3 (Advanced):**
- Full cli.py recording logic
- Buffer and output segments
- Integration with probe detection systems
### Implementation Checklist
- [x] Create gst/rollingsum directory
- [x] Implement gstrollingsum.h
- [x] Implement gstrollingsum.c
- [x] Create CMakeLists.txt
- [x] Update gst/CMakeLists.txt
- [x] Build and test basic functionality
- [x] Test with idsueyesrc
- [x] Update README.md
- [x] Create feature branch
- [x] Commit and document
## References ## References
- [DESIGN_ROLLINGSUM.md](DESIGN_ROLLINGSUM.md) - Design document - Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
- [analyze_sma.py](analyze_sma.py) - Analysis tool - Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
- [scripts/analyze_sma.py](scripts/analyze_sma.py) - Analysis tool
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/ - GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
## Support ## Support
For issues or questions: For issues or questions:
1. Enable debug output (`GST_DEBUG=rollingsum:5`) 1. Enable debug output (`$env:GST_DEBUG="rollingsum:5"` in PowerShell)
2. Generate CSV log and analyze 2. Generate CSV log and analyze
3. Check this guide's troubleshooting section 3. Check this guide's troubleshooting section
4. Review debug output for errors/warnings 4. Review debug output for errors/warnings

125
dot_pause-play.svg Normal file
View File

@ -0,0 +1,125 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 14.0.2 (20251019.1705)
-->
<!-- Title: pipeline Pages: 1 -->
<svg width="1103pt" height="290pt"
viewBox="0.00 0.00 1103.00 290.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 285.85)">
<title>pipeline</title>
<polygon fill="white" stroke="none" points="-4,4 -4,-285.85 1099.2,-285.85 1099.2,4 -4,4"/>
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-268.35" font-family="sans" font-size="10.00">&lt;GstPipeline&gt;</text>
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-255.6" font-family="sans" font-size="10.00">pipeline0</text>
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-242.85" font-family="sans" font-size="10.00">[&gt;]</text>
<g id="clust1" class="cluster">
<title>cluster_node_udpsink0_000001FC4DD89DE0</title>
<path fill="#aaaaff" stroke="black" d="M886.45,-75.6C886.45,-75.6 1083.2,-75.6 1083.2,-75.6 1089.2,-75.6 1095.2,-81.6 1095.2,-87.6 1095.2,-87.6 1095.2,-215.6 1095.2,-215.6 1095.2,-221.6 1089.2,-227.6 1083.2,-227.6 1083.2,-227.6 886.45,-227.6 886.45,-227.6 880.45,-227.6 874.45,-221.6 874.45,-215.6 874.45,-215.6 874.45,-87.6 874.45,-87.6 874.45,-81.6 880.45,-75.6 886.45,-75.6"/>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-216" font-family="Bitstream Vera Sans" font-size="8.00">GstUDPSink</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-206.25" font-family="Bitstream Vera Sans" font-size="8.00">udpsink0</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-196.5" font-family="Bitstream Vera Sans" font-size="8.00">[&gt;]</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-186.75" font-family="Bitstream Vera Sans" font-size="8.00">last&#45;sample=((GstSample*) 000001FC4DE728F0)</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-177" font-family="Bitstream Vera Sans" font-size="8.00">used&#45;socket=((GSocket*) 000001FC4DD85E90)</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-167.25" font-family="Bitstream Vera Sans" font-size="8.00">used&#45;socket&#45;v6=((GSocket*) 000001FC4DD87A30)</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-157.5" font-family="Bitstream Vera Sans" font-size="8.00">clients=&quot;127.0.0.1:5000&quot;</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-147.75" font-family="Bitstream Vera Sans" font-size="8.00">host=&quot;127.0.0.1&quot;</text>
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-138" font-family="Bitstream Vera Sans" font-size="8.00">port=5000</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_node_udpsink0_000001FC4DD89DE0_sink</title>
</g>
<g id="clust3" class="cluster">
<title>cluster_node_queue0_000001FC4BB96850</title>
<path fill="#aaffaa" stroke="black" d="M524.2,-75.6C524.2,-75.6 673.2,-75.6 673.2,-75.6 679.2,-75.6 685.2,-81.6 685.2,-87.6 685.2,-87.6 685.2,-156.6 685.2,-156.6 685.2,-162.6 679.2,-168.6 673.2,-168.6 673.2,-168.6 524.2,-168.6 524.2,-168.6 518.2,-168.6 512.2,-162.6 512.2,-156.6 512.2,-156.6 512.2,-87.6 512.2,-87.6 512.2,-81.6 518.2,-75.6 524.2,-75.6"/>
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-157" font-family="Bitstream Vera Sans" font-size="8.00">GstQueue</text>
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-147.25" font-family="Bitstream Vera Sans" font-size="8.00">queue0</text>
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-137.5" font-family="Bitstream Vera Sans" font-size="8.00">[&gt;]</text>
</g>
<g id="clust4" class="cluster">
<title>cluster_node_queue0_000001FC4BB96850_sink</title>
</g>
<g id="clust5" class="cluster">
<title>cluster_node_queue0_000001FC4BB96850_src</title>
</g>
<g id="clust6" class="cluster">
<title>cluster_node_idsueyesrc0_000001FC4DD4B2C0</title>
<path fill="#ffaaaa" stroke="black" d="M81.98,-75.6C81.98,-75.6 241.98,-75.6 241.98,-75.6 247.98,-75.6 253.98,-81.6 253.98,-87.6 253.98,-87.6 253.98,-186.6 253.98,-186.6 253.98,-192.6 247.98,-198.6 241.98,-198.6 241.98,-198.6 81.98,-198.6 81.98,-198.6 75.98,-198.6 69.98,-192.6 69.98,-186.6 69.98,-186.6 69.98,-87.6 69.98,-87.6 69.98,-81.6 75.98,-75.6 81.98,-75.6"/>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-187" font-family="Bitstream Vera Sans" font-size="8.00">GstIdsueyeSrc</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-177.25" font-family="Bitstream Vera Sans" font-size="8.00">idsueyesrc0</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-167.5" font-family="Bitstream Vera Sans" font-size="8.00">[&gt;]</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-157.75" font-family="Bitstream Vera Sans" font-size="8.00">config&#45;file=&quot;ini/200fps&#45;2456x4pix&#45;cw.ini&quot;</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-148" font-family="Bitstream Vera Sans" font-size="8.00">exposure=3.237784</text>
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-138.25" font-family="Bitstream Vera Sans" font-size="8.00">framerate=300.000000</text>
</g>
<g id="clust7" class="cluster">
<title>cluster_node_idsueyesrc0_000001FC4DD4B2C0_src</title>
</g>
<!-- legend -->
<g id="node1" class="node">
<title>legend</title>
<polygon fill="lightgrey" stroke="black" points="322.95,-67.2 0,-67.2 0,0 322.95,0 322.95,-67.2"/>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-55.05" font-family="sans" font-size="9.00">Legend</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-43.05" font-family="sans" font-size="9.00">Element&#45;States: [~] void&#45;pending, [0] null, [&#45;] ready, [=] paused, [&gt;] playing</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-31.05" font-family="sans" font-size="9.00">Pad&#45;Activation: [&#45;] none, [&gt;] push, [&lt;] pull</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-19.05" font-family="sans" font-size="9.00">Pad&#45;Flags: [b]locked, [f]lushing, [b]locking, [E]OS; upper&#45;case is set</text>
<text xml:space="preserve" text-anchor="start" x="3.6" y="-7.05" font-family="sans" font-size="9.00">Pad&#45;Task: [T] has started task, [t] has paused task</text>
</g>
<!-- node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0 -->
<g id="node2" class="node">
<title>node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0</title>
<polygon fill="#aaaaff" stroke="black" points="1011.33,-115.6 957.33,-115.6 957.33,-91.6 1011.33,-91.6 1011.33,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="984.33" y="-107.05" font-family="sans" font-size="9.00">sink</text>
<text xml:space="preserve" text-anchor="middle" x="984.33" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb]</text>
</g>
<!-- node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0 -->
<g id="node3" class="node">
<title>node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0</title>
<polygon fill="#aaaaff" stroke="black" points="582.2,-115.6 528.2,-115.6 528.2,-91.6 582.2,-91.6 582.2,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="555.2" y="-107.05" font-family="sans" font-size="9.00">sink</text>
<text xml:space="preserve" text-anchor="middle" x="555.2" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb]</text>
</g>
<!-- node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70 -->
<g id="node4" class="node">
<title>node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70</title>
<polygon fill="#ffaaaa" stroke="black" points="669.2,-115.6 615.2,-115.6 615.2,-91.6 669.2,-91.6 669.2,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="642.2" y="-107.05" font-family="sans" font-size="9.00">src</text>
<text xml:space="preserve" text-anchor="middle" x="642.2" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb][T]</text>
</g>
<!-- node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0&#45;&gt;node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70 -->
<!-- node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70&#45;&gt;node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0 -->
<g id="edge2" class="edge">
<title>node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70&#45;&gt;node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0</title>
<path fill="none" stroke="black" d="M669.61,-103.6C729.66,-103.6 876.61,-103.6 945.95,-103.6"/>
<polygon fill="black" stroke="black" points="945.66,-107.1 955.66,-103.6 945.66,-100.1 945.66,-107.1"/>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-179.05" font-family="monospace" font-size="9.00">video/x&#45;raw</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-168.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;format: BGR</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-158.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;width: 2456</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-147.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;height: 4</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-137.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;interlace&#45;mode: progressive</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-126.55" font-family="monospace" font-size="9.00"> &#160;pixel&#45;aspect&#45;ratio: 1/1</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-116.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;colorimetry: sRGB</text>
<text xml:space="preserve" text-anchor="start" x="693.2" y="-105.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;framerate: 0/1</text>
</g>
<!-- node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0 -->
<g id="node5" class="node">
<title>node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0</title>
<polygon fill="#ffaaaa" stroke="black" points="188.48,-115.6 134.48,-115.6 134.48,-91.6 188.48,-91.6 188.48,-115.6"/>
<text xml:space="preserve" text-anchor="middle" x="161.48" y="-107.05" font-family="sans" font-size="9.00">src</text>
<text xml:space="preserve" text-anchor="middle" x="161.48" y="-95.05" font-family="sans" font-size="9.00">[&gt;][bfb][T]</text>
</g>
<!-- node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0&#45;&gt;node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0 -->
<g id="edge3" class="edge">
<title>node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0&#45;&gt;node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0</title>
<path fill="none" stroke="black" d="M188.79,-103.6C256.58,-103.6 437.44,-103.6 516.42,-103.6"/>
<polygon fill="black" stroke="black" points="516.23,-107.1 526.23,-103.6 516.23,-100.1 516.23,-107.1"/>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-179.05" font-family="monospace" font-size="9.00">video/x&#45;raw</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-168.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;format: BGR</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-158.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;width: 2456</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-147.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;height: 4</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-137.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;interlace&#45;mode: progressive</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-126.55" font-family="monospace" font-size="9.00"> &#160;pixel&#45;aspect&#45;ratio: 1/1</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-116.05" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;colorimetry: sRGB</text>
<text xml:space="preserve" text-anchor="start" x="330.95" y="-105.55" font-family="monospace" font-size="9.00"> &#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;framerate: 0/1</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 12 KiB

222
ini/200fps-2456x4pix-cw.ini Normal file
View File

@ -0,0 +1,222 @@
[Versions]
ueye_api_64.dll=4.93.1730
ueye_usb_64.sys=4.93.1314
ueye_boot_64.sys=4.93.1314
[Sensor]
Sensor=UI308xCP-C
Sensor bit depth=0
Sensor source gain=24
FPN correction mode=0
Black reference mode=0
Sensor digital gain=0
[Image size]
Start X=0
Start Y=0
Start X absolute=0
Start Y absolute=0
Width=2456
Height=4
Binning=0
Subsampling=0
[Scaler]
Mode=0
Factor=0.000000
[Multi AOI]
Enabled=0
Mode=0
x1=0
x2=0
x3=0
x4=0
y1=0
y2=0
y3=0
y4=0
[Shutter]
Mode=0
Linescan number=0
[Log Mode]
Mode=3
Manual value=0
Manual gain=0
[Timing]
Pixelclock=237
Extended pixelclock range=0
Framerate=200.151466
Exposure=4.903189
Long exposure=0
Dual exposure ratio=0
[Selected Converter]
IS_SET_CM_RGB32=2
IS_SET_CM_RGB24=2
IS_SET_CM_RGB16=2
IS_SET_CM_RGB15=2
IS_SET_CM_Y8=2
IS_SET_CM_RGB8=2
IS_SET_CM_BAYER=8
IS_SET_CM_UYVY=2
IS_SET_CM_UYVY_MONO=2
IS_SET_CM_UYVY_BAYER=2
IS_CM_CBYCRY_PACKED=0
IS_SET_CM_RGBY=8
IS_SET_CM_RGB30=2
IS_SET_CM_Y12=2
IS_SET_CM_BAYER12=8
IS_SET_CM_Y16=2
IS_SET_CM_BAYER16=8
IS_CM_BGR12_UNPACKED=2
IS_CM_BGRA12_UNPACKED=2
IS_CM_JPEG=0
IS_CM_SENSOR_RAW10=8
IS_CM_MONO10=2
IS_CM_BGR10_UNPACKED=2
IS_CM_RGBA8_PACKED=2
IS_CM_RGB8_PACKED=2
IS_CM_RGBY8_PACKED=8
IS_CM_RGB10V2_PACKED=8
IS_CM_RGB12_UNPACKED=2
IS_CM_RGBA12_UNPACKED=2
IS_CM_RGB10_UNPACKED=2
IS_CM_RGB8_PLANAR=2
[Parameters]
Colormode=1
Gamma=1.000000
Hardware Gamma=0
Blacklevel Mode=0
Blacklevel Offset=4
Hotpixel Mode=2
Hotpixel Threshold=0
Sensor Hotpixel=0
Adaptive hotpixel correction enable=0
Adaptive hotpixel correction mode=0
Adaptive hotpixel correction sensitivity=3
GlobalShutter=0
AllowRawWithLut=0
[Gain]
Master=0
Red=19
Green=0
Blue=33
GainBoost=1
[Processing]
EdgeEnhancementFactor=0
RopEffect=0
Whitebalance=0
Whitebalance Red=1.000000
Whitebalance Green=1.000000
Whitebalance Blue=1.000000
Color correction=4
Color_correction_factor=1.000000
Color_correction_satU=100
Color_correction_satV=100
Bayer Conversion=1
JpegCompression=0
NoiseMode=0
ImageEffect=0
LscModel=0
WideDynamicRange=0
[Auto features]
Auto Framerate control=0
Brightness exposure control=0
Brightness gain control=0
Auto Framerate Sensor control=0
Brightness exposure Sensor control=0
Brightness gain Sensor control=0
Brightness exposure Sensor control photometry=0
Brightness gain Sensor control photometry=0
Brightness control once=0
Brightness reference=128
Brightness speed=50
Brightness max gain=100
Brightness max exposure=4.903189
Brightness Aoi Left=0
Brightness Aoi Top=0
Brightness Aoi Width=2456
Brightness Aoi Height=4
Brightness Hysteresis=2
AutoImageControlMode=2
AutoImageControlPeakWhiteChannel=0
AutoImageControlExposureMinimum=0.000000
AutoImageControlPeakWhiteChannelMode=0
AutoImageControlPeakWhiteGranularity=0
Auto WB control=0
Auto WB type=2
Auto WB RGB color model=1
Auto WB RGB color temperature=5000
Auto WB offsetR=0
Auto WB offsetB=0
Auto WB gainMin=0
Auto WB gainMax=100
Auto WB speed=50
Auto WB Aoi Left=0
Auto WB Aoi Top=0
Auto WB Aoi Width=2456
Auto WB Aoi Height=4
Auto WB Once=0
Auto WB Hysteresis=2
Brightness Skip Frames Trigger Mode=4
Brightness Skip Frames Freerun Mode=4
Auto WB Skip Frames Trigger Mode=4
Auto WB Skip Frames Freerun Mode=4
[Trigger and Flash]
Trigger mode=0
Trigger timeout=200
Trigger delay=0
Trigger debounce mode=0
Trigger debounce delay time=1
Trigger burst size=1
Trigger prescaler frame=1
Trigger prescaler line=1
Trigger input=1
Flash strobe=0
Flash delay=0
Flash duration=0
Flash auto freerun=0
PWM mode=0
PWM frequency=20000000
PWM dutycycle=20000000
GPIO state=3
GPIO direction=0
GPIO1 Config=1
GPIO2 Config=1
[Vertical AOI Merge Mode]
Mode=0
Position=0
Additional Position=0
Height=2
[Level Controlled Trigger Mode]
Mode=0
[Memory]
Camera memory mode=1

39
network_guide.md Normal file
View File

@ -0,0 +1,39 @@
# how to send a line
real data
```powershell
gst-launch-1.0 idsueyesrc config-file=ini/200fps-2456x4pix-cw.ini exposure=5 framerate=300 `
! queue `
! udpsink host=127.0.0.1 port=5000
```
note: 5ms is bit too fast for us
### Python/OpenCV Receiver
```pwsh
uv run scripts/recv_raw_column.py
```
or rolling like
```pwsh
uv run .\scripts\recv_raw_rolling.py
```
See [`scripts/recv_raw_column.py`](scripts/recv_raw_column.py) for the Python implementation with debug options.
# demo data
## Sender (crop to first column, send raw over UDP)
```pwsh
gst-launch-1.0 -v `
videotestsrc pattern=smpte ! `
videocrop left=0 right=639 top=0 bottom=0 ! `
video/x-raw,format=RGB,width=1,height=640,framerate=30/1 ! `
udpsink host=127.0.0.1 port=5000
```
### GStreamer Receiver (raw UDP → display)
```pwsh
gst-launch-1.0 -v `
udpsrc port=5000 caps="video/x-raw,format=RGB,width=1,height=640,framerate=30/1" ! `
videoconvert ! `
autovideosink
```

View File

@ -18,11 +18,17 @@ import pandas as pd
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
import numpy as np import numpy as np
from pathlib import Path from pathlib import Path
from datetime import datetime
import shutil
def analyze_csv(csv_file: str = "output.csv"): def analyze_csv(csv_file: str = "output.csv"):
"""Analyze the rolling sum CSV data and generate insights.""" """Analyze the rolling sum CSV data and generate insights."""
# Create output directory
output_dir = Path("results/debug")
output_dir.mkdir(parents=True, exist_ok=True)
# Read the CSV # Read the CSV
try: try:
df = pd.read_csv(csv_file) df = pd.read_csv(csv_file)
@ -30,6 +36,12 @@ def analyze_csv(csv_file: str = "output.csv"):
print(f"Error: CSV file '{csv_file}' not found.") print(f"Error: CSV file '{csv_file}' not found.")
sys.exit(1) sys.exit(1)
# Copy input CSV to results directory with timestamp
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
csv_name = Path(csv_file).stem
archived_csv = output_dir / f"{csv_name}_{timestamp}.csv"
shutil.copy(csv_file, archived_csv)
print("=" * 80) print("=" * 80)
print(f"ROLLING SUM ANALYSIS - {csv_file}") print(f"ROLLING SUM ANALYSIS - {csv_file}")
print("=" * 80) print("=" * 80)
@ -98,16 +110,17 @@ def analyze_csv(csv_file: str = "output.csv"):
print() print()
# Create visualizations # Create visualizations
create_plots(df, csv_file) plot_file = create_plots(df, csv_file, output_dir, timestamp)
print("=" * 80) print("=" * 80)
print("PLOTS SAVED:") print("OUTPUT FILES:")
print(f" - {csv_file.replace('.csv', '_analysis.png')}") print(f" CSV Archive: {archived_csv}")
print(f" Analysis Plot: {plot_file}")
print("=" * 80) print("=" * 80)
def create_plots(df: pd.DataFrame, csv_file: str): def create_plots(df: pd.DataFrame, csv_file: str, output_dir: Path, timestamp: str) -> Path:
"""Create analysis plots.""" """Create analysis plots and return the output file path."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10)) fig, axes = plt.subplots(2, 2, figsize=(14, 10))
fig.suptitle(f'Rolling Sum Analysis - {csv_file}', fontsize=16, fontweight='bold') fig.suptitle(f'Rolling Sum Analysis - {csv_file}', fontsize=16, fontweight='bold')
@ -167,10 +180,13 @@ def create_plots(df: pd.DataFrame, csv_file: str):
plt.tight_layout() plt.tight_layout()
# Save the plot # Save the plot to results/debug
output_file = csv_file.replace('.csv', '_analysis.png') csv_name = Path(csv_file).stem
output_file = output_dir / f"{csv_name}_analysis_{timestamp}.png"
plt.savefig(output_file, dpi=150, bbox_inches='tight') plt.savefig(output_file, dpi=150, bbox_inches='tight')
print(f"\n✓ Saved analysis plot to: {output_file}\n") print(f"\n✓ Saved analysis plot to: {output_file}\n")
return output_file
if __name__ == "__main__": if __name__ == "__main__":

View File

@ -0,0 +1,61 @@
#!/usr/bin/env python3
# /// script
# requires-python = "<=3.10"
# dependencies = [
# "opencv-python",
# "numpy",
# ]
# ///
import socket
import numpy as np
import cv2
# Debug flag - set to True to see frame reception details
DEBUG = False
# Stream parameters (match your GStreamer sender)
WIDTH = 1
HEIGHT = 640 # Default videotestsrc height
CHANNELS = 3
FRAME_SIZE = WIDTH * HEIGHT * CHANNELS # bytes
UDP_IP = "0.0.0.0"
UDP_PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
print(f"Receiving raw {WIDTH}x{HEIGHT} RGB frames on UDP port {UDP_PORT}")
if DEBUG:
print(f"Expected frame size: {FRAME_SIZE} bytes")
cv2.namedWindow("Raw Column Stream", cv2.WINDOW_NORMAL)
frame_count = 0
while True:
data, addr = sock.recvfrom(65536)
if len(data) != FRAME_SIZE:
if DEBUG:
print(f"Received {len(data)} bytes (expected {FRAME_SIZE}), skipping...")
continue
if DEBUG:
frame_count += 1
if frame_count % 30 == 0:
print(f"Received {frame_count} frames")
frame = np.frombuffer(data, dtype=np.uint8).reshape((HEIGHT, WIDTH, CHANNELS))
# scale for visibility
frame_large = cv2.resize(
frame, (200, HEIGHT), interpolation=cv2.INTER_NEAREST
)
cv2.imshow("Raw Column Stream", frame_large)
if cv2.waitKey(1) == 27: # ESC to quit
break
cv2.destroyAllWindows()

View File

@ -0,0 +1,86 @@
#!/usr/bin/env python3
# /// script
# requires-python = "<=3.10"
# dependencies = [
# "opencv-python",
# "numpy",
# ]
# ///
import socket
import numpy as np
import cv2
# Debug flag - set to True to see frame reception details
DEBUG = False
# Rotation mode - set to rotate incoming data before display
# Options: None, cv2.ROTATE_90_CLOCKWISE, cv2.ROTATE_90_COUNTERCLOCKWISE, cv2.ROTATE_180
ROTATION = cv2.ROTATE_90_COUNTERCLOCKWISE # Rotate rows to columns
# Stream parameters (match your GStreamer sender)
COLUMN_WIDTH = 4 # Width from 200fps-2456x4pix-cw.ini
COLUMN_HEIGHT = 2456 # Height from 200fps-2456x4pix-cw.ini
CHANNELS = 3
FRAME_SIZE = COLUMN_WIDTH * COLUMN_HEIGHT * CHANNELS # bytes (29472)
# Display parameters
DISPLAY_WIDTH = 800 # Width of rolling display in pixels
DISPLAY_HEIGHT = COLUMN_HEIGHT
UDP_IP = "0.0.0.0"
UDP_PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
print(f"Receiving raw {COLUMN_WIDTH}x{COLUMN_HEIGHT} RGB columns on UDP port {UDP_PORT}")
print(f"Display width: {DISPLAY_WIDTH} pixels (rolling)")
if DEBUG:
print(f"Expected frame size: {FRAME_SIZE} bytes")
cv2.namedWindow("Rolling Column Stream", cv2.WINDOW_NORMAL)
# Initialize the rolling buffer
rolling_buffer = np.zeros((DISPLAY_HEIGHT, DISPLAY_WIDTH, CHANNELS), dtype=np.uint8)
current_column = 0
frame_count = 0
while True:
data, addr = sock.recvfrom(65536)
if len(data) != FRAME_SIZE:
if DEBUG:
print(f"Received {len(data)} bytes (expected {FRAME_SIZE}), skipping...")
continue
if DEBUG:
frame_count += 1
if frame_count % 30 == 0:
print(f"Received {frame_count} frames, current column: {current_column}")
# Parse the incoming data
frame = np.frombuffer(data, dtype=np.uint8).reshape((COLUMN_WIDTH, COLUMN_HEIGHT, CHANNELS))
# Apply rotation if configured
if ROTATION is not None:
rotated = cv2.rotate(frame, ROTATION)
else:
rotated = frame
# Extract only the first column for smoother rolling (1 pixel/frame)
column = rotated[:, 0:1, :]
# Insert the single column into the rolling buffer at the current position
rolling_buffer[:, current_column:current_column+1, :] = column
# Move to the next column position, wrapping around when reaching the end
current_column = (current_column + 1) % DISPLAY_WIDTH
# Display the rolling buffer
cv2.imshow("Rolling Column Stream", rolling_buffer)
if cv2.waitKey(1) == 27: # ESC to quit
break
cv2.destroyAllWindows()