yair/gst-plugin-linescan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: convert bash commands to PowerShell and merge ROLLINGSUM docs

Browse Source 
- Updated all commands in README.md and ROLLINGSUM_GUIDE.md to use PowerShell syntax
- Changed line continuation from backslash (\) to backtick ()
- Updated environment variable syntax to PowerShell format ()
- Merged DESIGN_ROLLINGSUM.md into ROLLINGSUM_GUIDE.md for comprehensive documentation
- Combined user guide with technical design details in single document
- Added table of contents and improved organization
This commit is contained in:
yair 2025-11-14 14:58:39 +02:00
parent d0467aaf65
commit cb1e5c7607
1 changed files with 274 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

290
ROLLINGSUM_GUIDE.md
View File

@ -1,9 +1,28 @@
# GStreamer Rolling Sum Plugin Guide # GStreamer Rolling Sum Plugin - Complete Documentation
## Table of Contents
- [Overview](#overview)
- [How It Works](#how-it-works)
- [Architecture & Design](#architecture--design)
- [Plugin Properties](#plugin-properties)
- [Basic Usage](#basic-usage)
- [Debugging](#debugging)
- [CSV Analysis](#csv-analysis)
- [Recommended Thresholds](#recommended-thresholds)
- [Troubleshooting](#troubleshooting)
- [Performance](#performance)
- [Integration Examples](#integration-examples)
- [Developer Guide](#developer-guide)
- [References](#references)
## Overview ## Overview
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames. The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames
**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
## How It Works ## How It Works
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column 1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
@ -12,15 +31,134 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold 4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
5. **CSV Logging**: Records all frame statistics for analysis 5. **CSV Logging**: Records all frame statistics for analysis
### Data Flow
```mermaid
graph TD
A[Video Frame] --> B[Extract Column]
B --> C[Calculate Column Mean]
C --> D[Store in Ring Buffer]
D --> E[Update Rolling Mean]
E --> F{Deviation > Threshold?}
F -->|Yes| G[DROP Frame]
F -->|No| H[PASS Frame]
C --> E
style G fill:#ff6b6b
style H fill:#51cf66
```
### Ring Buffer Operation
```mermaid
graph LR
subgraph Ring Buffer
A[0] --> B[1]
B --> C[2]
C --> D[...]
D --> E[N-1]
E ---|wrap| A
end
F[New Frame Mean] --> G[ring_index]
G --> A
H[Rolling Mean] --> I[Sum all values]
I --> J[Divide by count]
style G fill:#ffd43b
```
## Architecture & Design
### Base Class
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
- In-place transform (analysis only, no frame modification)
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
- Returns `GST_FLOW_OK` to pass frames
### Element Structure
```c
struct _GstRollingSum
{
GstBaseTransform element;
/* Properties */
gint window_size; // Number of frames in rolling window (default: 1000)
gint column_index; // Which column to analyze (default: 1, second column)
gint stride; // Row sampling stride (default: 1, every row)
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
gchar *csv_file; // CSV output file path (default: NULL)
/* State */
gdouble *ring_buffer; // Circular buffer of column means
gint ring_index; // Current position in ring buffer
gint frame_count; // Total frames processed
gdouble rolling_mean; // Current rolling mean
FILE *csv_fp; // CSV file pointer
};
```
### Algorithm (Simplified from cli.py)
**Per Frame Processing:**
1. **Extract column data:**
- Select column at column_index
- Sample every stride rows
- Calculate mean of sampled pixels: frame_mean
2. **Update ring buffer:**
- Store frame_mean in ring_buffer[ring_index]
- Increment ring_index (wrap around)
3. **Calculate rolling mean:**
- Sum values in ring buffer (up to window_size or frame_count)
- Divide by actual window size
4. **Calculate deviation:**
- deviation = abs(frame_mean - rolling_mean)
- normalized_deviation = deviation / 255.0 (for 8-bit video)
5. **Decision:**
- If normalized_deviation > threshold: DROP frame
- Else: PASS frame
**Key Simplifications from cli.py:**
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
- **No recording logic**: Just drop/pass, no buffering for output segments
- **No patience mechanism**: Immediate decision per frame
### Video Format Support
**Initial Implementation:**
- **Primary target**: Grayscale (GRAY8, GRAY16)
- **Secondary**: Bayer formats (common in machine vision)
**Caps Filter:**
```c
static GstStaticPadTemplate sink_template =
GST_STATIC_PAD_TEMPLATE ("sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS (
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
)
);
```
## Plugin Properties ## Plugin Properties
| Property | Type | Default | Description | | Property | Type | Default | Range | Description |
|----------|------|---------|-------------| |----------|------|---------|-------|-------------|
| `window-size` | int | 1000 | Number of frames in rolling window (1-100000) | | `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
| `column-index` | int | 1 | Which vertical column to analyze (0-based) | | `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) |
| `stride` | int | 1 | Row sampling stride (1 = every row) | | `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) |
| `threshold` | double | 0.5 | Normalized deviation threshold for dropping frames (0.0-1.0) | | `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames |
| `csv-file` | string | NULL | Path to CSV file for logging (NULL = no logging) | | `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) |
### Understanding Normalized Deviation ### Understanding Normalized Deviation
@ -53,6 +191,25 @@ gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! `
fakesink fakesink
``` ```
### Custom Configuration
```powershell
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! `
queue ! `
autovideosink
```
### With Format Conversion
```powershell
gst-launch-1.0 idsueyesrc ! `
videoconvert ! `
video/x-raw,format=GRAY8 ! `
rollingsum ! `
autovideosink
```
## Debugging ## Debugging
### Enable Debug Output ### Enable Debug Output
@ -77,11 +234,6 @@ set GST_DEBUG=rollingsum:5 && gst-launch-1.0 [pipeline...]
GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...] GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...]
``` ```
**PowerShell equivalent:**
```powershell
$env:GST_DEBUG="rollingsum:5"; gst-launch-1.0 [pipeline...]
```
### Debug Levels ### Debug Levels
| Level | Output | | Level | Output |
@ -264,13 +416,32 @@ WARNING rollingsum: Column index 1000 >= width 1224, using column 0
- Choose different `column-index` (avoid edges) - Choose different `column-index` (avoid edges)
- Use `stride=2` or higher for faster processing - Use `stride=2` or higher for faster processing
## Performance Tips ## Performance
### Performance Tips
1. **Larger window = more stable** but slower to adapt to scene changes 1. **Larger window = more stable** but slower to adapt to scene changes
2. **Stride > 1** reduces computation but less accurate column mean 2. **Stride > 1** reduces computation but less accurate column mean
3. **CSV logging** has minimal performance impact 3. **CSV logging** has minimal performance impact
4. **Debug level 5** can produce massive logs, use only when needed 4. **Debug level 5** can produce massive logs, use only when needed
### Memory Usage
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
- Minimal per-frame allocation
### CPU Usage
- Column extraction: O(height/stride)
- Rolling mean update: O(1) using incremental sum
- Very lightweight compared to full-frame processing
### Optimization Opportunities
1. **Incremental mean**: Track sum instead of recalculating
2. **SIMD**: Vectorize column summation
3. **Skip calculation**: Only recalc every N frames if baseline is stable
## Integration Examples ## Integration Examples
### Python Script Control ### Python Script Control
@ -310,7 +481,33 @@ recommended_threshold = df['normalized_deviation'].quantile(0.95)
print(f"Recommended threshold: {recommended_threshold:.6f}") print(f"Recommended threshold: {recommended_threshold:.6f}")
``` ```
## Developer Notes ## Developer Guide
### Implementation Files
**Directory Structure:**
```
gst/rollingsum/
├── CMakeLists.txt
├── gstrollingsum.c
└── gstrollingsum.h
```
**gstrollingsum.h:**
- Element type definitions
- Structure declarations
- Property enums
- Function prototypes
**gstrollingsum.c:**
- GObject methods (init, dispose, get/set properties)
- GstBaseTransform methods (transform_ip)
- Helper functions (extract_column_mean, update_rolling_mean)
- Plugin registration
**CMakeLists.txt:**
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
- Link GStreamer base and video libraries
### Adding New Features ### Adding New Features
@ -340,9 +537,70 @@ $env:GST_DEBUG="rollingsum:5"
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
``` ```
### Testing Strategy
**Unit Tests:**
- Ring buffer wrapping
- Mean calculation accuracy
- Threshold comparison logic
**Integration Tests:**
- Pipeline with videotestsrc
- Pipeline with idsueyesrc
- Frame drop verification
- Property changes during playback
**Test Cases:**
1. Static video (all frames similar) → all pass
2. Single bright frame → that frame drops
3. Gradual change → frames pass
4. Periodic pattern → pattern frames drop
### Integration with Existing Project
**Build System:**
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
```cmake
add_subdirectory (rollingsum)
```
**Documentation:**
Update [`README.md`](README.md):
- Add rollingsum to "Other elements" section
- Add pipeline example
### Future Enhancements
**Phase 2 (If Needed):**
- Add EMA baseline tracking (like cli.py)
- Add variance-based thresholds
- Support multiple columns or regions
- Add metadata output (tag frames with deviation values)
- RGB format support (analyze specific channel)
**Phase 3 (Advanced):**
- Full cli.py recording logic
- Buffer and output segments
- Integration with probe detection systems
### Implementation Checklist
- [x] Create gst/rollingsum directory
- [x] Implement gstrollingsum.h
- [x] Implement gstrollingsum.c
- [x] Create CMakeLists.txt
- [x] Update gst/CMakeLists.txt
- [x] Build and test basic functionality
- [x] Test with idsueyesrc
- [x] Update README.md
- [x] Create feature branch
- [x] Commit and document
## References ## References
- [DESIGN_ROLLINGSUM.md](DESIGN_ROLLINGSUM.md) - Design document - Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
- [analyze_sma.py](analyze_sma.py) - Analysis tool - [analyze_sma.py](analyze_sma.py) - Analysis tool
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/ - GStreamer documentation: https://gstreamer.freedesktop.org/documentation/