deprcat rollingsum filter
This commit is contained in:
yair
@@ -1,613 +0,0 @@
|
|||||||
# GStreamer Rolling Sum Plugin - Complete Documentation
|
|
||||||
|
|
||||||
## Table of Contents
|
|
||||||
- [Overview](#overview)
|
|
||||||
- [How It Works](#how-it-works)
|
|
||||||
- [Architecture & Design](#architecture--design)
|
|
||||||
- [Plugin Properties](#plugin-properties)
|
|
||||||
- [Basic Usage](#basic-usage)
|
|
||||||
- [Debugging](#debugging)
|
|
||||||
- [CSV Analysis](#csv-analysis)
|
|
||||||
- [Recommended Thresholds](#recommended-thresholds)
|
|
||||||
- [Troubleshooting](#troubleshooting)
|
|
||||||
- [Performance](#performance)
|
|
||||||
- [Integration Examples](#integration-examples)
|
|
||||||
- [Developer Guide](#developer-guide)
|
|
||||||
- [References](#references)
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
|
|
||||||
|
|
||||||
**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames
|
|
||||||
|
|
||||||
**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
|
|
||||||
|
|
||||||
## How It Works
|
|
||||||
|
|
||||||
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
|
|
||||||
2. **Rolling Window**: Maintains a circular buffer of recent column means
|
|
||||||
3. **Deviation Detection**: Calculates how much each frame deviates from the rolling mean
|
|
||||||
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
|
|
||||||
5. **CSV Logging**: Records all frame statistics for analysis
|
|
||||||
|
|
||||||
### Data Flow
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph TD
|
|
||||||
A[Video Frame] --> B[Extract Column]
|
|
||||||
B --> C[Calculate Column Mean]
|
|
||||||
C --> D[Store in Ring Buffer]
|
|
||||||
D --> E[Update Rolling Mean]
|
|
||||||
E --> F{Deviation > Threshold?}
|
|
||||||
F -->|Yes| G[DROP Frame]
|
|
||||||
F -->|No| H[PASS Frame]
|
|
||||||
C --> E
|
|
||||||
|
|
||||||
style G fill:#ff6b6b
|
|
||||||
style H fill:#51cf66
|
|
||||||
```
|
|
||||||
|
|
||||||
### Ring Buffer Operation
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph LR
|
|
||||||
subgraph Ring Buffer
|
|
||||||
A[0] --> B[1]
|
|
||||||
B --> C[2]
|
|
||||||
C --> D[...]
|
|
||||||
D --> E[N-1]
|
|
||||||
E ---|wrap| A
|
|
||||||
end
|
|
||||||
|
|
||||||
F[New Frame Mean] --> G[ring_index]
|
|
||||||
G --> A
|
|
||||||
|
|
||||||
H[Rolling Mean] --> I[Sum all values]
|
|
||||||
I --> J[Divide by count]
|
|
||||||
|
|
||||||
style G fill:#ffd43b
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture & Design
|
|
||||||
|
|
||||||
### Base Class
|
|
||||||
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
|
|
||||||
- In-place transform (analysis only, no frame modification)
|
|
||||||
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
|
|
||||||
- Returns `GST_FLOW_OK` to pass frames
|
|
||||||
|
|
||||||
### Element Structure
|
|
||||||
|
|
||||||
```c
|
|
||||||
struct _GstRollingSum
|
|
||||||
{
|
|
||||||
GstBaseTransform element;
|
|
||||||
|
|
||||||
/* Properties */
|
|
||||||
gint window_size; // Number of frames in rolling window (default: 1000)
|
|
||||||
gint column_index; // Which column to analyze (default: 1, second column)
|
|
||||||
gint stride; // Row sampling stride (default: 1, every row)
|
|
||||||
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
|
|
||||||
gchar *csv_file; // CSV output file path (default: NULL)
|
|
||||||
|
|
||||||
/* State */
|
|
||||||
gdouble *ring_buffer; // Circular buffer of column means
|
|
||||||
gint ring_index; // Current position in ring buffer
|
|
||||||
gint frame_count; // Total frames processed
|
|
||||||
gdouble rolling_mean; // Current rolling mean
|
|
||||||
FILE *csv_fp; // CSV file pointer
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
### Algorithm (Simplified from wissotsky's cli.py)
|
|
||||||
|
|
||||||
**Per Frame Processing:**
|
|
||||||
|
|
||||||
1. **Extract column data:**
|
|
||||||
- Select column at column_index
|
|
||||||
- Sample every stride rows
|
|
||||||
- Calculate mean of sampled pixels: frame_mean
|
|
||||||
|
|
||||||
2. **Update ring buffer:**
|
|
||||||
- Store frame_mean in ring_buffer[ring_index]
|
|
||||||
- Increment ring_index (wrap around)
|
|
||||||
|
|
||||||
3. **Calculate rolling mean:**
|
|
||||||
- Sum values in ring buffer (up to window_size or frame_count)
|
|
||||||
- Divide by actual window size
|
|
||||||
|
|
||||||
4. **Calculate deviation:**
|
|
||||||
- deviation = abs(frame_mean - rolling_mean)
|
|
||||||
- normalized_deviation = deviation / 255.0 (for 8-bit video)
|
|
||||||
|
|
||||||
5. **Decision:**
|
|
||||||
- If normalized_deviation > threshold: DROP frame
|
|
||||||
- Else: PASS frame
|
|
||||||
|
|
||||||
**Key Simplifications from cli.py:**
|
|
||||||
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
|
|
||||||
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
|
|
||||||
- **No recording logic**: Just drop/pass, no buffering for output segments
|
|
||||||
- **No patience mechanism**: Immediate decision per frame
|
|
||||||
|
|
||||||
### Video Format Support
|
|
||||||
|
|
||||||
**Initial Implementation:**
|
|
||||||
- **Primary target**: Grayscale (GRAY8, GRAY16)
|
|
||||||
- **Secondary**: Bayer formats (common in machine vision)
|
|
||||||
|
|
||||||
**Caps Filter:**
|
|
||||||
```c
|
|
||||||
static GstStaticPadTemplate sink_template =
|
|
||||||
GST_STATIC_PAD_TEMPLATE ("sink",
|
|
||||||
GST_PAD_SINK,
|
|
||||||
GST_PAD_ALWAYS,
|
|
||||||
GST_STATIC_CAPS (
|
|
||||||
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
|
|
||||||
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
|
|
||||||
)
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
## Plugin Properties
|
|
||||||
|
|
||||||
| Property | Type | Default | Range | Description |
|
|
||||||
|----------|------|---------|-------|-------------|
|
|
||||||
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
|
|
||||||
| `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) |
|
|
||||||
| `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) |
|
|
||||||
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames |
|
|
||||||
| `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) |
|
|
||||||
|
|
||||||
### Understanding Normalized Deviation
|
|
||||||
|
|
||||||
- **Range**: 0.0 to 1.0
|
|
||||||
- **Calculation**: `absolute_deviation / 255.0` (for 8-bit video)
|
|
||||||
- **Meaning**: Fraction of the full pixel range
|
|
||||||
- `0.001` = deviation of ~0.255 pixel values
|
|
||||||
- `0.01` = deviation of ~2.55 pixel values
|
|
||||||
- `0.1` = deviation of ~25.5 pixel values
|
|
||||||
|
|
||||||
## Basic Usage
|
|
||||||
|
|
||||||
### Simple Pipeline
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
|
|
||||||
videoconvert ! `
|
|
||||||
video/x-raw,format=GRAY8 ! `
|
|
||||||
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! `
|
|
||||||
autovideosink
|
|
||||||
```
|
|
||||||
|
|
||||||
### With CSV Logging
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! `
|
|
||||||
videoconvert ! `
|
|
||||||
video/x-raw,format=GRAY8 ! `
|
|
||||||
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! `
|
|
||||||
fakesink
|
|
||||||
```
|
|
||||||
|
|
||||||
### Custom Configuration
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
|
|
||||||
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! `
|
|
||||||
queue ! `
|
|
||||||
autovideosink
|
|
||||||
```
|
|
||||||
|
|
||||||
### With Format Conversion
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gst-launch-1.0 idsueyesrc ! `
|
|
||||||
videoconvert ! `
|
|
||||||
video/x-raw,format=GRAY8 ! `
|
|
||||||
rollingsum ! `
|
|
||||||
autovideosink
|
|
||||||
```
|
|
||||||
|
|
||||||
## Debugging
|
|
||||||
|
|
||||||
### Enable Debug Output
|
|
||||||
|
|
||||||
Use the `GST_DEBUG` environment variable to see detailed plugin operation:
|
|
||||||
|
|
||||||
#### Windows PowerShell
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
$env:GST_DEBUG="rollingsum:5"; gst-launch-1.0 [pipeline...]
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Windows CMD
|
|
||||||
|
|
||||||
```cmd
|
|
||||||
set GST_DEBUG=rollingsum:5 && gst-launch-1.0 [pipeline...]
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Linux/Mac
|
|
||||||
|
|
||||||
```bash
|
|
||||||
GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Debug Levels
|
|
||||||
|
|
||||||
| Level | Output |
|
|
||||||
|-------|--------|
|
|
||||||
| `rollingsum:1` | Errors only |
|
|
||||||
| `rollingsum:2` | Warnings |
|
|
||||||
| `rollingsum:3` | Info messages (file open/close) |
|
|
||||||
| `rollingsum:4` | Debug (caps negotiation) |
|
|
||||||
| `rollingsum:5` | Log (all frame processing) |
|
|
||||||
|
|
||||||
### Example Debug Output
|
|
||||||
|
|
||||||
```
|
|
||||||
0:00:04.029432200 DEBUG rollingsum gstrollingsum.c:436: Extracted column mean: 10.07
|
|
||||||
0:00:04.032257100 DEBUG rollingsum gstrollingsum.c:466: Frame 1: mean=10.07, rolling_mean=10.07, deviation=0.00 (normalized=0.0000)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Fields:**
|
|
||||||
- `Frame N`: Frame number
|
|
||||||
- `mean`: Current frame's column mean
|
|
||||||
- `rolling_mean`: Average of last N frames (window-size)
|
|
||||||
- `deviation`: Absolute difference
|
|
||||||
- `normalized`: Deviation as fraction of 255
|
|
||||||
|
|
||||||
### Common Debug Scenarios
|
|
||||||
|
|
||||||
#### 1. Verify Plugin Loaded
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gst-inspect-1.0 rollingsum
|
|
||||||
```
|
|
||||||
|
|
||||||
Should show plugin details. If not found, check `GST_PLUGIN_PATH`.
|
|
||||||
|
|
||||||
#### 2. Check CSV File Creation
|
|
||||||
|
|
||||||
Look for this in debug output:
|
|
||||||
```
|
|
||||||
INFO rollingsum: Opened CSV file: output.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 3. Monitor Frame Drops
|
|
||||||
|
|
||||||
Look for:
|
|
||||||
```
|
|
||||||
DEBUG rollingsum: Dropping frame 42: deviation 0.0005 > threshold 0.0002
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 4. Verify Caps Negotiation
|
|
||||||
|
|
||||||
```
|
|
||||||
DEBUG rollingsum: set_caps
|
|
||||||
DEBUG rollingsum: Video format: GRAY8, 1224x1026
|
|
||||||
```
|
|
||||||
|
|
||||||
## CSV Analysis
|
|
||||||
|
|
||||||
### CSV Format
|
|
||||||
|
|
||||||
The output CSV contains:
|
|
||||||
|
|
||||||
```csv
|
|
||||||
frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped
|
|
||||||
1,10.071150,10.071150,0.000000,0.000000,0
|
|
||||||
2,10.059454,10.065302,0.005848,0.000023,0
|
|
||||||
...
|
|
||||||
```
|
|
||||||
|
|
||||||
### Analyze Results
|
|
||||||
|
|
||||||
Use the included analysis script:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
uv run scripts/analyze_sma.py output.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output includes:**
|
|
||||||
- Statistical summary (min/max/mean/std)
|
|
||||||
- Threshold recommendations based on percentiles
|
|
||||||
- Standard deviation-based suggestions
|
|
||||||
- Visualization plots saved to `results/debug/`
|
|
||||||
- Archived CSV with timestamp in `results/debug/`
|
|
||||||
|
|
||||||
**Output files are automatically organized:**
|
|
||||||
- `results/debug/output_YYYYMMDD_HHMMSS.csv` - Archived CSV
|
|
||||||
- `results/debug/output_analysis_YYYYMMDD_HHMMSS.png` - Analysis plots
|
|
||||||
|
|
||||||
The `results/` directory is gitignored to keep your repository clean.
|
|
||||||
|
|
||||||
### Interpreting Results
|
|
||||||
|
|
||||||
The analysis provides threshold recommendations:
|
|
||||||
|
|
||||||
| Percentile | Description | Use Case |
|
|
||||||
|------------|-------------|----------|
|
|
||||||
| 99th | Drops top 1% | Very conservative, catch only extreme outliers |
|
|
||||||
| 95th | Drops top 5% | Conservative, good for quality control |
|
|
||||||
| 90th | Drops top 10% | Balanced, moderate filtering |
|
|
||||||
| 75th | Drops top 25% | Aggressive, maximum quality |
|
|
||||||
|
|
||||||
## Recommended Thresholds
|
|
||||||
|
|
||||||
Based on analysis of stable camera footage:
|
|
||||||
|
|
||||||
### For General Use
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
# Conservative (1-2% frame drop)
|
|
||||||
threshold=0.0003
|
|
||||||
|
|
||||||
# Moderate (5-10% frame drop)
|
|
||||||
threshold=0.0002
|
|
||||||
|
|
||||||
# Aggressive (20-25% frame drop)
|
|
||||||
threshold=0.0001
|
|
||||||
```
|
|
||||||
|
|
||||||
### For Specific Scenarios
|
|
||||||
|
|
||||||
**High-speed acquisition** (minimal processing):
|
|
||||||
```powershell
|
|
||||||
window-size=100 threshold=0.0005
|
|
||||||
```
|
|
||||||
|
|
||||||
**Quality-focused** (stable scenes):
|
|
||||||
```powershell
|
|
||||||
window-size=1000 threshold=0.0001
|
|
||||||
```
|
|
||||||
|
|
||||||
**Real-time monitoring** (fast response):
|
|
||||||
```powershell
|
|
||||||
window-size=50 threshold=0.0002
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### No frames being dropped (threshold too high)
|
|
||||||
|
|
||||||
**Symptom**: `dropped` column always 0 in CSV
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
1. Run with CSV logging
|
|
||||||
2. Analyze with `uv run scripts/analyze_sma.py output.csv`
|
|
||||||
3. Use recommended threshold from 90th-99th percentile
|
|
||||||
|
|
||||||
### Too many frames dropped (threshold too low)
|
|
||||||
|
|
||||||
**Symptom**: Most frames have `dropped=1`, choppy video
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
1. Increase threshold (try doubling current value)
|
|
||||||
2. Check if column_index is appropriate
|
|
||||||
3. Verify video is stable (not shaking/moving)
|
|
||||||
|
|
||||||
### CSV file not created
|
|
||||||
|
|
||||||
**Check**:
|
|
||||||
1. File path is writable
|
|
||||||
2. Look for "Opened CSV file" in debug output (`GST_DEBUG=rollingsum:3`)
|
|
||||||
3. Verify csv-file property is set correctly
|
|
||||||
|
|
||||||
### Column index out of range
|
|
||||||
|
|
||||||
**Symptom**:
|
|
||||||
```
|
|
||||||
WARNING rollingsum: Column index 1000 >= width 1224, using column 0
|
|
||||||
```
|
|
||||||
|
|
||||||
**Solution**: Set `column-index` to value < video width
|
|
||||||
|
|
||||||
### Inconsistent results
|
|
||||||
|
|
||||||
**Possible causes**:
|
|
||||||
1. Window size too small (< 50 frames)
|
|
||||||
2. Sampling moving/dynamic content
|
|
||||||
3. Column contains edge/artifact data
|
|
||||||
|
|
||||||
**Solutions**:
|
|
||||||
- Increase `window-size` to 500-1000
|
|
||||||
- Choose different `column-index` (avoid edges)
|
|
||||||
- Use `stride=2` or higher for faster processing
|
|
||||||
|
|
||||||
## Performance
|
|
||||||
|
|
||||||
### Performance Tips
|
|
||||||
|
|
||||||
1. **Larger window = more stable** but slower to adapt to scene changes
|
|
||||||
2. **Stride > 1** reduces computation but less accurate column mean
|
|
||||||
3. **CSV logging** has minimal performance impact
|
|
||||||
4. **Debug level 5** can produce massive logs, use only when needed
|
|
||||||
|
|
||||||
### Memory Usage
|
|
||||||
|
|
||||||
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
|
|
||||||
- Minimal per-frame allocation
|
|
||||||
|
|
||||||
### CPU Usage
|
|
||||||
|
|
||||||
- Column extraction: O(height/stride)
|
|
||||||
- Rolling mean update: O(1) using incremental sum
|
|
||||||
- Very lightweight compared to full-frame processing
|
|
||||||
|
|
||||||
### Optimization Opportunities
|
|
||||||
|
|
||||||
1. **Incremental mean**: Track sum instead of recalculating
|
|
||||||
2. **SIMD**: Vectorize column summation
|
|
||||||
3. **Skip calculation**: Only recalc every N frames if baseline is stable
|
|
||||||
|
|
||||||
## Integration Examples
|
|
||||||
|
|
||||||
### Python Script Control
|
|
||||||
|
|
||||||
```python
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
# Run pipeline with CSV logging
|
|
||||||
subprocess.run([
|
|
||||||
'gst-launch-1.0',
|
|
||||||
'idsueyesrc', 'config-file=config.ini',
|
|
||||||
'!', 'videoconvert',
|
|
||||||
'!', 'video/x-raw,format=GRAY8',
|
|
||||||
'!', 'rollingsum',
|
|
||||||
'window-size=1000',
|
|
||||||
'column-index=1',
|
|
||||||
'threshold=0.0002',
|
|
||||||
'csv-file=output.csv',
|
|
||||||
'!', 'fakesink'
|
|
||||||
])
|
|
||||||
|
|
||||||
# Analyze results
|
|
||||||
subprocess.run(['uv', 'run', 'scripts/analyze_sma.py', 'output.csv'])
|
|
||||||
```
|
|
||||||
|
|
||||||
### Adaptive Threshold
|
|
||||||
|
|
||||||
Use analysis results to set optimal threshold for next run:
|
|
||||||
|
|
||||||
```python
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
# Analyze previous run
|
|
||||||
df = pd.read_csv('output.csv')
|
|
||||||
recommended_threshold = df['normalized_deviation'].quantile(0.95)
|
|
||||||
|
|
||||||
print(f"Recommended threshold: {recommended_threshold:.6f}")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Developer Guide
|
|
||||||
|
|
||||||
### Implementation Files
|
|
||||||
|
|
||||||
**Directory Structure:**
|
|
||||||
```
|
|
||||||
gst/rollingsum/
|
|
||||||
├── CMakeLists.txt
|
|
||||||
├── gstrollingsum.c
|
|
||||||
└── gstrollingsum.h
|
|
||||||
```
|
|
||||||
|
|
||||||
**gstrollingsum.h:**
|
|
||||||
- Element type definitions
|
|
||||||
- Structure declarations
|
|
||||||
- Property enums
|
|
||||||
- Function prototypes
|
|
||||||
|
|
||||||
**gstrollingsum.c:**
|
|
||||||
- GObject methods (init, dispose, get/set properties)
|
|
||||||
- GstBaseTransform methods (transform_ip)
|
|
||||||
- Helper functions (extract_column_mean, update_rolling_mean)
|
|
||||||
- Plugin registration
|
|
||||||
|
|
||||||
**CMakeLists.txt:**
|
|
||||||
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
|
|
||||||
- Link GStreamer base and video libraries
|
|
||||||
|
|
||||||
### Adding New Features
|
|
||||||
|
|
||||||
Key files:
|
|
||||||
- [`gst/rollingsum/gstrollingsum.c`](gst/rollingsum/gstrollingsum.c) - Main implementation
|
|
||||||
- [`gst/rollingsum/gstrollingsum.h`](gst/rollingsum/gstrollingsum.h) - Header/structures
|
|
||||||
- [`gst/rollingsum/CMakeLists.txt`](gst/rollingsum/CMakeLists.txt) - Build config
|
|
||||||
|
|
||||||
### Rebuild After Changes
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
.\build.ps1 # Windows
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./build.sh # Linux
|
|
||||||
```
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
# Quick test
|
|
||||||
gst-inspect-1.0 rollingsum
|
|
||||||
|
|
||||||
# Full pipeline test with debug
|
|
||||||
$env:GST_DEBUG="rollingsum:5"
|
|
||||||
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
|
|
||||||
```
|
|
||||||
|
|
||||||
### Testing Strategy
|
|
||||||
|
|
||||||
**Unit Tests:**
|
|
||||||
- Ring buffer wrapping
|
|
||||||
- Mean calculation accuracy
|
|
||||||
- Threshold comparison logic
|
|
||||||
|
|
||||||
**Integration Tests:**
|
|
||||||
- Pipeline with videotestsrc
|
|
||||||
- Pipeline with idsueyesrc
|
|
||||||
- Frame drop verification
|
|
||||||
- Property changes during playback
|
|
||||||
|
|
||||||
**Test Cases:**
|
|
||||||
1. Static video (all frames similar) → all pass
|
|
||||||
2. Single bright frame → that frame drops
|
|
||||||
3. Gradual change → frames pass
|
|
||||||
4. Periodic pattern → pattern frames drop
|
|
||||||
|
|
||||||
### Integration with Existing Project
|
|
||||||
|
|
||||||
**Build System:**
|
|
||||||
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
|
|
||||||
```cmake
|
|
||||||
add_subdirectory (rollingsum)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Documentation:**
|
|
||||||
Update [`README.md`](README.md):
|
|
||||||
- Add rollingsum to "Other elements" section
|
|
||||||
- Add pipeline example
|
|
||||||
|
|
||||||
### Future Enhancements
|
|
||||||
|
|
||||||
**Phase 2 (If Needed):**
|
|
||||||
- Add EMA baseline tracking (like cli.py)
|
|
||||||
- Add variance-based thresholds
|
|
||||||
- Support multiple columns or regions
|
|
||||||
- Add metadata output (tag frames with deviation values)
|
|
||||||
- RGB format support (analyze specific channel)
|
|
||||||
|
|
||||||
**Phase 3 (Advanced):**
|
|
||||||
- Full cli.py recording logic
|
|
||||||
- Buffer and output segments
|
|
||||||
- Integration with probe detection systems
|
|
||||||
|
|
||||||
### Implementation Checklist
|
|
||||||
|
|
||||||
- [x] Create gst/rollingsum directory
|
|
||||||
- [x] Implement gstrollingsum.h
|
|
||||||
- [x] Implement gstrollingsum.c
|
|
||||||
- [x] Create CMakeLists.txt
|
|
||||||
- [x] Update gst/CMakeLists.txt
|
|
||||||
- [x] Build and test basic functionality
|
|
||||||
- [x] Test with idsueyesrc
|
|
||||||
- [x] Update README.md
|
|
||||||
- [x] Create feature branch
|
|
||||||
- [x] Commit and document
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
|
|
||||||
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
|
|
||||||
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
|
|
||||||
- [scripts/analyze_sma.py](scripts/analyze_sma.py) - Analysis tool
|
|
||||||
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
|
|
||||||
|
|
||||||
## Support
|
|
||||||
|
|
||||||
For issues or questions:
|
|
||||||
1. Enable debug output (`$env:GST_DEBUG="rollingsum:5"` in PowerShell)
|
|
||||||
2. Generate CSV log and analyze
|
|
||||||
3. Check this guide's troubleshooting section
|
|
||||||
4. Review debug output for errors/warnings
|
|
||||||
@@ -12,6 +12,5 @@ if (ENABLE_KLV)
|
|||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
add_subdirectory (misb)
|
add_subdirectory (misb)
|
||||||
add_subdirectory (rollingsum)
|
|
||||||
add_subdirectory (select)
|
add_subdirectory (select)
|
||||||
add_subdirectory (videoadjust)
|
add_subdirectory (videoadjust)
|
||||||
|
|||||||
@@ -1,28 +0,0 @@
|
|||||||
set (SOURCES
|
|
||||||
gstrollingsum.c
|
|
||||||
)
|
|
||||||
|
|
||||||
set (HEADERS
|
|
||||||
gstrollingsum.h)
|
|
||||||
|
|
||||||
include_directories (AFTER
|
|
||||||
${ORC_INCLUDE_DIR})
|
|
||||||
|
|
||||||
set (libname gstrollingsum)
|
|
||||||
|
|
||||||
add_library (${libname} MODULE
|
|
||||||
${SOURCES}
|
|
||||||
${HEADERS})
|
|
||||||
|
|
||||||
target_link_libraries (${libname}
|
|
||||||
${ORC_LIBRARIES}
|
|
||||||
${GLIB2_LIBRARIES}
|
|
||||||
${GOBJECT_LIBRARIES}
|
|
||||||
${GSTREAMER_LIBRARY}
|
|
||||||
${GSTREAMER_BASE_LIBRARY}
|
|
||||||
${GSTREAMER_VIDEO_LIBRARY})
|
|
||||||
|
|
||||||
if (WIN32)
|
|
||||||
install (FILES $<TARGET_PDB_FILE:${libname}> DESTINATION ${PDB_INSTALL_DIR} COMPONENT pdb OPTIONAL)
|
|
||||||
endif ()
|
|
||||||
install(TARGETS ${libname} LIBRARY DESTINATION ${PLUGIN_INSTALL_DIR})
|
|
||||||
@@ -1,535 +0,0 @@
|
|||||||
/* GStreamer
|
|
||||||
* Copyright (C) 2024 <your-name@your-email.com>
|
|
||||||
*
|
|
||||||
* This library is free software; you can redistribute it and/or
|
|
||||||
* modify it under the terms of the GNU Library General Public
|
|
||||||
* License as published by the Free Software Foundation; either
|
|
||||||
* version 2 of the License, or (at your option) any later version.
|
|
||||||
*
|
|
||||||
* This library is distributed in the hope that it will be useful,
|
|
||||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
||||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
||||||
* Library General Public License for more details.
|
|
||||||
*
|
|
||||||
* You should have received a copy of the GNU Library General Public
|
|
||||||
* License along with this library; if not, write to the
|
|
||||||
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
|
|
||||||
* Boston, MA 02111-1307, USA.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* SECTION:element-rollingsum
|
|
||||||
*
|
|
||||||
* Drops frames based on rolling mean analysis of a single column.
|
|
||||||
* Inspired by cli.py detection algorithm, simplified for real-time streaming.
|
|
||||||
*
|
|
||||||
* <refsect2>
|
|
||||||
* <title>Example launch line</title>
|
|
||||||
* |[
|
|
||||||
* gst-launch-1.0 idsueyesrc config-file=config.ini ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! autovideosink
|
|
||||||
* ]|
|
|
||||||
* </refsect2>
|
|
||||||
*/
|
|
||||||
|
|
||||||
#ifdef HAVE_CONFIG_H
|
|
||||||
#include "config.h"
|
|
||||||
#endif
|
|
||||||
|
|
||||||
#include "gstrollingsum.h"
|
|
||||||
#include <string.h>
|
|
||||||
#include <math.h>
|
|
||||||
#include <stdio.h>
|
|
||||||
|
|
||||||
enum
|
|
||||||
{
|
|
||||||
PROP_0,
|
|
||||||
PROP_WINDOW_SIZE,
|
|
||||||
PROP_COLUMN_INDEX,
|
|
||||||
PROP_STRIDE,
|
|
||||||
PROP_THRESHOLD,
|
|
||||||
PROP_CSV_FILENAME,
|
|
||||||
PROP_LAST
|
|
||||||
};
|
|
||||||
|
|
||||||
#define DEFAULT_PROP_WINDOW_SIZE 1000
|
|
||||||
#define DEFAULT_PROP_COLUMN_INDEX 1
|
|
||||||
#define DEFAULT_PROP_STRIDE 1
|
|
||||||
#define DEFAULT_PROP_THRESHOLD 0.5
|
|
||||||
#define DEFAULT_PROP_CSV_FILENAME NULL
|
|
||||||
|
|
||||||
/* Supported video formats */
|
|
||||||
#define SUPPORTED_CAPS \
|
|
||||||
GST_VIDEO_CAPS_MAKE("{ GRAY8, GRAY16_LE, GRAY16_BE }") ";" \
|
|
||||||
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
|
|
||||||
|
|
||||||
static GstStaticPadTemplate gst_rolling_sum_sink_template =
|
|
||||||
GST_STATIC_PAD_TEMPLATE ("sink",
|
|
||||||
GST_PAD_SINK,
|
|
||||||
GST_PAD_ALWAYS,
|
|
||||||
GST_STATIC_CAPS (SUPPORTED_CAPS)
|
|
||||||
);
|
|
||||||
|
|
||||||
static GstStaticPadTemplate gst_rolling_sum_src_template =
|
|
||||||
GST_STATIC_PAD_TEMPLATE ("src",
|
|
||||||
GST_PAD_SRC,
|
|
||||||
GST_PAD_ALWAYS,
|
|
||||||
GST_STATIC_CAPS (SUPPORTED_CAPS)
|
|
||||||
);
|
|
||||||
|
|
||||||
/* GObject vmethod declarations */
|
|
||||||
static void gst_rolling_sum_set_property (GObject * object, guint prop_id,
|
|
||||||
const GValue * value, GParamSpec * pspec);
|
|
||||||
static void gst_rolling_sum_get_property (GObject * object, guint prop_id,
|
|
||||||
GValue * value, GParamSpec * pspec);
|
|
||||||
static void gst_rolling_sum_dispose (GObject * object);
|
|
||||||
|
|
||||||
/* GstBaseTransform vmethod declarations */
|
|
||||||
static gboolean gst_rolling_sum_set_caps (GstBaseTransform * trans,
|
|
||||||
GstCaps * incaps, GstCaps * outcaps);
|
|
||||||
static GstFlowReturn gst_rolling_sum_transform_ip (GstBaseTransform * trans,
|
|
||||||
GstBuffer * buf);
|
|
||||||
|
|
||||||
/* GstRollingSum method declarations */
|
|
||||||
static void gst_rolling_sum_reset (GstRollingSum * filter);
|
|
||||||
static gdouble gst_rolling_sum_extract_column_mean (GstRollingSum * filter,
|
|
||||||
GstBuffer * buf);
|
|
||||||
|
|
||||||
/* setup debug */
|
|
||||||
GST_DEBUG_CATEGORY_STATIC (rolling_sum_debug);
|
|
||||||
#define GST_CAT_DEFAULT rolling_sum_debug
|
|
||||||
|
|
||||||
G_DEFINE_TYPE (GstRollingSum, gst_rolling_sum, GST_TYPE_BASE_TRANSFORM);
|
|
||||||
|
|
||||||
/************************************************************************/
|
|
||||||
/* GObject vmethod implementations */
|
|
||||||
/************************************************************************/
|
|
||||||
|
|
||||||
static void
|
|
||||||
gst_rolling_sum_dispose (GObject * object)
|
|
||||||
{
|
|
||||||
GstRollingSum *filter = GST_ROLLING_SUM (object);
|
|
||||||
|
|
||||||
GST_DEBUG ("dispose");
|
|
||||||
|
|
||||||
/* Close CSV file if open */
|
|
||||||
if (filter->csv_file) {
|
|
||||||
fclose (filter->csv_file);
|
|
||||||
filter->csv_file = NULL;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Free CSV filename */
|
|
||||||
if (filter->csv_filename) {
|
|
||||||
g_free (filter->csv_filename);
|
|
||||||
filter->csv_filename = NULL;
|
|
||||||
}
|
|
||||||
|
|
||||||
gst_rolling_sum_reset (filter);
|
|
||||||
|
|
||||||
/* chain up to the parent class */
|
|
||||||
G_OBJECT_CLASS (gst_rolling_sum_parent_class)->dispose (object);
|
|
||||||
}
|
|
||||||
|
|
||||||
static void
|
|
||||||
gst_rolling_sum_class_init (GstRollingSumClass * klass)
|
|
||||||
{
|
|
||||||
GObjectClass *gobject_class = G_OBJECT_CLASS (klass);
|
|
||||||
GstElementClass *gstelement_class = GST_ELEMENT_CLASS (klass);
|
|
||||||
GstBaseTransformClass *gstbasetransform_class =
|
|
||||||
GST_BASE_TRANSFORM_CLASS (klass);
|
|
||||||
|
|
||||||
GST_DEBUG_CATEGORY_INIT (rolling_sum_debug, "rollingsum", 0,
|
|
||||||
"Rolling Sum Filter");
|
|
||||||
|
|
||||||
GST_DEBUG ("class init");
|
|
||||||
|
|
||||||
/* Register GObject vmethods */
|
|
||||||
gobject_class->dispose = GST_DEBUG_FUNCPTR (gst_rolling_sum_dispose);
|
|
||||||
gobject_class->set_property = GST_DEBUG_FUNCPTR (gst_rolling_sum_set_property);
|
|
||||||
gobject_class->get_property = GST_DEBUG_FUNCPTR (gst_rolling_sum_get_property);
|
|
||||||
|
|
||||||
/* Install GObject properties */
|
|
||||||
g_object_class_install_property (gobject_class, PROP_WINDOW_SIZE,
|
|
||||||
g_param_spec_int ("window-size", "Window Size",
|
|
||||||
"Number of frames in rolling window", 1, 100000,
|
|
||||||
DEFAULT_PROP_WINDOW_SIZE,
|
|
||||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
|
||||||
GST_PARAM_MUTABLE_PLAYING));
|
|
||||||
|
|
||||||
g_object_class_install_property (gobject_class, PROP_COLUMN_INDEX,
|
|
||||||
g_param_spec_int ("column-index", "Column Index",
|
|
||||||
"Which vertical column to analyze (0-based)", 0, G_MAXINT,
|
|
||||||
DEFAULT_PROP_COLUMN_INDEX,
|
|
||||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
|
||||||
GST_PARAM_MUTABLE_PLAYING));
|
|
||||||
|
|
||||||
g_object_class_install_property (gobject_class, PROP_STRIDE,
|
|
||||||
g_param_spec_int ("stride", "Row Stride",
|
|
||||||
"Row sampling stride (1 = every row)", 1, G_MAXINT,
|
|
||||||
DEFAULT_PROP_STRIDE,
|
|
||||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
|
||||||
GST_PARAM_MUTABLE_PLAYING));
|
|
||||||
|
|
||||||
g_object_class_install_property (gobject_class, PROP_THRESHOLD,
|
|
||||||
g_param_spec_double ("threshold", "Threshold",
|
|
||||||
"Normalized deviation threshold for dropping frames", 0.0, 1.0,
|
|
||||||
DEFAULT_PROP_THRESHOLD,
|
|
||||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
|
||||||
GST_PARAM_MUTABLE_PLAYING));
|
|
||||||
|
|
||||||
g_object_class_install_property (gobject_class, PROP_CSV_FILENAME,
|
|
||||||
g_param_spec_string ("csv-file", "CSV File",
|
|
||||||
"Path to CSV file for logging frame data (NULL = no logging)",
|
|
||||||
DEFAULT_PROP_CSV_FILENAME,
|
|
||||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
|
||||||
GST_PARAM_MUTABLE_READY));
|
|
||||||
|
|
||||||
gst_element_class_add_pad_template (gstelement_class,
|
|
||||||
gst_static_pad_template_get (&gst_rolling_sum_sink_template));
|
|
||||||
gst_element_class_add_pad_template (gstelement_class,
|
|
||||||
gst_static_pad_template_get (&gst_rolling_sum_src_template));
|
|
||||||
|
|
||||||
gst_element_class_set_static_metadata (gstelement_class,
|
|
||||||
"Rolling sum filter", "Filter/Effect/Video",
|
|
||||||
"Drops frames based on rolling mean analysis of a single column",
|
|
||||||
"Your Name <your-email@example.com>");
|
|
||||||
|
|
||||||
/* Register GstBaseTransform vmethods */
|
|
||||||
gstbasetransform_class->set_caps =
|
|
||||||
GST_DEBUG_FUNCPTR (gst_rolling_sum_set_caps);
|
|
||||||
gstbasetransform_class->transform_ip =
|
|
||||||
GST_DEBUG_FUNCPTR (gst_rolling_sum_transform_ip);
|
|
||||||
}
|
|
||||||
|
|
||||||
static void
|
|
||||||
gst_rolling_sum_init (GstRollingSum * filter)
|
|
||||||
{
|
|
||||||
GST_DEBUG_OBJECT (filter, "init class instance");
|
|
||||||
|
|
||||||
filter->window_size = DEFAULT_PROP_WINDOW_SIZE;
|
|
||||||
filter->column_index = DEFAULT_PROP_COLUMN_INDEX;
|
|
||||||
filter->stride = DEFAULT_PROP_STRIDE;
|
|
||||||
filter->threshold = DEFAULT_PROP_THRESHOLD;
|
|
||||||
filter->csv_filename = NULL;
|
|
||||||
|
|
||||||
filter->ring_buffer = NULL;
|
|
||||||
filter->ring_index = 0;
|
|
||||||
filter->frame_count = 0;
|
|
||||||
filter->rolling_mean = 0.0;
|
|
||||||
filter->rolling_sum = 0.0;
|
|
||||||
filter->info_set = FALSE;
|
|
||||||
filter->csv_file = NULL;
|
|
||||||
|
|
||||||
gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE);
|
|
||||||
|
|
||||||
gst_rolling_sum_reset (filter);
|
|
||||||
}
|
|
||||||
|
|
||||||
static void
|
|
||||||
gst_rolling_sum_set_property (GObject * object, guint prop_id,
|
|
||||||
const GValue * value, GParamSpec * pspec)
|
|
||||||
{
|
|
||||||
GstRollingSum *filter = GST_ROLLING_SUM (object);
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter, "setting property %s", pspec->name);
|
|
||||||
|
|
||||||
switch (prop_id) {
|
|
||||||
case PROP_WINDOW_SIZE:
|
|
||||||
{
|
|
||||||
gint new_size = g_value_get_int (value);
|
|
||||||
if (new_size != filter->window_size) {
|
|
||||||
filter->window_size = new_size;
|
|
||||||
/* Reallocate ring buffer */
|
|
||||||
gst_rolling_sum_reset (filter);
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
case PROP_COLUMN_INDEX:
|
|
||||||
filter->column_index = g_value_get_int (value);
|
|
||||||
break;
|
|
||||||
case PROP_STRIDE:
|
|
||||||
filter->stride = g_value_get_int (value);
|
|
||||||
break;
|
|
||||||
case PROP_THRESHOLD:
|
|
||||||
filter->threshold = g_value_get_double (value);
|
|
||||||
break;
|
|
||||||
case PROP_CSV_FILENAME:
|
|
||||||
{
|
|
||||||
const gchar *filename = g_value_get_string (value);
|
|
||||||
|
|
||||||
/* Close old file if open */
|
|
||||||
if (filter->csv_file) {
|
|
||||||
fclose (filter->csv_file);
|
|
||||||
filter->csv_file = NULL;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Free old filename */
|
|
||||||
if (filter->csv_filename) {
|
|
||||||
g_free (filter->csv_filename);
|
|
||||||
filter->csv_filename = NULL;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Set new filename and open file */
|
|
||||||
if (filename && filename[0] != '\0') {
|
|
||||||
filter->csv_filename = g_strdup (filename);
|
|
||||||
filter->csv_file = fopen (filter->csv_filename, "w");
|
|
||||||
|
|
||||||
if (filter->csv_file) {
|
|
||||||
/* Write CSV header */
|
|
||||||
fprintf (filter->csv_file, "frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped\n");
|
|
||||||
fflush (filter->csv_file);
|
|
||||||
GST_INFO_OBJECT (filter, "Opened CSV file: %s", filter->csv_filename);
|
|
||||||
} else {
|
|
||||||
GST_ERROR_OBJECT (filter, "Failed to open CSV file: %s", filter->csv_filename);
|
|
||||||
g_free (filter->csv_filename);
|
|
||||||
filter->csv_filename = NULL;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
static void
|
|
||||||
gst_rolling_sum_get_property (GObject * object, guint prop_id, GValue * value,
|
|
||||||
GParamSpec * pspec)
|
|
||||||
{
|
|
||||||
GstRollingSum *filter = GST_ROLLING_SUM (object);
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter, "getting property %s", pspec->name);
|
|
||||||
|
|
||||||
switch (prop_id) {
|
|
||||||
case PROP_WINDOW_SIZE:
|
|
||||||
g_value_set_int (value, filter->window_size);
|
|
||||||
break;
|
|
||||||
case PROP_COLUMN_INDEX:
|
|
||||||
g_value_set_int (value, filter->column_index);
|
|
||||||
break;
|
|
||||||
case PROP_STRIDE:
|
|
||||||
g_value_set_int (value, filter->stride);
|
|
||||||
break;
|
|
||||||
case PROP_THRESHOLD:
|
|
||||||
g_value_set_double (value, filter->threshold);
|
|
||||||
break;
|
|
||||||
case PROP_CSV_FILENAME:
|
|
||||||
g_value_set_string (value, filter->csv_filename);
|
|
||||||
break;
|
|
||||||
default:
|
|
||||||
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
static gboolean
|
|
||||||
gst_rolling_sum_set_caps (GstBaseTransform * trans, GstCaps * incaps,
|
|
||||||
GstCaps * outcaps)
|
|
||||||
{
|
|
||||||
GstRollingSum *filter = GST_ROLLING_SUM (trans);
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter, "set_caps");
|
|
||||||
|
|
||||||
if (!gst_video_info_from_caps (&filter->video_info, incaps)) {
|
|
||||||
GST_ERROR_OBJECT (filter, "Failed to parse caps");
|
|
||||||
return FALSE;
|
|
||||||
}
|
|
||||||
|
|
||||||
filter->info_set = TRUE;
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter, "Video format: %s, %dx%d",
|
|
||||||
gst_video_format_to_string (GST_VIDEO_INFO_FORMAT (&filter->video_info)),
|
|
||||||
GST_VIDEO_INFO_WIDTH (&filter->video_info),
|
|
||||||
GST_VIDEO_INFO_HEIGHT (&filter->video_info));
|
|
||||||
|
|
||||||
return TRUE;
|
|
||||||
}
|
|
||||||
|
|
||||||
static gdouble
|
|
||||||
gst_rolling_sum_extract_column_mean (GstRollingSum * filter, GstBuffer * buf)
|
|
||||||
{
|
|
||||||
GstMapInfo map;
|
|
||||||
gdouble sum = 0.0;
|
|
||||||
gint count = 0;
|
|
||||||
gint width, height, stride_bytes;
|
|
||||||
gint row, col_offset;
|
|
||||||
guint8 *data;
|
|
||||||
GstVideoFormat format;
|
|
||||||
|
|
||||||
if (!filter->info_set) {
|
|
||||||
GST_WARNING_OBJECT (filter, "Video info not set yet");
|
|
||||||
return 0.0;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!gst_buffer_map (buf, &map, GST_MAP_READ)) {
|
|
||||||
GST_ERROR_OBJECT (filter, "Failed to map buffer");
|
|
||||||
return 0.0;
|
|
||||||
}
|
|
||||||
|
|
||||||
data = map.data;
|
|
||||||
width = GST_VIDEO_INFO_WIDTH (&filter->video_info);
|
|
||||||
height = GST_VIDEO_INFO_HEIGHT (&filter->video_info);
|
|
||||||
stride_bytes = GST_VIDEO_INFO_PLANE_STRIDE (&filter->video_info, 0);
|
|
||||||
format = GST_VIDEO_INFO_FORMAT (&filter->video_info);
|
|
||||||
|
|
||||||
/* Check column index is valid */
|
|
||||||
if (filter->column_index >= width) {
|
|
||||||
GST_WARNING_OBJECT (filter, "Column index %d >= width %d, using column 0",
|
|
||||||
filter->column_index, width);
|
|
||||||
filter->column_index = 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Calculate column offset based on format */
|
|
||||||
if (format == GST_VIDEO_FORMAT_GRAY8) {
|
|
||||||
col_offset = filter->column_index;
|
|
||||||
|
|
||||||
/* Sum column values with stride */
|
|
||||||
for (row = 0; row < height; row += filter->stride) {
|
|
||||||
sum += data[row * stride_bytes + col_offset];
|
|
||||||
count++;
|
|
||||||
}
|
|
||||||
} else if (format == GST_VIDEO_FORMAT_GRAY16_LE ||
|
|
||||||
format == GST_VIDEO_FORMAT_GRAY16_BE) {
|
|
||||||
col_offset = filter->column_index * 2;
|
|
||||||
|
|
||||||
/* Sum column values with stride */
|
|
||||||
for (row = 0; row < height; row += filter->stride) {
|
|
||||||
guint16 pixel_value;
|
|
||||||
guint8 *pixel_ptr = &data[row * stride_bytes + col_offset];
|
|
||||||
|
|
||||||
if (format == GST_VIDEO_FORMAT_GRAY16_LE) {
|
|
||||||
pixel_value = pixel_ptr[0] | (pixel_ptr[1] << 8);
|
|
||||||
} else {
|
|
||||||
pixel_value = (pixel_ptr[0] << 8) | pixel_ptr[1];
|
|
||||||
}
|
|
||||||
|
|
||||||
sum += pixel_value;
|
|
||||||
count++;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
/* For Bayer formats, treat as GRAY8 */
|
|
||||||
col_offset = filter->column_index;
|
|
||||||
|
|
||||||
for (row = 0; row < height; row += filter->stride) {
|
|
||||||
sum += data[row * stride_bytes + col_offset];
|
|
||||||
count++;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
gst_buffer_unmap (buf, &map);
|
|
||||||
|
|
||||||
return count > 0 ? sum / count : 0.0;
|
|
||||||
}
|
|
||||||
|
|
||||||
static GstFlowReturn
|
|
||||||
gst_rolling_sum_transform_ip (GstBaseTransform * trans, GstBuffer * buf)
|
|
||||||
{
|
|
||||||
GstRollingSum *filter = GST_ROLLING_SUM (trans);
|
|
||||||
gdouble frame_mean, deviation, old_value;
|
|
||||||
gint effective_window_size;
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter, "transform_ip called, frame_count=%d", filter->frame_count);
|
|
||||||
|
|
||||||
/* Extract column mean from current frame */
|
|
||||||
frame_mean = gst_rolling_sum_extract_column_mean (filter, buf);
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter, "Extracted column mean: %.2f", frame_mean);
|
|
||||||
|
|
||||||
/* Store in ring buffer */
|
|
||||||
old_value = filter->ring_buffer[filter->ring_index];
|
|
||||||
filter->ring_buffer[filter->ring_index] = frame_mean;
|
|
||||||
|
|
||||||
/* Update rolling sum efficiently */
|
|
||||||
if (filter->frame_count < filter->window_size) {
|
|
||||||
/* Still filling the buffer */
|
|
||||||
filter->rolling_sum += frame_mean;
|
|
||||||
filter->frame_count++;
|
|
||||||
effective_window_size = filter->frame_count;
|
|
||||||
} else {
|
|
||||||
/* Buffer is full, replace old value */
|
|
||||||
filter->rolling_sum = filter->rolling_sum - old_value + frame_mean;
|
|
||||||
effective_window_size = filter->window_size;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Update rolling mean */
|
|
||||||
filter->rolling_mean = filter->rolling_sum / effective_window_size;
|
|
||||||
|
|
||||||
/* Calculate deviation */
|
|
||||||
deviation = fabs(frame_mean - filter->rolling_mean);
|
|
||||||
|
|
||||||
/* Normalize deviation (assuming 8-bit equivalent range) */
|
|
||||||
gdouble normalized_deviation = deviation / 255.0;
|
|
||||||
|
|
||||||
GST_DEBUG_OBJECT (filter,
|
|
||||||
"Frame %d: mean=%.2f, rolling_mean=%.2f, deviation=%.2f (normalized=%.4f)",
|
|
||||||
filter->frame_count, frame_mean, filter->rolling_mean, deviation,
|
|
||||||
normalized_deviation);
|
|
||||||
|
|
||||||
/* Advance ring buffer index */
|
|
||||||
filter->ring_index = (filter->ring_index + 1) % filter->window_size;
|
|
||||||
|
|
||||||
/* Decision: drop or pass frame */
|
|
||||||
gboolean dropped = FALSE;
|
|
||||||
if (normalized_deviation > filter->threshold) {
|
|
||||||
GST_DEBUG_OBJECT (filter,
|
|
||||||
"Dropping frame %d: deviation %.4f > threshold %.4f",
|
|
||||||
filter->frame_count, normalized_deviation, filter->threshold);
|
|
||||||
dropped = TRUE;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Write to CSV if file is open */
|
|
||||||
if (filter->csv_file) {
|
|
||||||
fprintf (filter->csv_file, "%d,%.6f,%.6f,%.6f,%.6f,%d\n",
|
|
||||||
filter->frame_count, frame_mean, filter->rolling_mean,
|
|
||||||
deviation, normalized_deviation, dropped ? 1 : 0);
|
|
||||||
fflush (filter->csv_file);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (dropped) {
|
|
||||||
return GST_BASE_TRANSFORM_FLOW_DROPPED;
|
|
||||||
}
|
|
||||||
|
|
||||||
return GST_FLOW_OK;
|
|
||||||
}
|
|
||||||
|
|
||||||
static void
|
|
||||||
gst_rolling_sum_reset (GstRollingSum * filter)
|
|
||||||
{
|
|
||||||
GST_DEBUG_OBJECT (filter, "reset");
|
|
||||||
|
|
||||||
/* Free old ring buffer if exists */
|
|
||||||
if (filter->ring_buffer) {
|
|
||||||
g_free (filter->ring_buffer);
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Allocate new ring buffer */
|
|
||||||
filter->ring_buffer = g_new0 (gdouble, filter->window_size);
|
|
||||||
filter->ring_index = 0;
|
|
||||||
filter->frame_count = 0;
|
|
||||||
filter->rolling_mean = 0.0;
|
|
||||||
filter->rolling_sum = 0.0;
|
|
||||||
}
|
|
||||||
|
|
||||||
static gboolean
|
|
||||||
plugin_init (GstPlugin * plugin)
|
|
||||||
{
|
|
||||||
GST_DEBUG_CATEGORY_INIT (GST_CAT_DEFAULT, "rollingsum", 0, "rollingsum");
|
|
||||||
|
|
||||||
GST_DEBUG ("plugin_init");
|
|
||||||
|
|
||||||
GST_CAT_INFO (GST_CAT_DEFAULT, "registering rollingsum element");
|
|
||||||
|
|
||||||
if (!gst_element_register (plugin, "rollingsum", GST_RANK_NONE,
|
|
||||||
GST_TYPE_ROLLING_SUM)) {
|
|
||||||
return FALSE;
|
|
||||||
}
|
|
||||||
|
|
||||||
return TRUE;
|
|
||||||
}
|
|
||||||
|
|
||||||
GST_PLUGIN_DEFINE (GST_VERSION_MAJOR,
|
|
||||||
GST_VERSION_MINOR,
|
|
||||||
rollingsum,
|
|
||||||
"Filter that drops frames based on rolling mean analysis",
|
|
||||||
plugin_init, GST_PACKAGE_VERSION, GST_PACKAGE_LICENSE, GST_PACKAGE_NAME,
|
|
||||||
GST_PACKAGE_ORIGIN);
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
/* GStreamer
|
|
||||||
* Copyright (C) 2024 <your-name@your-email.com>
|
|
||||||
*
|
|
||||||
* This library is free software; you can redistribute it and/or
|
|
||||||
* modify it under the terms of the GNU Library General Public
|
|
||||||
* License as published by the Free Software Foundation; either
|
|
||||||
* version 2 of the License, or (at your option) any later version.
|
|
||||||
*
|
|
||||||
* This library is distributed in the hope that it will be useful,
|
|
||||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
||||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
||||||
* Library General Public License for more details.
|
|
||||||
*
|
|
||||||
* You should have received a copy of the GNU Library General Public
|
|
||||||
* License along with this library; if not, write to the
|
|
||||||
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
|
|
||||||
* Boston, MA 02111-1307, USA.
|
|
||||||
*/
|
|
||||||
|
|
||||||
#ifndef __GST_ROLLING_SUM_H__
|
|
||||||
#define __GST_ROLLING_SUM_H__
|
|
||||||
|
|
||||||
#include <gst/base/gstbasetransform.h>
|
|
||||||
#include <gst/video/video.h>
|
|
||||||
#include <stdio.h>
|
|
||||||
|
|
||||||
G_BEGIN_DECLS
|
|
||||||
|
|
||||||
#define GST_TYPE_ROLLING_SUM \
|
|
||||||
(gst_rolling_sum_get_type())
|
|
||||||
#define GST_ROLLING_SUM(obj) \
|
|
||||||
(G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_ROLLING_SUM,GstRollingSum))
|
|
||||||
#define GST_ROLLING_SUM_CLASS(klass) \
|
|
||||||
(G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_ROLLING_SUM,GstRollingSumClass))
|
|
||||||
#define GST_IS_ROLLING_SUM(obj) \
|
|
||||||
(G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_ROLLING_SUM))
|
|
||||||
#define GST_IS_ROLLING_SUM_CLASS(klass) \
|
|
||||||
(G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_ROLLING_SUM))
|
|
||||||
|
|
||||||
typedef struct _GstRollingSum GstRollingSum;
|
|
||||||
typedef struct _GstRollingSumClass GstRollingSumClass;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* GstRollingSum:
|
|
||||||
* @element: the parent element.
|
|
||||||
*
|
|
||||||
* The opaque GstRollingSum data structure.
|
|
||||||
*/
|
|
||||||
struct _GstRollingSum
|
|
||||||
{
|
|
||||||
GstBaseTransform element;
|
|
||||||
|
|
||||||
/* Properties */
|
|
||||||
gint window_size; /* Number of frames in rolling window */
|
|
||||||
gint column_index; /* Which column to analyze (0-based) */
|
|
||||||
gint stride; /* Row sampling stride */
|
|
||||||
gdouble threshold; /* Deviation threshold for dropping frames */
|
|
||||||
gchar *csv_filename; /* CSV output filename (NULL = no CSV) */
|
|
||||||
|
|
||||||
/* State */
|
|
||||||
gdouble *ring_buffer; /* Circular buffer of column means */
|
|
||||||
gint ring_index; /* Current position in ring buffer */
|
|
||||||
gint frame_count; /* Total frames processed */
|
|
||||||
gdouble rolling_mean; /* Current rolling mean */
|
|
||||||
gdouble rolling_sum; /* Current rolling sum for efficient mean update */
|
|
||||||
FILE *csv_file; /* CSV file handle */
|
|
||||||
|
|
||||||
/* Video format info */
|
|
||||||
GstVideoInfo video_info;
|
|
||||||
gboolean info_set;
|
|
||||||
};
|
|
||||||
|
|
||||||
struct _GstRollingSumClass
|
|
||||||
{
|
|
||||||
GstBaseTransformClass parent_class;
|
|
||||||
};
|
|
||||||
|
|
||||||
GType gst_rolling_sum_get_type(void);
|
|
||||||
|
|
||||||
G_END_DECLS
|
|
||||||
|
|
||||||
#endif /* __GST_ROLLING_SUM_H__ */
|
|
||||||
Reference in New Issue
Block a user