Compare commits
5 Commits
89f904cd71
...
bfc8756a2a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bfc8756a2a | ||
|
|
11b279cb08 | ||
|
|
411310e4f1 | ||
|
|
11350301b9 | ||
|
|
ddb30e00e1 |
@@ -1,613 +0,0 @@
|
||||
# GStreamer Rolling Sum Plugin - Complete Documentation
|
||||
|
||||
## Table of Contents
|
||||
- [Overview](#overview)
|
||||
- [How It Works](#how-it-works)
|
||||
- [Architecture & Design](#architecture--design)
|
||||
- [Plugin Properties](#plugin-properties)
|
||||
- [Basic Usage](#basic-usage)
|
||||
- [Debugging](#debugging)
|
||||
- [CSV Analysis](#csv-analysis)
|
||||
- [Recommended Thresholds](#recommended-thresholds)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Performance](#performance)
|
||||
- [Integration Examples](#integration-examples)
|
||||
- [Developer Guide](#developer-guide)
|
||||
- [References](#references)
|
||||
|
||||
## Overview
|
||||
|
||||
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
|
||||
|
||||
**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames
|
||||
|
||||
**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
|
||||
2. **Rolling Window**: Maintains a circular buffer of recent column means
|
||||
3. **Deviation Detection**: Calculates how much each frame deviates from the rolling mean
|
||||
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
|
||||
5. **CSV Logging**: Records all frame statistics for analysis
|
||||
|
||||
### Data Flow
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Video Frame] --> B[Extract Column]
|
||||
B --> C[Calculate Column Mean]
|
||||
C --> D[Store in Ring Buffer]
|
||||
D --> E[Update Rolling Mean]
|
||||
E --> F{Deviation > Threshold?}
|
||||
F -->|Yes| G[DROP Frame]
|
||||
F -->|No| H[PASS Frame]
|
||||
C --> E
|
||||
|
||||
style G fill:#ff6b6b
|
||||
style H fill:#51cf66
|
||||
```
|
||||
|
||||
### Ring Buffer Operation
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Ring Buffer
|
||||
A[0] --> B[1]
|
||||
B --> C[2]
|
||||
C --> D[...]
|
||||
D --> E[N-1]
|
||||
E ---|wrap| A
|
||||
end
|
||||
|
||||
F[New Frame Mean] --> G[ring_index]
|
||||
G --> A
|
||||
|
||||
H[Rolling Mean] --> I[Sum all values]
|
||||
I --> J[Divide by count]
|
||||
|
||||
style G fill:#ffd43b
|
||||
```
|
||||
|
||||
## Architecture & Design
|
||||
|
||||
### Base Class
|
||||
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
|
||||
- In-place transform (analysis only, no frame modification)
|
||||
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
|
||||
- Returns `GST_FLOW_OK` to pass frames
|
||||
|
||||
### Element Structure
|
||||
|
||||
```c
|
||||
struct _GstRollingSum
|
||||
{
|
||||
GstBaseTransform element;
|
||||
|
||||
/* Properties */
|
||||
gint window_size; // Number of frames in rolling window (default: 1000)
|
||||
gint column_index; // Which column to analyze (default: 1, second column)
|
||||
gint stride; // Row sampling stride (default: 1, every row)
|
||||
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
|
||||
gchar *csv_file; // CSV output file path (default: NULL)
|
||||
|
||||
/* State */
|
||||
gdouble *ring_buffer; // Circular buffer of column means
|
||||
gint ring_index; // Current position in ring buffer
|
||||
gint frame_count; // Total frames processed
|
||||
gdouble rolling_mean; // Current rolling mean
|
||||
FILE *csv_fp; // CSV file pointer
|
||||
};
|
||||
```
|
||||
|
||||
### Algorithm (Simplified from wissotsky's cli.py)
|
||||
|
||||
**Per Frame Processing:**
|
||||
|
||||
1. **Extract column data:**
|
||||
- Select column at column_index
|
||||
- Sample every stride rows
|
||||
- Calculate mean of sampled pixels: frame_mean
|
||||
|
||||
2. **Update ring buffer:**
|
||||
- Store frame_mean in ring_buffer[ring_index]
|
||||
- Increment ring_index (wrap around)
|
||||
|
||||
3. **Calculate rolling mean:**
|
||||
- Sum values in ring buffer (up to window_size or frame_count)
|
||||
- Divide by actual window size
|
||||
|
||||
4. **Calculate deviation:**
|
||||
- deviation = abs(frame_mean - rolling_mean)
|
||||
- normalized_deviation = deviation / 255.0 (for 8-bit video)
|
||||
|
||||
5. **Decision:**
|
||||
- If normalized_deviation > threshold: DROP frame
|
||||
- Else: PASS frame
|
||||
|
||||
**Key Simplifications from cli.py:**
|
||||
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
|
||||
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
|
||||
- **No recording logic**: Just drop/pass, no buffering for output segments
|
||||
- **No patience mechanism**: Immediate decision per frame
|
||||
|
||||
### Video Format Support
|
||||
|
||||
**Initial Implementation:**
|
||||
- **Primary target**: Grayscale (GRAY8, GRAY16)
|
||||
- **Secondary**: Bayer formats (common in machine vision)
|
||||
|
||||
**Caps Filter:**
|
||||
```c
|
||||
static GstStaticPadTemplate sink_template =
|
||||
GST_STATIC_PAD_TEMPLATE ("sink",
|
||||
GST_PAD_SINK,
|
||||
GST_PAD_ALWAYS,
|
||||
GST_STATIC_CAPS (
|
||||
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
|
||||
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
|
||||
)
|
||||
);
|
||||
```
|
||||
|
||||
## Plugin Properties
|
||||
|
||||
| Property | Type | Default | Range | Description |
|
||||
|----------|------|---------|-------|-------------|
|
||||
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
|
||||
| `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) |
|
||||
| `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) |
|
||||
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames |
|
||||
| `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) |
|
||||
|
||||
### Understanding Normalized Deviation
|
||||
|
||||
- **Range**: 0.0 to 1.0
|
||||
- **Calculation**: `absolute_deviation / 255.0` (for 8-bit video)
|
||||
- **Meaning**: Fraction of the full pixel range
|
||||
- `0.001` = deviation of ~0.255 pixel values
|
||||
- `0.01` = deviation of ~2.55 pixel values
|
||||
- `0.1` = deviation of ~25.5 pixel values
|
||||
|
||||
## Basic Usage
|
||||
|
||||
### Simple Pipeline
|
||||
|
||||
```powershell
|
||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
|
||||
videoconvert ! `
|
||||
video/x-raw,format=GRAY8 ! `
|
||||
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! `
|
||||
autovideosink
|
||||
```
|
||||
|
||||
### With CSV Logging
|
||||
|
||||
```powershell
|
||||
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! `
|
||||
videoconvert ! `
|
||||
video/x-raw,format=GRAY8 ! `
|
||||
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! `
|
||||
fakesink
|
||||
```
|
||||
|
||||
### Custom Configuration
|
||||
|
||||
```powershell
|
||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
|
||||
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! `
|
||||
queue ! `
|
||||
autovideosink
|
||||
```
|
||||
|
||||
### With Format Conversion
|
||||
|
||||
```powershell
|
||||
gst-launch-1.0 idsueyesrc ! `
|
||||
videoconvert ! `
|
||||
video/x-raw,format=GRAY8 ! `
|
||||
rollingsum ! `
|
||||
autovideosink
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### Enable Debug Output
|
||||
|
||||
Use the `GST_DEBUG` environment variable to see detailed plugin operation:
|
||||
|
||||
#### Windows PowerShell
|
||||
|
||||
```powershell
|
||||
$env:GST_DEBUG="rollingsum:5"; gst-launch-1.0 [pipeline...]
|
||||
```
|
||||
|
||||
#### Windows CMD
|
||||
|
||||
```cmd
|
||||
set GST_DEBUG=rollingsum:5 && gst-launch-1.0 [pipeline...]
|
||||
```
|
||||
|
||||
#### Linux/Mac
|
||||
|
||||
```bash
|
||||
GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...]
|
||||
```
|
||||
|
||||
### Debug Levels
|
||||
|
||||
| Level | Output |
|
||||
|-------|--------|
|
||||
| `rollingsum:1` | Errors only |
|
||||
| `rollingsum:2` | Warnings |
|
||||
| `rollingsum:3` | Info messages (file open/close) |
|
||||
| `rollingsum:4` | Debug (caps negotiation) |
|
||||
| `rollingsum:5` | Log (all frame processing) |
|
||||
|
||||
### Example Debug Output
|
||||
|
||||
```
|
||||
0:00:04.029432200 DEBUG rollingsum gstrollingsum.c:436: Extracted column mean: 10.07
|
||||
0:00:04.032257100 DEBUG rollingsum gstrollingsum.c:466: Frame 1: mean=10.07, rolling_mean=10.07, deviation=0.00 (normalized=0.0000)
|
||||
```
|
||||
|
||||
**Key Fields:**
|
||||
- `Frame N`: Frame number
|
||||
- `mean`: Current frame's column mean
|
||||
- `rolling_mean`: Average of last N frames (window-size)
|
||||
- `deviation`: Absolute difference
|
||||
- `normalized`: Deviation as fraction of 255
|
||||
|
||||
### Common Debug Scenarios
|
||||
|
||||
#### 1. Verify Plugin Loaded
|
||||
|
||||
```powershell
|
||||
gst-inspect-1.0 rollingsum
|
||||
```
|
||||
|
||||
Should show plugin details. If not found, check `GST_PLUGIN_PATH`.
|
||||
|
||||
#### 2. Check CSV File Creation
|
||||
|
||||
Look for this in debug output:
|
||||
```
|
||||
INFO rollingsum: Opened CSV file: output.csv
|
||||
```
|
||||
|
||||
#### 3. Monitor Frame Drops
|
||||
|
||||
Look for:
|
||||
```
|
||||
DEBUG rollingsum: Dropping frame 42: deviation 0.0005 > threshold 0.0002
|
||||
```
|
||||
|
||||
#### 4. Verify Caps Negotiation
|
||||
|
||||
```
|
||||
DEBUG rollingsum: set_caps
|
||||
DEBUG rollingsum: Video format: GRAY8, 1224x1026
|
||||
```
|
||||
|
||||
## CSV Analysis
|
||||
|
||||
### CSV Format
|
||||
|
||||
The output CSV contains:
|
||||
|
||||
```csv
|
||||
frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped
|
||||
1,10.071150,10.071150,0.000000,0.000000,0
|
||||
2,10.059454,10.065302,0.005848,0.000023,0
|
||||
...
|
||||
```
|
||||
|
||||
### Analyze Results
|
||||
|
||||
Use the included analysis script:
|
||||
|
||||
```powershell
|
||||
uv run scripts/analyze_sma.py output.csv
|
||||
```
|
||||
|
||||
**Output includes:**
|
||||
- Statistical summary (min/max/mean/std)
|
||||
- Threshold recommendations based on percentiles
|
||||
- Standard deviation-based suggestions
|
||||
- Visualization plots saved to `results/debug/`
|
||||
- Archived CSV with timestamp in `results/debug/`
|
||||
|
||||
**Output files are automatically organized:**
|
||||
- `results/debug/output_YYYYMMDD_HHMMSS.csv` - Archived CSV
|
||||
- `results/debug/output_analysis_YYYYMMDD_HHMMSS.png` - Analysis plots
|
||||
|
||||
The `results/` directory is gitignored to keep your repository clean.
|
||||
|
||||
### Interpreting Results
|
||||
|
||||
The analysis provides threshold recommendations:
|
||||
|
||||
| Percentile | Description | Use Case |
|
||||
|------------|-------------|----------|
|
||||
| 99th | Drops top 1% | Very conservative, catch only extreme outliers |
|
||||
| 95th | Drops top 5% | Conservative, good for quality control |
|
||||
| 90th | Drops top 10% | Balanced, moderate filtering |
|
||||
| 75th | Drops top 25% | Aggressive, maximum quality |
|
||||
|
||||
## Recommended Thresholds
|
||||
|
||||
Based on analysis of stable camera footage:
|
||||
|
||||
### For General Use
|
||||
|
||||
```powershell
|
||||
# Conservative (1-2% frame drop)
|
||||
threshold=0.0003
|
||||
|
||||
# Moderate (5-10% frame drop)
|
||||
threshold=0.0002
|
||||
|
||||
# Aggressive (20-25% frame drop)
|
||||
threshold=0.0001
|
||||
```
|
||||
|
||||
### For Specific Scenarios
|
||||
|
||||
**High-speed acquisition** (minimal processing):
|
||||
```powershell
|
||||
window-size=100 threshold=0.0005
|
||||
```
|
||||
|
||||
**Quality-focused** (stable scenes):
|
||||
```powershell
|
||||
window-size=1000 threshold=0.0001
|
||||
```
|
||||
|
||||
**Real-time monitoring** (fast response):
|
||||
```powershell
|
||||
window-size=50 threshold=0.0002
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No frames being dropped (threshold too high)
|
||||
|
||||
**Symptom**: `dropped` column always 0 in CSV
|
||||
|
||||
**Solution**:
|
||||
1. Run with CSV logging
|
||||
2. Analyze with `uv run scripts/analyze_sma.py output.csv`
|
||||
3. Use recommended threshold from 90th-99th percentile
|
||||
|
||||
### Too many frames dropped (threshold too low)
|
||||
|
||||
**Symptom**: Most frames have `dropped=1`, choppy video
|
||||
|
||||
**Solution**:
|
||||
1. Increase threshold (try doubling current value)
|
||||
2. Check if column_index is appropriate
|
||||
3. Verify video is stable (not shaking/moving)
|
||||
|
||||
### CSV file not created
|
||||
|
||||
**Check**:
|
||||
1. File path is writable
|
||||
2. Look for "Opened CSV file" in debug output (`GST_DEBUG=rollingsum:3`)
|
||||
3. Verify csv-file property is set correctly
|
||||
|
||||
### Column index out of range
|
||||
|
||||
**Symptom**:
|
||||
```
|
||||
WARNING rollingsum: Column index 1000 >= width 1224, using column 0
|
||||
```
|
||||
|
||||
**Solution**: Set `column-index` to value < video width
|
||||
|
||||
### Inconsistent results
|
||||
|
||||
**Possible causes**:
|
||||
1. Window size too small (< 50 frames)
|
||||
2. Sampling moving/dynamic content
|
||||
3. Column contains edge/artifact data
|
||||
|
||||
**Solutions**:
|
||||
- Increase `window-size` to 500-1000
|
||||
- Choose different `column-index` (avoid edges)
|
||||
- Use `stride=2` or higher for faster processing
|
||||
|
||||
## Performance
|
||||
|
||||
### Performance Tips
|
||||
|
||||
1. **Larger window = more stable** but slower to adapt to scene changes
|
||||
2. **Stride > 1** reduces computation but less accurate column mean
|
||||
3. **CSV logging** has minimal performance impact
|
||||
4. **Debug level 5** can produce massive logs, use only when needed
|
||||
|
||||
### Memory Usage
|
||||
|
||||
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
|
||||
- Minimal per-frame allocation
|
||||
|
||||
### CPU Usage
|
||||
|
||||
- Column extraction: O(height/stride)
|
||||
- Rolling mean update: O(1) using incremental sum
|
||||
- Very lightweight compared to full-frame processing
|
||||
|
||||
### Optimization Opportunities
|
||||
|
||||
1. **Incremental mean**: Track sum instead of recalculating
|
||||
2. **SIMD**: Vectorize column summation
|
||||
3. **Skip calculation**: Only recalc every N frames if baseline is stable
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Python Script Control
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
|
||||
# Run pipeline with CSV logging
|
||||
subprocess.run([
|
||||
'gst-launch-1.0',
|
||||
'idsueyesrc', 'config-file=config.ini',
|
||||
'!', 'videoconvert',
|
||||
'!', 'video/x-raw,format=GRAY8',
|
||||
'!', 'rollingsum',
|
||||
'window-size=1000',
|
||||
'column-index=1',
|
||||
'threshold=0.0002',
|
||||
'csv-file=output.csv',
|
||||
'!', 'fakesink'
|
||||
])
|
||||
|
||||
# Analyze results
|
||||
subprocess.run(['uv', 'run', 'scripts/analyze_sma.py', 'output.csv'])
|
||||
```
|
||||
|
||||
### Adaptive Threshold
|
||||
|
||||
Use analysis results to set optimal threshold for next run:
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
|
||||
# Analyze previous run
|
||||
df = pd.read_csv('output.csv')
|
||||
recommended_threshold = df['normalized_deviation'].quantile(0.95)
|
||||
|
||||
print(f"Recommended threshold: {recommended_threshold:.6f}")
|
||||
```
|
||||
|
||||
## Developer Guide
|
||||
|
||||
### Implementation Files
|
||||
|
||||
**Directory Structure:**
|
||||
```
|
||||
gst/rollingsum/
|
||||
├── CMakeLists.txt
|
||||
├── gstrollingsum.c
|
||||
└── gstrollingsum.h
|
||||
```
|
||||
|
||||
**gstrollingsum.h:**
|
||||
- Element type definitions
|
||||
- Structure declarations
|
||||
- Property enums
|
||||
- Function prototypes
|
||||
|
||||
**gstrollingsum.c:**
|
||||
- GObject methods (init, dispose, get/set properties)
|
||||
- GstBaseTransform methods (transform_ip)
|
||||
- Helper functions (extract_column_mean, update_rolling_mean)
|
||||
- Plugin registration
|
||||
|
||||
**CMakeLists.txt:**
|
||||
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
|
||||
- Link GStreamer base and video libraries
|
||||
|
||||
### Adding New Features
|
||||
|
||||
Key files:
|
||||
- [`gst/rollingsum/gstrollingsum.c`](gst/rollingsum/gstrollingsum.c) - Main implementation
|
||||
- [`gst/rollingsum/gstrollingsum.h`](gst/rollingsum/gstrollingsum.h) - Header/structures
|
||||
- [`gst/rollingsum/CMakeLists.txt`](gst/rollingsum/CMakeLists.txt) - Build config
|
||||
|
||||
### Rebuild After Changes
|
||||
|
||||
```powershell
|
||||
.\build.ps1 # Windows
|
||||
```
|
||||
|
||||
```bash
|
||||
./build.sh # Linux
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```powershell
|
||||
# Quick test
|
||||
gst-inspect-1.0 rollingsum
|
||||
|
||||
# Full pipeline test with debug
|
||||
$env:GST_DEBUG="rollingsum:5"
|
||||
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
|
||||
```
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
**Unit Tests:**
|
||||
- Ring buffer wrapping
|
||||
- Mean calculation accuracy
|
||||
- Threshold comparison logic
|
||||
|
||||
**Integration Tests:**
|
||||
- Pipeline with videotestsrc
|
||||
- Pipeline with idsueyesrc
|
||||
- Frame drop verification
|
||||
- Property changes during playback
|
||||
|
||||
**Test Cases:**
|
||||
1. Static video (all frames similar) → all pass
|
||||
2. Single bright frame → that frame drops
|
||||
3. Gradual change → frames pass
|
||||
4. Periodic pattern → pattern frames drop
|
||||
|
||||
### Integration with Existing Project
|
||||
|
||||
**Build System:**
|
||||
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
|
||||
```cmake
|
||||
add_subdirectory (rollingsum)
|
||||
```
|
||||
|
||||
**Documentation:**
|
||||
Update [`README.md`](README.md):
|
||||
- Add rollingsum to "Other elements" section
|
||||
- Add pipeline example
|
||||
|
||||
### Future Enhancements
|
||||
|
||||
**Phase 2 (If Needed):**
|
||||
- Add EMA baseline tracking (like cli.py)
|
||||
- Add variance-based thresholds
|
||||
- Support multiple columns or regions
|
||||
- Add metadata output (tag frames with deviation values)
|
||||
- RGB format support (analyze specific channel)
|
||||
|
||||
**Phase 3 (Advanced):**
|
||||
- Full cli.py recording logic
|
||||
- Buffer and output segments
|
||||
- Integration with probe detection systems
|
||||
|
||||
### Implementation Checklist
|
||||
|
||||
- [x] Create gst/rollingsum directory
|
||||
- [x] Implement gstrollingsum.h
|
||||
- [x] Implement gstrollingsum.c
|
||||
- [x] Create CMakeLists.txt
|
||||
- [x] Update gst/CMakeLists.txt
|
||||
- [x] Build and test basic functionality
|
||||
- [x] Test with idsueyesrc
|
||||
- [x] Update README.md
|
||||
- [x] Create feature branch
|
||||
- [x] Commit and document
|
||||
|
||||
## References
|
||||
|
||||
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
|
||||
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
|
||||
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
|
||||
- [scripts/analyze_sma.py](scripts/analyze_sma.py) - Analysis tool
|
||||
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Enable debug output (`$env:GST_DEBUG="rollingsum:5"` in PowerShell)
|
||||
2. Generate CSV log and analyze
|
||||
3. Check this guide's troubleshooting section
|
||||
4. Review debug output for errors/warnings
|
||||
@@ -12,6 +12,5 @@ if (ENABLE_KLV)
|
||||
endif ()
|
||||
|
||||
add_subdirectory (misb)
|
||||
add_subdirectory (rollingsum)
|
||||
add_subdirectory (select)
|
||||
add_subdirectory (videoadjust)
|
||||
|
||||
@@ -720,7 +720,7 @@ gst_intervalometer_update_camera_settings (GstIntervalometer * filter,
|
||||
/* Check deadband zone - if enabled and brightness is within tolerance, skip adjustments */
|
||||
abs_error = fabs(filter->target_brightness - brightness);
|
||||
if (filter->brightness_deadband > 0.0 && abs_error < filter->brightness_deadband) {
|
||||
GST_DEBUG_OBJECT (filter, "Within deadband zone (error=%.2f < %.2f), skipping adjustment",
|
||||
GST_LOG_OBJECT (filter, "Within deadband zone (error=%.2f < %.2f), skipping adjustment",
|
||||
abs_error, filter->brightness_deadband);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
set (SOURCES
|
||||
gstrollingsum.c
|
||||
)
|
||||
|
||||
set (HEADERS
|
||||
gstrollingsum.h)
|
||||
|
||||
include_directories (AFTER
|
||||
${ORC_INCLUDE_DIR})
|
||||
|
||||
set (libname gstrollingsum)
|
||||
|
||||
add_library (${libname} MODULE
|
||||
${SOURCES}
|
||||
${HEADERS})
|
||||
|
||||
target_link_libraries (${libname}
|
||||
${ORC_LIBRARIES}
|
||||
${GLIB2_LIBRARIES}
|
||||
${GOBJECT_LIBRARIES}
|
||||
${GSTREAMER_LIBRARY}
|
||||
${GSTREAMER_BASE_LIBRARY}
|
||||
${GSTREAMER_VIDEO_LIBRARY})
|
||||
|
||||
if (WIN32)
|
||||
install (FILES $<TARGET_PDB_FILE:${libname}> DESTINATION ${PDB_INSTALL_DIR} COMPONENT pdb OPTIONAL)
|
||||
endif ()
|
||||
install(TARGETS ${libname} LIBRARY DESTINATION ${PLUGIN_INSTALL_DIR})
|
||||
@@ -1,535 +0,0 @@
|
||||
/* GStreamer
|
||||
* Copyright (C) 2024 <your-name@your-email.com>
|
||||
*
|
||||
* This library is free software; you can redistribute it and/or
|
||||
* modify it under the terms of the GNU Library General Public
|
||||
* License as published by the Free Software Foundation; either
|
||||
* version 2 of the License, or (at your option) any later version.
|
||||
*
|
||||
* This library is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
||||
* Library General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Library General Public
|
||||
* License along with this library; if not, write to the
|
||||
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
|
||||
* Boston, MA 02111-1307, USA.
|
||||
*/
|
||||
|
||||
/**
|
||||
* SECTION:element-rollingsum
|
||||
*
|
||||
* Drops frames based on rolling mean analysis of a single column.
|
||||
* Inspired by cli.py detection algorithm, simplified for real-time streaming.
|
||||
*
|
||||
* <refsect2>
|
||||
* <title>Example launch line</title>
|
||||
* |[
|
||||
* gst-launch-1.0 idsueyesrc config-file=config.ini ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! autovideosink
|
||||
* ]|
|
||||
* </refsect2>
|
||||
*/
|
||||
|
||||
#ifdef HAVE_CONFIG_H
|
||||
#include "config.h"
|
||||
#endif
|
||||
|
||||
#include "gstrollingsum.h"
|
||||
#include <string.h>
|
||||
#include <math.h>
|
||||
#include <stdio.h>
|
||||
|
||||
enum
|
||||
{
|
||||
PROP_0,
|
||||
PROP_WINDOW_SIZE,
|
||||
PROP_COLUMN_INDEX,
|
||||
PROP_STRIDE,
|
||||
PROP_THRESHOLD,
|
||||
PROP_CSV_FILENAME,
|
||||
PROP_LAST
|
||||
};
|
||||
|
||||
#define DEFAULT_PROP_WINDOW_SIZE 1000
|
||||
#define DEFAULT_PROP_COLUMN_INDEX 1
|
||||
#define DEFAULT_PROP_STRIDE 1
|
||||
#define DEFAULT_PROP_THRESHOLD 0.5
|
||||
#define DEFAULT_PROP_CSV_FILENAME NULL
|
||||
|
||||
/* Supported video formats */
|
||||
#define SUPPORTED_CAPS \
|
||||
GST_VIDEO_CAPS_MAKE("{ GRAY8, GRAY16_LE, GRAY16_BE }") ";" \
|
||||
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
|
||||
|
||||
static GstStaticPadTemplate gst_rolling_sum_sink_template =
|
||||
GST_STATIC_PAD_TEMPLATE ("sink",
|
||||
GST_PAD_SINK,
|
||||
GST_PAD_ALWAYS,
|
||||
GST_STATIC_CAPS (SUPPORTED_CAPS)
|
||||
);
|
||||
|
||||
static GstStaticPadTemplate gst_rolling_sum_src_template =
|
||||
GST_STATIC_PAD_TEMPLATE ("src",
|
||||
GST_PAD_SRC,
|
||||
GST_PAD_ALWAYS,
|
||||
GST_STATIC_CAPS (SUPPORTED_CAPS)
|
||||
);
|
||||
|
||||
/* GObject vmethod declarations */
|
||||
static void gst_rolling_sum_set_property (GObject * object, guint prop_id,
|
||||
const GValue * value, GParamSpec * pspec);
|
||||
static void gst_rolling_sum_get_property (GObject * object, guint prop_id,
|
||||
GValue * value, GParamSpec * pspec);
|
||||
static void gst_rolling_sum_dispose (GObject * object);
|
||||
|
||||
/* GstBaseTransform vmethod declarations */
|
||||
static gboolean gst_rolling_sum_set_caps (GstBaseTransform * trans,
|
||||
GstCaps * incaps, GstCaps * outcaps);
|
||||
static GstFlowReturn gst_rolling_sum_transform_ip (GstBaseTransform * trans,
|
||||
GstBuffer * buf);
|
||||
|
||||
/* GstRollingSum method declarations */
|
||||
static void gst_rolling_sum_reset (GstRollingSum * filter);
|
||||
static gdouble gst_rolling_sum_extract_column_mean (GstRollingSum * filter,
|
||||
GstBuffer * buf);
|
||||
|
||||
/* setup debug */
|
||||
GST_DEBUG_CATEGORY_STATIC (rolling_sum_debug);
|
||||
#define GST_CAT_DEFAULT rolling_sum_debug
|
||||
|
||||
G_DEFINE_TYPE (GstRollingSum, gst_rolling_sum, GST_TYPE_BASE_TRANSFORM);
|
||||
|
||||
/************************************************************************/
|
||||
/* GObject vmethod implementations */
|
||||
/************************************************************************/
|
||||
|
||||
static void
|
||||
gst_rolling_sum_dispose (GObject * object)
|
||||
{
|
||||
GstRollingSum *filter = GST_ROLLING_SUM (object);
|
||||
|
||||
GST_DEBUG ("dispose");
|
||||
|
||||
/* Close CSV file if open */
|
||||
if (filter->csv_file) {
|
||||
fclose (filter->csv_file);
|
||||
filter->csv_file = NULL;
|
||||
}
|
||||
|
||||
/* Free CSV filename */
|
||||
if (filter->csv_filename) {
|
||||
g_free (filter->csv_filename);
|
||||
filter->csv_filename = NULL;
|
||||
}
|
||||
|
||||
gst_rolling_sum_reset (filter);
|
||||
|
||||
/* chain up to the parent class */
|
||||
G_OBJECT_CLASS (gst_rolling_sum_parent_class)->dispose (object);
|
||||
}
|
||||
|
||||
static void
|
||||
gst_rolling_sum_class_init (GstRollingSumClass * klass)
|
||||
{
|
||||
GObjectClass *gobject_class = G_OBJECT_CLASS (klass);
|
||||
GstElementClass *gstelement_class = GST_ELEMENT_CLASS (klass);
|
||||
GstBaseTransformClass *gstbasetransform_class =
|
||||
GST_BASE_TRANSFORM_CLASS (klass);
|
||||
|
||||
GST_DEBUG_CATEGORY_INIT (rolling_sum_debug, "rollingsum", 0,
|
||||
"Rolling Sum Filter");
|
||||
|
||||
GST_DEBUG ("class init");
|
||||
|
||||
/* Register GObject vmethods */
|
||||
gobject_class->dispose = GST_DEBUG_FUNCPTR (gst_rolling_sum_dispose);
|
||||
gobject_class->set_property = GST_DEBUG_FUNCPTR (gst_rolling_sum_set_property);
|
||||
gobject_class->get_property = GST_DEBUG_FUNCPTR (gst_rolling_sum_get_property);
|
||||
|
||||
/* Install GObject properties */
|
||||
g_object_class_install_property (gobject_class, PROP_WINDOW_SIZE,
|
||||
g_param_spec_int ("window-size", "Window Size",
|
||||
"Number of frames in rolling window", 1, 100000,
|
||||
DEFAULT_PROP_WINDOW_SIZE,
|
||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
||||
GST_PARAM_MUTABLE_PLAYING));
|
||||
|
||||
g_object_class_install_property (gobject_class, PROP_COLUMN_INDEX,
|
||||
g_param_spec_int ("column-index", "Column Index",
|
||||
"Which vertical column to analyze (0-based)", 0, G_MAXINT,
|
||||
DEFAULT_PROP_COLUMN_INDEX,
|
||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
||||
GST_PARAM_MUTABLE_PLAYING));
|
||||
|
||||
g_object_class_install_property (gobject_class, PROP_STRIDE,
|
||||
g_param_spec_int ("stride", "Row Stride",
|
||||
"Row sampling stride (1 = every row)", 1, G_MAXINT,
|
||||
DEFAULT_PROP_STRIDE,
|
||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
||||
GST_PARAM_MUTABLE_PLAYING));
|
||||
|
||||
g_object_class_install_property (gobject_class, PROP_THRESHOLD,
|
||||
g_param_spec_double ("threshold", "Threshold",
|
||||
"Normalized deviation threshold for dropping frames", 0.0, 1.0,
|
||||
DEFAULT_PROP_THRESHOLD,
|
||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
||||
GST_PARAM_MUTABLE_PLAYING));
|
||||
|
||||
g_object_class_install_property (gobject_class, PROP_CSV_FILENAME,
|
||||
g_param_spec_string ("csv-file", "CSV File",
|
||||
"Path to CSV file for logging frame data (NULL = no logging)",
|
||||
DEFAULT_PROP_CSV_FILENAME,
|
||||
G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE |
|
||||
GST_PARAM_MUTABLE_READY));
|
||||
|
||||
gst_element_class_add_pad_template (gstelement_class,
|
||||
gst_static_pad_template_get (&gst_rolling_sum_sink_template));
|
||||
gst_element_class_add_pad_template (gstelement_class,
|
||||
gst_static_pad_template_get (&gst_rolling_sum_src_template));
|
||||
|
||||
gst_element_class_set_static_metadata (gstelement_class,
|
||||
"Rolling sum filter", "Filter/Effect/Video",
|
||||
"Drops frames based on rolling mean analysis of a single column",
|
||||
"Your Name <your-email@example.com>");
|
||||
|
||||
/* Register GstBaseTransform vmethods */
|
||||
gstbasetransform_class->set_caps =
|
||||
GST_DEBUG_FUNCPTR (gst_rolling_sum_set_caps);
|
||||
gstbasetransform_class->transform_ip =
|
||||
GST_DEBUG_FUNCPTR (gst_rolling_sum_transform_ip);
|
||||
}
|
||||
|
||||
static void
|
||||
gst_rolling_sum_init (GstRollingSum * filter)
|
||||
{
|
||||
GST_DEBUG_OBJECT (filter, "init class instance");
|
||||
|
||||
filter->window_size = DEFAULT_PROP_WINDOW_SIZE;
|
||||
filter->column_index = DEFAULT_PROP_COLUMN_INDEX;
|
||||
filter->stride = DEFAULT_PROP_STRIDE;
|
||||
filter->threshold = DEFAULT_PROP_THRESHOLD;
|
||||
filter->csv_filename = NULL;
|
||||
|
||||
filter->ring_buffer = NULL;
|
||||
filter->ring_index = 0;
|
||||
filter->frame_count = 0;
|
||||
filter->rolling_mean = 0.0;
|
||||
filter->rolling_sum = 0.0;
|
||||
filter->info_set = FALSE;
|
||||
filter->csv_file = NULL;
|
||||
|
||||
gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE);
|
||||
|
||||
gst_rolling_sum_reset (filter);
|
||||
}
|
||||
|
||||
static void
|
||||
gst_rolling_sum_set_property (GObject * object, guint prop_id,
|
||||
const GValue * value, GParamSpec * pspec)
|
||||
{
|
||||
GstRollingSum *filter = GST_ROLLING_SUM (object);
|
||||
|
||||
GST_DEBUG_OBJECT (filter, "setting property %s", pspec->name);
|
||||
|
||||
switch (prop_id) {
|
||||
case PROP_WINDOW_SIZE:
|
||||
{
|
||||
gint new_size = g_value_get_int (value);
|
||||
if (new_size != filter->window_size) {
|
||||
filter->window_size = new_size;
|
||||
/* Reallocate ring buffer */
|
||||
gst_rolling_sum_reset (filter);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case PROP_COLUMN_INDEX:
|
||||
filter->column_index = g_value_get_int (value);
|
||||
break;
|
||||
case PROP_STRIDE:
|
||||
filter->stride = g_value_get_int (value);
|
||||
break;
|
||||
case PROP_THRESHOLD:
|
||||
filter->threshold = g_value_get_double (value);
|
||||
break;
|
||||
case PROP_CSV_FILENAME:
|
||||
{
|
||||
const gchar *filename = g_value_get_string (value);
|
||||
|
||||
/* Close old file if open */
|
||||
if (filter->csv_file) {
|
||||
fclose (filter->csv_file);
|
||||
filter->csv_file = NULL;
|
||||
}
|
||||
|
||||
/* Free old filename */
|
||||
if (filter->csv_filename) {
|
||||
g_free (filter->csv_filename);
|
||||
filter->csv_filename = NULL;
|
||||
}
|
||||
|
||||
/* Set new filename and open file */
|
||||
if (filename && filename[0] != '\0') {
|
||||
filter->csv_filename = g_strdup (filename);
|
||||
filter->csv_file = fopen (filter->csv_filename, "w");
|
||||
|
||||
if (filter->csv_file) {
|
||||
/* Write CSV header */
|
||||
fprintf (filter->csv_file, "frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped\n");
|
||||
fflush (filter->csv_file);
|
||||
GST_INFO_OBJECT (filter, "Opened CSV file: %s", filter->csv_filename);
|
||||
} else {
|
||||
GST_ERROR_OBJECT (filter, "Failed to open CSV file: %s", filter->csv_filename);
|
||||
g_free (filter->csv_filename);
|
||||
filter->csv_filename = NULL;
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
default:
|
||||
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
static void
|
||||
gst_rolling_sum_get_property (GObject * object, guint prop_id, GValue * value,
|
||||
GParamSpec * pspec)
|
||||
{
|
||||
GstRollingSum *filter = GST_ROLLING_SUM (object);
|
||||
|
||||
GST_DEBUG_OBJECT (filter, "getting property %s", pspec->name);
|
||||
|
||||
switch (prop_id) {
|
||||
case PROP_WINDOW_SIZE:
|
||||
g_value_set_int (value, filter->window_size);
|
||||
break;
|
||||
case PROP_COLUMN_INDEX:
|
||||
g_value_set_int (value, filter->column_index);
|
||||
break;
|
||||
case PROP_STRIDE:
|
||||
g_value_set_int (value, filter->stride);
|
||||
break;
|
||||
case PROP_THRESHOLD:
|
||||
g_value_set_double (value, filter->threshold);
|
||||
break;
|
||||
case PROP_CSV_FILENAME:
|
||||
g_value_set_string (value, filter->csv_filename);
|
||||
break;
|
||||
default:
|
||||
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
static gboolean
|
||||
gst_rolling_sum_set_caps (GstBaseTransform * trans, GstCaps * incaps,
|
||||
GstCaps * outcaps)
|
||||
{
|
||||
GstRollingSum *filter = GST_ROLLING_SUM (trans);
|
||||
|
||||
GST_DEBUG_OBJECT (filter, "set_caps");
|
||||
|
||||
if (!gst_video_info_from_caps (&filter->video_info, incaps)) {
|
||||
GST_ERROR_OBJECT (filter, "Failed to parse caps");
|
||||
return FALSE;
|
||||
}
|
||||
|
||||
filter->info_set = TRUE;
|
||||
|
||||
GST_DEBUG_OBJECT (filter, "Video format: %s, %dx%d",
|
||||
gst_video_format_to_string (GST_VIDEO_INFO_FORMAT (&filter->video_info)),
|
||||
GST_VIDEO_INFO_WIDTH (&filter->video_info),
|
||||
GST_VIDEO_INFO_HEIGHT (&filter->video_info));
|
||||
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
static gdouble
|
||||
gst_rolling_sum_extract_column_mean (GstRollingSum * filter, GstBuffer * buf)
|
||||
{
|
||||
GstMapInfo map;
|
||||
gdouble sum = 0.0;
|
||||
gint count = 0;
|
||||
gint width, height, stride_bytes;
|
||||
gint row, col_offset;
|
||||
guint8 *data;
|
||||
GstVideoFormat format;
|
||||
|
||||
if (!filter->info_set) {
|
||||
GST_WARNING_OBJECT (filter, "Video info not set yet");
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
if (!gst_buffer_map (buf, &map, GST_MAP_READ)) {
|
||||
GST_ERROR_OBJECT (filter, "Failed to map buffer");
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
data = map.data;
|
||||
width = GST_VIDEO_INFO_WIDTH (&filter->video_info);
|
||||
height = GST_VIDEO_INFO_HEIGHT (&filter->video_info);
|
||||
stride_bytes = GST_VIDEO_INFO_PLANE_STRIDE (&filter->video_info, 0);
|
||||
format = GST_VIDEO_INFO_FORMAT (&filter->video_info);
|
||||
|
||||
/* Check column index is valid */
|
||||
if (filter->column_index >= width) {
|
||||
GST_WARNING_OBJECT (filter, "Column index %d >= width %d, using column 0",
|
||||
filter->column_index, width);
|
||||
filter->column_index = 0;
|
||||
}
|
||||
|
||||
/* Calculate column offset based on format */
|
||||
if (format == GST_VIDEO_FORMAT_GRAY8) {
|
||||
col_offset = filter->column_index;
|
||||
|
||||
/* Sum column values with stride */
|
||||
for (row = 0; row < height; row += filter->stride) {
|
||||
sum += data[row * stride_bytes + col_offset];
|
||||
count++;
|
||||
}
|
||||
} else if (format == GST_VIDEO_FORMAT_GRAY16_LE ||
|
||||
format == GST_VIDEO_FORMAT_GRAY16_BE) {
|
||||
col_offset = filter->column_index * 2;
|
||||
|
||||
/* Sum column values with stride */
|
||||
for (row = 0; row < height; row += filter->stride) {
|
||||
guint16 pixel_value;
|
||||
guint8 *pixel_ptr = &data[row * stride_bytes + col_offset];
|
||||
|
||||
if (format == GST_VIDEO_FORMAT_GRAY16_LE) {
|
||||
pixel_value = pixel_ptr[0] | (pixel_ptr[1] << 8);
|
||||
} else {
|
||||
pixel_value = (pixel_ptr[0] << 8) | pixel_ptr[1];
|
||||
}
|
||||
|
||||
sum += pixel_value;
|
||||
count++;
|
||||
}
|
||||
} else {
|
||||
/* For Bayer formats, treat as GRAY8 */
|
||||
col_offset = filter->column_index;
|
||||
|
||||
for (row = 0; row < height; row += filter->stride) {
|
||||
sum += data[row * stride_bytes + col_offset];
|
||||
count++;
|
||||
}
|
||||
}
|
||||
|
||||
gst_buffer_unmap (buf, &map);
|
||||
|
||||
return count > 0 ? sum / count : 0.0;
|
||||
}
|
||||
|
||||
static GstFlowReturn
|
||||
gst_rolling_sum_transform_ip (GstBaseTransform * trans, GstBuffer * buf)
|
||||
{
|
||||
GstRollingSum *filter = GST_ROLLING_SUM (trans);
|
||||
gdouble frame_mean, deviation, old_value;
|
||||
gint effective_window_size;
|
||||
|
||||
GST_DEBUG_OBJECT (filter, "transform_ip called, frame_count=%d", filter->frame_count);
|
||||
|
||||
/* Extract column mean from current frame */
|
||||
frame_mean = gst_rolling_sum_extract_column_mean (filter, buf);
|
||||
|
||||
GST_DEBUG_OBJECT (filter, "Extracted column mean: %.2f", frame_mean);
|
||||
|
||||
/* Store in ring buffer */
|
||||
old_value = filter->ring_buffer[filter->ring_index];
|
||||
filter->ring_buffer[filter->ring_index] = frame_mean;
|
||||
|
||||
/* Update rolling sum efficiently */
|
||||
if (filter->frame_count < filter->window_size) {
|
||||
/* Still filling the buffer */
|
||||
filter->rolling_sum += frame_mean;
|
||||
filter->frame_count++;
|
||||
effective_window_size = filter->frame_count;
|
||||
} else {
|
||||
/* Buffer is full, replace old value */
|
||||
filter->rolling_sum = filter->rolling_sum - old_value + frame_mean;
|
||||
effective_window_size = filter->window_size;
|
||||
}
|
||||
|
||||
/* Update rolling mean */
|
||||
filter->rolling_mean = filter->rolling_sum / effective_window_size;
|
||||
|
||||
/* Calculate deviation */
|
||||
deviation = fabs(frame_mean - filter->rolling_mean);
|
||||
|
||||
/* Normalize deviation (assuming 8-bit equivalent range) */
|
||||
gdouble normalized_deviation = deviation / 255.0;
|
||||
|
||||
GST_DEBUG_OBJECT (filter,
|
||||
"Frame %d: mean=%.2f, rolling_mean=%.2f, deviation=%.2f (normalized=%.4f)",
|
||||
filter->frame_count, frame_mean, filter->rolling_mean, deviation,
|
||||
normalized_deviation);
|
||||
|
||||
/* Advance ring buffer index */
|
||||
filter->ring_index = (filter->ring_index + 1) % filter->window_size;
|
||||
|
||||
/* Decision: drop or pass frame */
|
||||
gboolean dropped = FALSE;
|
||||
if (normalized_deviation > filter->threshold) {
|
||||
GST_DEBUG_OBJECT (filter,
|
||||
"Dropping frame %d: deviation %.4f > threshold %.4f",
|
||||
filter->frame_count, normalized_deviation, filter->threshold);
|
||||
dropped = TRUE;
|
||||
}
|
||||
|
||||
/* Write to CSV if file is open */
|
||||
if (filter->csv_file) {
|
||||
fprintf (filter->csv_file, "%d,%.6f,%.6f,%.6f,%.6f,%d\n",
|
||||
filter->frame_count, frame_mean, filter->rolling_mean,
|
||||
deviation, normalized_deviation, dropped ? 1 : 0);
|
||||
fflush (filter->csv_file);
|
||||
}
|
||||
|
||||
if (dropped) {
|
||||
return GST_BASE_TRANSFORM_FLOW_DROPPED;
|
||||
}
|
||||
|
||||
return GST_FLOW_OK;
|
||||
}
|
||||
|
||||
static void
|
||||
gst_rolling_sum_reset (GstRollingSum * filter)
|
||||
{
|
||||
GST_DEBUG_OBJECT (filter, "reset");
|
||||
|
||||
/* Free old ring buffer if exists */
|
||||
if (filter->ring_buffer) {
|
||||
g_free (filter->ring_buffer);
|
||||
}
|
||||
|
||||
/* Allocate new ring buffer */
|
||||
filter->ring_buffer = g_new0 (gdouble, filter->window_size);
|
||||
filter->ring_index = 0;
|
||||
filter->frame_count = 0;
|
||||
filter->rolling_mean = 0.0;
|
||||
filter->rolling_sum = 0.0;
|
||||
}
|
||||
|
||||
static gboolean
|
||||
plugin_init (GstPlugin * plugin)
|
||||
{
|
||||
GST_DEBUG_CATEGORY_INIT (GST_CAT_DEFAULT, "rollingsum", 0, "rollingsum");
|
||||
|
||||
GST_DEBUG ("plugin_init");
|
||||
|
||||
GST_CAT_INFO (GST_CAT_DEFAULT, "registering rollingsum element");
|
||||
|
||||
if (!gst_element_register (plugin, "rollingsum", GST_RANK_NONE,
|
||||
GST_TYPE_ROLLING_SUM)) {
|
||||
return FALSE;
|
||||
}
|
||||
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
GST_PLUGIN_DEFINE (GST_VERSION_MAJOR,
|
||||
GST_VERSION_MINOR,
|
||||
rollingsum,
|
||||
"Filter that drops frames based on rolling mean analysis",
|
||||
plugin_init, GST_PACKAGE_VERSION, GST_PACKAGE_LICENSE, GST_PACKAGE_NAME,
|
||||
GST_PACKAGE_ORIGIN);
|
||||
@@ -1,82 +0,0 @@
|
||||
/* GStreamer
|
||||
* Copyright (C) 2024 <your-name@your-email.com>
|
||||
*
|
||||
* This library is free software; you can redistribute it and/or
|
||||
* modify it under the terms of the GNU Library General Public
|
||||
* License as published by the Free Software Foundation; either
|
||||
* version 2 of the License, or (at your option) any later version.
|
||||
*
|
||||
* This library is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
||||
* Library General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Library General Public
|
||||
* License along with this library; if not, write to the
|
||||
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
|
||||
* Boston, MA 02111-1307, USA.
|
||||
*/
|
||||
|
||||
#ifndef __GST_ROLLING_SUM_H__
|
||||
#define __GST_ROLLING_SUM_H__
|
||||
|
||||
#include <gst/base/gstbasetransform.h>
|
||||
#include <gst/video/video.h>
|
||||
#include <stdio.h>
|
||||
|
||||
G_BEGIN_DECLS
|
||||
|
||||
#define GST_TYPE_ROLLING_SUM \
|
||||
(gst_rolling_sum_get_type())
|
||||
#define GST_ROLLING_SUM(obj) \
|
||||
(G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_ROLLING_SUM,GstRollingSum))
|
||||
#define GST_ROLLING_SUM_CLASS(klass) \
|
||||
(G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_ROLLING_SUM,GstRollingSumClass))
|
||||
#define GST_IS_ROLLING_SUM(obj) \
|
||||
(G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_ROLLING_SUM))
|
||||
#define GST_IS_ROLLING_SUM_CLASS(klass) \
|
||||
(G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_ROLLING_SUM))
|
||||
|
||||
typedef struct _GstRollingSum GstRollingSum;
|
||||
typedef struct _GstRollingSumClass GstRollingSumClass;
|
||||
|
||||
/**
|
||||
* GstRollingSum:
|
||||
* @element: the parent element.
|
||||
*
|
||||
* The opaque GstRollingSum data structure.
|
||||
*/
|
||||
struct _GstRollingSum
|
||||
{
|
||||
GstBaseTransform element;
|
||||
|
||||
/* Properties */
|
||||
gint window_size; /* Number of frames in rolling window */
|
||||
gint column_index; /* Which column to analyze (0-based) */
|
||||
gint stride; /* Row sampling stride */
|
||||
gdouble threshold; /* Deviation threshold for dropping frames */
|
||||
gchar *csv_filename; /* CSV output filename (NULL = no CSV) */
|
||||
|
||||
/* State */
|
||||
gdouble *ring_buffer; /* Circular buffer of column means */
|
||||
gint ring_index; /* Current position in ring buffer */
|
||||
gint frame_count; /* Total frames processed */
|
||||
gdouble rolling_mean; /* Current rolling mean */
|
||||
gdouble rolling_sum; /* Current rolling sum for efficient mean update */
|
||||
FILE *csv_file; /* CSV file handle */
|
||||
|
||||
/* Video format info */
|
||||
GstVideoInfo video_info;
|
||||
gboolean info_set;
|
||||
};
|
||||
|
||||
struct _GstRollingSumClass
|
||||
{
|
||||
GstBaseTransformClass parent_class;
|
||||
};
|
||||
|
||||
GType gst_rolling_sum_get_type(void);
|
||||
|
||||
G_END_DECLS
|
||||
|
||||
#endif /* __GST_ROLLING_SUM_H__ */
|
||||
321
scripts/append_signals.py
Normal file
321
scripts/append_signals.py
Normal file
@@ -0,0 +1,321 @@
|
||||
#!/usr/bin/env -S uv run
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# dependencies = [
|
||||
# "pillow",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
"""
|
||||
Create scrolling panorama videos from linescan image sequences.
|
||||
|
||||
This script stitches linescan images horizontally into a wide panorama,
|
||||
then creates a scrolling video that pans left-to-right across the stitched image.
|
||||
The scroll speed is calibrated to the linescan capture rate (750 lines/second by default).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def find_frames(directory: Path, pattern: str = "*.jpeg") -> List[Path]:
|
||||
"""Find all frame files matching the pattern, sorted by name."""
|
||||
frames = sorted(directory.glob(pattern))
|
||||
if not frames:
|
||||
frames = sorted(directory.glob("*.jpg"))
|
||||
if not frames:
|
||||
frames = sorted(directory.glob("*.png"))
|
||||
return frames
|
||||
|
||||
|
||||
def stitch_images_horizontally(
|
||||
frames: List[Path],
|
||||
output_file: Path,
|
||||
max_frames: int = None
|
||||
) -> tuple[int, int]:
|
||||
"""
|
||||
Stitch images horizontally into a single wide image.
|
||||
Returns (width, height) of stitched image.
|
||||
"""
|
||||
if max_frames is not None and max_frames > 0:
|
||||
frames = frames[:max_frames]
|
||||
|
||||
print(f"Loading {len(frames)} images for stitching...")
|
||||
|
||||
# Load first image to get dimensions
|
||||
first_img = Image.open(frames[0])
|
||||
img_width, img_height = first_img.size
|
||||
|
||||
# Calculate total width
|
||||
total_width = img_width * len(frames)
|
||||
|
||||
print(f"Creating stitched image: {total_width}x{img_height} pixels")
|
||||
print(f" Individual frame size: {img_width}x{img_height}")
|
||||
print(f" Total frames: {len(frames)}")
|
||||
|
||||
# Create large canvas
|
||||
stitched = Image.new('RGB', (total_width, img_height))
|
||||
|
||||
# Paste images
|
||||
x_offset = 0
|
||||
for i, frame_path in enumerate(frames):
|
||||
if i % 50 == 0:
|
||||
print(f" Stitching frame {i+1}/{len(frames)}...")
|
||||
img = Image.open(frame_path)
|
||||
stitched.paste(img, (x_offset, 0))
|
||||
x_offset += img_width
|
||||
|
||||
print(f"Saving stitched image to {output_file}...")
|
||||
stitched.save(output_file, quality=95)
|
||||
|
||||
return total_width, img_height
|
||||
|
||||
|
||||
def create_scrolling_video(
|
||||
stitched_image: Path,
|
||||
output_file: Path,
|
||||
total_width: int,
|
||||
height: int,
|
||||
output_width: int = 1920,
|
||||
scroll_speed: float = 750.0,
|
||||
fps: int = 30,
|
||||
crf: int = 18,
|
||||
lines_per_second: float = 750.0
|
||||
) -> None:
|
||||
"""
|
||||
Create scrolling video from stitched image.
|
||||
|
||||
Args:
|
||||
stitched_image: Path to stitched horizontal image
|
||||
output_file: Output video file
|
||||
total_width: Total width of stitched image
|
||||
height: Height of image
|
||||
output_width: Width of output video viewport
|
||||
scroll_speed: Scroll speed multiplier (1.0 = real-time @ 750 lines/sec)
|
||||
fps: Output video framerate
|
||||
crf: Quality (lower = better)
|
||||
lines_per_second: Linescan capture rate (pixels per second)
|
||||
"""
|
||||
# Calculate scroll parameters
|
||||
# At scroll_speed=1.0, we scroll at the actual capture rate
|
||||
pixels_per_second = lines_per_second * scroll_speed
|
||||
pixels_per_frame = pixels_per_second / fps
|
||||
|
||||
# Calculate video duration
|
||||
scrollable_width = total_width - output_width
|
||||
duration = scrollable_width / pixels_per_second
|
||||
total_frames = int(duration * fps)
|
||||
|
||||
print(f"\nScrolling video parameters:")
|
||||
print(f" Output viewport: {output_width}x{height}")
|
||||
print(f" Scroll speed: {scroll_speed}x real-time ({pixels_per_second:.1f} pixels/sec)")
|
||||
print(f" Pixels per frame: {pixels_per_frame:.2f}")
|
||||
print(f" Video duration: {duration:.2f} seconds")
|
||||
print(f" Total frames: {total_frames}")
|
||||
print(f" Output FPS: {fps}")
|
||||
|
||||
# FFmpeg crop filter with horizontal scrolling
|
||||
# crop=w:h:x:y where x moves from 0 to (total_width - output_width)
|
||||
# The 't' variable represents time in seconds
|
||||
crop_expr = f"crop={output_width}:{height}:min({pixels_per_second}*t\\,{total_width-output_width}):0"
|
||||
|
||||
cmd = [
|
||||
"ffmpeg",
|
||||
"-y",
|
||||
"-loop", "1",
|
||||
"-i", str(stitched_image),
|
||||
"-vf", crop_expr,
|
||||
"-t", str(duration),
|
||||
"-r", str(fps),
|
||||
"-c:v", "libx264",
|
||||
"-crf", str(crf),
|
||||
"-pix_fmt", "yuv420p",
|
||||
"-movflags", "+faststart",
|
||||
str(output_file)
|
||||
]
|
||||
|
||||
print(f"\nRunning ffmpeg...")
|
||||
print(f"Command: {' '.join(cmd)}\n")
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"\n✓ Scrolling video created successfully: {output_file}")
|
||||
|
||||
# Show file size
|
||||
size_mb = output_file.stat().st_size / (1024 * 1024)
|
||||
print(f" File size: {size_mb:.2f} MB")
|
||||
print(f" Duration: {duration:.2f} seconds")
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Error: ffmpeg command failed with return code {e.returncode}")
|
||||
sys.exit(1)
|
||||
except FileNotFoundError:
|
||||
print("Error: ffmpeg not found. Please install ffmpeg and ensure it's in your PATH.")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Stitch linescan images horizontally and create scrolling video",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
# Real-time playback (1.0x speed, 750 lines/second)
|
||||
python append_signals.py results/20251122/bumpy-filter
|
||||
|
||||
# Slow motion (0.5x speed)
|
||||
python append_signals.py results/20251122/bumpy-filter --scroll-speed 0.5
|
||||
|
||||
# Fast playback (2x speed)
|
||||
python append_signals.py results/20251122/bumpy-filter --scroll-speed 2.0
|
||||
|
||||
# Process only first 100 frames
|
||||
python append_signals.py results/20251122/bumpy-filter --max-frames 100
|
||||
|
||||
# Higher FPS for smoother scrolling
|
||||
python append_signals.py results/20251122/bumpy-filter --fps 60
|
||||
|
||||
# Custom output width (4K)
|
||||
python append_signals.py results/20251122/bumpy-filter --width 3840
|
||||
"""
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"input_dir",
|
||||
type=Path,
|
||||
help="Directory containing the linescan frame sequence"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"-o", "--output",
|
||||
type=Path,
|
||||
help="Output video file (default: {folder_name}_{timestamp}.mp4 in input dir)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--scroll-speed",
|
||||
type=float,
|
||||
default=1.0,
|
||||
help="Scroll speed multiplier (1.0 = real-time at 750 lines/sec, default: 1.0)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--lines-per-second",
|
||||
type=float,
|
||||
default=750.0,
|
||||
help="Linescan capture rate in lines/pixels per second (default: 750)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--fps",
|
||||
type=int,
|
||||
default=30,
|
||||
help="Output video framerate (default: 30)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--width",
|
||||
type=int,
|
||||
default=1920,
|
||||
help="Output video viewport width in pixels (default: 1920)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--crf",
|
||||
type=int,
|
||||
default=18,
|
||||
help="Constant Rate Factor for quality, lower=better (default: 18, range: 0-51)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--pattern",
|
||||
type=str,
|
||||
default="*.jpeg",
|
||||
help="File pattern to match frames (default: *.jpeg)"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--max-frames",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Maximum number of frames to process (default: all frames)"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Validate input directory
|
||||
if not args.input_dir.exists():
|
||||
print(f"Error: Input directory does not exist: {args.input_dir}")
|
||||
sys.exit(1)
|
||||
|
||||
if not args.input_dir.is_dir():
|
||||
print(f"Error: Input path is not a directory: {args.input_dir}")
|
||||
sys.exit(1)
|
||||
|
||||
# Find frames
|
||||
frames = find_frames(args.input_dir, args.pattern)
|
||||
|
||||
if not frames:
|
||||
print(f"Error: No frames found in {args.input_dir} matching pattern {args.pattern}")
|
||||
sys.exit(1)
|
||||
|
||||
total_frames = len(frames)
|
||||
if args.max_frames is not None and args.max_frames > 0:
|
||||
frames = frames[:args.max_frames]
|
||||
print(f"Found {total_frames} frames, processing first {len(frames)} frames")
|
||||
else:
|
||||
print(f"Found {len(frames)} frames in {args.input_dir}")
|
||||
|
||||
# Set output file (one folder up from input directory)
|
||||
if args.output is None:
|
||||
unix_time = int(time.time())
|
||||
folder_name = args.input_dir.name
|
||||
output_filename = f"{folder_name}_scroll_{unix_time}.mp4"
|
||||
output_file = args.input_dir.parent / output_filename
|
||||
else:
|
||||
output_file = args.output
|
||||
|
||||
# Validate CRF range
|
||||
if not 0 <= args.crf <= 51:
|
||||
print("Error: CRF must be between 0 and 51")
|
||||
sys.exit(1)
|
||||
|
||||
# Create temporary stitched image
|
||||
stitched_filename = f"stitched_temp_{int(time.time())}.jpg"
|
||||
stitched_path = args.input_dir / stitched_filename
|
||||
|
||||
try:
|
||||
# Stitch images
|
||||
total_width, img_height = stitch_images_horizontally(
|
||||
frames=frames,
|
||||
output_file=stitched_path,
|
||||
max_frames=args.max_frames
|
||||
)
|
||||
|
||||
# Create scrolling video
|
||||
create_scrolling_video(
|
||||
stitched_image=stitched_path,
|
||||
output_file=output_file,
|
||||
total_width=total_width,
|
||||
height=img_height,
|
||||
output_width=args.width,
|
||||
scroll_speed=args.scroll_speed,
|
||||
fps=args.fps,
|
||||
crf=args.crf,
|
||||
lines_per_second=args.lines_per_second
|
||||
)
|
||||
|
||||
finally:
|
||||
# Clean up temporary stitched image
|
||||
if stitched_path.exists():
|
||||
print(f"\nCleaning up temporary file: {stitched_filename}")
|
||||
stitched_path.unlink()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user