diff --git a/ROLLINGSUM_GUIDE.md b/ROLLINGSUM_GUIDE.md deleted file mode 100644 index 0f29f1b..0000000 --- a/ROLLINGSUM_GUIDE.md +++ /dev/null @@ -1,613 +0,0 @@ -# GStreamer Rolling Sum Plugin - Complete Documentation - -## Table of Contents -- [Overview](#overview) -- [How It Works](#how-it-works) -- [Architecture & Design](#architecture--design) -- [Plugin Properties](#plugin-properties) -- [Basic Usage](#basic-usage) -- [Debugging](#debugging) -- [CSV Analysis](#csv-analysis) -- [Recommended Thresholds](#recommended-thresholds) -- [Troubleshooting](#troubleshooting) -- [Performance](#performance) -- [Integration Examples](#integration-examples) -- [Developer Guide](#developer-guide) -- [References](#references) - -## Overview - -The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames. - -**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames - -**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline. - -## How It Works - -1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column -2. **Rolling Window**: Maintains a circular buffer of recent column means -3. **Deviation Detection**: Calculates how much each frame deviates from the rolling mean -4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold -5. **CSV Logging**: Records all frame statistics for analysis - -### Data Flow - -```mermaid -graph TD - A[Video Frame] --> B[Extract Column] - B --> C[Calculate Column Mean] - C --> D[Store in Ring Buffer] - D --> E[Update Rolling Mean] - E --> F{Deviation > Threshold?} - F -->|Yes| G[DROP Frame] - F -->|No| H[PASS Frame] - C --> E - - style G fill:#ff6b6b - style H fill:#51cf66 -``` - -### Ring Buffer Operation - -```mermaid -graph LR - subgraph Ring Buffer - A[0] --> B[1] - B --> C[2] - C --> D[...] - D --> E[N-1] - E ---|wrap| A - end - - F[New Frame Mean] --> G[ring_index] - G --> A - - H[Rolling Mean] --> I[Sum all values] - I --> J[Divide by count] - - style G fill:#ffd43b -``` - -## Architecture & Design - -### Base Class -- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c)) -- In-place transform (analysis only, no frame modification) -- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames -- Returns `GST_FLOW_OK` to pass frames - -### Element Structure - -```c -struct _GstRollingSum -{ - GstBaseTransform element; - - /* Properties */ - gint window_size; // Number of frames in rolling window (default: 1000) - gint column_index; // Which column to analyze (default: 1, second column) - gint stride; // Row sampling stride (default: 1, every row) - gdouble threshold; // Deviation threshold for dropping (default: 0.5) - gchar *csv_file; // CSV output file path (default: NULL) - - /* State */ - gdouble *ring_buffer; // Circular buffer of column means - gint ring_index; // Current position in ring buffer - gint frame_count; // Total frames processed - gdouble rolling_mean; // Current rolling mean - FILE *csv_fp; // CSV file pointer -}; -``` - -### Algorithm (Simplified from wissotsky's cli.py) - -**Per Frame Processing:** - -1. **Extract column data:** - - Select column at column_index - - Sample every stride rows - - Calculate mean of sampled pixels: frame_mean - -2. **Update ring buffer:** - - Store frame_mean in ring_buffer[ring_index] - - Increment ring_index (wrap around) - -3. **Calculate rolling mean:** - - Sum values in ring buffer (up to window_size or frame_count) - - Divide by actual window size - -4. **Calculate deviation:** - - deviation = abs(frame_mean - rolling_mean) - - normalized_deviation = deviation / 255.0 (for 8-bit video) - -5. **Decision:** - - If normalized_deviation > threshold: DROP frame - - Else: PASS frame - -**Key Simplifications from cli.py:** -- **No EMA tracking**: Use simple rolling mean instead of exponential moving average -- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection -- **No recording logic**: Just drop/pass, no buffering for output segments -- **No patience mechanism**: Immediate decision per frame - -### Video Format Support - -**Initial Implementation:** -- **Primary target**: Grayscale (GRAY8, GRAY16) -- **Secondary**: Bayer formats (common in machine vision) - -**Caps Filter:** -```c -static GstStaticPadTemplate sink_template = - GST_STATIC_PAD_TEMPLATE ("sink", - GST_PAD_SINK, - GST_PAD_ALWAYS, - GST_STATIC_CAPS ( - "video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; " - "video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}" - ) - ); -``` - -## Plugin Properties - -| Property | Type | Default | Range | Description | -|----------|------|---------|-------|-------------| -| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window | -| `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) | -| `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) | -| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames | -| `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) | - -### Understanding Normalized Deviation - -- **Range**: 0.0 to 1.0 -- **Calculation**: `absolute_deviation / 255.0` (for 8-bit video) -- **Meaning**: Fraction of the full pixel range - - `0.001` = deviation of ~0.255 pixel values - - `0.01` = deviation of ~2.55 pixel values - - `0.1` = deviation of ~25.5 pixel values - -## Basic Usage - -### Simple Pipeline - -```powershell -gst-launch-1.0 idsueyesrc config-file=config.ini ! ` - videoconvert ! ` - video/x-raw,format=GRAY8 ! ` - rollingsum window-size=1000 column-index=1 threshold=0.0002 ! ` - autovideosink -``` - -### With CSV Logging - -```powershell -gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! ` - videoconvert ! ` - video/x-raw,format=GRAY8 ! ` - rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! ` - fakesink -``` - -### Custom Configuration - -```powershell -gst-launch-1.0 idsueyesrc config-file=config.ini ! ` - rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! ` - queue ! ` - autovideosink -``` - -### With Format Conversion - -```powershell -gst-launch-1.0 idsueyesrc ! ` - videoconvert ! ` - video/x-raw,format=GRAY8 ! ` - rollingsum ! ` - autovideosink -``` - -## Debugging - -### Enable Debug Output - -Use the `GST_DEBUG` environment variable to see detailed plugin operation: - -#### Windows PowerShell - -```powershell -$env:GST_DEBUG="rollingsum:5"; gst-launch-1.0 [pipeline...] -``` - -#### Windows CMD - -```cmd -set GST_DEBUG=rollingsum:5 && gst-launch-1.0 [pipeline...] -``` - -#### Linux/Mac - -```bash -GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...] -``` - -### Debug Levels - -| Level | Output | -|-------|--------| -| `rollingsum:1` | Errors only | -| `rollingsum:2` | Warnings | -| `rollingsum:3` | Info messages (file open/close) | -| `rollingsum:4` | Debug (caps negotiation) | -| `rollingsum:5` | Log (all frame processing) | - -### Example Debug Output - -``` -0:00:04.029432200 DEBUG rollingsum gstrollingsum.c:436: Extracted column mean: 10.07 -0:00:04.032257100 DEBUG rollingsum gstrollingsum.c:466: Frame 1: mean=10.07, rolling_mean=10.07, deviation=0.00 (normalized=0.0000) -``` - -**Key Fields:** -- `Frame N`: Frame number -- `mean`: Current frame's column mean -- `rolling_mean`: Average of last N frames (window-size) -- `deviation`: Absolute difference -- `normalized`: Deviation as fraction of 255 - -### Common Debug Scenarios - -#### 1. Verify Plugin Loaded - -```powershell -gst-inspect-1.0 rollingsum -``` - -Should show plugin details. If not found, check `GST_PLUGIN_PATH`. - -#### 2. Check CSV File Creation - -Look for this in debug output: -``` -INFO rollingsum: Opened CSV file: output.csv -``` - -#### 3. Monitor Frame Drops - -Look for: -``` -DEBUG rollingsum: Dropping frame 42: deviation 0.0005 > threshold 0.0002 -``` - -#### 4. Verify Caps Negotiation - -``` -DEBUG rollingsum: set_caps -DEBUG rollingsum: Video format: GRAY8, 1224x1026 -``` - -## CSV Analysis - -### CSV Format - -The output CSV contains: - -```csv -frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped -1,10.071150,10.071150,0.000000,0.000000,0 -2,10.059454,10.065302,0.005848,0.000023,0 -... -``` - -### Analyze Results - -Use the included analysis script: - -```powershell -uv run scripts/analyze_sma.py output.csv -``` - -**Output includes:** -- Statistical summary (min/max/mean/std) -- Threshold recommendations based on percentiles -- Standard deviation-based suggestions -- Visualization plots saved to `results/debug/` -- Archived CSV with timestamp in `results/debug/` - -**Output files are automatically organized:** -- `results/debug/output_YYYYMMDD_HHMMSS.csv` - Archived CSV -- `results/debug/output_analysis_YYYYMMDD_HHMMSS.png` - Analysis plots - -The `results/` directory is gitignored to keep your repository clean. - -### Interpreting Results - -The analysis provides threshold recommendations: - -| Percentile | Description | Use Case | -|------------|-------------|----------| -| 99th | Drops top 1% | Very conservative, catch only extreme outliers | -| 95th | Drops top 5% | Conservative, good for quality control | -| 90th | Drops top 10% | Balanced, moderate filtering | -| 75th | Drops top 25% | Aggressive, maximum quality | - -## Recommended Thresholds - -Based on analysis of stable camera footage: - -### For General Use - -```powershell -# Conservative (1-2% frame drop) -threshold=0.0003 - -# Moderate (5-10% frame drop) -threshold=0.0002 - -# Aggressive (20-25% frame drop) -threshold=0.0001 -``` - -### For Specific Scenarios - -**High-speed acquisition** (minimal processing): -```powershell -window-size=100 threshold=0.0005 -``` - -**Quality-focused** (stable scenes): -```powershell -window-size=1000 threshold=0.0001 -``` - -**Real-time monitoring** (fast response): -```powershell -window-size=50 threshold=0.0002 -``` - -## Troubleshooting - -### No frames being dropped (threshold too high) - -**Symptom**: `dropped` column always 0 in CSV - -**Solution**: -1. Run with CSV logging -2. Analyze with `uv run scripts/analyze_sma.py output.csv` -3. Use recommended threshold from 90th-99th percentile - -### Too many frames dropped (threshold too low) - -**Symptom**: Most frames have `dropped=1`, choppy video - -**Solution**: -1. Increase threshold (try doubling current value) -2. Check if column_index is appropriate -3. Verify video is stable (not shaking/moving) - -### CSV file not created - -**Check**: -1. File path is writable -2. Look for "Opened CSV file" in debug output (`GST_DEBUG=rollingsum:3`) -3. Verify csv-file property is set correctly - -### Column index out of range - -**Symptom**: -``` -WARNING rollingsum: Column index 1000 >= width 1224, using column 0 -``` - -**Solution**: Set `column-index` to value < video width - -### Inconsistent results - -**Possible causes**: -1. Window size too small (< 50 frames) -2. Sampling moving/dynamic content -3. Column contains edge/artifact data - -**Solutions**: -- Increase `window-size` to 500-1000 -- Choose different `column-index` (avoid edges) -- Use `stride=2` or higher for faster processing - -## Performance - -### Performance Tips - -1. **Larger window = more stable** but slower to adapt to scene changes -2. **Stride > 1** reduces computation but less accurate column mean -3. **CSV logging** has minimal performance impact -4. **Debug level 5** can produce massive logs, use only when needed - -### Memory Usage - -- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames -- Minimal per-frame allocation - -### CPU Usage - -- Column extraction: O(height/stride) -- Rolling mean update: O(1) using incremental sum -- Very lightweight compared to full-frame processing - -### Optimization Opportunities - -1. **Incremental mean**: Track sum instead of recalculating -2. **SIMD**: Vectorize column summation -3. **Skip calculation**: Only recalc every N frames if baseline is stable - -## Integration Examples - -### Python Script Control - -```python -import subprocess - -# Run pipeline with CSV logging -subprocess.run([ - 'gst-launch-1.0', - 'idsueyesrc', 'config-file=config.ini', - '!', 'videoconvert', - '!', 'video/x-raw,format=GRAY8', - '!', 'rollingsum', - 'window-size=1000', - 'column-index=1', - 'threshold=0.0002', - 'csv-file=output.csv', - '!', 'fakesink' -]) - -# Analyze results -subprocess.run(['uv', 'run', 'scripts/analyze_sma.py', 'output.csv']) -``` - -### Adaptive Threshold - -Use analysis results to set optimal threshold for next run: - -```python -import pandas as pd - -# Analyze previous run -df = pd.read_csv('output.csv') -recommended_threshold = df['normalized_deviation'].quantile(0.95) - -print(f"Recommended threshold: {recommended_threshold:.6f}") -``` - -## Developer Guide - -### Implementation Files - -**Directory Structure:** -``` -gst/rollingsum/ -├── CMakeLists.txt -├── gstrollingsum.c -└── gstrollingsum.h -``` - -**gstrollingsum.h:** -- Element type definitions -- Structure declarations -- Property enums -- Function prototypes - -**gstrollingsum.c:** -- GObject methods (init, dispose, get/set properties) -- GstBaseTransform methods (transform_ip) -- Helper functions (extract_column_mean, update_rolling_mean) -- Plugin registration - -**CMakeLists.txt:** -- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt)) -- Link GStreamer base and video libraries - -### Adding New Features - -Key files: -- [`gst/rollingsum/gstrollingsum.c`](gst/rollingsum/gstrollingsum.c) - Main implementation -- [`gst/rollingsum/gstrollingsum.h`](gst/rollingsum/gstrollingsum.h) - Header/structures -- [`gst/rollingsum/CMakeLists.txt`](gst/rollingsum/CMakeLists.txt) - Build config - -### Rebuild After Changes - -```powershell -.\build.ps1 # Windows -``` - -```bash -./build.sh # Linux -``` - -### Testing - -```powershell -# Quick test -gst-inspect-1.0 rollingsum - -# Full pipeline test with debug -$env:GST_DEBUG="rollingsum:5" -gst-launch-1.0 videotestsrc ! rollingsum ! fakesink -``` - -### Testing Strategy - -**Unit Tests:** -- Ring buffer wrapping -- Mean calculation accuracy -- Threshold comparison logic - -**Integration Tests:** -- Pipeline with videotestsrc -- Pipeline with idsueyesrc -- Frame drop verification -- Property changes during playback - -**Test Cases:** -1. Static video (all frames similar) → all pass -2. Single bright frame → that frame drops -3. Gradual change → frames pass -4. Periodic pattern → pattern frames drop - -### Integration with Existing Project - -**Build System:** -Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt): -```cmake -add_subdirectory (rollingsum) -``` - -**Documentation:** -Update [`README.md`](README.md): -- Add rollingsum to "Other elements" section -- Add pipeline example - -### Future Enhancements - -**Phase 2 (If Needed):** -- Add EMA baseline tracking (like cli.py) -- Add variance-based thresholds -- Support multiple columns or regions -- Add metadata output (tag frames with deviation values) -- RGB format support (analyze specific channel) - -**Phase 3 (Advanced):** -- Full cli.py recording logic -- Buffer and output segments -- Integration with probe detection systems - -### Implementation Checklist - -- [x] Create gst/rollingsum directory -- [x] Implement gstrollingsum.h -- [x] Implement gstrollingsum.c -- [x] Create CMakeLists.txt -- [x] Update gst/CMakeLists.txt -- [x] Build and test basic functionality -- [x] Test with idsueyesrc -- [x] Update README.md -- [x] Create feature branch -- [x] Commit and document - -## References - -- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison) -- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c) -- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html) -- [scripts/analyze_sma.py](scripts/analyze_sma.py) - Analysis tool -- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/ - -## Support - -For issues or questions: -1. Enable debug output (`$env:GST_DEBUG="rollingsum:5"` in PowerShell) -2. Generate CSV log and analyze -3. Check this guide's troubleshooting section -4. Review debug output for errors/warnings \ No newline at end of file diff --git a/gst/CMakeLists.txt b/gst/CMakeLists.txt index b22603c..1fbbb5e 100644 --- a/gst/CMakeLists.txt +++ b/gst/CMakeLists.txt @@ -12,6 +12,5 @@ if (ENABLE_KLV) endif () add_subdirectory (misb) -add_subdirectory (rollingsum) add_subdirectory (select) add_subdirectory (videoadjust) diff --git a/gst/rollingsum/CMakeLists.txt b/gst/rollingsum/CMakeLists.txt deleted file mode 100644 index e71e0b2..0000000 --- a/gst/rollingsum/CMakeLists.txt +++ /dev/null @@ -1,28 +0,0 @@ -set (SOURCES - gstrollingsum.c - ) - -set (HEADERS - gstrollingsum.h) - -include_directories (AFTER - ${ORC_INCLUDE_DIR}) - -set (libname gstrollingsum) - -add_library (${libname} MODULE - ${SOURCES} - ${HEADERS}) - -target_link_libraries (${libname} - ${ORC_LIBRARIES} - ${GLIB2_LIBRARIES} - ${GOBJECT_LIBRARIES} - ${GSTREAMER_LIBRARY} - ${GSTREAMER_BASE_LIBRARY} - ${GSTREAMER_VIDEO_LIBRARY}) - -if (WIN32) - install (FILES $ DESTINATION ${PDB_INSTALL_DIR} COMPONENT pdb OPTIONAL) -endif () -install(TARGETS ${libname} LIBRARY DESTINATION ${PLUGIN_INSTALL_DIR}) \ No newline at end of file diff --git a/gst/rollingsum/gstrollingsum.c b/gst/rollingsum/gstrollingsum.c deleted file mode 100644 index eb532bd..0000000 --- a/gst/rollingsum/gstrollingsum.c +++ /dev/null @@ -1,535 +0,0 @@ -/* GStreamer - * Copyright (C) 2024 - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Library General Public - * License as published by the Free Software Foundation; either - * version 2 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Library General Public License for more details. - * - * You should have received a copy of the GNU Library General Public - * License along with this library; if not, write to the - * Free Software Foundation, Inc., 59 Temple Place - Suite 330, - * Boston, MA 02111-1307, USA. - */ - -/** - * SECTION:element-rollingsum - * - * Drops frames based on rolling mean analysis of a single column. - * Inspired by cli.py detection algorithm, simplified for real-time streaming. - * - * - * Example launch line - * |[ - * gst-launch-1.0 idsueyesrc config-file=config.ini ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! autovideosink - * ]| - * - */ - -#ifdef HAVE_CONFIG_H -#include "config.h" -#endif - -#include "gstrollingsum.h" -#include -#include -#include - -enum -{ - PROP_0, - PROP_WINDOW_SIZE, - PROP_COLUMN_INDEX, - PROP_STRIDE, - PROP_THRESHOLD, - PROP_CSV_FILENAME, - PROP_LAST -}; - -#define DEFAULT_PROP_WINDOW_SIZE 1000 -#define DEFAULT_PROP_COLUMN_INDEX 1 -#define DEFAULT_PROP_STRIDE 1 -#define DEFAULT_PROP_THRESHOLD 0.5 -#define DEFAULT_PROP_CSV_FILENAME NULL - -/* Supported video formats */ -#define SUPPORTED_CAPS \ - GST_VIDEO_CAPS_MAKE("{ GRAY8, GRAY16_LE, GRAY16_BE }") ";" \ - "video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}" - -static GstStaticPadTemplate gst_rolling_sum_sink_template = -GST_STATIC_PAD_TEMPLATE ("sink", - GST_PAD_SINK, - GST_PAD_ALWAYS, - GST_STATIC_CAPS (SUPPORTED_CAPS) - ); - -static GstStaticPadTemplate gst_rolling_sum_src_template = -GST_STATIC_PAD_TEMPLATE ("src", - GST_PAD_SRC, - GST_PAD_ALWAYS, - GST_STATIC_CAPS (SUPPORTED_CAPS) - ); - -/* GObject vmethod declarations */ -static void gst_rolling_sum_set_property (GObject * object, guint prop_id, - const GValue * value, GParamSpec * pspec); -static void gst_rolling_sum_get_property (GObject * object, guint prop_id, - GValue * value, GParamSpec * pspec); -static void gst_rolling_sum_dispose (GObject * object); - -/* GstBaseTransform vmethod declarations */ -static gboolean gst_rolling_sum_set_caps (GstBaseTransform * trans, - GstCaps * incaps, GstCaps * outcaps); -static GstFlowReturn gst_rolling_sum_transform_ip (GstBaseTransform * trans, - GstBuffer * buf); - -/* GstRollingSum method declarations */ -static void gst_rolling_sum_reset (GstRollingSum * filter); -static gdouble gst_rolling_sum_extract_column_mean (GstRollingSum * filter, - GstBuffer * buf); - -/* setup debug */ -GST_DEBUG_CATEGORY_STATIC (rolling_sum_debug); -#define GST_CAT_DEFAULT rolling_sum_debug - -G_DEFINE_TYPE (GstRollingSum, gst_rolling_sum, GST_TYPE_BASE_TRANSFORM); - -/************************************************************************/ -/* GObject vmethod implementations */ -/************************************************************************/ - -static void -gst_rolling_sum_dispose (GObject * object) -{ - GstRollingSum *filter = GST_ROLLING_SUM (object); - - GST_DEBUG ("dispose"); - - /* Close CSV file if open */ - if (filter->csv_file) { - fclose (filter->csv_file); - filter->csv_file = NULL; - } - - /* Free CSV filename */ - if (filter->csv_filename) { - g_free (filter->csv_filename); - filter->csv_filename = NULL; - } - - gst_rolling_sum_reset (filter); - - /* chain up to the parent class */ - G_OBJECT_CLASS (gst_rolling_sum_parent_class)->dispose (object); -} - -static void -gst_rolling_sum_class_init (GstRollingSumClass * klass) -{ - GObjectClass *gobject_class = G_OBJECT_CLASS (klass); - GstElementClass *gstelement_class = GST_ELEMENT_CLASS (klass); - GstBaseTransformClass *gstbasetransform_class = - GST_BASE_TRANSFORM_CLASS (klass); - - GST_DEBUG_CATEGORY_INIT (rolling_sum_debug, "rollingsum", 0, - "Rolling Sum Filter"); - - GST_DEBUG ("class init"); - - /* Register GObject vmethods */ - gobject_class->dispose = GST_DEBUG_FUNCPTR (gst_rolling_sum_dispose); - gobject_class->set_property = GST_DEBUG_FUNCPTR (gst_rolling_sum_set_property); - gobject_class->get_property = GST_DEBUG_FUNCPTR (gst_rolling_sum_get_property); - - /* Install GObject properties */ - g_object_class_install_property (gobject_class, PROP_WINDOW_SIZE, - g_param_spec_int ("window-size", "Window Size", - "Number of frames in rolling window", 1, 100000, - DEFAULT_PROP_WINDOW_SIZE, - G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE | - GST_PARAM_MUTABLE_PLAYING)); - - g_object_class_install_property (gobject_class, PROP_COLUMN_INDEX, - g_param_spec_int ("column-index", "Column Index", - "Which vertical column to analyze (0-based)", 0, G_MAXINT, - DEFAULT_PROP_COLUMN_INDEX, - G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE | - GST_PARAM_MUTABLE_PLAYING)); - - g_object_class_install_property (gobject_class, PROP_STRIDE, - g_param_spec_int ("stride", "Row Stride", - "Row sampling stride (1 = every row)", 1, G_MAXINT, - DEFAULT_PROP_STRIDE, - G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE | - GST_PARAM_MUTABLE_PLAYING)); - - g_object_class_install_property (gobject_class, PROP_THRESHOLD, - g_param_spec_double ("threshold", "Threshold", - "Normalized deviation threshold for dropping frames", 0.0, 1.0, - DEFAULT_PROP_THRESHOLD, - G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE | - GST_PARAM_MUTABLE_PLAYING)); - - g_object_class_install_property (gobject_class, PROP_CSV_FILENAME, - g_param_spec_string ("csv-file", "CSV File", - "Path to CSV file for logging frame data (NULL = no logging)", - DEFAULT_PROP_CSV_FILENAME, - G_PARAM_STATIC_STRINGS | G_PARAM_READWRITE | - GST_PARAM_MUTABLE_READY)); - - gst_element_class_add_pad_template (gstelement_class, - gst_static_pad_template_get (&gst_rolling_sum_sink_template)); - gst_element_class_add_pad_template (gstelement_class, - gst_static_pad_template_get (&gst_rolling_sum_src_template)); - - gst_element_class_set_static_metadata (gstelement_class, - "Rolling sum filter", "Filter/Effect/Video", - "Drops frames based on rolling mean analysis of a single column", - "Your Name "); - - /* Register GstBaseTransform vmethods */ - gstbasetransform_class->set_caps = - GST_DEBUG_FUNCPTR (gst_rolling_sum_set_caps); - gstbasetransform_class->transform_ip = - GST_DEBUG_FUNCPTR (gst_rolling_sum_transform_ip); -} - -static void -gst_rolling_sum_init (GstRollingSum * filter) -{ - GST_DEBUG_OBJECT (filter, "init class instance"); - - filter->window_size = DEFAULT_PROP_WINDOW_SIZE; - filter->column_index = DEFAULT_PROP_COLUMN_INDEX; - filter->stride = DEFAULT_PROP_STRIDE; - filter->threshold = DEFAULT_PROP_THRESHOLD; - filter->csv_filename = NULL; - - filter->ring_buffer = NULL; - filter->ring_index = 0; - filter->frame_count = 0; - filter->rolling_mean = 0.0; - filter->rolling_sum = 0.0; - filter->info_set = FALSE; - filter->csv_file = NULL; - - gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE); - - gst_rolling_sum_reset (filter); -} - -static void -gst_rolling_sum_set_property (GObject * object, guint prop_id, - const GValue * value, GParamSpec * pspec) -{ - GstRollingSum *filter = GST_ROLLING_SUM (object); - - GST_DEBUG_OBJECT (filter, "setting property %s", pspec->name); - - switch (prop_id) { - case PROP_WINDOW_SIZE: - { - gint new_size = g_value_get_int (value); - if (new_size != filter->window_size) { - filter->window_size = new_size; - /* Reallocate ring buffer */ - gst_rolling_sum_reset (filter); - } - break; - } - case PROP_COLUMN_INDEX: - filter->column_index = g_value_get_int (value); - break; - case PROP_STRIDE: - filter->stride = g_value_get_int (value); - break; - case PROP_THRESHOLD: - filter->threshold = g_value_get_double (value); - break; - case PROP_CSV_FILENAME: - { - const gchar *filename = g_value_get_string (value); - - /* Close old file if open */ - if (filter->csv_file) { - fclose (filter->csv_file); - filter->csv_file = NULL; - } - - /* Free old filename */ - if (filter->csv_filename) { - g_free (filter->csv_filename); - filter->csv_filename = NULL; - } - - /* Set new filename and open file */ - if (filename && filename[0] != '\0') { - filter->csv_filename = g_strdup (filename); - filter->csv_file = fopen (filter->csv_filename, "w"); - - if (filter->csv_file) { - /* Write CSV header */ - fprintf (filter->csv_file, "frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped\n"); - fflush (filter->csv_file); - GST_INFO_OBJECT (filter, "Opened CSV file: %s", filter->csv_filename); - } else { - GST_ERROR_OBJECT (filter, "Failed to open CSV file: %s", filter->csv_filename); - g_free (filter->csv_filename); - filter->csv_filename = NULL; - } - } - break; - } - default: - G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec); - break; - } -} - -static void -gst_rolling_sum_get_property (GObject * object, guint prop_id, GValue * value, - GParamSpec * pspec) -{ - GstRollingSum *filter = GST_ROLLING_SUM (object); - - GST_DEBUG_OBJECT (filter, "getting property %s", pspec->name); - - switch (prop_id) { - case PROP_WINDOW_SIZE: - g_value_set_int (value, filter->window_size); - break; - case PROP_COLUMN_INDEX: - g_value_set_int (value, filter->column_index); - break; - case PROP_STRIDE: - g_value_set_int (value, filter->stride); - break; - case PROP_THRESHOLD: - g_value_set_double (value, filter->threshold); - break; - case PROP_CSV_FILENAME: - g_value_set_string (value, filter->csv_filename); - break; - default: - G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec); - break; - } -} - -static gboolean -gst_rolling_sum_set_caps (GstBaseTransform * trans, GstCaps * incaps, - GstCaps * outcaps) -{ - GstRollingSum *filter = GST_ROLLING_SUM (trans); - - GST_DEBUG_OBJECT (filter, "set_caps"); - - if (!gst_video_info_from_caps (&filter->video_info, incaps)) { - GST_ERROR_OBJECT (filter, "Failed to parse caps"); - return FALSE; - } - - filter->info_set = TRUE; - - GST_DEBUG_OBJECT (filter, "Video format: %s, %dx%d", - gst_video_format_to_string (GST_VIDEO_INFO_FORMAT (&filter->video_info)), - GST_VIDEO_INFO_WIDTH (&filter->video_info), - GST_VIDEO_INFO_HEIGHT (&filter->video_info)); - - return TRUE; -} - -static gdouble -gst_rolling_sum_extract_column_mean (GstRollingSum * filter, GstBuffer * buf) -{ - GstMapInfo map; - gdouble sum = 0.0; - gint count = 0; - gint width, height, stride_bytes; - gint row, col_offset; - guint8 *data; - GstVideoFormat format; - - if (!filter->info_set) { - GST_WARNING_OBJECT (filter, "Video info not set yet"); - return 0.0; - } - - if (!gst_buffer_map (buf, &map, GST_MAP_READ)) { - GST_ERROR_OBJECT (filter, "Failed to map buffer"); - return 0.0; - } - - data = map.data; - width = GST_VIDEO_INFO_WIDTH (&filter->video_info); - height = GST_VIDEO_INFO_HEIGHT (&filter->video_info); - stride_bytes = GST_VIDEO_INFO_PLANE_STRIDE (&filter->video_info, 0); - format = GST_VIDEO_INFO_FORMAT (&filter->video_info); - - /* Check column index is valid */ - if (filter->column_index >= width) { - GST_WARNING_OBJECT (filter, "Column index %d >= width %d, using column 0", - filter->column_index, width); - filter->column_index = 0; - } - - /* Calculate column offset based on format */ - if (format == GST_VIDEO_FORMAT_GRAY8) { - col_offset = filter->column_index; - - /* Sum column values with stride */ - for (row = 0; row < height; row += filter->stride) { - sum += data[row * stride_bytes + col_offset]; - count++; - } - } else if (format == GST_VIDEO_FORMAT_GRAY16_LE || - format == GST_VIDEO_FORMAT_GRAY16_BE) { - col_offset = filter->column_index * 2; - - /* Sum column values with stride */ - for (row = 0; row < height; row += filter->stride) { - guint16 pixel_value; - guint8 *pixel_ptr = &data[row * stride_bytes + col_offset]; - - if (format == GST_VIDEO_FORMAT_GRAY16_LE) { - pixel_value = pixel_ptr[0] | (pixel_ptr[1] << 8); - } else { - pixel_value = (pixel_ptr[0] << 8) | pixel_ptr[1]; - } - - sum += pixel_value; - count++; - } - } else { - /* For Bayer formats, treat as GRAY8 */ - col_offset = filter->column_index; - - for (row = 0; row < height; row += filter->stride) { - sum += data[row * stride_bytes + col_offset]; - count++; - } - } - - gst_buffer_unmap (buf, &map); - - return count > 0 ? sum / count : 0.0; -} - -static GstFlowReturn -gst_rolling_sum_transform_ip (GstBaseTransform * trans, GstBuffer * buf) -{ - GstRollingSum *filter = GST_ROLLING_SUM (trans); - gdouble frame_mean, deviation, old_value; - gint effective_window_size; - - GST_DEBUG_OBJECT (filter, "transform_ip called, frame_count=%d", filter->frame_count); - - /* Extract column mean from current frame */ - frame_mean = gst_rolling_sum_extract_column_mean (filter, buf); - - GST_DEBUG_OBJECT (filter, "Extracted column mean: %.2f", frame_mean); - - /* Store in ring buffer */ - old_value = filter->ring_buffer[filter->ring_index]; - filter->ring_buffer[filter->ring_index] = frame_mean; - - /* Update rolling sum efficiently */ - if (filter->frame_count < filter->window_size) { - /* Still filling the buffer */ - filter->rolling_sum += frame_mean; - filter->frame_count++; - effective_window_size = filter->frame_count; - } else { - /* Buffer is full, replace old value */ - filter->rolling_sum = filter->rolling_sum - old_value + frame_mean; - effective_window_size = filter->window_size; - } - - /* Update rolling mean */ - filter->rolling_mean = filter->rolling_sum / effective_window_size; - - /* Calculate deviation */ - deviation = fabs(frame_mean - filter->rolling_mean); - - /* Normalize deviation (assuming 8-bit equivalent range) */ - gdouble normalized_deviation = deviation / 255.0; - - GST_DEBUG_OBJECT (filter, - "Frame %d: mean=%.2f, rolling_mean=%.2f, deviation=%.2f (normalized=%.4f)", - filter->frame_count, frame_mean, filter->rolling_mean, deviation, - normalized_deviation); - - /* Advance ring buffer index */ - filter->ring_index = (filter->ring_index + 1) % filter->window_size; - - /* Decision: drop or pass frame */ - gboolean dropped = FALSE; - if (normalized_deviation > filter->threshold) { - GST_DEBUG_OBJECT (filter, - "Dropping frame %d: deviation %.4f > threshold %.4f", - filter->frame_count, normalized_deviation, filter->threshold); - dropped = TRUE; - } - - /* Write to CSV if file is open */ - if (filter->csv_file) { - fprintf (filter->csv_file, "%d,%.6f,%.6f,%.6f,%.6f,%d\n", - filter->frame_count, frame_mean, filter->rolling_mean, - deviation, normalized_deviation, dropped ? 1 : 0); - fflush (filter->csv_file); - } - - if (dropped) { - return GST_BASE_TRANSFORM_FLOW_DROPPED; - } - - return GST_FLOW_OK; -} - -static void -gst_rolling_sum_reset (GstRollingSum * filter) -{ - GST_DEBUG_OBJECT (filter, "reset"); - - /* Free old ring buffer if exists */ - if (filter->ring_buffer) { - g_free (filter->ring_buffer); - } - - /* Allocate new ring buffer */ - filter->ring_buffer = g_new0 (gdouble, filter->window_size); - filter->ring_index = 0; - filter->frame_count = 0; - filter->rolling_mean = 0.0; - filter->rolling_sum = 0.0; -} - -static gboolean -plugin_init (GstPlugin * plugin) -{ - GST_DEBUG_CATEGORY_INIT (GST_CAT_DEFAULT, "rollingsum", 0, "rollingsum"); - - GST_DEBUG ("plugin_init"); - - GST_CAT_INFO (GST_CAT_DEFAULT, "registering rollingsum element"); - - if (!gst_element_register (plugin, "rollingsum", GST_RANK_NONE, - GST_TYPE_ROLLING_SUM)) { - return FALSE; - } - - return TRUE; -} - -GST_PLUGIN_DEFINE (GST_VERSION_MAJOR, - GST_VERSION_MINOR, - rollingsum, - "Filter that drops frames based on rolling mean analysis", - plugin_init, GST_PACKAGE_VERSION, GST_PACKAGE_LICENSE, GST_PACKAGE_NAME, - GST_PACKAGE_ORIGIN); \ No newline at end of file diff --git a/gst/rollingsum/gstrollingsum.h b/gst/rollingsum/gstrollingsum.h deleted file mode 100644 index 7545c80..0000000 --- a/gst/rollingsum/gstrollingsum.h +++ /dev/null @@ -1,82 +0,0 @@ -/* GStreamer - * Copyright (C) 2024 - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Library General Public - * License as published by the Free Software Foundation; either - * version 2 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Library General Public License for more details. - * - * You should have received a copy of the GNU Library General Public - * License along with this library; if not, write to the - * Free Software Foundation, Inc., 59 Temple Place - Suite 330, - * Boston, MA 02111-1307, USA. - */ - -#ifndef __GST_ROLLING_SUM_H__ -#define __GST_ROLLING_SUM_H__ - -#include -#include -#include - -G_BEGIN_DECLS - -#define GST_TYPE_ROLLING_SUM \ - (gst_rolling_sum_get_type()) -#define GST_ROLLING_SUM(obj) \ - (G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_ROLLING_SUM,GstRollingSum)) -#define GST_ROLLING_SUM_CLASS(klass) \ - (G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_ROLLING_SUM,GstRollingSumClass)) -#define GST_IS_ROLLING_SUM(obj) \ - (G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_ROLLING_SUM)) -#define GST_IS_ROLLING_SUM_CLASS(klass) \ - (G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_ROLLING_SUM)) - -typedef struct _GstRollingSum GstRollingSum; -typedef struct _GstRollingSumClass GstRollingSumClass; - -/** - * GstRollingSum: - * @element: the parent element. - * - * The opaque GstRollingSum data structure. - */ -struct _GstRollingSum -{ - GstBaseTransform element; - - /* Properties */ - gint window_size; /* Number of frames in rolling window */ - gint column_index; /* Which column to analyze (0-based) */ - gint stride; /* Row sampling stride */ - gdouble threshold; /* Deviation threshold for dropping frames */ - gchar *csv_filename; /* CSV output filename (NULL = no CSV) */ - - /* State */ - gdouble *ring_buffer; /* Circular buffer of column means */ - gint ring_index; /* Current position in ring buffer */ - gint frame_count; /* Total frames processed */ - gdouble rolling_mean; /* Current rolling mean */ - gdouble rolling_sum; /* Current rolling sum for efficient mean update */ - FILE *csv_file; /* CSV file handle */ - - /* Video format info */ - GstVideoInfo video_info; - gboolean info_set; -}; - -struct _GstRollingSumClass -{ - GstBaseTransformClass parent_class; -}; - -GType gst_rolling_sum_get_type(void); - -G_END_DECLS - -#endif /* __GST_ROLLING_SUM_H__ */ \ No newline at end of file