Compare commits
10 Commits
c783de425a
...
e58e2319a3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e58e2319a3 | ||
|
|
4d35d16c72 | ||
|
|
2699913e92 | ||
|
|
72e7091c61 | ||
|
|
94f7c04dc6 | ||
|
|
ee4fd76e13 | ||
|
|
1c7ca124b1 | ||
|
|
cb1e5c7607 | ||
|
|
d0467aaf65 | ||
|
|
44083222ee |
3
.gitignore
vendored
3
.gitignore
vendored
@ -38,3 +38,6 @@ ipch/
|
|||||||
|
|
||||||
*.mkv
|
*.mkv
|
||||||
*.raw
|
*.raw
|
||||||
|
*.dot
|
||||||
|
gst_plugs/
|
||||||
|
results/
|
||||||
@ -1,290 +0,0 @@
|
|||||||
# Rolling Sum Filter - Design Document
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
A GStreamer element that drops frames based on rolling mean analysis of a single column of pixels. Inspired by the detection algorithm in `cli.py`, simplified for real-time streaming.
|
|
||||||
|
|
||||||
## Element Name
|
|
||||||
|
|
||||||
**`rollingsum`** - Transform element that analyzes pixel values and selectively drops frames
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Base Class
|
|
||||||
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
|
|
||||||
- In-place transform (analysis only, no frame modification)
|
|
||||||
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
|
|
||||||
- Returns `GST_FLOW_OK` to pass frames
|
|
||||||
|
|
||||||
### Element Structure
|
|
||||||
|
|
||||||
```c
|
|
||||||
struct _GstRollingSum
|
|
||||||
{
|
|
||||||
GstBaseTransform element;
|
|
||||||
|
|
||||||
/* Properties */
|
|
||||||
gint window_size; // Number of frames in rolling window (default: 1000)
|
|
||||||
gint column_index; // Which column to analyze (default: 1, second column)
|
|
||||||
gint stride; // Row sampling stride (default: 1, every row)
|
|
||||||
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
|
|
||||||
|
|
||||||
/* State */
|
|
||||||
gdouble *ring_buffer; // Circular buffer of column means
|
|
||||||
gint ring_index; // Current position in ring buffer
|
|
||||||
gint frame_count; // Total frames processed
|
|
||||||
gdouble rolling_mean; // Current rolling mean
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
## Algorithm (Simplified from cli.py)
|
|
||||||
|
|
||||||
### Per Frame Processing
|
|
||||||
|
|
||||||
```
|
|
||||||
1. Extract column data:
|
|
||||||
- Select column at column_index
|
|
||||||
- Sample every stride rows
|
|
||||||
- Calculate mean of sampled pixels: frame_mean
|
|
||||||
|
|
||||||
2. Update ring buffer:
|
|
||||||
- Store frame_mean in ring_buffer[ring_index]
|
|
||||||
- Increment ring_index (wrap around)
|
|
||||||
|
|
||||||
3. Calculate rolling mean:
|
|
||||||
- Sum values in ring buffer (up to window_size or frame_count)
|
|
||||||
- Divide by actual window size
|
|
||||||
|
|
||||||
4. Calculate deviation:
|
|
||||||
- deviation = abs(frame_mean - rolling_mean)
|
|
||||||
|
|
||||||
5. Decision:
|
|
||||||
- If deviation > threshold: DROP frame
|
|
||||||
- Else: PASS frame
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Simplifications from cli.py
|
|
||||||
|
|
||||||
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
|
|
||||||
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
|
|
||||||
- **No recording logic**: Just drop/pass, no buffering for output segments
|
|
||||||
- **No patience mechanism**: Immediate decision per frame
|
|
||||||
|
|
||||||
## Properties
|
|
||||||
|
|
||||||
| Property | Type | Default | Range | Description |
|
|
||||||
|----------|------|---------|-------|-------------|
|
|
||||||
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
|
|
||||||
| `column-index` | int | 1 | 0-width | Which vertical column to analyze |
|
|
||||||
| `stride` | int | 1 | 1-height | Row sampling stride (1=all rows) |
|
|
||||||
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping |
|
|
||||||
|
|
||||||
## Data Structures
|
|
||||||
|
|
||||||
### Ring Buffer
|
|
||||||
- Array of doubles sized to `window_size`
|
|
||||||
- Stores column mean for each processed frame
|
|
||||||
- Circular wrapping via modulo: `ring_index % window_size`
|
|
||||||
|
|
||||||
### Frame Analysis
|
|
||||||
```
|
|
||||||
For column_index=1, stride=1, frame height=H:
|
|
||||||
- Extract pixels: frame[:, column_index]
|
|
||||||
- Sample: frame[::stride, column_index]
|
|
||||||
- Count: H / stride samples
|
|
||||||
- Mean: sum(samples) / count
|
|
||||||
```
|
|
||||||
|
|
||||||
## Implementation Files
|
|
||||||
|
|
||||||
### Directory Structure
|
|
||||||
```
|
|
||||||
gst/rollingsum/
|
|
||||||
├── CMakeLists.txt
|
|
||||||
├── gstrollingsum.c
|
|
||||||
└── gstrollingsum.h
|
|
||||||
```
|
|
||||||
|
|
||||||
### gstrollingsum.h
|
|
||||||
- Element type definitions
|
|
||||||
- Structure declarations
|
|
||||||
- Property enums
|
|
||||||
- Function prototypes
|
|
||||||
|
|
||||||
### gstrollingsum.c
|
|
||||||
- GObject methods (init, dispose, get/set properties)
|
|
||||||
- GstBaseTransform methods (transform_ip)
|
|
||||||
- Helper functions (extract_column_mean, update_rolling_mean)
|
|
||||||
- Plugin registration
|
|
||||||
|
|
||||||
### CMakeLists.txt
|
|
||||||
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
|
|
||||||
- Link GStreamer base and video libraries
|
|
||||||
|
|
||||||
## Video Format Support
|
|
||||||
|
|
||||||
### Initial Implementation
|
|
||||||
- **Primary target**: Grayscale (GRAY8, GRAY16)
|
|
||||||
- **Secondary**: Bayer formats (common in machine vision)
|
|
||||||
|
|
||||||
### Caps Filter
|
|
||||||
```c
|
|
||||||
static GstStaticPadTemplate sink_template =
|
|
||||||
GST_STATIC_PAD_TEMPLATE ("sink",
|
|
||||||
GST_PAD_SINK,
|
|
||||||
GST_PAD_ALWAYS,
|
|
||||||
GST_STATIC_CAPS (
|
|
||||||
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
|
|
||||||
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
|
|
||||||
)
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
## Pipeline Usage Examples
|
|
||||||
|
|
||||||
### Basic Usage
|
|
||||||
```bash
|
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
|
|
||||||
rollingsum window-size=1000 column-index=1 threshold=0.5 ! \
|
|
||||||
autovideosink
|
|
||||||
```
|
|
||||||
|
|
||||||
### Custom Configuration
|
|
||||||
```bash
|
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
|
|
||||||
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! \
|
|
||||||
queue ! \
|
|
||||||
autovideosink
|
|
||||||
```
|
|
||||||
|
|
||||||
### With Format Conversion
|
|
||||||
```bash
|
|
||||||
gst-launch-1.0 idsueyesrc ! \
|
|
||||||
videoconvert ! \
|
|
||||||
video/x-raw,format=GRAY8 ! \
|
|
||||||
rollingsum ! \
|
|
||||||
autovideosink
|
|
||||||
```
|
|
||||||
|
|
||||||
## Performance Considerations
|
|
||||||
|
|
||||||
### Memory Usage
|
|
||||||
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
|
|
||||||
- Minimal per-frame allocation
|
|
||||||
|
|
||||||
### CPU Usage
|
|
||||||
- Column extraction: O(height/stride)
|
|
||||||
- Rolling mean update: O(1) using incremental sum
|
|
||||||
- Very lightweight compared to full-frame processing
|
|
||||||
|
|
||||||
### Optimization Opportunities
|
|
||||||
1. **Incremental mean**: Track sum instead of recalculating
|
|
||||||
2. **SIMD**: Vectorize column summation
|
|
||||||
3. **Skip calculation**: Only recalc every N frames if baseline is stable
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
### Unit Tests
|
|
||||||
- Ring buffer wrapping
|
|
||||||
- Mean calculation accuracy
|
|
||||||
- Threshold comparison logic
|
|
||||||
|
|
||||||
### Integration Tests
|
|
||||||
- Pipeline with videotestsrc
|
|
||||||
- Pipeline with idsueyesrc
|
|
||||||
- Frame drop verification
|
|
||||||
- Property changes during playback
|
|
||||||
|
|
||||||
### Test Cases
|
|
||||||
1. Static video (all frames similar) → all pass
|
|
||||||
2. Single bright frame → that frame drops
|
|
||||||
3. Gradual change → frames pass
|
|
||||||
4. Periodic pattern → pattern frames drop
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
### Phase 2 (If Needed)
|
|
||||||
- Add EMA baseline tracking (like cli.py)
|
|
||||||
- Add variance-based thresholds
|
|
||||||
- Support multiple columns or regions
|
|
||||||
- Add metadata output (tag frames with deviation values)
|
|
||||||
- RGB format support (analyze specific channel)
|
|
||||||
|
|
||||||
### Phase 3 (Advanced)
|
|
||||||
- Full cli.py recording logic
|
|
||||||
- Buffer and output segments
|
|
||||||
- Integration with probe detection systems
|
|
||||||
|
|
||||||
## Integration with Existing Project
|
|
||||||
|
|
||||||
### Build System
|
|
||||||
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
|
|
||||||
```cmake
|
|
||||||
add_subdirectory (rollingsum)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Documentation
|
|
||||||
Update [`README.md`](README.md):
|
|
||||||
- Add rollingsum to "Other elements" section
|
|
||||||
- Add pipeline example
|
|
||||||
|
|
||||||
## Mermaid Diagram: Data Flow
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph TD
|
|
||||||
A[Video Frame] --> B[Extract Column]
|
|
||||||
B --> C[Calculate Column Mean]
|
|
||||||
C --> D[Store in Ring Buffer]
|
|
||||||
D --> E[Update Rolling Mean]
|
|
||||||
E --> F{Deviation > Threshold?}
|
|
||||||
F -->|Yes| G[DROP Frame]
|
|
||||||
F -->|No| H[PASS Frame]
|
|
||||||
C --> E
|
|
||||||
|
|
||||||
style G fill:#ff6b6b
|
|
||||||
style H fill:#51cf66
|
|
||||||
```
|
|
||||||
|
|
||||||
## Mermaid Diagram: Ring Buffer Operation
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph LR
|
|
||||||
subgraph Ring Buffer
|
|
||||||
A[0] --> B[1]
|
|
||||||
B --> C[2]
|
|
||||||
C --> D[...]
|
|
||||||
D --> E[N-1]
|
|
||||||
E ---|wrap| A
|
|
||||||
end
|
|
||||||
|
|
||||||
F[New Frame Mean] --> G[ring_index]
|
|
||||||
G --> A
|
|
||||||
|
|
||||||
H[Rolling Mean] --> I[Sum all values]
|
|
||||||
I --> J[Divide by count]
|
|
||||||
|
|
||||||
style G fill:#ffd43b
|
|
||||||
```
|
|
||||||
|
|
||||||
## Implementation Checklist
|
|
||||||
|
|
||||||
- [ ] Create gst/rollingsum directory
|
|
||||||
- [ ] Implement gstrollingsum.h
|
|
||||||
- [ ] Implement gstrollingsum.c
|
|
||||||
- [ ] Create CMakeLists.txt
|
|
||||||
- [ ] Update gst/CMakeLists.txt
|
|
||||||
- [ ] Build and test basic functionality
|
|
||||||
- [ ] Test with idsueyesrc
|
|
||||||
- [ ] Update README.md
|
|
||||||
- [ ] Create feature branch
|
|
||||||
- [ ] Commit and document
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
|
|
||||||
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
|
|
||||||
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
|
|
||||||
17
LICENSE
17
LICENSE
@ -1,17 +0,0 @@
|
|||||||
Copyright (C) <2003> David Schleef <ds@schleef.org>
|
|
||||||
Copyright (C) 2016-2019 Tim-Philipp Müller <tim@centricular.com>
|
|
||||||
Copyright (C) 2016-2017 PlayGineering Ltd.
|
|
||||||
|
|
||||||
This library is free software; you can redistribute it and/or
|
|
||||||
modify it under the terms of the GNU Lesser General Public
|
|
||||||
License as published by the Free Software Foundation; either
|
|
||||||
version 2.1 of the License, or (at your option) any later version.
|
|
||||||
|
|
||||||
This library is distributed in the hope that it will be useful,
|
|
||||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
||||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
||||||
Lesser General Public License for more details.
|
|
||||||
|
|
||||||
You should have received a copy of the GNU Lesser General Public
|
|
||||||
License along with this library; if not, write to the Free Software
|
|
||||||
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
|
|
||||||
24
README.md
24
README.md
@ -19,13 +19,13 @@ GStreamer plugins for IDS uEye cameras with frame analysis capabilities.
|
|||||||
## Usage Examples
|
## Usage Examples
|
||||||
|
|
||||||
### Basic capture from IDS uEye camera
|
### Basic capture from IDS uEye camera
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! queue ! autovideosink
|
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! queue ! autovideosink
|
||||||
```
|
```
|
||||||
|
|
||||||
### Frame filtering based on column analysis
|
### Frame filtering based on column analysis
|
||||||
Drop frames when column mean deviates from rolling baseline by more than 0.5:
|
Drop frames when column mean deviates from rolling baseline by more than 0.5:
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! queue ! autovideosink
|
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 threshold=0.5 ! queue ! autovideosink
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -34,17 +34,17 @@ gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.in
|
|||||||
### Additional rollingsum examples
|
### Additional rollingsum examples
|
||||||
|
|
||||||
Analyze column 320 with larger window:
|
Analyze column 320 with larger window:
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=5000 column-index=320 threshold=0.3 ! queue ! autovideosink
|
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=5000 column-index=320 threshold=0.3 ! queue ! autovideosink
|
||||||
```
|
```
|
||||||
|
|
||||||
Use stride for faster processing (sample every 2 rows):
|
Use stride for faster processing (sample every 2 rows):
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 stride=2 threshold=0.5 ! queue ! autovideosink
|
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum window-size=1000 column-index=1 stride=2 threshold=0.5 ! queue ! autovideosink
|
||||||
```
|
```
|
||||||
|
|
||||||
Lower threshold for more sensitive detection:
|
Lower threshold for more sensitive detection:
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum threshold=0.2 ! queue ! autovideosink
|
gst-launch-1.0 idsueyesrc config-file=ini/whole-presacler64_autoexp-binningx2.ini exposure=0.5 ! videoconvert ! video/x-raw,format=GRAY8 ! rollingsum threshold=0.2 ! queue ! autovideosink
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -81,7 +81,7 @@ If you prefer to build manually:
|
|||||||
3. Install [IDS uEye SDK](https://www.ids-imaging.com) (default path: `C:\Program Files\IDS\uEye\Develop`)
|
3. Install [IDS uEye SDK](https://www.ids-imaging.com) (default path: `C:\Program Files\IDS\uEye\Develop`)
|
||||||
4. Run:
|
4. Run:
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
git clone https://github.com/joshdoe/gst-plugins-vision.git
|
git clone https://github.com/joshdoe/gst-plugins-vision.git
|
||||||
cd gst-plugins-vision
|
cd gst-plugins-vision
|
||||||
mkdir build
|
mkdir build
|
||||||
@ -101,7 +101,7 @@ The `build.ps1` script automatically copies plugins to `$env:GST_PLUGIN_PATH` if
|
|||||||
|
|
||||||
### Verify Installation
|
### Verify Installation
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
gst-inspect-1.0 idsueyesrc
|
gst-inspect-1.0 idsueyesrc
|
||||||
gst-inspect-1.0 rollingsum
|
gst-inspect-1.0 rollingsum
|
||||||
```
|
```
|
||||||
@ -109,3 +109,13 @@ gst-inspect-1.0 rollingsum
|
|||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [Rolling Sum Filter Design](DESIGN_ROLLINGSUM.md)
|
- [Rolling Sum Filter Design](DESIGN_ROLLINGSUM.md)
|
||||||
|
|
||||||
|
## Debugging
|
||||||
|
add `$env:GST_DEBUG_DUMP_DOT_DIR='.'`
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
to get dotfile, and view using https://dreampuf.github.io/GraphvizOnline/, or
|
||||||
|
```pwsh
|
||||||
|
dot -Tsvg C:\dev\gst-plugins-vision\0.00.02.922833100-gst-launch.PAUSED_PLAYING.dot -o same.svg
|
||||||
|
```
|
||||||
@ -1,9 +1,28 @@
|
|||||||
# GStreamer Rolling Sum Plugin Guide
|
# GStreamer Rolling Sum Plugin - Complete Documentation
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
- [Overview](#overview)
|
||||||
|
- [How It Works](#how-it-works)
|
||||||
|
- [Architecture & Design](#architecture--design)
|
||||||
|
- [Plugin Properties](#plugin-properties)
|
||||||
|
- [Basic Usage](#basic-usage)
|
||||||
|
- [Debugging](#debugging)
|
||||||
|
- [CSV Analysis](#csv-analysis)
|
||||||
|
- [Recommended Thresholds](#recommended-thresholds)
|
||||||
|
- [Troubleshooting](#troubleshooting)
|
||||||
|
- [Performance](#performance)
|
||||||
|
- [Integration Examples](#integration-examples)
|
||||||
|
- [Developer Guide](#developer-guide)
|
||||||
|
- [References](#references)
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
|
The `rollingsum` plugin analyzes video frames in real-time by tracking the mean pixel intensity of a specific column across frames. It maintains a rolling window of these values and can drop frames that deviate significantly from the rolling mean, useful for detecting and filtering unstable or anomalous frames.
|
||||||
|
|
||||||
|
**Element Name:** `rollingsum` - Transform element that analyzes pixel values and selectively drops frames
|
||||||
|
|
||||||
|
**Purpose:** Monitor a vertical column of pixels in video frames, calculate the rolling mean over a time window, and drop frames when the current frame's column mean deviates significantly from the rolling mean baseline.
|
||||||
|
|
||||||
## How It Works
|
## How It Works
|
||||||
|
|
||||||
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
|
1. **Column Analysis**: Extracts mean pixel intensity from a specified vertical column
|
||||||
@ -12,15 +31,134 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
|
|||||||
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
|
4. **Frame Filtering**: Optionally drops frames exceeding the deviation threshold
|
||||||
5. **CSV Logging**: Records all frame statistics for analysis
|
5. **CSV Logging**: Records all frame statistics for analysis
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[Video Frame] --> B[Extract Column]
|
||||||
|
B --> C[Calculate Column Mean]
|
||||||
|
C --> D[Store in Ring Buffer]
|
||||||
|
D --> E[Update Rolling Mean]
|
||||||
|
E --> F{Deviation > Threshold?}
|
||||||
|
F -->|Yes| G[DROP Frame]
|
||||||
|
F -->|No| H[PASS Frame]
|
||||||
|
C --> E
|
||||||
|
|
||||||
|
style G fill:#ff6b6b
|
||||||
|
style H fill:#51cf66
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ring Buffer Operation
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
subgraph Ring Buffer
|
||||||
|
A[0] --> B[1]
|
||||||
|
B --> C[2]
|
||||||
|
C --> D[...]
|
||||||
|
D --> E[N-1]
|
||||||
|
E ---|wrap| A
|
||||||
|
end
|
||||||
|
|
||||||
|
F[New Frame Mean] --> G[ring_index]
|
||||||
|
G --> A
|
||||||
|
|
||||||
|
H[Rolling Mean] --> I[Sum all values]
|
||||||
|
I --> J[Divide by count]
|
||||||
|
|
||||||
|
style G fill:#ffd43b
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture & Design
|
||||||
|
|
||||||
|
### Base Class
|
||||||
|
- Inherits from `GstBaseTransform` (similar to [`select`](gst/select/gstselect.c))
|
||||||
|
- In-place transform (analysis only, no frame modification)
|
||||||
|
- Returns `GST_BASE_TRANSFORM_FLOW_DROPPED` to drop frames
|
||||||
|
- Returns `GST_FLOW_OK` to pass frames
|
||||||
|
|
||||||
|
### Element Structure
|
||||||
|
|
||||||
|
```c
|
||||||
|
struct _GstRollingSum
|
||||||
|
{
|
||||||
|
GstBaseTransform element;
|
||||||
|
|
||||||
|
/* Properties */
|
||||||
|
gint window_size; // Number of frames in rolling window (default: 1000)
|
||||||
|
gint column_index; // Which column to analyze (default: 1, second column)
|
||||||
|
gint stride; // Row sampling stride (default: 1, every row)
|
||||||
|
gdouble threshold; // Deviation threshold for dropping (default: 0.5)
|
||||||
|
gchar *csv_file; // CSV output file path (default: NULL)
|
||||||
|
|
||||||
|
/* State */
|
||||||
|
gdouble *ring_buffer; // Circular buffer of column means
|
||||||
|
gint ring_index; // Current position in ring buffer
|
||||||
|
gint frame_count; // Total frames processed
|
||||||
|
gdouble rolling_mean; // Current rolling mean
|
||||||
|
FILE *csv_fp; // CSV file pointer
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### Algorithm (Simplified from cli.py)
|
||||||
|
|
||||||
|
**Per Frame Processing:**
|
||||||
|
|
||||||
|
1. **Extract column data:**
|
||||||
|
- Select column at column_index
|
||||||
|
- Sample every stride rows
|
||||||
|
- Calculate mean of sampled pixels: frame_mean
|
||||||
|
|
||||||
|
2. **Update ring buffer:**
|
||||||
|
- Store frame_mean in ring_buffer[ring_index]
|
||||||
|
- Increment ring_index (wrap around)
|
||||||
|
|
||||||
|
3. **Calculate rolling mean:**
|
||||||
|
- Sum values in ring buffer (up to window_size or frame_count)
|
||||||
|
- Divide by actual window size
|
||||||
|
|
||||||
|
4. **Calculate deviation:**
|
||||||
|
- deviation = abs(frame_mean - rolling_mean)
|
||||||
|
- normalized_deviation = deviation / 255.0 (for 8-bit video)
|
||||||
|
|
||||||
|
5. **Decision:**
|
||||||
|
- If normalized_deviation > threshold: DROP frame
|
||||||
|
- Else: PASS frame
|
||||||
|
|
||||||
|
**Key Simplifications from cli.py:**
|
||||||
|
- **No EMA tracking**: Use simple rolling mean instead of exponential moving average
|
||||||
|
- **No variance tracking**: Use fixed threshold instead of dynamic variance-based detection
|
||||||
|
- **No recording logic**: Just drop/pass, no buffering for output segments
|
||||||
|
- **No patience mechanism**: Immediate decision per frame
|
||||||
|
|
||||||
|
### Video Format Support
|
||||||
|
|
||||||
|
**Initial Implementation:**
|
||||||
|
- **Primary target**: Grayscale (GRAY8, GRAY16)
|
||||||
|
- **Secondary**: Bayer formats (common in machine vision)
|
||||||
|
|
||||||
|
**Caps Filter:**
|
||||||
|
```c
|
||||||
|
static GstStaticPadTemplate sink_template =
|
||||||
|
GST_STATIC_PAD_TEMPLATE ("sink",
|
||||||
|
GST_PAD_SINK,
|
||||||
|
GST_PAD_ALWAYS,
|
||||||
|
GST_STATIC_CAPS (
|
||||||
|
"video/x-raw, format=(string){GRAY8,GRAY16_LE,GRAY16_BE}; "
|
||||||
|
"video/x-bayer, format=(string){bggr,grbg,gbrg,rggb}"
|
||||||
|
)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
## Plugin Properties
|
## Plugin Properties
|
||||||
|
|
||||||
| Property | Type | Default | Description |
|
| Property | Type | Default | Range | Description |
|
||||||
|----------|------|---------|-------------|
|
|----------|------|---------|-------|-------------|
|
||||||
| `window-size` | int | 1000 | Number of frames in rolling window (1-100000) |
|
| `window-size` | int | 1000 | 1-100000 | Number of frames in rolling window |
|
||||||
| `column-index` | int | 1 | Which vertical column to analyze (0-based) |
|
| `column-index` | int | 1 | 0-width | Which vertical column to analyze (0-based) |
|
||||||
| `stride` | int | 1 | Row sampling stride (1 = every row) |
|
| `stride` | int | 1 | 1-height | Row sampling stride (1 = every row) |
|
||||||
| `threshold` | double | 0.5 | Normalized deviation threshold for dropping frames (0.0-1.0) |
|
| `threshold` | double | 0.5 | 0.0-1.0 | Normalized deviation threshold for dropping frames |
|
||||||
| `csv-file` | string | NULL | Path to CSV file for logging (NULL = no logging) |
|
| `csv-file` | string | NULL | - | Path to CSV file for logging (NULL = no logging) |
|
||||||
|
|
||||||
### Understanding Normalized Deviation
|
### Understanding Normalized Deviation
|
||||||
|
|
||||||
@ -35,24 +173,43 @@ The `rollingsum` plugin analyzes video frames in real-time by tracking the mean
|
|||||||
|
|
||||||
### Simple Pipeline
|
### Simple Pipeline
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini ! \
|
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
|
||||||
videoconvert ! \
|
videoconvert ! `
|
||||||
video/x-raw,format=GRAY8 ! \
|
video/x-raw,format=GRAY8 ! `
|
||||||
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! \
|
rollingsum window-size=1000 column-index=1 threshold=0.0002 ! `
|
||||||
autovideosink
|
autovideosink
|
||||||
```
|
```
|
||||||
|
|
||||||
### With CSV Logging
|
### With CSV Logging
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! \
|
gst-launch-1.0 idsueyesrc config-file=config.ini exposure=0.5 ! `
|
||||||
videoconvert ! \
|
videoconvert ! `
|
||||||
video/x-raw,format=GRAY8 ! \
|
video/x-raw,format=GRAY8 ! `
|
||||||
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! \
|
rollingsum window-size=1000 column-index=1 threshold=0.0002 csv-file=output.csv ! `
|
||||||
fakesink
|
fakesink
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Custom Configuration
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
gst-launch-1.0 idsueyesrc config-file=config.ini ! `
|
||||||
|
rollingsum window-size=5000 column-index=320 stride=2 threshold=0.3 ! `
|
||||||
|
queue ! `
|
||||||
|
autovideosink
|
||||||
|
```
|
||||||
|
|
||||||
|
### With Format Conversion
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
gst-launch-1.0 idsueyesrc ! `
|
||||||
|
videoconvert ! `
|
||||||
|
video/x-raw,format=GRAY8 ! `
|
||||||
|
rollingsum ! `
|
||||||
|
autovideosink
|
||||||
|
```
|
||||||
|
|
||||||
## Debugging
|
## Debugging
|
||||||
|
|
||||||
### Enable Debug Output
|
### Enable Debug Output
|
||||||
@ -105,7 +262,7 @@ GST_DEBUG=rollingsum:5 gst-launch-1.0 [pipeline...]
|
|||||||
|
|
||||||
#### 1. Verify Plugin Loaded
|
#### 1. Verify Plugin Loaded
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
gst-inspect-1.0 rollingsum
|
gst-inspect-1.0 rollingsum
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -149,15 +306,22 @@ frame,column_mean,rolling_mean,deviation,normalized_deviation,dropped
|
|||||||
|
|
||||||
Use the included analysis script:
|
Use the included analysis script:
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
uv run analyze_sma.py output.csv
|
uv run scripts/analyze_sma.py output.csv
|
||||||
```
|
```
|
||||||
|
|
||||||
**Output includes:**
|
**Output includes:**
|
||||||
- Statistical summary (min/max/mean/std)
|
- Statistical summary (min/max/mean/std)
|
||||||
- Threshold recommendations based on percentiles
|
- Threshold recommendations based on percentiles
|
||||||
- Standard deviation-based suggestions
|
- Standard deviation-based suggestions
|
||||||
- Visualization plots (`output_analysis.png`)
|
- Visualization plots saved to `results/debug/`
|
||||||
|
- Archived CSV with timestamp in `results/debug/`
|
||||||
|
|
||||||
|
**Output files are automatically organized:**
|
||||||
|
- `results/debug/output_YYYYMMDD_HHMMSS.csv` - Archived CSV
|
||||||
|
- `results/debug/output_analysis_YYYYMMDD_HHMMSS.png` - Analysis plots
|
||||||
|
|
||||||
|
The `results/` directory is gitignored to keep your repository clean.
|
||||||
|
|
||||||
### Interpreting Results
|
### Interpreting Results
|
||||||
|
|
||||||
@ -176,7 +340,7 @@ Based on analysis of stable camera footage:
|
|||||||
|
|
||||||
### For General Use
|
### For General Use
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
# Conservative (1-2% frame drop)
|
# Conservative (1-2% frame drop)
|
||||||
threshold=0.0003
|
threshold=0.0003
|
||||||
|
|
||||||
@ -190,17 +354,17 @@ threshold=0.0001
|
|||||||
### For Specific Scenarios
|
### For Specific Scenarios
|
||||||
|
|
||||||
**High-speed acquisition** (minimal processing):
|
**High-speed acquisition** (minimal processing):
|
||||||
```bash
|
```powershell
|
||||||
window-size=100 threshold=0.0005
|
window-size=100 threshold=0.0005
|
||||||
```
|
```
|
||||||
|
|
||||||
**Quality-focused** (stable scenes):
|
**Quality-focused** (stable scenes):
|
||||||
```bash
|
```powershell
|
||||||
window-size=1000 threshold=0.0001
|
window-size=1000 threshold=0.0001
|
||||||
```
|
```
|
||||||
|
|
||||||
**Real-time monitoring** (fast response):
|
**Real-time monitoring** (fast response):
|
||||||
```bash
|
```powershell
|
||||||
window-size=50 threshold=0.0002
|
window-size=50 threshold=0.0002
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -212,7 +376,7 @@ window-size=50 threshold=0.0002
|
|||||||
|
|
||||||
**Solution**:
|
**Solution**:
|
||||||
1. Run with CSV logging
|
1. Run with CSV logging
|
||||||
2. Analyze with `uv run analyze_sma.py output.csv`
|
2. Analyze with `uv run scripts/analyze_sma.py output.csv`
|
||||||
3. Use recommended threshold from 90th-99th percentile
|
3. Use recommended threshold from 90th-99th percentile
|
||||||
|
|
||||||
### Too many frames dropped (threshold too low)
|
### Too many frames dropped (threshold too low)
|
||||||
@ -252,13 +416,32 @@ WARNING rollingsum: Column index 1000 >= width 1224, using column 0
|
|||||||
- Choose different `column-index` (avoid edges)
|
- Choose different `column-index` (avoid edges)
|
||||||
- Use `stride=2` or higher for faster processing
|
- Use `stride=2` or higher for faster processing
|
||||||
|
|
||||||
## Performance Tips
|
## Performance
|
||||||
|
|
||||||
|
### Performance Tips
|
||||||
|
|
||||||
1. **Larger window = more stable** but slower to adapt to scene changes
|
1. **Larger window = more stable** but slower to adapt to scene changes
|
||||||
2. **Stride > 1** reduces computation but less accurate column mean
|
2. **Stride > 1** reduces computation but less accurate column mean
|
||||||
3. **CSV logging** has minimal performance impact
|
3. **CSV logging** has minimal performance impact
|
||||||
4. **Debug level 5** can produce massive logs, use only when needed
|
4. **Debug level 5** can produce massive logs, use only when needed
|
||||||
|
|
||||||
|
### Memory Usage
|
||||||
|
|
||||||
|
- Ring buffer: `window_size * sizeof(double)` = ~8KB for default 1000 frames
|
||||||
|
- Minimal per-frame allocation
|
||||||
|
|
||||||
|
### CPU Usage
|
||||||
|
|
||||||
|
- Column extraction: O(height/stride)
|
||||||
|
- Rolling mean update: O(1) using incremental sum
|
||||||
|
- Very lightweight compared to full-frame processing
|
||||||
|
|
||||||
|
### Optimization Opportunities
|
||||||
|
|
||||||
|
1. **Incremental mean**: Track sum instead of recalculating
|
||||||
|
2. **SIMD**: Vectorize column summation
|
||||||
|
3. **Skip calculation**: Only recalc every N frames if baseline is stable
|
||||||
|
|
||||||
## Integration Examples
|
## Integration Examples
|
||||||
|
|
||||||
### Python Script Control
|
### Python Script Control
|
||||||
@ -281,7 +464,7 @@ subprocess.run([
|
|||||||
])
|
])
|
||||||
|
|
||||||
# Analyze results
|
# Analyze results
|
||||||
subprocess.run(['uv', 'run', 'analyze_sma.py', 'output.csv'])
|
subprocess.run(['uv', 'run', 'scripts/analyze_sma.py', 'output.csv'])
|
||||||
```
|
```
|
||||||
|
|
||||||
### Adaptive Threshold
|
### Adaptive Threshold
|
||||||
@ -298,7 +481,33 @@ recommended_threshold = df['normalized_deviation'].quantile(0.95)
|
|||||||
print(f"Recommended threshold: {recommended_threshold:.6f}")
|
print(f"Recommended threshold: {recommended_threshold:.6f}")
|
||||||
```
|
```
|
||||||
|
|
||||||
## Developer Notes
|
## Developer Guide
|
||||||
|
|
||||||
|
### Implementation Files
|
||||||
|
|
||||||
|
**Directory Structure:**
|
||||||
|
```
|
||||||
|
gst/rollingsum/
|
||||||
|
├── CMakeLists.txt
|
||||||
|
├── gstrollingsum.c
|
||||||
|
└── gstrollingsum.h
|
||||||
|
```
|
||||||
|
|
||||||
|
**gstrollingsum.h:**
|
||||||
|
- Element type definitions
|
||||||
|
- Structure declarations
|
||||||
|
- Property enums
|
||||||
|
- Function prototypes
|
||||||
|
|
||||||
|
**gstrollingsum.c:**
|
||||||
|
- GObject methods (init, dispose, get/set properties)
|
||||||
|
- GstBaseTransform methods (transform_ip)
|
||||||
|
- Helper functions (extract_column_mean, update_rolling_mean)
|
||||||
|
- Plugin registration
|
||||||
|
|
||||||
|
**CMakeLists.txt:**
|
||||||
|
- Build configuration (copy from [`gst/select/CMakeLists.txt`](gst/select/CMakeLists.txt))
|
||||||
|
- Link GStreamer base and video libraries
|
||||||
|
|
||||||
### Adding New Features
|
### Adding New Features
|
||||||
|
|
||||||
@ -309,14 +518,17 @@ Key files:
|
|||||||
|
|
||||||
### Rebuild After Changes
|
### Rebuild After Changes
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
.\build.ps1 # Windows
|
.\build.ps1 # Windows
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
./build.sh # Linux
|
./build.sh # Linux
|
||||||
```
|
```
|
||||||
|
|
||||||
### Testing
|
### Testing
|
||||||
|
|
||||||
```bash
|
```powershell
|
||||||
# Quick test
|
# Quick test
|
||||||
gst-inspect-1.0 rollingsum
|
gst-inspect-1.0 rollingsum
|
||||||
|
|
||||||
@ -325,16 +537,77 @@ $env:GST_DEBUG="rollingsum:5"
|
|||||||
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
|
gst-launch-1.0 videotestsrc ! rollingsum ! fakesink
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Testing Strategy
|
||||||
|
|
||||||
|
**Unit Tests:**
|
||||||
|
- Ring buffer wrapping
|
||||||
|
- Mean calculation accuracy
|
||||||
|
- Threshold comparison logic
|
||||||
|
|
||||||
|
**Integration Tests:**
|
||||||
|
- Pipeline with videotestsrc
|
||||||
|
- Pipeline with idsueyesrc
|
||||||
|
- Frame drop verification
|
||||||
|
- Property changes during playback
|
||||||
|
|
||||||
|
**Test Cases:**
|
||||||
|
1. Static video (all frames similar) → all pass
|
||||||
|
2. Single bright frame → that frame drops
|
||||||
|
3. Gradual change → frames pass
|
||||||
|
4. Periodic pattern → pattern frames drop
|
||||||
|
|
||||||
|
### Integration with Existing Project
|
||||||
|
|
||||||
|
**Build System:**
|
||||||
|
Update [`gst/CMakeLists.txt`](gst/CMakeLists.txt):
|
||||||
|
```cmake
|
||||||
|
add_subdirectory (rollingsum)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Documentation:**
|
||||||
|
Update [`README.md`](README.md):
|
||||||
|
- Add rollingsum to "Other elements" section
|
||||||
|
- Add pipeline example
|
||||||
|
|
||||||
|
### Future Enhancements
|
||||||
|
|
||||||
|
**Phase 2 (If Needed):**
|
||||||
|
- Add EMA baseline tracking (like cli.py)
|
||||||
|
- Add variance-based thresholds
|
||||||
|
- Support multiple columns or regions
|
||||||
|
- Add metadata output (tag frames with deviation values)
|
||||||
|
- RGB format support (analyze specific channel)
|
||||||
|
|
||||||
|
**Phase 3 (Advanced):**
|
||||||
|
- Full cli.py recording logic
|
||||||
|
- Buffer and output segments
|
||||||
|
- Integration with probe detection systems
|
||||||
|
|
||||||
|
### Implementation Checklist
|
||||||
|
|
||||||
|
- [x] Create gst/rollingsum directory
|
||||||
|
- [x] Implement gstrollingsum.h
|
||||||
|
- [x] Implement gstrollingsum.c
|
||||||
|
- [x] Create CMakeLists.txt
|
||||||
|
- [x] Update gst/CMakeLists.txt
|
||||||
|
- [x] Build and test basic functionality
|
||||||
|
- [x] Test with idsueyesrc
|
||||||
|
- [x] Update README.md
|
||||||
|
- [x] Create feature branch
|
||||||
|
- [x] Commit and document
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|
||||||
- [DESIGN_ROLLINGSUM.md](DESIGN_ROLLINGSUM.md) - Design document
|
- Original algorithm: `cli.py` lines 64-79 (column extraction and mean comparison)
|
||||||
- [analyze_sma.py](analyze_sma.py) - Analysis tool
|
- Template element: [`gst/select/gstselect.c`](gst/select/gstselect.c)
|
||||||
|
- GStreamer base transform: [GstBaseTransform documentation](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html)
|
||||||
|
- [scripts/analyze_sma.py](scripts/analyze_sma.py) - Analysis tool
|
||||||
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
|
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
|
||||||
|
|
||||||
## Support
|
## Support
|
||||||
|
|
||||||
For issues or questions:
|
For issues or questions:
|
||||||
1. Enable debug output (`GST_DEBUG=rollingsum:5`)
|
1. Enable debug output (`$env:GST_DEBUG="rollingsum:5"` in PowerShell)
|
||||||
2. Generate CSV log and analyze
|
2. Generate CSV log and analyze
|
||||||
3. Check this guide's troubleshooting section
|
3. Check this guide's troubleshooting section
|
||||||
4. Review debug output for errors/warnings
|
4. Review debug output for errors/warnings
|
||||||
125
dot_pause-play.svg
Normal file
125
dot_pause-play.svg
Normal file
@ -0,0 +1,125 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||||||
|
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
|
||||||
|
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||||
|
<!-- Generated by graphviz version 14.0.2 (20251019.1705)
|
||||||
|
-->
|
||||||
|
<!-- Title: pipeline Pages: 1 -->
|
||||||
|
<svg width="1103pt" height="290pt"
|
||||||
|
viewBox="0.00 0.00 1103.00 290.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
|
||||||
|
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 285.85)">
|
||||||
|
<title>pipeline</title>
|
||||||
|
<polygon fill="white" stroke="none" points="-4,4 -4,-285.85 1099.2,-285.85 1099.2,4 -4,4"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-268.35" font-family="sans" font-size="10.00"><GstPipeline></text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-255.6" font-family="sans" font-size="10.00">pipeline0</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="547.6" y="-242.85" font-family="sans" font-size="10.00">[>]</text>
|
||||||
|
<g id="clust1" class="cluster">
|
||||||
|
<title>cluster_node_udpsink0_000001FC4DD89DE0</title>
|
||||||
|
<path fill="#aaaaff" stroke="black" d="M886.45,-75.6C886.45,-75.6 1083.2,-75.6 1083.2,-75.6 1089.2,-75.6 1095.2,-81.6 1095.2,-87.6 1095.2,-87.6 1095.2,-215.6 1095.2,-215.6 1095.2,-221.6 1089.2,-227.6 1083.2,-227.6 1083.2,-227.6 886.45,-227.6 886.45,-227.6 880.45,-227.6 874.45,-221.6 874.45,-215.6 874.45,-215.6 874.45,-87.6 874.45,-87.6 874.45,-81.6 880.45,-75.6 886.45,-75.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-216" font-family="Bitstream Vera Sans" font-size="8.00">GstUDPSink</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-206.25" font-family="Bitstream Vera Sans" font-size="8.00">udpsink0</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-196.5" font-family="Bitstream Vera Sans" font-size="8.00">[>]</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-186.75" font-family="Bitstream Vera Sans" font-size="8.00">last-sample=((GstSample*) 000001FC4DE728F0)</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-177" font-family="Bitstream Vera Sans" font-size="8.00">used-socket=((GSocket*) 000001FC4DD85E90)</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-167.25" font-family="Bitstream Vera Sans" font-size="8.00">used-socket-v6=((GSocket*) 000001FC4DD87A30)</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-157.5" font-family="Bitstream Vera Sans" font-size="8.00">clients="127.0.0.1:5000"</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-147.75" font-family="Bitstream Vera Sans" font-size="8.00">host="127.0.0.1"</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.83" y="-138" font-family="Bitstream Vera Sans" font-size="8.00">port=5000</text>
|
||||||
|
</g>
|
||||||
|
<g id="clust2" class="cluster">
|
||||||
|
<title>cluster_node_udpsink0_000001FC4DD89DE0_sink</title>
|
||||||
|
</g>
|
||||||
|
<g id="clust3" class="cluster">
|
||||||
|
<title>cluster_node_queue0_000001FC4BB96850</title>
|
||||||
|
<path fill="#aaffaa" stroke="black" d="M524.2,-75.6C524.2,-75.6 673.2,-75.6 673.2,-75.6 679.2,-75.6 685.2,-81.6 685.2,-87.6 685.2,-87.6 685.2,-156.6 685.2,-156.6 685.2,-162.6 679.2,-168.6 673.2,-168.6 673.2,-168.6 524.2,-168.6 524.2,-168.6 518.2,-168.6 512.2,-162.6 512.2,-156.6 512.2,-156.6 512.2,-87.6 512.2,-87.6 512.2,-81.6 518.2,-75.6 524.2,-75.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-157" font-family="Bitstream Vera Sans" font-size="8.00">GstQueue</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-147.25" font-family="Bitstream Vera Sans" font-size="8.00">queue0</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="598.7" y="-137.5" font-family="Bitstream Vera Sans" font-size="8.00">[>]</text>
|
||||||
|
</g>
|
||||||
|
<g id="clust4" class="cluster">
|
||||||
|
<title>cluster_node_queue0_000001FC4BB96850_sink</title>
|
||||||
|
</g>
|
||||||
|
<g id="clust5" class="cluster">
|
||||||
|
<title>cluster_node_queue0_000001FC4BB96850_src</title>
|
||||||
|
</g>
|
||||||
|
<g id="clust6" class="cluster">
|
||||||
|
<title>cluster_node_idsueyesrc0_000001FC4DD4B2C0</title>
|
||||||
|
<path fill="#ffaaaa" stroke="black" d="M81.98,-75.6C81.98,-75.6 241.98,-75.6 241.98,-75.6 247.98,-75.6 253.98,-81.6 253.98,-87.6 253.98,-87.6 253.98,-186.6 253.98,-186.6 253.98,-192.6 247.98,-198.6 241.98,-198.6 241.98,-198.6 81.98,-198.6 81.98,-198.6 75.98,-198.6 69.98,-192.6 69.98,-186.6 69.98,-186.6 69.98,-87.6 69.98,-87.6 69.98,-81.6 75.98,-75.6 81.98,-75.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-187" font-family="Bitstream Vera Sans" font-size="8.00">GstIdsueyeSrc</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-177.25" font-family="Bitstream Vera Sans" font-size="8.00">idsueyesrc0</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-167.5" font-family="Bitstream Vera Sans" font-size="8.00">[>]</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-157.75" font-family="Bitstream Vera Sans" font-size="8.00">config-file="ini/200fps-2456x4pix-cw.ini"</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-148" font-family="Bitstream Vera Sans" font-size="8.00">exposure=3.237784</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.98" y="-138.25" font-family="Bitstream Vera Sans" font-size="8.00">framerate=300.000000</text>
|
||||||
|
</g>
|
||||||
|
<g id="clust7" class="cluster">
|
||||||
|
<title>cluster_node_idsueyesrc0_000001FC4DD4B2C0_src</title>
|
||||||
|
</g>
|
||||||
|
<!-- legend -->
|
||||||
|
<g id="node1" class="node">
|
||||||
|
<title>legend</title>
|
||||||
|
<polygon fill="lightgrey" stroke="black" points="322.95,-67.2 0,-67.2 0,0 322.95,0 322.95,-67.2"/>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="3.6" y="-55.05" font-family="sans" font-size="9.00">Legend</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="3.6" y="-43.05" font-family="sans" font-size="9.00">Element-States: [~] void-pending, [0] null, [-] ready, [=] paused, [>] playing</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="3.6" y="-31.05" font-family="sans" font-size="9.00">Pad-Activation: [-] none, [>] push, [<] pull</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="3.6" y="-19.05" font-family="sans" font-size="9.00">Pad-Flags: [b]locked, [f]lushing, [b]locking, [E]OS; upper-case is set</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="3.6" y="-7.05" font-family="sans" font-size="9.00">Pad-Task: [T] has started task, [t] has paused task</text>
|
||||||
|
</g>
|
||||||
|
<!-- node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0 -->
|
||||||
|
<g id="node2" class="node">
|
||||||
|
<title>node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0</title>
|
||||||
|
<polygon fill="#aaaaff" stroke="black" points="1011.33,-115.6 957.33,-115.6 957.33,-91.6 1011.33,-91.6 1011.33,-115.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.33" y="-107.05" font-family="sans" font-size="9.00">sink</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="984.33" y="-95.05" font-family="sans" font-size="9.00">[>][bfb]</text>
|
||||||
|
</g>
|
||||||
|
<!-- node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0 -->
|
||||||
|
<g id="node3" class="node">
|
||||||
|
<title>node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0</title>
|
||||||
|
<polygon fill="#aaaaff" stroke="black" points="582.2,-115.6 528.2,-115.6 528.2,-91.6 582.2,-91.6 582.2,-115.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="555.2" y="-107.05" font-family="sans" font-size="9.00">sink</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="555.2" y="-95.05" font-family="sans" font-size="9.00">[>][bfb]</text>
|
||||||
|
</g>
|
||||||
|
<!-- node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70 -->
|
||||||
|
<g id="node4" class="node">
|
||||||
|
<title>node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70</title>
|
||||||
|
<polygon fill="#ffaaaa" stroke="black" points="669.2,-115.6 615.2,-115.6 615.2,-91.6 669.2,-91.6 669.2,-115.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="642.2" y="-107.05" font-family="sans" font-size="9.00">src</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="642.2" y="-95.05" font-family="sans" font-size="9.00">[>][bfb][T]</text>
|
||||||
|
</g>
|
||||||
|
<!-- node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0->node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70 -->
|
||||||
|
<!-- node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70->node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0 -->
|
||||||
|
<g id="edge2" class="edge">
|
||||||
|
<title>node_queue0_000001FC4BB96850_node_src_000001FC4DD60E70->node_udpsink0_000001FC4DD89DE0_node_sink_000001FC4DD609D0</title>
|
||||||
|
<path fill="none" stroke="black" d="M669.61,-103.6C729.66,-103.6 876.61,-103.6 945.95,-103.6"/>
|
||||||
|
<polygon fill="black" stroke="black" points="945.66,-107.1 955.66,-103.6 945.66,-100.1 945.66,-107.1"/>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-179.05" font-family="monospace" font-size="9.00">video/x-raw</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-168.55" font-family="monospace" font-size="9.00">              format: BGR</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-158.05" font-family="monospace" font-size="9.00">               width: 2456</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-147.55" font-family="monospace" font-size="9.00">              height: 4</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-137.05" font-family="monospace" font-size="9.00">      interlace-mode: progressive</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-126.55" font-family="monospace" font-size="9.00">  pixel-aspect-ratio: 1/1</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-116.05" font-family="monospace" font-size="9.00">         colorimetry: sRGB</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="693.2" y="-105.55" font-family="monospace" font-size="9.00">           framerate: 0/1</text>
|
||||||
|
</g>
|
||||||
|
<!-- node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0 -->
|
||||||
|
<g id="node5" class="node">
|
||||||
|
<title>node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0</title>
|
||||||
|
<polygon fill="#ffaaaa" stroke="black" points="188.48,-115.6 134.48,-115.6 134.48,-91.6 188.48,-91.6 188.48,-115.6"/>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.48" y="-107.05" font-family="sans" font-size="9.00">src</text>
|
||||||
|
<text xml:space="preserve" text-anchor="middle" x="161.48" y="-95.05" font-family="sans" font-size="9.00">[>][bfb][T]</text>
|
||||||
|
</g>
|
||||||
|
<!-- node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0->node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0 -->
|
||||||
|
<g id="edge3" class="edge">
|
||||||
|
<title>node_idsueyesrc0_000001FC4DD4B2C0_node_src_000001FC4BB7ECA0->node_queue0_000001FC4BB96850_node_sink_000001FC4DD5F9A0</title>
|
||||||
|
<path fill="none" stroke="black" d="M188.79,-103.6C256.58,-103.6 437.44,-103.6 516.42,-103.6"/>
|
||||||
|
<polygon fill="black" stroke="black" points="516.23,-107.1 526.23,-103.6 516.23,-100.1 516.23,-107.1"/>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-179.05" font-family="monospace" font-size="9.00">video/x-raw</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-168.55" font-family="monospace" font-size="9.00">              format: BGR</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-158.05" font-family="monospace" font-size="9.00">               width: 2456</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-147.55" font-family="monospace" font-size="9.00">              height: 4</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-137.05" font-family="monospace" font-size="9.00">      interlace-mode: progressive</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-126.55" font-family="monospace" font-size="9.00">  pixel-aspect-ratio: 1/1</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-116.05" font-family="monospace" font-size="9.00">         colorimetry: sRGB</text>
|
||||||
|
<text xml:space="preserve" text-anchor="start" x="330.95" y="-105.55" font-family="monospace" font-size="9.00">           framerate: 0/1</text>
|
||||||
|
</g>
|
||||||
|
</g>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 12 KiB |
222
ini/200fps-2456x4pix-cw.ini
Normal file
222
ini/200fps-2456x4pix-cw.ini
Normal file
@ -0,0 +1,222 @@
|
|||||||
|
[Versions]
|
||||||
|
ueye_api_64.dll=4.93.1730
|
||||||
|
ueye_usb_64.sys=4.93.1314
|
||||||
|
ueye_boot_64.sys=4.93.1314
|
||||||
|
|
||||||
|
|
||||||
|
[Sensor]
|
||||||
|
Sensor=UI308xCP-C
|
||||||
|
Sensor bit depth=0
|
||||||
|
Sensor source gain=24
|
||||||
|
FPN correction mode=0
|
||||||
|
Black reference mode=0
|
||||||
|
Sensor digital gain=0
|
||||||
|
|
||||||
|
|
||||||
|
[Image size]
|
||||||
|
Start X=0
|
||||||
|
Start Y=0
|
||||||
|
Start X absolute=0
|
||||||
|
Start Y absolute=0
|
||||||
|
Width=2456
|
||||||
|
Height=4
|
||||||
|
Binning=0
|
||||||
|
Subsampling=0
|
||||||
|
|
||||||
|
|
||||||
|
[Scaler]
|
||||||
|
Mode=0
|
||||||
|
Factor=0.000000
|
||||||
|
|
||||||
|
|
||||||
|
[Multi AOI]
|
||||||
|
Enabled=0
|
||||||
|
Mode=0
|
||||||
|
x1=0
|
||||||
|
x2=0
|
||||||
|
x3=0
|
||||||
|
x4=0
|
||||||
|
y1=0
|
||||||
|
y2=0
|
||||||
|
y3=0
|
||||||
|
y4=0
|
||||||
|
|
||||||
|
|
||||||
|
[Shutter]
|
||||||
|
Mode=0
|
||||||
|
Linescan number=0
|
||||||
|
|
||||||
|
|
||||||
|
[Log Mode]
|
||||||
|
Mode=3
|
||||||
|
Manual value=0
|
||||||
|
Manual gain=0
|
||||||
|
|
||||||
|
|
||||||
|
[Timing]
|
||||||
|
Pixelclock=237
|
||||||
|
Extended pixelclock range=0
|
||||||
|
Framerate=200.151466
|
||||||
|
Exposure=4.903189
|
||||||
|
Long exposure=0
|
||||||
|
Dual exposure ratio=0
|
||||||
|
|
||||||
|
|
||||||
|
[Selected Converter]
|
||||||
|
IS_SET_CM_RGB32=2
|
||||||
|
IS_SET_CM_RGB24=2
|
||||||
|
IS_SET_CM_RGB16=2
|
||||||
|
IS_SET_CM_RGB15=2
|
||||||
|
IS_SET_CM_Y8=2
|
||||||
|
IS_SET_CM_RGB8=2
|
||||||
|
IS_SET_CM_BAYER=8
|
||||||
|
IS_SET_CM_UYVY=2
|
||||||
|
IS_SET_CM_UYVY_MONO=2
|
||||||
|
IS_SET_CM_UYVY_BAYER=2
|
||||||
|
IS_CM_CBYCRY_PACKED=0
|
||||||
|
IS_SET_CM_RGBY=8
|
||||||
|
IS_SET_CM_RGB30=2
|
||||||
|
IS_SET_CM_Y12=2
|
||||||
|
IS_SET_CM_BAYER12=8
|
||||||
|
IS_SET_CM_Y16=2
|
||||||
|
IS_SET_CM_BAYER16=8
|
||||||
|
IS_CM_BGR12_UNPACKED=2
|
||||||
|
IS_CM_BGRA12_UNPACKED=2
|
||||||
|
IS_CM_JPEG=0
|
||||||
|
IS_CM_SENSOR_RAW10=8
|
||||||
|
IS_CM_MONO10=2
|
||||||
|
IS_CM_BGR10_UNPACKED=2
|
||||||
|
IS_CM_RGBA8_PACKED=2
|
||||||
|
IS_CM_RGB8_PACKED=2
|
||||||
|
IS_CM_RGBY8_PACKED=8
|
||||||
|
IS_CM_RGB10V2_PACKED=8
|
||||||
|
IS_CM_RGB12_UNPACKED=2
|
||||||
|
IS_CM_RGBA12_UNPACKED=2
|
||||||
|
IS_CM_RGB10_UNPACKED=2
|
||||||
|
IS_CM_RGB8_PLANAR=2
|
||||||
|
|
||||||
|
|
||||||
|
[Parameters]
|
||||||
|
Colormode=1
|
||||||
|
Gamma=1.000000
|
||||||
|
Hardware Gamma=0
|
||||||
|
Blacklevel Mode=0
|
||||||
|
Blacklevel Offset=4
|
||||||
|
Hotpixel Mode=2
|
||||||
|
Hotpixel Threshold=0
|
||||||
|
Sensor Hotpixel=0
|
||||||
|
Adaptive hotpixel correction enable=0
|
||||||
|
Adaptive hotpixel correction mode=0
|
||||||
|
Adaptive hotpixel correction sensitivity=3
|
||||||
|
GlobalShutter=0
|
||||||
|
AllowRawWithLut=0
|
||||||
|
|
||||||
|
|
||||||
|
[Gain]
|
||||||
|
Master=0
|
||||||
|
Red=19
|
||||||
|
Green=0
|
||||||
|
Blue=33
|
||||||
|
GainBoost=1
|
||||||
|
|
||||||
|
|
||||||
|
[Processing]
|
||||||
|
EdgeEnhancementFactor=0
|
||||||
|
RopEffect=0
|
||||||
|
Whitebalance=0
|
||||||
|
Whitebalance Red=1.000000
|
||||||
|
Whitebalance Green=1.000000
|
||||||
|
Whitebalance Blue=1.000000
|
||||||
|
Color correction=4
|
||||||
|
Color_correction_factor=1.000000
|
||||||
|
Color_correction_satU=100
|
||||||
|
Color_correction_satV=100
|
||||||
|
Bayer Conversion=1
|
||||||
|
JpegCompression=0
|
||||||
|
NoiseMode=0
|
||||||
|
ImageEffect=0
|
||||||
|
LscModel=0
|
||||||
|
WideDynamicRange=0
|
||||||
|
|
||||||
|
|
||||||
|
[Auto features]
|
||||||
|
Auto Framerate control=0
|
||||||
|
Brightness exposure control=0
|
||||||
|
Brightness gain control=0
|
||||||
|
Auto Framerate Sensor control=0
|
||||||
|
Brightness exposure Sensor control=0
|
||||||
|
Brightness gain Sensor control=0
|
||||||
|
Brightness exposure Sensor control photometry=0
|
||||||
|
Brightness gain Sensor control photometry=0
|
||||||
|
Brightness control once=0
|
||||||
|
Brightness reference=128
|
||||||
|
Brightness speed=50
|
||||||
|
Brightness max gain=100
|
||||||
|
Brightness max exposure=4.903189
|
||||||
|
Brightness Aoi Left=0
|
||||||
|
Brightness Aoi Top=0
|
||||||
|
Brightness Aoi Width=2456
|
||||||
|
Brightness Aoi Height=4
|
||||||
|
Brightness Hysteresis=2
|
||||||
|
AutoImageControlMode=2
|
||||||
|
AutoImageControlPeakWhiteChannel=0
|
||||||
|
AutoImageControlExposureMinimum=0.000000
|
||||||
|
AutoImageControlPeakWhiteChannelMode=0
|
||||||
|
AutoImageControlPeakWhiteGranularity=0
|
||||||
|
Auto WB control=0
|
||||||
|
Auto WB type=2
|
||||||
|
Auto WB RGB color model=1
|
||||||
|
Auto WB RGB color temperature=5000
|
||||||
|
Auto WB offsetR=0
|
||||||
|
Auto WB offsetB=0
|
||||||
|
Auto WB gainMin=0
|
||||||
|
Auto WB gainMax=100
|
||||||
|
Auto WB speed=50
|
||||||
|
Auto WB Aoi Left=0
|
||||||
|
Auto WB Aoi Top=0
|
||||||
|
Auto WB Aoi Width=2456
|
||||||
|
Auto WB Aoi Height=4
|
||||||
|
Auto WB Once=0
|
||||||
|
Auto WB Hysteresis=2
|
||||||
|
Brightness Skip Frames Trigger Mode=4
|
||||||
|
Brightness Skip Frames Freerun Mode=4
|
||||||
|
Auto WB Skip Frames Trigger Mode=4
|
||||||
|
Auto WB Skip Frames Freerun Mode=4
|
||||||
|
|
||||||
|
|
||||||
|
[Trigger and Flash]
|
||||||
|
Trigger mode=0
|
||||||
|
Trigger timeout=200
|
||||||
|
Trigger delay=0
|
||||||
|
Trigger debounce mode=0
|
||||||
|
Trigger debounce delay time=1
|
||||||
|
Trigger burst size=1
|
||||||
|
Trigger prescaler frame=1
|
||||||
|
Trigger prescaler line=1
|
||||||
|
Trigger input=1
|
||||||
|
Flash strobe=0
|
||||||
|
Flash delay=0
|
||||||
|
Flash duration=0
|
||||||
|
Flash auto freerun=0
|
||||||
|
PWM mode=0
|
||||||
|
PWM frequency=20000000
|
||||||
|
PWM dutycycle=20000000
|
||||||
|
GPIO state=3
|
||||||
|
GPIO direction=0
|
||||||
|
GPIO1 Config=1
|
||||||
|
GPIO2 Config=1
|
||||||
|
|
||||||
|
|
||||||
|
[Vertical AOI Merge Mode]
|
||||||
|
Mode=0
|
||||||
|
Position=0
|
||||||
|
Additional Position=0
|
||||||
|
Height=2
|
||||||
|
|
||||||
|
|
||||||
|
[Level Controlled Trigger Mode]
|
||||||
|
Mode=0
|
||||||
|
|
||||||
|
|
||||||
|
[Memory]
|
||||||
|
Camera memory mode=1
|
||||||
39
network_guide.md
Normal file
39
network_guide.md
Normal file
@ -0,0 +1,39 @@
|
|||||||
|
# how to send a line
|
||||||
|
|
||||||
|
real data
|
||||||
|
```powershell
|
||||||
|
gst-launch-1.0 idsueyesrc config-file=ini/200fps-2456x4pix-cw.ini exposure=5 framerate=300 `
|
||||||
|
! queue `
|
||||||
|
! udpsink host=127.0.0.1 port=5000
|
||||||
|
```
|
||||||
|
|
||||||
|
note: 5ms is bit too fast for us
|
||||||
|
|
||||||
|
### Python/OpenCV Receiver
|
||||||
|
```pwsh
|
||||||
|
uv run scripts/recv_raw_column.py
|
||||||
|
```
|
||||||
|
or rolling like
|
||||||
|
```pwsh
|
||||||
|
uv run .\scripts\recv_raw_rolling.py
|
||||||
|
```
|
||||||
|
See [`scripts/recv_raw_column.py`](scripts/recv_raw_column.py) for the Python implementation with debug options.
|
||||||
|
|
||||||
|
|
||||||
|
# demo data
|
||||||
|
## Sender (crop to first column, send raw over UDP)
|
||||||
|
```pwsh
|
||||||
|
gst-launch-1.0 -v `
|
||||||
|
videotestsrc pattern=smpte ! `
|
||||||
|
videocrop left=0 right=639 top=0 bottom=0 ! `
|
||||||
|
video/x-raw,format=RGB,width=1,height=640,framerate=30/1 ! `
|
||||||
|
udpsink host=127.0.0.1 port=5000
|
||||||
|
```
|
||||||
|
|
||||||
|
### GStreamer Receiver (raw UDP → display)
|
||||||
|
```pwsh
|
||||||
|
gst-launch-1.0 -v `
|
||||||
|
udpsrc port=5000 caps="video/x-raw,format=RGB,width=1,height=640,framerate=30/1" ! `
|
||||||
|
videoconvert ! `
|
||||||
|
autovideosink
|
||||||
|
```
|
||||||
@ -18,11 +18,17 @@ import pandas as pd
|
|||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from datetime import datetime
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
|
||||||
def analyze_csv(csv_file: str = "output.csv"):
|
def analyze_csv(csv_file: str = "output.csv"):
|
||||||
"""Analyze the rolling sum CSV data and generate insights."""
|
"""Analyze the rolling sum CSV data and generate insights."""
|
||||||
|
|
||||||
|
# Create output directory
|
||||||
|
output_dir = Path("results/debug")
|
||||||
|
output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
# Read the CSV
|
# Read the CSV
|
||||||
try:
|
try:
|
||||||
df = pd.read_csv(csv_file)
|
df = pd.read_csv(csv_file)
|
||||||
@ -30,6 +36,12 @@ def analyze_csv(csv_file: str = "output.csv"):
|
|||||||
print(f"Error: CSV file '{csv_file}' not found.")
|
print(f"Error: CSV file '{csv_file}' not found.")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Copy input CSV to results directory with timestamp
|
||||||
|
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||||
|
csv_name = Path(csv_file).stem
|
||||||
|
archived_csv = output_dir / f"{csv_name}_{timestamp}.csv"
|
||||||
|
shutil.copy(csv_file, archived_csv)
|
||||||
|
|
||||||
print("=" * 80)
|
print("=" * 80)
|
||||||
print(f"ROLLING SUM ANALYSIS - {csv_file}")
|
print(f"ROLLING SUM ANALYSIS - {csv_file}")
|
||||||
print("=" * 80)
|
print("=" * 80)
|
||||||
@ -98,16 +110,17 @@ def analyze_csv(csv_file: str = "output.csv"):
|
|||||||
print()
|
print()
|
||||||
|
|
||||||
# Create visualizations
|
# Create visualizations
|
||||||
create_plots(df, csv_file)
|
plot_file = create_plots(df, csv_file, output_dir, timestamp)
|
||||||
|
|
||||||
print("=" * 80)
|
print("=" * 80)
|
||||||
print("PLOTS SAVED:")
|
print("OUTPUT FILES:")
|
||||||
print(f" - {csv_file.replace('.csv', '_analysis.png')}")
|
print(f" CSV Archive: {archived_csv}")
|
||||||
|
print(f" Analysis Plot: {plot_file}")
|
||||||
print("=" * 80)
|
print("=" * 80)
|
||||||
|
|
||||||
|
|
||||||
def create_plots(df: pd.DataFrame, csv_file: str):
|
def create_plots(df: pd.DataFrame, csv_file: str, output_dir: Path, timestamp: str) -> Path:
|
||||||
"""Create analysis plots."""
|
"""Create analysis plots and return the output file path."""
|
||||||
|
|
||||||
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
|
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
|
||||||
fig.suptitle(f'Rolling Sum Analysis - {csv_file}', fontsize=16, fontweight='bold')
|
fig.suptitle(f'Rolling Sum Analysis - {csv_file}', fontsize=16, fontweight='bold')
|
||||||
@ -167,10 +180,13 @@ def create_plots(df: pd.DataFrame, csv_file: str):
|
|||||||
|
|
||||||
plt.tight_layout()
|
plt.tight_layout()
|
||||||
|
|
||||||
# Save the plot
|
# Save the plot to results/debug
|
||||||
output_file = csv_file.replace('.csv', '_analysis.png')
|
csv_name = Path(csv_file).stem
|
||||||
|
output_file = output_dir / f"{csv_name}_analysis_{timestamp}.png"
|
||||||
plt.savefig(output_file, dpi=150, bbox_inches='tight')
|
plt.savefig(output_file, dpi=150, bbox_inches='tight')
|
||||||
print(f"\n✓ Saved analysis plot to: {output_file}\n")
|
print(f"\n✓ Saved analysis plot to: {output_file}\n")
|
||||||
|
|
||||||
|
return output_file
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
61
scripts/recv_raw_column.py
Normal file
61
scripts/recv_raw_column.py
Normal file
@ -0,0 +1,61 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
# /// script
|
||||||
|
# requires-python = "<=3.10"
|
||||||
|
# dependencies = [
|
||||||
|
# "opencv-python",
|
||||||
|
# "numpy",
|
||||||
|
# ]
|
||||||
|
# ///
|
||||||
|
|
||||||
|
import socket
|
||||||
|
import numpy as np
|
||||||
|
import cv2
|
||||||
|
|
||||||
|
# Debug flag - set to True to see frame reception details
|
||||||
|
DEBUG = False
|
||||||
|
|
||||||
|
# Stream parameters (match your GStreamer sender)
|
||||||
|
WIDTH = 1
|
||||||
|
HEIGHT = 640 # Default videotestsrc height
|
||||||
|
CHANNELS = 3
|
||||||
|
FRAME_SIZE = WIDTH * HEIGHT * CHANNELS # bytes
|
||||||
|
|
||||||
|
UDP_IP = "0.0.0.0"
|
||||||
|
UDP_PORT = 5000
|
||||||
|
|
||||||
|
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||||
|
sock.bind((UDP_IP, UDP_PORT))
|
||||||
|
|
||||||
|
print(f"Receiving raw {WIDTH}x{HEIGHT} RGB frames on UDP port {UDP_PORT}")
|
||||||
|
if DEBUG:
|
||||||
|
print(f"Expected frame size: {FRAME_SIZE} bytes")
|
||||||
|
|
||||||
|
cv2.namedWindow("Raw Column Stream", cv2.WINDOW_NORMAL)
|
||||||
|
|
||||||
|
frame_count = 0
|
||||||
|
while True:
|
||||||
|
data, addr = sock.recvfrom(65536)
|
||||||
|
|
||||||
|
if len(data) != FRAME_SIZE:
|
||||||
|
if DEBUG:
|
||||||
|
print(f"Received {len(data)} bytes (expected {FRAME_SIZE}), skipping...")
|
||||||
|
continue
|
||||||
|
|
||||||
|
if DEBUG:
|
||||||
|
frame_count += 1
|
||||||
|
if frame_count % 30 == 0:
|
||||||
|
print(f"Received {frame_count} frames")
|
||||||
|
|
||||||
|
frame = np.frombuffer(data, dtype=np.uint8).reshape((HEIGHT, WIDTH, CHANNELS))
|
||||||
|
|
||||||
|
# scale for visibility
|
||||||
|
frame_large = cv2.resize(
|
||||||
|
frame, (200, HEIGHT), interpolation=cv2.INTER_NEAREST
|
||||||
|
)
|
||||||
|
|
||||||
|
cv2.imshow("Raw Column Stream", frame_large)
|
||||||
|
|
||||||
|
if cv2.waitKey(1) == 27: # ESC to quit
|
||||||
|
break
|
||||||
|
|
||||||
|
cv2.destroyAllWindows()
|
||||||
86
scripts/recv_raw_rolling.py
Normal file
86
scripts/recv_raw_rolling.py
Normal file
@ -0,0 +1,86 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
# /// script
|
||||||
|
# requires-python = "<=3.10"
|
||||||
|
# dependencies = [
|
||||||
|
# "opencv-python",
|
||||||
|
# "numpy",
|
||||||
|
# ]
|
||||||
|
# ///
|
||||||
|
|
||||||
|
import socket
|
||||||
|
import numpy as np
|
||||||
|
import cv2
|
||||||
|
|
||||||
|
# Debug flag - set to True to see frame reception details
|
||||||
|
DEBUG = False
|
||||||
|
|
||||||
|
# Rotation mode - set to rotate incoming data before display
|
||||||
|
# Options: None, cv2.ROTATE_90_CLOCKWISE, cv2.ROTATE_90_COUNTERCLOCKWISE, cv2.ROTATE_180
|
||||||
|
ROTATION = cv2.ROTATE_90_COUNTERCLOCKWISE # Rotate rows to columns
|
||||||
|
|
||||||
|
# Stream parameters (match your GStreamer sender)
|
||||||
|
COLUMN_WIDTH = 4 # Width from 200fps-2456x4pix-cw.ini
|
||||||
|
COLUMN_HEIGHT = 2456 # Height from 200fps-2456x4pix-cw.ini
|
||||||
|
CHANNELS = 3
|
||||||
|
FRAME_SIZE = COLUMN_WIDTH * COLUMN_HEIGHT * CHANNELS # bytes (29472)
|
||||||
|
|
||||||
|
# Display parameters
|
||||||
|
DISPLAY_WIDTH = 800 # Width of rolling display in pixels
|
||||||
|
DISPLAY_HEIGHT = COLUMN_HEIGHT
|
||||||
|
|
||||||
|
UDP_IP = "0.0.0.0"
|
||||||
|
UDP_PORT = 5000
|
||||||
|
|
||||||
|
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||||
|
sock.bind((UDP_IP, UDP_PORT))
|
||||||
|
|
||||||
|
print(f"Receiving raw {COLUMN_WIDTH}x{COLUMN_HEIGHT} RGB columns on UDP port {UDP_PORT}")
|
||||||
|
print(f"Display width: {DISPLAY_WIDTH} pixels (rolling)")
|
||||||
|
if DEBUG:
|
||||||
|
print(f"Expected frame size: {FRAME_SIZE} bytes")
|
||||||
|
|
||||||
|
cv2.namedWindow("Rolling Column Stream", cv2.WINDOW_NORMAL)
|
||||||
|
|
||||||
|
# Initialize the rolling buffer
|
||||||
|
rolling_buffer = np.zeros((DISPLAY_HEIGHT, DISPLAY_WIDTH, CHANNELS), dtype=np.uint8)
|
||||||
|
current_column = 0
|
||||||
|
frame_count = 0
|
||||||
|
|
||||||
|
while True:
|
||||||
|
data, addr = sock.recvfrom(65536)
|
||||||
|
|
||||||
|
if len(data) != FRAME_SIZE:
|
||||||
|
if DEBUG:
|
||||||
|
print(f"Received {len(data)} bytes (expected {FRAME_SIZE}), skipping...")
|
||||||
|
continue
|
||||||
|
|
||||||
|
if DEBUG:
|
||||||
|
frame_count += 1
|
||||||
|
if frame_count % 30 == 0:
|
||||||
|
print(f"Received {frame_count} frames, current column: {current_column}")
|
||||||
|
|
||||||
|
# Parse the incoming data
|
||||||
|
frame = np.frombuffer(data, dtype=np.uint8).reshape((COLUMN_WIDTH, COLUMN_HEIGHT, CHANNELS))
|
||||||
|
|
||||||
|
# Apply rotation if configured
|
||||||
|
if ROTATION is not None:
|
||||||
|
rotated = cv2.rotate(frame, ROTATION)
|
||||||
|
else:
|
||||||
|
rotated = frame
|
||||||
|
|
||||||
|
# Extract only the first column for smoother rolling (1 pixel/frame)
|
||||||
|
column = rotated[:, 0:1, :]
|
||||||
|
|
||||||
|
# Insert the single column into the rolling buffer at the current position
|
||||||
|
rolling_buffer[:, current_column:current_column+1, :] = column
|
||||||
|
|
||||||
|
# Move to the next column position, wrapping around when reaching the end
|
||||||
|
current_column = (current_column + 1) % DISPLAY_WIDTH
|
||||||
|
|
||||||
|
# Display the rolling buffer
|
||||||
|
cv2.imshow("Rolling Column Stream", rolling_buffer)
|
||||||
|
|
||||||
|
if cv2.waitKey(1) == 27: # ESC to quit
|
||||||
|
break
|
||||||
|
|
||||||
|
cv2.destroyAllWindows()
|
||||||
Loading…
x
Reference in New Issue
Block a user