Add brightness temporal smoothing to reduce oscillation from moving objects
- Added brightness-smoothing parameter (0-1, default 0.1) - Implements exponential moving average to filter transient brightness changes - Samples brightness every frame but smooths before adjusting exposure - Reduces oscillation from people/cars/birds moving through scene - Updated DEBUG.md with complete implementation details Recommended settings for dawn/dusk time-lapse: ramp-rate=vslow update-interval=1000 brightness-smoothing=0.1
This commit is contained in:
@@ -352,7 +352,107 @@ Frame 199: 0.282ms brightness 117
|
|||||||
- Proper min/max values queried from camera hardware
|
- Proper min/max values queried from camera hardware
|
||||||
- All values stay within valid range [0.019 - 19.943] ms
|
- All values stay within valid range [0.019 - 19.943] ms
|
||||||
|
|
||||||
The intervalometer auto-exposure system is now fully functional and flicker-free!
|
The intervalometer auto-exposure system is now fully functional!
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Remaining Issue: Oscillation from Moving Objects
|
||||||
|
|
||||||
|
### Problem Description
|
||||||
|
|
||||||
|
Even with `vslow` ramp rate and 1000ms updates, small oscillations occur when:
|
||||||
|
- Objects move through the scene (people, cars, birds)
|
||||||
|
- Temporarily changing the average brightness
|
||||||
|
- Algorithm reacts to transient changes, not background light
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```
|
||||||
|
Frame 100: brightness 128 → exposure 0.500ms
|
||||||
|
Frame 150: person walks by → brightness 140 → exposure starts increasing
|
||||||
|
Frame 200: person gone → brightness 128 → exposure starts decreasing
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates a "breathing" effect as the algorithm chases temporary brightness changes.
|
||||||
|
|
||||||
|
### Root Cause: No Temporal Filtering
|
||||||
|
|
||||||
|
Currently, the algorithm:
|
||||||
|
1. ✅ Calculates brightness every frame
|
||||||
|
2. ❌ Uses that single frame's brightness directly for exposure decisions
|
||||||
|
3. ❌ Reacts to transient objects instead of background lighting trend
|
||||||
|
|
||||||
|
### Solution: Decouple Sampling from Adjustment
|
||||||
|
|
||||||
|
**YASS approach** (which we should adopt):
|
||||||
|
- **Sample brightness:** Every frame (high temporal resolution)
|
||||||
|
- **Smooth brightness:** Exponential moving average or rolling average
|
||||||
|
- **Adjust exposure:** Only based on the smoothed brightness value
|
||||||
|
|
||||||
|
This filters out transient changes while staying responsive to actual lighting changes.
|
||||||
|
|
||||||
|
### Proposed Implementation
|
||||||
|
|
||||||
|
Add brightness smoothing with exponential moving average (EMA):
|
||||||
|
|
||||||
|
```c
|
||||||
|
/* In GstIntervalometer struct, add: */
|
||||||
|
gdouble smoothed_brightness; /* Exponentially smoothed brightness */
|
||||||
|
gdouble brightness_alpha; /* Smoothing factor (0-1) */
|
||||||
|
|
||||||
|
/* In transform_ip, replace direct brightness use: */
|
||||||
|
// Current (reacts to every frame):
|
||||||
|
gst_intervalometer_update_camera_settings(filter, brightness);
|
||||||
|
|
||||||
|
// Improved (reacts to trend):
|
||||||
|
filter->smoothed_brightness = (brightness_alpha * brightness) +
|
||||||
|
((1.0 - brightness_alpha) * filter->smoothed_brightness);
|
||||||
|
gst_intervalometer_update_camera_settings(filter, filter->smoothed_brightness);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
- `brightness_alpha = 0.1` → heavily smoothed (recommended for time-lapse)
|
||||||
|
- `brightness_alpha = 0.3` → moderately smoothed
|
||||||
|
- `brightness_alpha = 0.5` → lightly smoothed
|
||||||
|
- `brightness_alpha = 1.0` → no smoothing (current behavior)
|
||||||
|
|
||||||
|
With alpha=0.1:
|
||||||
|
- New frame contributes 10%
|
||||||
|
- History contributes 90%
|
||||||
|
- Effectively ~10 frame averaging
|
||||||
|
- Transient objects have minimal impact
|
||||||
|
|
||||||
|
### Alternative: Configurable Brightness Averaging Window
|
||||||
|
|
||||||
|
Add a new property `brightness-window` (number of frames to average):
|
||||||
|
|
||||||
|
```c
|
||||||
|
/* Rolling average over N frames */
|
||||||
|
guint brightness_window; /* e.g., 50 frames = 1 second at 50fps */
|
||||||
|
gdouble brightness_history[256]; /* Circular buffer */
|
||||||
|
guint brightness_index; /* Current position in buffer */
|
||||||
|
```
|
||||||
|
|
||||||
|
This gives users direct control: "average brightness over last N frames"
|
||||||
|
|
||||||
|
### Recommendation
|
||||||
|
|
||||||
|
**For dawn/dusk time-lapse with moving objects:**
|
||||||
|
```bash
|
||||||
|
intervalometer enabled=true camera-element=cam \
|
||||||
|
ramp-rate=vslow \
|
||||||
|
update-interval=1000 \
|
||||||
|
brightness-alpha=0.1 # (new parameter - to be implemented)
|
||||||
|
```
|
||||||
|
|
||||||
|
Or once averaging is implemented:
|
||||||
|
```bash
|
||||||
|
intervalometer enabled=true camera-element=cam \
|
||||||
|
ramp-rate=vslow \
|
||||||
|
update-interval=1000 \
|
||||||
|
brightness-window=50 # Average over 50 frames (1 sec at 50fps)
|
||||||
|
```
|
||||||
|
|
||||||
|
This will make the algorithm **ignore transient brightness spikes** from moving objects and focus on the **actual background lighting trend**.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -61,7 +61,8 @@ enum
|
|||||||
PROP_RAMP_RATE,
|
PROP_RAMP_RATE,
|
||||||
PROP_LOG_FILE,
|
PROP_LOG_FILE,
|
||||||
PROP_CAMERA_ELEMENT,
|
PROP_CAMERA_ELEMENT,
|
||||||
PROP_UPDATE_INTERVAL
|
PROP_UPDATE_INTERVAL,
|
||||||
|
PROP_BRIGHTNESS_SMOOTHING
|
||||||
};
|
};
|
||||||
|
|
||||||
#define DEFAULT_PROP_ENABLED TRUE
|
#define DEFAULT_PROP_ENABLED TRUE
|
||||||
@@ -75,6 +76,7 @@ enum
|
|||||||
#define DEFAULT_PROP_LOG_FILE ""
|
#define DEFAULT_PROP_LOG_FILE ""
|
||||||
#define DEFAULT_PROP_CAMERA_ELEMENT ""
|
#define DEFAULT_PROP_CAMERA_ELEMENT ""
|
||||||
#define DEFAULT_PROP_UPDATE_INTERVAL 100 /* Update every 100ms (10 Hz) */
|
#define DEFAULT_PROP_UPDATE_INTERVAL 100 /* Update every 100ms (10 Hz) */
|
||||||
|
#define DEFAULT_PROP_BRIGHTNESS_SMOOTHING 0.1 /* 10% new, 90% history - heavy smoothing for time-lapse */
|
||||||
|
|
||||||
/* GStreamer boilerplate */
|
/* GStreamer boilerplate */
|
||||||
#define gst_intervalometer_parent_class parent_class
|
#define gst_intervalometer_parent_class parent_class
|
||||||
@@ -232,6 +234,14 @@ gst_intervalometer_class_init (GstIntervalometerClass * klass)
|
|||||||
0, 10000, DEFAULT_PROP_UPDATE_INTERVAL,
|
0, 10000, DEFAULT_PROP_UPDATE_INTERVAL,
|
||||||
G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
|
G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
|
||||||
|
|
||||||
|
g_object_class_install_property (gobject_class, PROP_BRIGHTNESS_SMOOTHING,
|
||||||
|
g_param_spec_double ("brightness-smoothing", "Brightness Smoothing",
|
||||||
|
"Exponential smoothing factor for brightness (0=no smoothing, 1=no history). "
|
||||||
|
"Lower values (0.05-0.2) filter out transient objects like people/cars. "
|
||||||
|
"Recommended: 0.1 for dawn/dusk time-lapse",
|
||||||
|
0.0, 1.0, DEFAULT_PROP_BRIGHTNESS_SMOOTHING,
|
||||||
|
G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
|
||||||
|
|
||||||
/* Set element metadata */
|
/* Set element metadata */
|
||||||
gst_element_class_add_pad_template (gstelement_class,
|
gst_element_class_add_pad_template (gstelement_class,
|
||||||
gst_static_pad_template_get (&gst_intervalometer_sink_template));
|
gst_static_pad_template_get (&gst_intervalometer_sink_template));
|
||||||
@@ -267,6 +277,7 @@ gst_intervalometer_init (GstIntervalometer * filter)
|
|||||||
filter->log_file = g_strdup (DEFAULT_PROP_LOG_FILE);
|
filter->log_file = g_strdup (DEFAULT_PROP_LOG_FILE);
|
||||||
filter->camera_element_name = g_strdup (DEFAULT_PROP_CAMERA_ELEMENT);
|
filter->camera_element_name = g_strdup (DEFAULT_PROP_CAMERA_ELEMENT);
|
||||||
filter->update_interval = DEFAULT_PROP_UPDATE_INTERVAL;
|
filter->update_interval = DEFAULT_PROP_UPDATE_INTERVAL;
|
||||||
|
filter->brightness_smoothing = DEFAULT_PROP_BRIGHTNESS_SMOOTHING;
|
||||||
|
|
||||||
/* Initialize internal state */
|
/* Initialize internal state */
|
||||||
filter->camera_src = NULL;
|
filter->camera_src = NULL;
|
||||||
@@ -282,6 +293,8 @@ gst_intervalometer_init (GstIntervalometer * filter)
|
|||||||
filter->log_header_written = FALSE;
|
filter->log_header_written = FALSE;
|
||||||
filter->video_info_valid = FALSE;
|
filter->video_info_valid = FALSE;
|
||||||
filter->ramp_step = 1.0;
|
filter->ramp_step = 1.0;
|
||||||
|
filter->smoothed_brightness = DEFAULT_PROP_TARGET_BRIGHTNESS;
|
||||||
|
filter->brightness_initialized = FALSE;
|
||||||
|
|
||||||
/* Set in-place transform */
|
/* Set in-place transform */
|
||||||
gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE);
|
gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE);
|
||||||
@@ -334,6 +347,9 @@ gst_intervalometer_set_property (GObject * object, guint prop_id,
|
|||||||
case PROP_UPDATE_INTERVAL:
|
case PROP_UPDATE_INTERVAL:
|
||||||
filter->update_interval = g_value_get_uint (value);
|
filter->update_interval = g_value_get_uint (value);
|
||||||
break;
|
break;
|
||||||
|
case PROP_BRIGHTNESS_SMOOTHING:
|
||||||
|
filter->brightness_smoothing = g_value_get_double (value);
|
||||||
|
break;
|
||||||
default:
|
default:
|
||||||
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
||||||
break;
|
break;
|
||||||
@@ -380,6 +396,9 @@ gst_intervalometer_get_property (GObject * object, guint prop_id,
|
|||||||
case PROP_UPDATE_INTERVAL:
|
case PROP_UPDATE_INTERVAL:
|
||||||
g_value_set_uint (value, filter->update_interval);
|
g_value_set_uint (value, filter->update_interval);
|
||||||
break;
|
break;
|
||||||
|
case PROP_BRIGHTNESS_SMOOTHING:
|
||||||
|
g_value_set_double (value, filter->brightness_smoothing);
|
||||||
|
break;
|
||||||
default:
|
default:
|
||||||
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
|
||||||
break;
|
break;
|
||||||
@@ -532,9 +551,21 @@ gst_intervalometer_transform_ip (GstBaseTransform * trans, GstBuffer * buf)
|
|||||||
/* Always calculate brightness for logging */
|
/* Always calculate brightness for logging */
|
||||||
brightness = gst_intervalometer_calculate_brightness (filter, buf);
|
brightness = gst_intervalometer_calculate_brightness (filter, buf);
|
||||||
|
|
||||||
|
/* Apply exponential moving average for brightness smoothing */
|
||||||
|
if (!filter->brightness_initialized) {
|
||||||
|
/* Initialize smoothed brightness on first frame */
|
||||||
|
filter->smoothed_brightness = brightness;
|
||||||
|
filter->brightness_initialized = TRUE;
|
||||||
|
} else {
|
||||||
|
/* Exponential moving average: new_value = alpha * current + (1-alpha) * old */
|
||||||
|
filter->smoothed_brightness = (filter->brightness_smoothing * brightness) +
|
||||||
|
((1.0 - filter->brightness_smoothing) * filter->smoothed_brightness);
|
||||||
|
}
|
||||||
|
|
||||||
/* Only update camera settings at the specified interval */
|
/* Only update camera settings at the specified interval */
|
||||||
if (should_update) {
|
if (should_update) {
|
||||||
gst_intervalometer_update_camera_settings (filter, brightness);
|
/* Use smoothed brightness to filter out transient object movements */
|
||||||
|
gst_intervalometer_update_camera_settings (filter, filter->smoothed_brightness);
|
||||||
filter->last_update_time = now;
|
filter->last_update_time = now;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -74,6 +74,7 @@ struct _GstIntervalometer
|
|||||||
gchar *log_file; /* CSV log file path */
|
gchar *log_file; /* CSV log file path */
|
||||||
gchar *camera_element_name; /* Name of upstream idsueyesrc element */
|
gchar *camera_element_name; /* Name of upstream idsueyesrc element */
|
||||||
guint update_interval; /* Update interval in milliseconds */
|
guint update_interval; /* Update interval in milliseconds */
|
||||||
|
gdouble brightness_smoothing; /* Brightness smoothing factor (0-1, 0=no smoothing) */
|
||||||
|
|
||||||
/* Internal state */
|
/* Internal state */
|
||||||
GstElement *camera_src; /* Reference to upstream camera element */
|
GstElement *camera_src; /* Reference to upstream camera element */
|
||||||
@@ -94,6 +95,10 @@ struct _GstIntervalometer
|
|||||||
GstVideoInfo video_info;
|
GstVideoInfo video_info;
|
||||||
gboolean video_info_valid;
|
gboolean video_info_valid;
|
||||||
|
|
||||||
|
/* Brightness smoothing */
|
||||||
|
gdouble smoothed_brightness; /* Exponentially smoothed brightness value */
|
||||||
|
gboolean brightness_initialized; /* Whether smoothed_brightness has been initialized */
|
||||||
|
|
||||||
/* Ramping parameters */
|
/* Ramping parameters */
|
||||||
gdouble ramp_step; /* Current ramping step size */
|
gdouble ramp_step; /* Current ramping step size */
|
||||||
};
|
};
|
||||||
|
|||||||
Reference in New Issue
Block a user